" :stream sock))
Best Regards,
--
Aycan iRiCAN
http://people.core.gen.tr/~aycan.irican/
From sionescu at common-lisp.net Fri Jun 1 13:21:16 2007
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Fri, 1 Jun 2007 15:21:16 +0200
Subject: [iolib-devel] event handlers not working
In-Reply-To: <465FF7B7.9090700@core.gen.tr>
References: <465FF7B7.9090700@core.gen.tr>
Message-ID: <20070601132116.GA18793@universe.org>
On Fri, Jun 01, 2007 at 01:40:55PM +0300, Aycan iRiCAN wrote:
>I'm trying to write a simple example. But I cannot make the event
>handler work. Could you please check this code (mostly stolen from
>philip-jose).
you forgot to run the event loop :)
I've attached a slightly modified version of the code
--
(sign :name "Stelian Ionescu" :aka "fe[nl]ix"
:quote "Quidquid latine dictum sit, altum videtur.")
-------------- next part --------------
;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp -*-
(in-package :common-lisp-user)
(defpackage :iolib-test
(:use #:common-lisp #:net.sockets #:iomux #:bordeaux-threads))
(in-package :iolib-test)
(defvar *server*)
(defvar *event-base* (make-instance 'iomux:event-base))
(defvar *event-loop-thread*)
(defvar *server-event*)
(defun handle-connection (sock handler)
(unwind-protect
(progn
(funcall handler sock)
(finish-output sock))
(close sock)))
(defun make-server (port)
(make-socket :address-family :internet :type :stream
:connect :passive :reuse-address t
:local-host +ipv4-unspecified+ :local-port port
:ipv6 nil))
(defun make-connection-worker (socket handler)
#'(lambda ()
(handle-connection socket handler)))
(defun connection-spawner (listener-socket handler)
#'(lambda (fd evtype)
(if (eql evtype :read)
(let ((connection (accept-connection listener-socket)))
(make-thread (make-connection-worker connection handler)
:name "connection worker"))
(format t "Error on FD ~A" fd))))
(defun add-multi-threaded-server (event-base listener-socket handler)
(add-fd event-base (socket-fd listener-socket)
:read (connection-spawner listener-socket handler)
:persistent t))
(defun start-server (port)
(prog1 (setf *server* (make-server port))
(make-thread #'(lambda () (event-dispatch *event-base*))
:name "event loop")
(setf *server-event* (add-multi-threaded-server *event-base* *server*
#'test-handler))))
(defun stop-server ()
(remove-event *event-base* *server-event*)
(close *server*)
(setf *server* nil)
(setf *server-event* nil))
(defun test-handler (sock)
(write "Test
Hello, World!
"
:stream sock)
(terpri sock))
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
From sionescu at common-lisp.net Fri Jun 1 14:27:27 2007
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Fri, 1 Jun 2007 16:27:27 +0200
Subject: [iolib-devel] event handlers not working
In-Reply-To: <20070601132116.GA18793@universe.org>
References: <465FF7B7.9090700@core.gen.tr>
<20070601132116.GA18793@universe.org>
Message-ID: <20070601142727.GA20183@universe.org>
On Fri, Jun 01, 2007 at 03:21:16PM +0200, Stelian Ionescu wrote:
>On Fri, Jun 01, 2007 at 01:40:55PM +0300, Aycan iRiCAN wrote:
>>I'm trying to write a simple example. But I cannot make the event
>>handler work. Could you please check this code (mostly stolen from
>>philip-jose).
>you forgot to run the event loop :)
>I've attached a slightly modified version of the code
I forgot to say that you'll need the live sources of iolib-posix and iolib,
although I'll soon make a new release
see http://common-lisp.net/project/iolib/download.shtml
--
(sign :name "Stelian Ionescu" :aka "fe[nl]ix"
:quote "Quidquid latine dictum sit, altum videtur.")
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
From sionescu at common-lisp.net Sun Jun 3 03:30:51 2007
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Sun, 3 Jun 2007 05:30:51 +0200
Subject: [iolib-devel] New releases: iolib-posix-0.5.4 and iolib-0.5.3
Message-ID: <20070603033051.GA309014@universe.org>
I've just made a new round of releases.
Changes WRT the previous version are:
* NET.SOCKETS: TCP socket options for Linux and FreeBSD have been added,
so now you can set options like TCP_MAXSEG and TCP_NODELAY
* NET.SOCKETS: calling BIND-ADDRESS on active sockets now works,
although probably it is seldom needed
* NET.SOCKETS: the function MAKE-SOCKET have been renamed to
CREATE-SOCKET, and an implementation of MAKE-SOCKET compatible with
Allegro 8 has been added basically you can now do something like this:
(make-socket :address-family :internet :type :stream
:connect :passive :reuse-address t
:local-host host :local-port port)
* NET.SOCKETS: it is now possible to specify an active socket's external
format in MAKE-SOCKET and setting it through (SETF EXTERNAL-FORMAT-OF)
works better; accepted values are:
- keywords: either :default or the name of an encoding, :utf-8 or
:ascii for instance
- lists specifying the encoding and the end-of-line flavour(:unix,
:mac or :dos): '(:utf-8 :line-terminator :dos)
- EXTERNAL-FORMAT structures, as returned by IOENC:MAKE-EXTERNAL-FORMAT
* a modified version of CL-SMTP has been added in package NET.SMTP-CLIENT
the system depends on CL-BASE64 and has only one external symbol: the
function SEND-EMAIL
* NET.SOCKETS: the INVALID-ADDRESS condition has been eliminated, the address
conversion functions(like DOTTED-TO-VECTOR, VECTOR-TO-COLON-SEPARATED ...)
now signal only TYPE-ERROR or PARSE-ERROR
* IO.MULTIPLEX: EVENT-DISPATCH has been improved, and two more functions
have been exported: EXIT-EVENT-LOOP and EVENT-BASE-EMPTY-P(which
returns T when no events are being monitored by an event base)
A special variable *MAXIMUM-EVENT-LOOP-TIMEOUT* has been exported: it
serves to limit the poll time of the event loop in case a thread other
than the one of the event loop accesses the event base. Defaults to
1 (in seconds) but can also be bound to NIL which means no timeout(or
infinity)
--
(sign :name "Stelian Ionescu" :aka "fe[nl]ix"
:quote "Quidquid latine dictum sit, altum videtur.")
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
From aycan.irican at core.gen.tr Mon Jun 4 10:04:13 2007
From: aycan.irican at core.gen.tr (Aycan iRiCAN)
Date: Mon, 04 Jun 2007 13:04:13 +0300
Subject: [iolib-devel] crash with the current iolib
Message-ID: <4663E39D.4050504@core.gen.tr>
Hi,
First I want to thank you for your great effort. I tried to stress the
current iolib. Attached is a simple HTTP reply generator. I used sbcl
1.0.5 VM on a 32 bit AMD desktop machine. I used siege with lot's of
concurrent connections. Here is the first inspection:
This is my stupid test server with current iolib (with a small content
size).
aycan at zen ~ $ siege -b -c10 -t5S http://localhost:8001/
** siege 2.61
** Preparing 10 concurrent users for battle.
The server is now under siege...
Lifting the server siege.. done.
Transactions: 6996 hits
Availability: 100.00 %
Elapsed time: 4.86 secs
Data transferred: 615648 bytes
Response time: 0.01 secs
Transaction rate: 1439.51 trans/sec
Throughput: 126676.54 bytes/sec
Concurrency: 9.53
Successful transactions: 6996
Failed transactions: 0
Longest transaction: 4.25
Shortest transaction: 0.00
I got 1490 transactions per second. The performance is great (I got
around 500 trans/sec with a threaded version) but when I repeat the
test, SBCL crashes silently and eats all my cpu. I don't get any errors
or some output. Also the :timeout parameter to add-fd should not be nil
by default, if so I got type errors with lengthy content sizes.
Here is the same siege parameters applied to *apache-tomcat-6-0-13* (the
content size is around 7600 bytes):
aycan at zen ~ $ siege -b -c100 -t5S http://localhost:8080/
** siege 2.61
** Preparing 100 concurrent users for battle.
The server is now under siege...
Lifting the server siege.. done.
Transactions: 16083 hits
Availability: 100.00 %
Elapsed time: 6.71 secs
Data transferred: 118274382 bytes
Response time: 0.00 secs
Transaction rate: 2396.87 trans/sec
Throughput: 17626584.40 bytes/sec
Concurrency: 10.63
Successful transactions: 16083
Failed transactions: 0
Longest transaction: 4.18
Shortest transaction: 0.00
Is it possible to outperform java io performance?
--
aycan
--
Aycan iRiCAN
C0R3 Computer Security Group
http://people.core.gen.tr/~aycan.irican/
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: iolib-problem.lisp
URL:
From sionescu at common-lisp.net Tue Jun 5 00:46:04 2007
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Tue, 5 Jun 2007 02:46:04 +0200
Subject: [iolib-devel] crash with the current iolib
In-Reply-To: <4663E39D.4050504@core.gen.tr>
References: <4663E39D.4050504@core.gen.tr>
Message-ID: <20070605004604.GB21972@universe.org>
On Mon, Jun 04, 2007 at 01:04:13PM +0300, Aycan iRiCAN wrote:
>Hi,
>
>First I want to thank you for your great effort. I tried to stress the
>current iolib. Attached is a simple HTTP reply generator. I used sbcl
>1.0.5 VM on a 32 bit AMD desktop machine. I used siege with lot's of
>concurrent connections. Here is the first inspection:
I've attached your code which I modified a bit in order to be able to
debug it more easily
>I got 1490 transactions per second. The performance is great (I got
>around 500 trans/sec with a threaded version) but when I repeat the
>test, SBCL crashes silently and eats all my cpu. I don't get any errors
>or some output.
it crashes here too with the latest SBCL, with a memory-fault-error(most
of the time, I also got various types of heap corruption) - I had to
run SBCL directly from the terminal because indeed Slime seems not to be
able to cope with a memory fault in the REPL thread. On my computer SBCL
crashed directly if I run the test for just 7 seconds
I've also tried CMUCL which is a bit sturdier(but essentially it crashes
too), while CLISP had no problems(although it is slower that SBCL or CMUCL):
hechee at blackhole iolib $ siege -b -c10 -t400S http://localhost:8001/
** siege 2.65
** Preparing 10 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 216139 hits
Availability: 100.00 %
Elapsed time: 400.81 secs
Data transferred: 30.30 MB
Response time: 0.02 secs
Transaction rate: 539.26 trans/sec
Throughput: 0.08 MB/sec
Concurrency: 9.95
Successful transactions: 216139
Failed transactions: 0
Longest transaction: 9.06
Shortest transaction: 0.00
>Also the :timeout parameter to add-fd should not be nil
>by default, if so I got type errors with lengthy content sizes.
I believe that it's better if the users of the library specify
themselves the timeout both for the simplicity of the library code and
for the fact that it's difficult to select a default timeout that is ok
for everyone
>Is it possible to outperform java io performance?
once the features will all be implemented and we get to optimize the
code, I think it can certainly be done
--
(sign :name "Stelian Ionescu" :aka "fe[nl]ix"
:quote "Quidquid latine dictum sit, altum videtur.")
-------------- next part --------------
(in-package :common-lisp-user)
(declaim (optimize (speed 3) (safety 1)))
(eval-when (:compile-toplevel :load-toplevel)
(asdf:oos 'asdf:load-op :iolib))
(defpackage :iolib-problem
(:nicknames :iop)
(:use #:common-lisp #:net.sockets #:iomux)
(:export #:start-server #:stop-server))
(in-package :iolib-problem)
(defparameter *server* nil)
(defparameter *event-base* (make-instance 'event-base))
(defparameter *event-loop-thread* nil)
(defparameter *server-event* nil)
(defparameter *handlers* (make-hash-table :test #'eql))
(defparameter +maxconn+ 50)
(defparameter *currcon* 0)
(defun make-server (port)
(make-socket :address-family :internet :type :stream
:connect :passive :reuse-address t
:local-host +ipv4-unspecified+ :local-port port
:external-format '(:ascii :line-terminator :dos) :ipv6 nil))
(defun close-handler (event-base connection)
(remove-event event-base (gethash (socket-fd connection)
*handlers*))
(close connection))
(defun make-client-handler (event-base connection request-handler)
(declare (type function request-handler))
(lambda (fd evtype)
(declare (ignore fd))
(cond
((eq evtype :read)
(ignore-errors
(unwind-protect
(progn
(funcall request-handler connection)
(finish-output connection))
(close-handler event-base connection)
(decf *currcon*))))
(t (close-handler event-base connection)))))
(defun add-connection-handler (event-base connection request-handler)
(declare (type function request-handler))
(setf (gethash (socket-fd connection) *handlers*)
(add-fd event-base (socket-fd connection)
:read (make-client-handler event-base connection request-handler)
:persistent t :timeout 1)))
(defun make-listener-handler (event-base listener-socket request-handler)
(lambda (fd evtype)
(declare (ignore fd))
(case evtype
(:read
(loop :do (if (< *currcon* +maxconn+)
(let ((connection (accept-connection listener-socket :wait nil)))
(cond (connection
(add-connection-handler event-base connection request-handler)
(incf *currcon*))
(t (loop-finish))))
(loop-finish))))
(otherwise (break)))))
(defun add-single-threaded-server (event-base listener-socket request-handler)
(add-fd event-base (socket-fd listener-socket)
:read (make-listener-handler event-base listener-socket request-handler)
:persistent t))
(defparameter *response-body* "TestAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA")
(defun test-handler (sock)
(declare (type stream sock))
(read-line sock)(read-line sock)
(format sock "HTTP/1.1 200 OK~%Content-Type: text/html~%Content-Length: ~A~%~%~A"
(length *response-body*) *response-body*))
(defun start-server (port)
(setf *server* (make-server port)
*server-event* (add-single-threaded-server *event-base* *server* #'test-handler)
*currcon* 0)
(unwind-protect (event-dispatch *event-base*)
(stop-server)))
(defun stop-server ()
(remove-event *event-base* *server-event*)
(close *server*)
(setf *server* nil
*server-event* nil))
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
From binghe.lisp at gmail.com Wed Jun 6 17:19:29 2007
From: binghe.lisp at gmail.com (Chun Tian (binghe))
Date: Thu, 07 Jun 2007 01:19:29 +0800
Subject: [iolib-devel] Bug in asdf-additions-0.5.2: lispworks spell wrong
Message-ID: <4666ECA1.2010207@gmail.com>
Hi, iolib developers
There's a small bug in asdf-additions-0.5.2: you spell "lispworks" to
"lisworks", this caused cannot compile iolib in LispWorks, actually
didn't start.
A trivial patch in attach.
Chun Tian (binghe)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: asdf-additions-1.diff
Type: text/x-patch
Size: 1130 bytes
Desc: not available
URL:
From sionescu at common-lisp.net Wed Jun 6 17:33:57 2007
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Wed, 6 Jun 2007 19:33:57 +0200
Subject: [iolib-devel] Bug in asdf-additions-0.5.2: lispworks spell wrong
In-Reply-To: <4666ECA1.2010207@gmail.com>
References: <4666ECA1.2010207@gmail.com>
Message-ID: <20070606173357.GA23073@universe.org>
On Thu, Jun 07, 2007 at 01:19:29AM +0800, Chun Tian (binghe) wrote:
>Hi, iolib developers
>
>There's a small bug in asdf-additions-0.5.2: you spell "lispworks" to
>"lisworks", this caused cannot compile iolib in LispWorks, actually
>didn't start.
>
>A trivial patch in attach.
thanks, fixed :)
plase remember though to send unified diffs next time because they're
much easier to read
--
(sign :name "Stelian Ionescu" :aka "fe[nl]ix"
:quote "Quidquid latine dictum sit, altum videtur.")
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
From binghe.lisp at gmail.com Wed Jun 6 19:56:24 2007
From: binghe.lisp at gmail.com (Chun Tian)
Date: Thu, 7 Jun 2007 03:56:24 +0800
Subject: [iolib-devel] (gcc-cpu-flags) in asdf-additions/unix-dso.lisp
Message-ID:
Hi, iolib developers
I'm not sure why need a (gcc-cpu-flags) function to detect a gcc compile
flag (-m32/-m64) and use this to compile C files.
First, use (cffi:foreign-type-size :int) to guess is wrong at least on
amd64 Linux: (cffi:foreign-type-size :int) return 4 on amd64 Linux, so
you guess wrong to 32-bit.
Second, if you guess wrong, a 64-bit Lisp process will can not load a
32-bit library.
If I disable this (gcc-cpu-flags), gcc with no -m32/-m64 can always do
the right thing on both 32 and 64-bit platform, and the Lisp process can
load this library. (I'm just doing this on Debian GNU/Linux amd64 and
LispWorks 5.0.2 Enterprise Edition for AMD64 Linux.) Am I right?
Thanks.
Chun Tian (binghe)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
-------------- next part --------------
A non-text attachment was scrubbed...
Name: asdf-additions-2.patch
Type: text/x-patch
Size: 877 bytes
Desc: not available
URL:
From fahree at gmail.com Thu Jun 7 01:16:10 2007
From: fahree at gmail.com (=?ISO-8859-1?Q?Far=E9?=)
Date: Wed, 6 Jun 2007 21:16:10 -0400
Subject: [iolib-devel] (gcc-cpu-flags) in asdf-additions/unix-dso.lisp
In-Reply-To:
References:
Message-ID: <653bea160706061816t413607a3xc1d18dda35629453@mail.gmail.com>
Wrong wrong wrong!
The -m32 or -m64 are both needed, because you may be running on a
machine where the default distribution is of one type, but your Lisp
process is of a different size!
Here at ITA, we casually compile 32-bit Lisp executable on 64-bit
machines, and the reverse is not unconceivable either.
[ Fran?ois-Ren? ?VB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
When everything seems to be going against you, remember that the airplane takes
off against the wind, not with it. -- Henry Ford
On 06/06/07, Chun Tian wrote:
> Hi, iolib developers
>
> I'm not sure why need a (gcc-cpu-flags) function to detect a gcc compile
> flag (-m32/-m64) and use this to compile C files.
>
> First, use (cffi:foreign-type-size :int) to guess is wrong at least on
>
> amd64 Linux: (cffi:foreign-type-size :int) return 4 on amd64 Linux, so
> you guess wrong to 32-bit.
>
> Second, if you guess wrong, a 64-bit Lisp process will can not load a
> 32-bit library.
>
> If I disable this (gcc-cpu-flags), gcc with no -m32/-m64 can always do
>
> the right thing on both 32 and 64-bit platform, and the Lisp process can
> load this library. (I'm just doing this on Debian GNU/Linux amd64 and
> LispWorks 5.0.2 Enterprise Edition for AMD64 Linux.) Am I right?
From binghe.lisp at gmail.com Thu Jun 7 06:10:35 2007
From: binghe.lisp at gmail.com (Chun Tian binghe)
Date: Thu, 07 Jun 2007 14:10:35 +0800
Subject: [iolib-devel] Re: (gcc-cpu-flags) in asdf-additions/unix-dso.lisp
References:
<653bea160706061816t413607a3xc1d18dda35629453@mail.gmail.com>
Message-ID: <87sl94pa5g.fsf@binghe-laptop.people.163.org>
"Far?" writes:
OK, but why not use (cffi:foreign-type-size :pointer) instead of
(cffi:foreign-type-size :int) to determine 32/64 bit OS?
I think (cffi:foreign-type-size :pointer) can always do the right thing
than (cffi:foreign-type-size :int)
> Wrong wrong wrong!
>
> The -m32 or -m64 are both needed, because you may be running on a
> machine where the default distribution is of one type, but your Lisp
> process is of a different size!
>
> Here at ITA, we casually compile 32-bit Lisp executable on 64-bit
> machines, and the reverse is not unconceivable either.
>
> [ Fran?ois-Ren? ?VB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
> When everything seems to be going against you, remember that the airplane takes
> off against the wind, not with it. -- Henry Ford
>
>
> On 06/06/07, Chun Tian wrote:
>> Hi, iolib developers
>>
>> I'm not sure why need a (gcc-cpu-flags) function to detect a gcc compile
>> flag (-m32/-m64) and use this to compile C files.
>>
>> First, use (cffi:foreign-type-size :int) to guess is wrong at least on
>>
>> amd64 Linux: (cffi:foreign-type-size :int) return 4 on amd64 Linux, so
>> you guess wrong to 32-bit.
>>
>> Second, if you guess wrong, a 64-bit Lisp process will can not load a
>> 32-bit library.
>>
>> If I disable this (gcc-cpu-flags), gcc with no -m32/-m64 can always do
>>
>> the right thing on both 32 and 64-bit platform, and the Lisp process can
>> load this library. (I'm just doing this on Debian GNU/Linux amd64 and
>> LispWorks 5.0.2 Enterprise Edition for AMD64 Linux.) Am I right?
--
From sionescu at common-lisp.net Thu Jun 7 12:29:59 2007
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Thu, 7 Jun 2007 14:29:59 +0200
Subject: [iolib-devel] Re: (gcc-cpu-flags) in asdf-additions/unix-dso.lisp
In-Reply-To: <87sl94pa5g.fsf@binghe-laptop.people.163.org>
References:
<653bea160706061816t413607a3xc1d18dda35629453@mail.gmail.com>
<87sl94pa5g.fsf@binghe-laptop.people.163.org>
Message-ID: <20070607122959.GA32819@universe.org>
On Thu, Jun 07, 2007 at 02:10:35PM +0800, Chun Tian (binghe) wrote:
>"Far?" writes:
>
>OK, but why not use (cffi:foreign-type-size :pointer) instead of
>(cffi:foreign-type-size :int) to determine 32/64 bit OS?
>
>I think (cffi:foreign-type-size :pointer) can always do the right thing
>than (cffi:foreign-type-size :int)
you're correct: I've just committed the fix :)
--
(sign :name "Stelian Ionescu" :aka "fe[nl]ix"
:quote "Quidquid latine dictum sit, altum videtur.")
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
From binghe.lisp at gmail.com Thu Jun 7 18:20:24 2007
From: binghe.lisp at gmail.com (Chun Tian binghe)
Date: Fri, 08 Jun 2007 02:20:24 +0800
Subject: [iolib-devel] #+lispworks missing in iolib/io.streams/pkgdcl.lisp
Message-ID: <87tztjhbiv.fsf@binghe-laptop.people.163.org>
Hi, IOLib
The line 31 of iolib/io.streams/pkgdcl.lisp miss a "lispworks" feature. This caused io.streams package cannot import symbols from lispworks' stream package, and cause a error when I use net.smtp-client:
Error: The class # cannot be finalized because the following superclass is not defined: #.
1 (abort) Return to level 0.
2 Return to top loop level 0.
Type :b for backtrace, :c