From vsedach at gmail.com  Sun Dec  6 22:23:47 2009
From: vsedach at gmail.com (Vladimir Sedach)
Date: Sun, 6 Dec 2009 17:23:47 -0500
Subject: [iolib-devel] Patches to make multiplexer use easier
In-Reply-To: <c07e645b0911210304p29c4a8b9k59abfd44a81b6ae4@mail.gmail.com>
References: <c07e645b0911160015v1a68ad75kb0a73258f80e16ef@mail.gmail.com>
	<c07e645b0911161211i6e62b9ebge6fc7ad585c27b9b@mail.gmail.com>
	<c07e645b0911210304p29c4a8b9k59abfd44a81b6ae4@mail.gmail.com>
Message-ID: <c07e645b0912061423g531045cbpa5804f3d593b78de@mail.gmail.com>

Stelian, any updates on this? Are you planning to apply these patches,
or is there something you would like changed?

Vladimir

2009/11/21 Vladimir Sedach <vsedach at gmail.com>:
> Here's a patch that finishes adding the interface to the other
> multiplexer backends. Now it should be possible to use all of them for
> simple readiness notification on a set of integer fds.
>
> For example use, take a look at: http://github.com/vsedach/HTTP-DOHC
>
> Vladimir
>
> 2009/11/16 Vladimir Sedach <vsedach at gmail.com>:
>> Here's a corrected patch for 0005 that supersedes the one in my
>> previous message.
>>
>> Vladimir
>>
>> 2009/11/16 Vladimir Sedach <vsedach at gmail.com>:
>>> Hello,
>>>
>>> Here are some patches that make it easier to just use the multiplexer
>>> object without having to go through the provided event loop/timer
>>> machinery. This is done in three steps: make the monitor/harvest etc.
>>> methods on multiplexer accept just file descriptors as well as
>>> FD-ENTRYs (note that I've only done that for epoll in this set of
>>> patches, others to come!), export the handler methods, and add an
>>> optional flags parameter to the MONITOR-FD method (really useful for
>>> epoll, the other backends I'm not sure about).
>>>
>>> Let me know what you think. I'd like to get these patches (or at least
>>> the equivalent functionality of being able to use the multiplexer
>>> without having to make FD-ENTRYs) into IOlib as I'm about to release
>>> some software that uses it.
>>>
>>> Thanks,
>>> Vladimir
>>>
>>
>



From sionescu at cddr.org  Sat Dec 12 18:23:38 2009
From: sionescu at cddr.org (Stelian Ionescu)
Date: Sat, 12 Dec 2009 19:23:38 +0100
Subject: [iolib-devel] Patches to make multiplexer use easier
In-Reply-To: <c07e645b0912061423g531045cbpa5804f3d593b78de@mail.gmail.com>
References: <c07e645b0911160015v1a68ad75kb0a73258f80e16ef@mail.gmail.com>
	<c07e645b0911161211i6e62b9ebge6fc7ad585c27b9b@mail.gmail.com>
	<c07e645b0911210304p29c4a8b9k59abfd44a81b6ae4@mail.gmail.com>
	<c07e645b0912061423g531045cbpa5804f3d593b78de@mail.gmail.com>
Message-ID: <1260642219.26732.20.camel@blackhole.cddr.org>

On Sun, 2009-12-06 at 17:23 -0500, Vladimir Sedach wrote:
> Stelian, any updates on this? Are you planning to apply these patches,
> or is there something you would like changed?

I've merged your first patch(added readv too).

As for the rest, I'm not inclined to merge them because ultimately I
want the muxer to be callback-only and not to expose the details of the
OS-specific mechanisms, which I consider to be only internals.

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/iolib-devel/attachments/20091212/92428b8a/attachment.sig>

From vsedach at gmail.com  Sat Dec 12 23:52:31 2009
From: vsedach at gmail.com (Vladimir Sedach)
Date: Sat, 12 Dec 2009 18:52:31 -0500
Subject: [iolib-devel] Patches to make multiplexer use easier
In-Reply-To: <1260642219.26732.20.camel@blackhole.cddr.org>
References: <c07e645b0911160015v1a68ad75kb0a73258f80e16ef@mail.gmail.com>
	<c07e645b0911161211i6e62b9ebge6fc7ad585c27b9b@mail.gmail.com>
	<c07e645b0911210304p29c4a8b9k59abfd44a81b6ae4@mail.gmail.com>
	<c07e645b0912061423g531045cbpa5804f3d593b78de@mail.gmail.com>
	<1260642219.26732.20.camel@blackhole.cddr.org>
Message-ID: <c07e645b0912121552x38a191d3m91b58ca3a5074be7@mail.gmail.com>

> As for the rest, I'm not inclined to merge them because ultimately I
> want the muxer to be callback-only and not to expose the details of the
> OS-specific mechanisms, which I consider to be only internals.

I don't mind the interface being based on callbacks instead of
HARVEST-EVENTS, but I also don't see how the current interface is
simpler or offers more than just exposing
(UN)MONITOR-FD/UPDATE-FD/HARVEST-EVENTS and a generic mechanism to
make a MULTIPLEXER. I don't get event-loop.lisp or why I would need
it, and I definitely don't want synthetic timers in my networking
code.

Vladimir

> --
> Stelian Ionescu a.k.a. fe[nl]ix
> Quidquid latine dictum sit, altum videtur.
> http://common-lisp.net/project/iolib
>
> _______________________________________________
> IOLib-devel mailing list
> IOLib-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/iolib-devel
>
>



From sionescu at common-lisp.net  Sun Dec 13 10:00:05 2009
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Sun, 13 Dec 2009 05:00:05 -0500
Subject: [iolib-devel] New patches: 12-Dec-2009
Message-ID: <E1NJlFF-0000RL-FH@common-lisp.net>


commit 8da6620f8b76a809d489bb3c7312c3394e845c8f
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sat Dec 12 00:18:20 2009 +0100

    Use %SYS-CLEARENV in (SETF ENVIRONMENT).

 src/os/os-unix.lisp |   12 ++++--------
 1 files changed, 4 insertions(+), 8 deletions(-)

commit 7e528c7b51f07f14e0974d53afbcd1eb4f5ad567
Author: Vladimir Sedach <vsedach at gmail.com>
Date:   Thu Dec 10 19:34:58 2009 +0100

    Add syscalls readv(2) and writev(2).

 src/syscalls/ffi-functions-unix.lisp |   14 ++++++++++++++
 src/syscalls/pkgdcl.lisp             |    2 ++
 2 files changed, 16 insertions(+), 0 deletions(-)

commit deeaa58946c6a38d1e0057f8233ef56f68160f92
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Thu Dec 10 12:29:12 2009 +0100

    Add CLEAR-ENVIRONMENT.

 src/os/os-unix.lisp                  |   14 +++++++++++++-
 src/os/pkgdcl.lisp                   |    1 +
 src/syscalls/ffi-functions-unix.lisp |   12 ++++++++++++
 src/syscalls/pkgdcl.lisp             |    1 +
 4 files changed, 27 insertions(+), 1 deletions(-)

commit f649ab8620d040019aec72b1bc411a30b81457b8
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Wed Dec 9 11:16:14 2009 +0100

    Print empty environments better.

 src/os/os-unix.lisp |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

commit 50c78442fd26e94d2f76760a52b948e744dd89c3
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Wed Dec 9 00:03:47 2009 +0100

    Fix %SYS-GETENV.

 src/syscalls/ffi-functions-unix.lisp |   14 ++++----------
 1 files changed, 4 insertions(+), 10 deletions(-)

commit 806cd30fda78a929a1a76bd7db5ac91a73101d7e
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Tue Dec 8 23:45:07 2009 +0100

    Style fixes.

 src/os/os-unix.lisp |   18 +++++++++---------
 1 files changed, 9 insertions(+), 9 deletions(-)

commit 17b15977376e5e42ffa5f3df13dba61f3398836d
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Tue Dec 8 23:42:52 2009 +0100

    Add print method for instances of ENVIRONMENT.

 src/os/os-unix.lisp |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

commit 06e73a8930329478a62202ef14c90effca0717e9
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Tue Dec 8 23:12:55 2009 +0100

    Guard against null OS environment in ENVIRONMENT.

 src/os/os-unix.lisp |   20 +++++++++++---------
 1 files changed, 11 insertions(+), 9 deletions(-)

commit b21ab65e892ef338a936239571e03af61d5f11b1
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Tue Dec 8 19:44:00 2009 +0100

    Clean environment code.

 src/os/os-unix.lisp |   37 ++++++++++++++++++++++++-------------
 1 files changed, 24 insertions(+), 13 deletions(-)

commit ab382219e662686f4070933a1464a282317165b3
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sat Dec 5 00:51:21 2009 +0100

    Inline %ENVAR, (SETF %ENVAR) and %REMVAR.

 src/os/os-unix.lisp |   19 +++++--------------
 1 files changed, 5 insertions(+), 14 deletions(-)

commit fecdb57dd817c2fa3cc4af18c1cb22030f7dc177
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sat Dec 5 00:20:11 2009 +0100

    ENVIRONMENT no longer copies it argument, fix docstrings.

 src/os/os-unix.lisp |   18 ++++--------------
 1 files changed, 4 insertions(+), 14 deletions(-)

commit f9d69861075fa66fa792273ec7a8770c06140813
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sat Dec 5 00:15:08 2009 +0100

    Better type checking.

 src/os/os-unix.lisp |   48 ++++++++++++++++++++++--------------------------
 1 files changed, 22 insertions(+), 26 deletions(-)

commit 5f2c8ba77bad2b03cbb93ea4307c51b6cef91a8a
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sat Dec 5 00:12:22 2009 +0100

    Fix docstring.

 src/os/os-unix.lisp |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

commit 3d34dc938aadc6a11fe1334dd908aaf546375660
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sun Nov 15 17:35:12 2009 +0100

    Add .gitattributes

 .gitattributes |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

commit fb58ba050ba6f423934e5fba652c9f65c28e95d5
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Wed Nov 11 14:59:44 2009 +0100

    Update my email address.

 src/iolib.asd                 |    4 ++--
 src/iolib.base.asd            |    4 ++--
 src/iolib.multiplex.asd       |    4 ++--
 src/iolib.os.asd              |    2 +-
 src/iolib.pathnames.asd       |    2 +-
 src/iolib.sockets.asd         |    4 ++--
 src/iolib.streams.asd         |    2 +-
 src/iolib.syscalls.asd        |    2 +-
 src/iolib.trivial-sockets.asd |    2 +-
 src/iolib.zeta-streams.asd    |    2 +-
 src/sockets/grovel.lisp       |    2 +-
 11 files changed, 15 insertions(+), 15 deletions(-)


An updated tarball of IOLib's source can be downloaded here:
http://common-lisp.net/project/iolib/files/snapshots/iolib-20091212.tar.gz

Gitweb URL:
http://repo.or.cz/w/iolib.git



From sionescu at common-lisp.net  Mon Dec 14 13:49:32 2009
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Mon, 14 Dec 2009 14:49:32 +0100
Subject: [iolib-devel] Patches to make multiplexer use easier
In-Reply-To: <c07e645b0912121552x38a191d3m91b58ca3a5074be7@mail.gmail.com>
References: <c07e645b0911160015v1a68ad75kb0a73258f80e16ef@mail.gmail.com>	<c07e645b0911161211i6e62b9ebge6fc7ad585c27b9b@mail.gmail.com>	<c07e645b0911210304p29c4a8b9k59abfd44a81b6ae4@mail.gmail.com>	<c07e645b0912061423g531045cbpa5804f3d593b78de@mail.gmail.com>	<1260642219.26732.20.camel@blackhole.cddr.org>
	<c07e645b0912121552x38a191d3m91b58ca3a5074be7@mail.gmail.com>
Message-ID: <4B26426C.8080202@common-lisp.net>

On 12/13/2009 12:52 AM, Vladimir Sedach wrote:
>> As for the rest, I'm not inclined to merge them because ultimately I
>> want the muxer to be callback-only and not to expose the details of the
>> OS-specific mechanisms, which I consider to be only internals.
> 
> I don't mind the interface being based on callbacks instead of
> HARVEST-EVENTS, but I also don't see how the current interface is
> simpler or offers more than just exposing
> (UN)MONITOR-FD/UPDATE-FD/HARVEST-EVENTS and a generic mechanism to
> make a MULTIPLEXER. I don't get event-loop.lisp or why I would need
> it, and I definitely don't want synthetic timers in my networking
> code.

It hides the fact that with epoll() you need to set/unset FD flags,
while with kqueue() you add/remove, and select()/poll() have not state
whatsoever. Furthermore, while for the moment only FD events and timers
are supported, I'm planning to add at least two more: signals and file
change notification.

As for the timers, you're not forced to use them, although when working
with the non-blocking sockets IMO they're essential.

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib



From vsedach at gmail.com  Tue Dec 15 05:24:27 2009
From: vsedach at gmail.com (Vladimir Sedach)
Date: Tue, 15 Dec 2009 00:24:27 -0500
Subject: [iolib-devel] Patches to make multiplexer use easier
In-Reply-To: <4B26426C.8080202@common-lisp.net>
References: <c07e645b0911160015v1a68ad75kb0a73258f80e16ef@mail.gmail.com>
	<c07e645b0911161211i6e62b9ebge6fc7ad585c27b9b@mail.gmail.com>
	<c07e645b0911210304p29c4a8b9k59abfd44a81b6ae4@mail.gmail.com>
	<c07e645b0912061423g531045cbpa5804f3d593b78de@mail.gmail.com>
	<1260642219.26732.20.camel@blackhole.cddr.org>
	<c07e645b0912121552x38a191d3m91b58ca3a5074be7@mail.gmail.com>
	<4B26426C.8080202@common-lisp.net>
Message-ID: <c07e645b0912142124j378d21cfx86ac881b20775da7@mail.gmail.com>

> It hides the fact that with epoll() you need to set/unset FD flags,
> while with kqueue() you add/remove, and select()/poll() have not state
> whatsoever.

But that's exactly what the multiplexer abstracts.

> Furthermore, while for the moment only FD events and timers
> are supported, I'm planning to add at least two more: signals and file
> change notification.

Ok, but I think I'm not the only person that just wants a portable
interface to event-driven IO for sockets with minimal overhead.

> As for the timers, you're not forced to use them, although when working
> with the non-blocking sockets IMO they're essential.

For HTTP servers it makes more sense to clean up any timed out
connections in the server loop. I suspect that's the case with servers
for most other protocols. Is there a difference for clients?

Before we get any further, I'd like to see example code of how the
event loop and timers work (documentation might help too). Maybe I'm
misunderstanding something.

Vladimir

> --
> Stelian Ionescu a.k.a. fe[nl]ix
> Quidquid latine dictum sit, altum videtur.
> http://common-lisp.net/project/iolib
>
> _______________________________________________
> IOLib-devel mailing list
> IOLib-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/iolib-devel
>



From sionescu at common-lisp.net  Tue Dec 15 10:00:06 2009
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Tue, 15 Dec 2009 05:00:06 -0500
Subject: [iolib-devel] New patches: 14-Dec-2009
Message-ID: <E1NKUCM-0008FE-Oe@common-lisp.net>


commit 8fbea177223d77ef218720d63719c303e030a1a6
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sun Dec 13 16:26:18 2009 +0100

    Fix docstring of (SETF DIRECTORY).

 src/os/os-unix.lisp |    9 ++++-----
 1 files changed, 4 insertions(+), 5 deletions(-)

commit 9c4d0e155be879cf39e9e885ab4e7c75a92ef009
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sun Dec 13 00:33:21 2009 +0100

    Better variable names.

 src/os/os-unix.lisp |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)


An updated tarball of IOLib's source can be downloaded here:
http://common-lisp.net/project/iolib/files/snapshots/iolib-20091214.tar.gz

Gitweb URL:
http://repo.or.cz/w/iolib.git



From sionescu at common-lisp.net  Fri Dec 18 10:00:07 2009
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Fri, 18 Dec 2009 05:00:07 -0500
Subject: [iolib-devel] New patches: 17-Dec-2009
Message-ID: <E1NLZd1-00085j-GM@common-lisp.net>


commit b80f088b820b2e8d271ef84d6d27894319a55c38
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Thu Dec 17 12:18:31 2009 +0100

    Don't define dirfd() and fdopendir() on OSX.

 src/syscalls/ffi-functions-unix.lisp |    2 ++
 src/syscalls/pkgdcl.lisp             |    4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)


An updated tarball of IOLib's source can be downloaded here:
http://common-lisp.net/project/iolib/files/snapshots/iolib-20091217.tar.gz

Gitweb URL:
http://repo.or.cz/w/iolib.git



From sionescu at common-lisp.net  Tue Dec 22 10:00:07 2009
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Tue, 22 Dec 2009 05:00:07 -0500
Subject: [iolib-devel] New patches: 21-Dec-2009
Message-ID: <E1NN1XD-0008RO-RU@common-lisp.net>


commit 4768f6368de6c5d01c7c1f2b5802c52133071ee5
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Mon Dec 21 21:07:59 2009 +0100

    Grovel ENOTSUP too.

 src/syscalls/ffi-types-unix.lisp |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

commit ab3b5b3293133937a9b11360ac844aba1f31d2b7
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Mon Dec 21 15:36:20 2009 +0100

    Oops, wrong fix.

 examples/echo-server.lisp |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

commit d71ca0f64fbc3c00eaf57b48766b4f43eb2d34fa
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Mon Dec 21 15:26:52 2009 +0100

    Remove superfluous constant +FAR-INTO-THE-FUTURE+.

 src/multiplex/timers.lisp |    6 +-----
 1 files changed, 1 insertions(+), 5 deletions(-)

commit d164bef6aca1c8d681f6b74363a4b0c1338a522b
Author: Nikodemus Siivola <nikodemus at random-state.net>
Date:   Mon Dec 21 15:25:53 2009 +0100

    Fix computation of the event loop timeout in case of no timers.

 src/multiplex/event-loop.lisp |   12 ++++++++----
 src/multiplex/scheduler.lisp  |    6 ++----
 2 files changed, 10 insertions(+), 8 deletions(-)

commit 42dc0304632fde15aeb635a8f792da0e5277d7c6
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Mon Dec 21 13:49:15 2009 +0100

    Minor fixes in echo-server example.

 examples/echo-server.lisp |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

commit 839106a075b6ba68c63d319ff2379bf047c35030
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Mon Dec 21 13:40:35 2009 +0100

    Guard against NULL max-step.
    
    Thanks to Nikodemus Siivola.

 src/multiplex/event-loop.lisp |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

commit abff7bc2ec258a421392bc59c2d9267e43674a3a
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Mon Dec 21 13:11:44 2009 +0100

    Use better defaults for *MINIMUM-EVENT-LOOP-STEP* and *MAXIMUM-EVENT-LOOP-STEP*.

 src/multiplex/event-loop.lisp |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)


An updated tarball of IOLib's source can be downloaded here:
http://common-lisp.net/project/iolib/files/snapshots/iolib-20091221.tar.gz

Gitweb URL:
http://repo.or.cz/w/iolib.git



From marsijanin at gmail.com  Thu Dec 24 17:58:42 2009
From: marsijanin at gmail.com (marsijanin at gmail.com)
Date: Thu, 24 Dec 2009 20:58:42 +0300
Subject: [iolib-devel] Extension for serial devices support.
In-Reply-To: <1258733925.4239.5.camel@blackhole.cddr.org> (Stelian Ionescu's
	message of "Fri, 20 Nov 2009 17:18:45 +0100")
References: <87skco6g8b.fsf@fobos.mars>
	<1258733925.4239.5.camel@blackhole.cddr.org>
Message-ID: <87vdfw45kd.fsf@gmail.com>

So, Stelian, are you planning to incorporate my serial devices extension
into IOLib main tree or not? I still don't mind if you will, but you
still don't give an answer. 

Best,

Nikolay V. Razbegaev



From psilord at cs.wisc.edu  Thu Dec 24 17:55:01 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Thu, 24 Dec 2009 11:55:01 -0600
Subject: [iolib-devel] compilation error
Message-ID: <20091224175501.GA21896@cs.wisc.edu>

Hello,

I'm *very* new to Common Lisp and have only just started understanding
how to use packages and asdf. I have a client/server application in mind,
of which I've already written a fair amount (using the asdf module system),
and I would like to use iolib to multiplex the i/o.

I'm using sbcl 1.0.18.debian on an ubuntu 9.04 x86 box.

I initially used asdf to set up iolib and it picked the iolib-0.6.0
package and downloaded/compiled a bunch of denpendencies.  There were
some problems compiling it concerning its dependencies, which I'm sorry
to say I did not record or what I did to get around them--but it had to
do with cffi-grovel not being a package asdf new about, and I believe I
had to install cffi by hand before iolib would finish compiling becaue
the grovel module was now part of the cffi.

So, I started looking around for an example for how to use iolib,
and found one online in the current sources for iolib called
echo-server.lisp. It took a while for me to realize that net.sockets in
the iolib-0.6.0 version was moved to iolib.sockets in the current sources, and
after noticing that some other API changes happened and whatnot, I removed the
iolib-0.6.0 site and system from my ~/.sbcl/{site,system} directory, removed
the asd file symlinks, and git checked out the head of iolib.

I put that checkout into the above mentioned place, hand set up the asd
symlinks, and tried (require 'iolib) again. 

This is what I got:

* (require 'iolib)

... compile lines ...

debugger invoked on a SB-PCL::INITARG-ERROR in thread #<THREAD "initial thread" RUNNING {A834819}>:
  Invalid initialization argument:
    :SONAME
  in call for class #<STANDARD-CLASS CFFI-GROVEL:WRAPPER-FILE>.
See also:
  The ANSI Standard, Section 7.1.2

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

(SB-PCL::CHECK-RI-INITARGS
 #<CFFI-GROVEL:WRAPPER-FILE  {AC30651}>
 (:NAME "ffi-wrappers" :PATHNAME "ffi-wrappers-unix" :PARENT
  #<ASDF:SYSTEM "iolib.syscalls" {AC27D91}> :SONAME "libiolib-syscalls"))
0] 

I don't know how to get around this error.

I apologize for my gross misunderstanding of how to do all of this. I'm in the
process of figuring it out.

Thank you.

-pete



From sionescu at cddr.org  Thu Dec 24 18:28:56 2009
From: sionescu at cddr.org (Stelian Ionescu)
Date: Thu, 24 Dec 2009 19:28:56 +0100
Subject: [iolib-devel] Extension for serial devices support.
In-Reply-To: <87bpiuoewd.fsf@fobos.mars>
References: <87skco6g8b.fsf@fobos.mars>
	<1258733925.4239.5.camel@blackhole.cddr.org>
	<87bpiuoewd.fsf@fobos.mars>
Message-ID: <1261679336.32594.11.camel@blackhole.cddr.org>

On Sun, 2009-11-22 at 16:41 +0300, marsijanin at gmail.com wrote:
> Hi, Stelian.
> 
> I guess that incorporation will be best solution. IOLib.termios is my
> first experience in code sharing and cffi coding. I currently
> haven't much time to support this my little IOLib extension because of
> deadline at my work, but I hope, that it should not be a big problem in the
> future.  Besides, I believe that it will be a chance to learn more
> about git for me.
> 
> Anyway there are still two questions about incorporation: how exactly
> you are going to incorporate and to collaborate? Will the incorporation
> be just coping files and calling "git add" or there is a way to import mercurial
> project into git? I don't suppose, that history of my little part is
> really important -- just asking. And after that -- how should we
> collaborate? Should I just send patches into mailing list or will you
> give me write access to the project repository?

You'll need to get an account on common-lisp.net after which I'll give
you write access to the repository.

I don't think there's a way to automatically import the patches, it will
probably have to be done manually

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/iolib-devel/attachments/20091224/74773e6b/attachment.sig>

From sionescu at cddr.org  Thu Dec 24 18:46:13 2009
From: sionescu at cddr.org (Stelian Ionescu)
Date: Thu, 24 Dec 2009 19:46:13 +0100
Subject: [iolib-devel] compilation error
In-Reply-To: <20091224175501.GA21896@cs.wisc.edu>
References: <20091224175501.GA21896@cs.wisc.edu>
Message-ID: <1261680373.32594.29.camel@blackhole.cddr.org>

On Thu, 2009-12-24 at 11:55 -0600, Peter Keller wrote:
> Hello,
> 
> I'm *very* new to Common Lisp and have only just started understanding
> how to use packages and asdf. I have a client/server application in mind,
> of which I've already written a fair amount (using the asdf module system),
> and I would like to use iolib to multiplex the i/o.
> 
> I'm using sbcl 1.0.18.debian on an ubuntu 9.04 x86 box.

sbcl 1.0.18 may be too old (17 months old ATM) so I'd advise you to try
with a more recent version.

> 
> I initially used asdf to set up iolib and it picked the iolib-0.6.0
> package and downloaded/compiled a bunch of denpendencies.  There were
> some problems compiling it concerning its dependencies, which I'm sorry
> to say I did not record or what I did to get around them--but it had to
> do with cffi-grovel not being a package asdf new about, and I believe I
> had to install cffi by hand before iolib would finish compiling becaue
> the grovel module was now part of the cffi.
> 
> So, I started looking around for an example for how to use iolib,
> and found one online in the current sources for iolib called
> echo-server.lisp. It took a while for me to realize that net.sockets in
> the iolib-0.6.0 version was moved to iolib.sockets in the current sources, and
> after noticing that some other API changes happened and whatnot, I removed the
> iolib-0.6.0 site and system from my ~/.sbcl/{site,system} directory, removed
> the asd file symlinks, and git checked out the head of iolib.

No need to apologize. A new release is long overdue and the current HEAD
needs CFFI HEAD(http://common-lisp.net/project/cffi/darcs/cffi/). I'll
try to arrange a new release of CFFI and IOlib before the end of the
vacation

Usually, whenever I switch from requiring a released version to the live
version of some dependency, I update the download page
(http://common-lisp.net/project/iolib/download.shtml)

If you need assistance with IOlib, you can also find me on
irc.freenode.net, channel #iolib :)

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/iolib-devel/attachments/20091224/a6115b23/attachment.sig>

From psilord at cs.wisc.edu  Thu Dec 24 20:41:26 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Thu, 24 Dec 2009 14:41:26 -0600
Subject: [iolib-devel] compilation error
In-Reply-To: <1261680373.32594.29.camel@blackhole.cddr.org>
References: <20091224175501.GA21896@cs.wisc.edu>
	<1261680373.32594.29.camel@blackhole.cddr.org>
Message-ID: <20091224204126.GA26238@cs.wisc.edu>

On Thu, Dec 24, 2009 at 07:46:13PM +0100, Stelian Ionescu wrote:
> sbcl 1.0.18 may be too old (17 months old ATM) so I'd advise you to try
> with a more recent version.

Another poster mentioned to me to use clbuild to get a newer copy of
sbcl and the various supported packages. I used it, installed ALL of
the supported packages, and it worked *almost* out of the box.

Oddly, enough, it failed to compile iolib :), but this time I figured out why.

Basically, you have some 8-bit ascii in the documentation string for the
method in iolib/src/sockets/base-sockets.lisp:

(defgeneric socket-option (socket option-name)
  (:documentation "Returns the value(s) of OS option OPTION-NAME on SOCKET.
  For a complete list of supported options see <C2><AB>iolib.sockets/socket-options.lisp<C2><BB>."))

sbcl complained about an unreadable character in the stream and
stopped. Once I removed the << >> marks from the documentation string,
it worked and iolib compiled and loaded. I was easily able to get the library
I am working on as well to set up and run.

> If you need assistance with IOlib, you can also find me on
> irc.freenode.net, channel #iolib :)

Ok, my main interest is figuring out how in Common Lisp to write a
nonblocking multplexed i/o server and the client it speaks to.

I have a project that I need to scale to tens of thousands of clients
talking with a single server.

Thank you.

-pete



From sionescu at cddr.org  Thu Dec 24 21:04:02 2009
From: sionescu at cddr.org (Stelian Ionescu)
Date: Thu, 24 Dec 2009 22:04:02 +0100
Subject: [iolib-devel] compilation error
In-Reply-To: <20091224204126.GA26238@cs.wisc.edu>
References: <20091224175501.GA21896@cs.wisc.edu>
	<1261680373.32594.29.camel@blackhole.cddr.org>
	<20091224204126.GA26238@cs.wisc.edu>
Message-ID: <1261688642.32594.48.camel@blackhole.cddr.org>

On Thu, 2009-12-24 at 14:41 -0600, Peter Keller wrote:
> On Thu, Dec 24, 2009 at 07:46:13PM +0100, Stelian Ionescu wrote:
> > sbcl 1.0.18 may be too old (17 months old ATM) so I'd advise you to try
> > with a more recent version.
> 
> Another poster mentioned to me to use clbuild to get a newer copy of
> sbcl and the various supported packages. I used it, installed ALL of
> the supported packages, and it worked *almost* out of the box.
> 
> Oddly, enough, it failed to compile iolib :), but this time I figured out why.
> 
> Basically, you have some 8-bit ascii in the documentation string for the
> method in iolib/src/sockets/base-sockets.lisp:
> 
> (defgeneric socket-option (socket option-name)
>   (:documentation "Returns the value(s) of OS option OPTION-NAME on SOCKET.
>   For a complete list of supported options see <C2><AB>iolib.sockets/socket-options.lisp<C2><BB>."))
> 

That's UTF-8 for #\? and #\?. You can make SBCL user UTF-8 by default:
either use a UTF-8 locale or add
(setf sb-impl::*default-external-format* :utf-8) to your ~/.sbclrc

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/iolib-devel/attachments/20091224/535e7308/attachment.sig>

From psilord at cs.wisc.edu  Fri Dec 25 01:19:59 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Thu, 24 Dec 2009 19:19:59 -0600
Subject: [iolib-devel] compilation error
In-Reply-To: <1261688642.32594.48.camel@blackhole.cddr.org>
References: <20091224175501.GA21896@cs.wisc.edu>
	<1261680373.32594.29.camel@blackhole.cddr.org>
	<20091224204126.GA26238@cs.wisc.edu>
	<1261688642.32594.48.camel@blackhole.cddr.org>
Message-ID: <20091225011959.GA26653@cs.wisc.edu>

On Thu, Dec 24, 2009 at 10:04:02PM +0100, Stelian Ionescu wrote:
> That's UTF-8 for #\? and #\?. You can make SBCL user UTF-8 by default:
> either use a UTF-8 locale or add
> (setf sb-impl::*default-external-format* :utf-8) to your ~/.sbclrc

This fixed seemed to work. Thank you.

After a long bout of experimenting with the very reasonable tool
clbuild and some gnashing of teeth at Limp, the SLIME equivalent for vim,
I now have my workflow back up with the newest sbcl.

So, after all that, now I have to check out to see if the echo-server.lisp
program works. :)

Thank you.

-pete



From psilord at cs.wisc.edu  Sat Dec 26 20:05:28 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Sat, 26 Dec 2009 14:05:28 -0600
Subject: [iolib-devel] compilation error
In-Reply-To: <20091225011959.GA26653@cs.wisc.edu>
References: <20091224175501.GA21896@cs.wisc.edu>
	<1261680373.32594.29.camel@blackhole.cddr.org>
	<20091224204126.GA26238@cs.wisc.edu>
	<1261688642.32594.48.camel@blackhole.cddr.org>
	<20091225011959.GA26653@cs.wisc.edu>
Message-ID: <20091226200528.GA7636@cs.wisc.edu>

On Thu, Dec 24, 2009 at 07:19:59PM -0600, Peter Keller wrote:
> So, after all that, now I have to check out to see if the echo-server.lisp
> program works. :)

Ok, after some experimentation and writing my own code based off of the
echo-server.lisp code, I've found a bug in the echo-server.lisp code:

In function make-listener-handler, it will timeout after a certain amount
of time, this clause is supposed to fire:

      (when (eql :timeout event)
        (warn "Got a server timeout!")
        (return))


However, it won't because the (eql :timeout event) should be 
(eql :timeout exception) instead. 

The current behavior of the bug is after 15 seconds, the server becomes
completely unresponsive. By default this isn't seen because the timeout
for the server itself is 10 seconds.

Also, unrelated, the (defvar *sockets* (make-hash-table)) at the top of
the file should be (defvar *sockets* nil) instead.

Now, a question, is there a means by which I can have iomux:event-dispatch
return control to me for a while at specified intervals, and then go
back to multiplexing i/o? This would allow me to not have to put the
server machinery into the read/write functions on the streams themselves
or be beholden to my processing only on communication boundaries. If I
can get a single threaded solution, then I wouldn't have to worry about
the usual things with threads concerning data races and whatnot.

I'm looking forward to a new release of iolib, especially to get an
updated API description on the online manual.

Thank you.

-pete



From sionescu at cddr.org  Sat Dec 26 21:36:48 2009
From: sionescu at cddr.org (Stelian Ionescu)
Date: Sat, 26 Dec 2009 22:36:48 +0100
Subject: [iolib-devel] compilation error
In-Reply-To: <20091226200528.GA7636@cs.wisc.edu>
References: <20091224175501.GA21896@cs.wisc.edu>
	<1261680373.32594.29.camel@blackhole.cddr.org>
	<20091224204126.GA26238@cs.wisc.edu>
	<1261688642.32594.48.camel@blackhole.cddr.org>
	<20091225011959.GA26653@cs.wisc.edu>
	<20091226200528.GA7636@cs.wisc.edu>
Message-ID: <1261863408.14411.7.camel@blackhole.cddr.org>

On Sat, 2009-12-26 at 14:05 -0600, Peter Keller wrote:
> On Thu, Dec 24, 2009 at 07:19:59PM -0600, Peter Keller wrote:
> > So, after all that, now I have to check out to see if the echo-server.lisp
> > program works. :)
> 
> Ok, after some experimentation and writing my own code based off of the
> echo-server.lisp code, I've found a bug in the echo-server.lisp code:
> 
> In function make-listener-handler, it will timeout after a certain amount
> of time, this clause is supposed to fire:
> 
>       (when (eql :timeout event)
>         (warn "Got a server timeout!")
>         (return))
> 
> 
> However, it won't because the (eql :timeout event) should be 
> (eql :timeout exception) instead. 
> 
> The current behavior of the bug is after 15 seconds, the server becomes
> completely unresponsive. By default this isn't seen because the timeout
> for the server itself is 10 seconds.
> 
> Also, unrelated, the (defvar *sockets* (make-hash-table)) at the top of
> the file should be (defvar *sockets* nil) instead.

Thanks. I've updated the example

> Now, a question, is there a means by which I can have iomux:event-dispatch
> return control to me for a while at specified intervals, and then go
> back to multiplexing i/o? This would allow me to not have to put the
> server machinery into the read/write functions on the streams themselves
> or be beholden to my processing only on communication boundaries. If I
> can get a single threaded solution, then I wouldn't have to worry about
> the usual things with threads concerning data races and whatnot.

(event-dispatch *base* :one-shot t)


-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/iolib-devel/attachments/20091226/1eb4582b/attachment.sig>

From psilord at cs.wisc.edu  Sun Dec 27 03:50:00 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Sat, 26 Dec 2009 21:50:00 -0600
Subject: [iolib-devel] compilation error
In-Reply-To: <1261863408.14411.7.camel@blackhole.cddr.org>
References: <20091224175501.GA21896@cs.wisc.edu>
	<1261680373.32594.29.camel@blackhole.cddr.org>
	<20091224204126.GA26238@cs.wisc.edu>
	<1261688642.32594.48.camel@blackhole.cddr.org>
	<20091225011959.GA26653@cs.wisc.edu>
	<20091226200528.GA7636@cs.wisc.edu>
	<1261863408.14411.7.camel@blackhole.cddr.org>
Message-ID: <20091227035000.GA17366@cs.wisc.edu>

Hello,

On Sat, Dec 26, 2009 at 10:36:48PM +0100, Stelian Ionescu wrote:
> > Now, a question, is there a means by which I can have iomux:event-dispatch
> > return control to me for a while at specified intervals, and then go
> > back to multiplexing i/o? This would allow me to not have to put the
> > server machinery into the read/write functions on the streams themselves
> > or be beholden to my processing only on communication boundaries. If I
> > can get a single threaded solution, then I wouldn't have to worry about
> > the usual things with threads concerning data races and whatnot.
> 
> (event-dispatch *base* :one-shot t)

Two questions, A. will returning from the event-dispatch function using
the above close all connections in the mean time, and B. why is it bad
architecture (at least it feels like it) to put my server's background
work as the timeout code path of make-listener-handler and set the
timeout to something like .01?

Also, I could be potentially sending megabytes to gigabytes of data back
and forth between the clients and server, so I assume there a nonblocking
interface to reading and writing the stream in an unsigned byte sequence
oriented fashion once established? 

Thank you.

-pete



From sionescu at common-lisp.net  Sun Dec 27 10:00:11 2009
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Sun, 27 Dec 2009 05:00:11 -0500
Subject: [iolib-devel] New patches: 26-Dec-2009
Message-ID: <E1NOpv1-0007LJ-IA@common-lisp.net>


commit b8a3c1a535009ba60a3f9156c9e01386efd7e7ba
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Sat Dec 26 22:36:11 2009 +0100

    Fix a couple of bugs in echo-server.lisp
    
    Thanks to Peter Keller.

 examples/echo-server.lisp |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)


An updated tarball of IOLib's source can be downloaded here:
http://common-lisp.net/project/iolib/files/snapshots/iolib-20091226.tar.gz

Gitweb URL:
http://repo.or.cz/w/iolib.git



From psilord at cs.wisc.edu  Mon Dec 28 08:32:14 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Mon, 28 Dec 2009 02:32:14 -0600
Subject: [iolib-devel] New patches: 26-Dec-2009
In-Reply-To: <E1NOpv1-0007LJ-IA@common-lisp.net>
References: <E1NOpv1-0007LJ-IA@common-lisp.net>
Message-ID: <20091228083214.GA9831@cs.wisc.edu>

On Sun, Dec 27, 2009 at 05:00:11AM -0500, Stelian Ionescu wrote:
> commit b8a3c1a535009ba60a3f9156c9e01386efd7e7ba
> Author: Stelian Ionescu <sionescu at cddr.org>
> Date:   Sat Dec 26 22:36:11 2009 +0100
> 
>     Fix a couple of bugs in echo-server.lisp
>     
>     Thanks to Peter Keller.
> 
>  examples/echo-server.lisp |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)

It turns out I found another unrelated bug. :)

To reproduce unreliably:

; run the server like this:
* (run-server :port 10000 :new-process nil :timeout 60)

In another xterm:
> telnet localhost 10000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
hello
1: hello
how
1: how

1: 
are
1: are
you
telnet> close
Connection closed.

Then:
> telnet localhost 10000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
<hangs>


Transcript of what SBCL 1.0.33.30 does:

* (run-server :port 10000 :new-process nil :timeout 60)
Accepted client 1
1: hello
1: how
1: 
1: are
Accepted client 2
1: you
1: closing

debugger invoked on a TYPE-ERROR in thread #<THREAD "initial thread" RUNNING
                                             {AA09C79}>:
  The value NIL is not of type IOLIB.MULTIPLEX::FD-HANDLER.

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

(IOLIB.MULTIPLEX::%DISPATCH-EVENT
 #S(IOLIB.MULTIPLEX::FD-ENTRY
    :FD 8
    :READ-HANDLER NIL
    :WRITE-HANDLER NIL
    :ERROR-CALLBACK NIL)
 :WRITE
 :ERROR
 1478256.382031317d0)
0] * 

Trouble is, it doesn't happen all the time, so maybe a thread race condition?
It only happened once for me so far in my testing. Had I known it wasn't 
easily reproducable, I would have gotten a full backtrace.

Thank you.

-pete






From psilord at cs.wisc.edu  Mon Dec 28 08:37:14 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Mon, 28 Dec 2009 02:37:14 -0600
Subject: [iolib-devel] New patches: 26-Dec-2009
In-Reply-To: <20091228083214.GA9831@cs.wisc.edu>
References: <E1NOpv1-0007LJ-IA@common-lisp.net>
	<20091228083214.GA9831@cs.wisc.edu>
Message-ID: <20091228083714.GA10006@cs.wisc.edu>

On Mon, Dec 28, 2009 at 02:32:14AM -0600, Peter Keller wrote:
> It only happened once for me so far in my testing. Had I known it wasn't 
> easily reproducable, I would have gotten a full backtrace.

It figures I'd get the failure 5 seconds after sending the emai:

0] backtrace

0: (IOLIB.MULTIPLEX::%DISPATCH-EVENT
    #S(IOLIB.MULTIPLEX::FD-ENTRY
       :FD 8
       :READ-HANDLER NIL
       :WRITE-HANDLER NIL
       :ERROR-CALLBACK NIL)
    :WRITE
    :ERROR
    1478938.390490213d0)
1: (IOLIB.MULTIPLEX::%HANDLE-ONE-FD
    #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B2080C1}>
    (8 (:ERROR :READ :WRITE))
    1478938.390490213d0
    #<unavailable argument>)
2: (IOLIB.MULTIPLEX::DISPATCH-FD-EVENTS-ONCE
    #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B2080C1}>
    7.825390378013253d0
    1478938.390490213d0)
3: ((SB-PCL::FAST-METHOD IOLIB.MULTIPLEX:EVENT-DISPATCH
     (IOLIB.MULTIPLEX:EVENT-BASE))
    #<unavailable argument>
    #<unavailable argument>
    #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B2080C1}>)[:EXTERNAL]
4: ((LAMBDA
        (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB-INT:&MORE
         SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.))
    #<unused argument>
    #<unused argument>
    #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B2080C1}>
    -303312229
    2)
5: ((FLET %RUN-SERVER))
6: (RUN-SERVER :HOST #/ip/0.0.0.0 :PORT 10000 :NEW-PROCESS NIL :TIMEOUT 3600)
7: (SB-INT:SIMPLE-EVAL-IN-LEXENV
    (RUN-SERVER :PORT 10000 :NEW-PROCESS NIL :TIMEOUT 3600)
    #<NULL-LEXENV>)
8: (SB-EXT:INTERACTIVE-EVAL
    (RUN-SERVER :PORT 10000 :NEW-PROCESS NIL :TIMEOUT 3600))[:EXTERNAL]
9: (SB-IMPL::REPL-FUN NIL)
10: ((LAMBDA ()))
11: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX #<CLOSURE (LAMBDA #) {AD29EAD}>)
12: (SB-IMPL::TOPLEVEL-REPL NIL)
13: (SB-IMPL::TOPLEVEL-INIT)
14: ((LABELS SB-IMPL::RESTART-LISP))

-pete





From sionescu at cddr.org  Mon Dec 28 15:39:19 2009
From: sionescu at cddr.org (Stelian Ionescu)
Date: Mon, 28 Dec 2009 16:39:19 +0100
Subject: [iolib-devel] New patches: 26-Dec-2009
In-Reply-To: <20091228083714.GA10006@cs.wisc.edu>
References: <E1NOpv1-0007LJ-IA@common-lisp.net>
	<20091228083214.GA9831@cs.wisc.edu>
	<20091228083714.GA10006@cs.wisc.edu>
Message-ID: <1262014759.32312.1.camel@blackhole.cddr.org>

On Mon, 2009-12-28 at 02:37 -0600, Peter Keller wrote:
> On Mon, Dec 28, 2009 at 02:32:14AM -0600, Peter Keller wrote:
> > It only happened once for me so far in my testing. Had I known it wasn't 
> > easily reproducable, I would have gotten a full backtrace.
> 
> It figures I'd get the failure 5 seconds after sending the emai:
> 
> 0] backtrace
> 
> 0: (IOLIB.MULTIPLEX::%DISPATCH-EVENT
>     #S(IOLIB.MULTIPLEX::FD-ENTRY
>        :FD 8
>        :READ-HANDLER NIL
>        :WRITE-HANDLER NIL
>        :ERROR-CALLBACK NIL)
>     :WRITE
>     :ERROR
>     1478938.390490213d0)
> 1: (IOLIB.MULTIPLEX::%HANDLE-ONE-FD
>     #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B2080C1}>
>     (8 (:ERROR :READ :WRITE))
>     1478938.390490213d0
>     #<unavailable argument>)

I think that one of the patches by David Lichteblau that I've just
committed fixes this. Please try again :)

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/iolib-devel/attachments/20091228/055482ed/attachment.sig>

From psilord at cs.wisc.edu  Mon Dec 28 19:09:47 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Mon, 28 Dec 2009 13:09:47 -0600
Subject: [iolib-devel] New patches: 26-Dec-2009
In-Reply-To: <1262014759.32312.1.camel@blackhole.cddr.org>
References: <E1NOpv1-0007LJ-IA@common-lisp.net>
	<20091228083214.GA9831@cs.wisc.edu>
	<20091228083714.GA10006@cs.wisc.edu>
	<1262014759.32312.1.camel@blackhole.cddr.org>
Message-ID: <20091228190947.GA23480@cs.wisc.edu>

On Mon, Dec 28, 2009 at 04:39:19PM +0100, Stelian Ionescu wrote:
> I think that one of the patches by David Lichteblau that I've just
> committed fixes this. Please try again :)

Ok, I've:
./clbuild update iolib
./clbuild recompile iolib

I stopped my running sbcl, started a new one, loaded my code, and hit
the same error. However, it is completely reproducable now, which means it is
probably in my code somewhere so I'll have to debug it a while.

I've included the stack trace, just in case it is in your library and you 
happen to see it.

Thank you.

-pete


  The value NIL is not of type IOLIB.MULTIPLEX::FD-HANDLER.

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

(IOLIB.MULTIPLEX::%DISPATCH-EVENT
 #S(IOLIB.MULTIPLEX::FD-ENTRY
    :FD 8
    :READ-HANDLER NIL
    :WRITE-HANDLER NIL
    :ERROR-CALLBACK NIL)
 :WRITE
 :ERROR
 1515890.136375863d0)
0] backtrace

0: (IOLIB.MULTIPLEX::%DISPATCH-EVENT
    #S(IOLIB.MULTIPLEX::FD-ENTRY
       :FD 8
       :READ-HANDLER NIL
       :WRITE-HANDLER NIL
       :ERROR-CALLBACK NIL)
    :WRITE
    :ERROR
    1515890.136375863d0)
1: (IOLIB.MULTIPLEX::%HANDLE-ONE-FD
    #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B8DCB29}>
    (8 (:ERROR :READ :WRITE))
    1515890.136375863d0
    #<unavailable argument>)
2: (IOLIB.MULTIPLEX::DISPATCH-FD-EVENTS-ONCE
    #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B8DCB29}>
    0.05363454087637365d0
    1515890.136375863d0)
3: ((SB-PCL::FAST-METHOD IOLIB.MULTIPLEX:EVENT-DISPATCH
     (IOLIB.MULTIPLEX:EVENT-BASE))
    #<unavailable argument>
    #<unavailable argument>
    #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B8DCB29}>)[:EXTERNAL]
4: ((LAMBDA
        (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB-INT:&MORE
         SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.))
    #<unused argument>
    #<unused argument>
    #<event base, 2 FDs monitored, using: #<epoll(4) multiplexer> {B8DCB29}>
    -303338815
    4)
5: (START-SERVER :HOST #/ip/0.0.0.0 :PORT NIL :TIMEOUT 86400)
6: (SB-INT:SIMPLE-EVAL-IN-LEXENV (START-SERVER) #<NULL-LEXENV>)
7: (INTERACTIVE-EVAL (START-SERVER))[:EXTERNAL]
8: (SB-IMPL::REPL-FUN NIL)
9: ((LAMBDA ()))
10: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX #<CLOSURE (LAMBDA #) {AD29EAD}>)
11: (SB-IMPL::TOPLEVEL-REPL NIL)
12: (SB-IMPL::TOPLEVEL-INIT)
13: ((LABELS SB-IMPL::RESTART-LISP))




From sionescu at common-lisp.net  Tue Dec 29 10:00:07 2009
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Tue, 29 Dec 2009 05:00:07 -0500
Subject: [iolib-devel] New patches: 28-Dec-2009
Message-ID: <E1NPYs3-0004xi-U3@common-lisp.net>


commit e0d819ff198c0db40310b122f8b7b1edb3644f26
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Mon Dec 28 16:42:21 2009 +0100

    Small fix: ignoring wrong variable.

 examples/echo-server.lisp |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

commit 01ab6520626dda4d765e562fceb045a77090dbd2
Author: David Lichteblau <david at lichteblau.com>
Date:   Mon Dec 28 16:30:03 2009 +0100

    Add %SYS-EXIT that calls _exit(2) not exit(3).

 src/syscalls/ffi-functions-unix.lisp |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

commit f2ace2ddb13de7116b7bb77a0951a53f1136bbd3
Author: David Lichteblau <david at lichteblau.com>
Date:   Mon Dec 28 16:25:15 2009 +0100

    Fix return format of HARVEST-SELECT-FD-ERRORS.

 src/multiplex/backend-select.lisp |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

commit a101999472d90e71cc34685a10af22b88c55bf9f
Author: David Lichteblau <david at lichteblau.com>
Date:   Mon Dec 28 16:14:51 2009 +0100

    Check if the FD-ENTRY has an error callback registered before dispatch.

 src/multiplex/event-loop.lisp |    5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)


An updated tarball of IOLib's source can be downloaded here:
http://common-lisp.net/project/iolib/files/snapshots/iolib-20091228.tar.gz

Gitweb URL:
http://repo.or.cz/w/iolib.git



From psilord at cs.wisc.edu  Tue Dec 29 18:41:13 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Tue, 29 Dec 2009 12:41:13 -0600
Subject: [iolib-devel] Broken code
Message-ID: <20091229184113.GA23254@cs.wisc.edu>

Hello,

I've attached an example program, loadable with (load "server.lisp"),
and runnable with (start-server). It uses ioib and alexandria.

Once the server is started, one can 'telnet localhost 10000'. As soon as
this happens you will see lisp forms automatically sent to the client,
plus a timeout form every sixty seconds. The server will emit some
information about the stream it is using before sending the data.

To see the bug, ctrl-} the telnet session on the client, and close
it with 'close'.  There should be an immediate error on the server.
This happens for me 100% of the time.

Thank you.

-pete
-------------- next part --------------
; sbcl.28005

(eval-when (:compile-toplevel :load-toplevel :execute)
  (asdf:oos 'asdf:load-op :iolib.sockets))

(eval-when (:compile-toplevel :load-toplevel :execute)
  (asdf:oos 'asdf:load-op :alexandria))

(use-package 'common-lisp)
(use-package 'iolib.sockets)
(use-package 'alexandria)

(defparameter *event-base* nil)
(defparameter *sockets* nil)
(defparameter *port* 10000)
(defparameter *count* 0)
(defvar *streams* nil)

(defun start-server (&key (host iolib.sockets:+ipv4-unspecified+) (port *port*) (timeout (* 60 60 24)))

  (setf *streams* (make-hash-table :test #'equal))

  (let ((iolib.sockets:*ipv6* nil))
       (unwind-protect
        (progn
         (setf *event-base* (make-instance 'iomux:event-base))
         (format t "About to start-my-server~%")
         (with-open-stream (sock (start-my-server host port))
                           (declare (ignorable sock))
                           (format t "Starting event-dispatch...~%")
                           (do ()
                               (nil nil)
                               ;; here is where I should stick one main loop
                               ;; iteration and then N i/o iterations.
                               (stdout-streams)
                               (transmit-work `(work ,(get-universal-time)))
                               (iomux:event-dispatch *event-base*
                                                     :timeout .10
                                                     :one-shot t))
                           (format t "Ending event-dispatch...~%")))
        (format t  "No new clients in ~A seconds: timed out. Terminating." 
                timeout)
        (close-all-sockets)
        (close *event-base*))))

(defun close-all-sockets ()
  (setf *streams* (make-hash-table :test #'equal))
  (map 'nil #'close-socket *sockets*))

(defun start-my-server (host port)
  (format t "Start-my-server called: ~A ~A~%" host port)
  (let ((socket
         (iolib.sockets:make-socket 
          :connect :passive 
          :address-family :internet
          :type :stream 
          :local-host host
          :local-port port 
          :backlog 5 
          :reuse-address t
          :external-format '(:utf-8 :eol-style :crlf)
          :ipv6 nil)))

       (format t "socket bound: ~A~%" socket)
       (setf *sockets* nil)
       (unwind-protect-case ()
                            (progn
                             (format t "Setting socket to nonblocking~%")
                             (setf (iolib.streams:fd-non-blocking socket) t)
                             (format t "set-io-handler called...~%")
                             (iomux:set-io-handler
                              *event-base*
                              (iolib.sockets:socket-os-fd socket)
                              :read 
                              (make-listener-handler socket)
                              :timeout 10))
                            (:abort 
                             (close socket)))
       (format t "start-my-server returning a socket!~%")
       socket))

(defun make-listener-handler (socket)
  (format t "Making listener handler...~%")
  #'(lambda (fd event exception)
            (declare (ignore fd event))
            (block nil
                   (when (eql :timeout exception)
                         (warn "Got a server timeout: ~A (time ~A)!" *count*
                               (get-universal-time))
                         (incf *count*)
                         (transmit-work '(a server timeout))
                         (return))
                   (let ((client (iolib.sockets:accept-connection socket)))
                        (format t "accepted connection: ~A~%" client)
                        (when client
                              (format t "set client to nonblocking~%")
                              (setf (iolib.streams:fd-non-blocking client) t)
                              (add-socket client)
                              (format t "Accepted a client~%")
                              (serve client))))))


(defun serve (socket)
  (format t "Serving the client and setting up i/o handler~%")
  (iomux:set-io-handler *event-base*
                        (iolib.sockets:socket-os-fd socket)
                        :read
                        (make-conversation socket (make-disconnector socket))))


(defun make-disconnector (socket)
  #'(lambda ()
            (format t "Disconnecting a socket~%")
            (remove-stream socket)
            (close-socket socket)))

(defun make-conversation (stream disconnector)
  (format t "Making an conversation function.~%")
  (add-stream stream)
  #'(lambda (fd event exception)
            (declare (ignore fd event exception))
            (handler-case
             (conversation stream disconnector)
             (end-of-file ()
                          (remove-stream stream)
                          (funcall disconnector)))))

;; read a lisp form from the stream and write 'ok back to the stream.
(defun conversation (stream disconnector)
  (let ((*read-eval* nil))
       (let ((form (read stream)))
            (cond ((equal form '(quit))
                   (funcall disconnector))
                  (t
                   (format t "Form from Client: ~A~%" form)
                   (format stream "~s~%" 'OK)
                   (ignore-some-conditions
                    (iolib.streams:hangup)
                    (finish-output stream)))))))


(defun close-socket (socket)
  (format t "close-socket: ~A~%" socket)
  (let ((fd (iolib.sockets:socket-os-fd socket)))
       (ignore-some-conditions (isys:syscall-error)
                               (iomux:remove-fd-handlers *event-base* fd))
       (remove-socket socket)
       (close socket)))

(defun remove-socket (socket)
  (format t "remove-socket ~s~%" socket)
  (removef *sockets* socket))

(defun add-socket (socket)
  (format t "add-socket ~s~%" socket)
  (push socket *sockets*))

(defun remove-stream (stream)
  (format t "remove-stream [count: ~A] ~s~%" (hash-table-count *streams*) stream)
  (remhash stream *streams*))

(defun add-stream (stream)
  (format t "add-stream [count: ~A] ~s~%" (hash-table-count *streams*) stream)
  (setf (gethash stream *streams*) stream))

(defun stdout-streams ()
  (when (> (hash-table-count *streams*) 0)
        (maphash
         #'(lambda (k v) 
                   (format t "State of stream ~s is ~s~%" k v))
         *streams*)))


(defun transmit-work (form)
  (when (> (hash-table-count *streams*) 0)
        (maphash
         #'(lambda (k v) 
                   (declare (ignore k))
                   ;(format t "Transmitting to ~s~%" k)
                   (format v "~s~%" form)
                   (finish-output v)
         *streams*)))

;; commented out.
;;(handler-bind
;;  ((isys:eintr (lambda (e) (invoke-restart
;;                            'ignore-syscall-error))))
;;  (send-to socket (cl-serializer:serialize 42)
;;           :remote-host "hostname" :remote-port
;;           9999))






From psilord at cs.wisc.edu  Wed Dec 30 01:13:55 2009
From: psilord at cs.wisc.edu (Peter Keller)
Date: Tue, 29 Dec 2009 19:13:55 -0600
Subject: [iolib-devel] Broken code, no syntax errors
Message-ID: <20091230011355.GA31311@cs.wisc.edu>

Hello,

Here is the same code as before, but fixed as to compile, sorry about that.

-pete

-------------- next part --------------
; sbcl.28005

(eval-when (:compile-toplevel :load-toplevel :execute)
  (asdf:oos 'asdf:load-op :iolib.sockets))

(eval-when (:compile-toplevel :load-toplevel :execute)
  (asdf:oos 'asdf:load-op :alexandria))

(use-package 'common-lisp)
(use-package 'iolib.sockets)
(use-package 'alexandria)

(defparameter *event-base* nil)
(defparameter *sockets* nil)
(defparameter *port* 10000)
(defparameter *count* 0)
(defvar *streams* nil)

(defun start-server (&key (host iolib.sockets:+ipv4-unspecified+) (port *port*) (timeout (* 60 60 24)))

  (setf *streams* (make-hash-table :test #'equal))

  (let ((iolib.sockets:*ipv6* nil))
       (unwind-protect
        (progn
         (setf *event-base* (make-instance 'iomux:event-base))
         (format t "About to start-my-server~%")
         (with-open-stream (sock (start-my-server host port))
                           (declare (ignorable sock))
                           (format t "Starting event-dispatch...~%")
                           (do ()
                               (nil nil)
                               ;; here is where I should stick one main loop
                               ;; iteration and then N i/o iterations.
                               (stdout-streams)
                               (transmit-work `(work ,(get-universal-time)))
                               (iomux:event-dispatch *event-base*
                                                     :timeout .10
                                                     :one-shot t))
                           (format t "Ending event-dispatch...~%")))
        (format t  "No new clients in ~A seconds: timed out. Terminating." 
                timeout)
        (close-all-sockets)
        (close *event-base*))))

(defun close-all-sockets ()
  (setf *streams* (make-hash-table :test #'equal))
  (map 'nil #'close-socket *sockets*))

(defun start-my-server (host port)
  (format t "Start-my-server called: ~A ~A~%" host port)
  (let ((socket
         (iolib.sockets:make-socket 
          :connect :passive 
          :address-family :internet
          :type :stream 
          :local-host host
          :local-port port 
          :backlog 5 
          :reuse-address t
          :external-format '(:utf-8 :eol-style :crlf)
          :ipv6 nil)))

       (format t "socket bound: ~A~%" socket)
       (setf *sockets* nil)
       (unwind-protect-case ()
                            (progn
                             (format t "Setting socket to nonblocking~%")
                             (setf (iolib.streams:fd-non-blocking socket) t)
                             (format t "set-io-handler called...~%")
                             (iomux:set-io-handler
                              *event-base*
                              (iolib.sockets:socket-os-fd socket)
                              :read 
                              (make-listener-handler socket)
                              :timeout 10))
                            (:abort 
                             (close socket)))
       (format t "start-my-server returning a socket!~%")
       socket))

(defun make-listener-handler (socket)
  (format t "Making listener handler...~%")
  #'(lambda (fd event exception)
            (declare (ignore fd event))
            (block nil
                   (when (eql :timeout exception)
                         (warn "Got a server timeout: ~A (time ~A)!" *count*
                               (get-universal-time))
                         (incf *count*)
                         (transmit-work '(a server timeout))
                         (return))
                   (let ((client (iolib.sockets:accept-connection socket)))
                        (format t "accepted connection: ~A~%" client)
                        (when client
                              (format t "set client to nonblocking~%")
                              (setf (iolib.streams:fd-non-blocking client) t)
                              (add-socket client)
                              (format t "Accepted a client~%")
                              (serve client))))))


(defun serve (socket)
  (format t "Serving the client and setting up i/o handler~%")
  (iomux:set-io-handler *event-base*
                        (iolib.sockets:socket-os-fd socket)
                        :read
                        (make-conversation socket (make-disconnector socket))))


(defun make-disconnector (socket)
  #'(lambda ()
            (format t "Disconnecting a socket~%")
            (remove-stream socket)
            (close-socket socket)))

(defun make-conversation (stream disconnector)
  (format t "Making an conversation function.~%")
  (add-stream stream)
  #'(lambda (fd event exception)
            (declare (ignore fd event exception))
            (handler-case
             (conversation stream disconnector)
             (end-of-file ()
                          (remove-stream stream)
                          (funcall disconnector)))))

;; read a lisp form from the stream and write 'ok back to the stream.
(defun conversation (stream disconnector)
  (let ((*read-eval* nil))
       (let ((form (read stream)))
            (cond ((equal form '(quit))
                   (funcall disconnector))
                  (t
                   (format t "Form from Client: ~A~%" form)
                   (format stream "~s~%" 'OK)
                   (ignore-some-conditions
                    (iolib.streams:hangup)
                    (finish-output stream)))))))


(defun close-socket (socket)
  (format t "close-socket: ~A~%" socket)
  (let ((fd (iolib.sockets:socket-os-fd socket)))
       (ignore-some-conditions (isys:syscall-error)
                               (iomux:remove-fd-handlers *event-base* fd))
       (remove-socket socket)
       (close socket)))

(defun remove-socket (socket)
  (format t "remove-socket ~s~%" socket)
  (removef *sockets* socket))

(defun add-socket (socket)
  (format t "add-socket ~s~%" socket)
  (push socket *sockets*))

(defun remove-stream (stream)
  (format t "remove-stream [count: ~A] ~s~%" (hash-table-count *streams*) stream)
  (remhash stream *streams*))

(defun add-stream (stream)
  (format t "add-stream [count: ~A] ~s~%" (hash-table-count *streams*) stream)
  (setf (gethash stream *streams*) stream))

(defun stdout-streams ()
  (when (> (hash-table-count *streams*) 0)
        (maphash
         #'(lambda (k v) 
                   (format t "State of stream ~s is ~s~%" k v))
         *streams*)))


(defun transmit-work (form)
  (when (> (hash-table-count *streams*) 0)
        (maphash
         #'(lambda (k v) 
                   (declare (ignore k))
                   ;(format t "Transmitting to ~s~%" k)
                   (format v "~s~%" form)
                   (finish-output v))
                   *streams*)))

;; commented out.
;;(handler-bind
;;  ((isys:eintr (lambda (e) (invoke-restart
;;                            'ignore-syscall-error))))
;;  (send-to socket (cl-serializer:serialize 42)
;;           :remote-host "hostname" :remote-port
;;           9999))

From sionescu at common-lisp.net  Wed Dec 30 10:00:08 2009
From: sionescu at common-lisp.net (Stelian Ionescu)
Date: Wed, 30 Dec 2009 05:00:08 -0500
Subject: [iolib-devel] New patches: 29-Dec-2009
Message-ID: <E1NPvLc-0000D2-GG@common-lisp.net>


commit 158f5c25c2209e859420c61e3c073f15b183708e
Author: Stelian Ionescu <sionescu at cddr.org>
Date:   Tue Dec 29 18:17:37 2009 +0100

    Export restarts IGNORE-SYSCALL-ERROR and RETRY-SYSCALL.

 src/sockets/pkgdcl.lisp |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)


An updated tarball of IOLib's source can be downloaded here:
http://common-lisp.net/project/iolib/files/snapshots/iolib-20091229.tar.gz

Gitweb URL:
http://repo.or.cz/w/iolib.git