From ehuels at gmail.com Fri Mar 2 21:44:42 2007 From: ehuels at gmail.com (Erik Huelsmann) Date: Fri, 2 Mar 2007 22:44:42 +0100 Subject: [drakma-devel] Re: Portability of Drakma In-Reply-To: References: Message-ID: On 2/25/07, Edi Weitz wrote: > On Sun, 25 Feb 2007 14:38:02 +0100, "Erik Huelsmann" wrote: > > >> - The code should have been tested successfully on at least the > >> Lisp/OS combinations that are currently supported by Drakma. > > > > Is there a list somewhere as a reference to what I'm getting into? > > No, unfortunately not. I myself use mostly LispWorks (Windows and > Linux/x86) and SBCL (Linux/x86). (LispWorks shouldn't be a problem > anyway as it's not affected by the switch.) > > I think that at least LispWorks, SBCL, AllegroCL, CMUCL, CLISP, and > OpenMCL should be supported, everything else being a bonus. Operating > systems: Windows (where applicable), Linux, OS X. > > If you don't want to test all of this for yourself, how about offering > a tarball of Drakma which uses usocket for download? Send an > announcement to this mailing list including an overview of what you've > tested and what not. We can ask "interested parties" to try it out > and we'll switch to the new version in four weeks, say, unless someone > objects. Does that sound OK? Sounds great. I did an initial port which works, but I then realised it doesn't exactly conform to the current usocket API :-( What I did was bind the socket-stream associated with the usocket object instead of the open-stream return value by trivial-sockets. That will work on all (current) implementations, but doesn't really conform with the API. The only benefit will be that Drakma runs on more platforms, but none of the additional usocket functionality can be used (as the usocket object itself can't be retrieved from the stream). This means I have 3 choices: 1) return both the stream and the socket object in the return values 2) use trivial-usocket: the trivial-sockets portability library which associates a usocket with a gray stream. Closing the stream will close the socket. But it adds another (rather useless IMO) call for every socket stream operation. Also, trivial-usocket has a smaller supported set of implementations than usocket does. 3) slightly violate/change the usocket API to require changing the stream to be a valid action for closing the socket. So, my initial reaction would be to change the http-request interface: still returning the http-stream return value, but return also a seventh value: the socket. On input we'd require the stream and the socket when the :stream key is supplied. But I could live with (3) as well. Do you have any preferences? bye, Erik. From edi at agharta.de Sun Mar 4 13:54:45 2007 From: edi at agharta.de (Edi Weitz) Date: Sun, 04 Mar 2007 14:54:45 +0100 Subject: [drakma-devel] Re: Portability of Drakma In-Reply-To: (Erik Huelsmann's message of "Fri, 2 Mar 2007 22:44:42 +0100") References: Message-ID: On Fri, 2 Mar 2007 22:44:42 +0100, "Erik Huelsmann" wrote: > This means I have 3 choices: > > 1) return both the stream and the socket object in the return values > 2) use trivial-usocket: the trivial-sockets portability library which > associates a usocket with a gray stream. Closing the stream will close > the socket. But it adds another (rather useless IMO) call for every > socket stream operation. Also, trivial-usocket has a smaller supported > set of implementations than usocket does. > 3) slightly violate/change the usocket API to require changing the > stream to be a valid action for closing the socket. > > So, my initial reaction would be to change the http-request interface: > still returning the http-stream return value, but return also a > seventh value: the socket. On input we'd require the stream and the > socket when the :stream key is supplied. > > But I could live with (3) as well. > > Do you have any preferences? I'd prefer (3) because it means we don't have to change Drakma and I won't have to do any work (for which I don't have time right now). But I can live with (1), if you provide a clean patch for that - including all the necessary updates to docstrings and the HTML documentation... What is the additional usocket functionality we'd lose if we opt for (3)? Thanks, Edi. From ehuels at gmail.com Sun Mar 4 16:07:01 2007 From: ehuels at gmail.com (Erik Huelsmann) Date: Sun, 4 Mar 2007 17:07:01 +0100 Subject: [drakma-devel] Re: Portability of Drakma In-Reply-To: References: Message-ID: On 3/4/07, Edi Weitz wrote: > On Fri, 2 Mar 2007 22:44:42 +0100, "Erik Huelsmann" wrote: > > > This means I have 3 choices: > > > > 1) return both the stream and the socket object in the return values > > 2) use trivial-usocket: the trivial-sockets portability library which > > associates a usocket with a gray stream. Closing the stream will close > > the socket. But it adds another (rather useless IMO) call for every > > socket stream operation. Also, trivial-usocket has a smaller supported > > set of implementations than usocket does. > > 3) slightly violate/change the usocket API to require changing the > > stream to be a valid action for closing the socket. > > > > So, my initial reaction would be to change the http-request interface: > > still returning the http-stream return value, but return also a > > seventh value: the socket. On input we'd require the stream and the > > socket when the :stream key is supplied. > > > > But I could live with (3) as well. > > > > Do you have any preferences? > > I'd prefer (3) because it means we don't have to change Drakma and I > won't have to do any work (for which I don't have time right now). I figured so much, so I've been working to make the API support (3). It looks like I'll be able to make it so. > But I can live with (1), if you provide a clean patch for that - > including all the necessary updates to docstrings and the HTML > documentation... I'll regard (3) as a minimal version. I can submit a patch for (1) which will ofcourse change the docs and docstrings. For now, I'll focus on (3) though. > What is the additional usocket functionality we'd lose if we opt for > (3)? It won't be possible to support any other function than reading/writing content on the socket. Other functionality includes retrieving the local or remote socket 'names' (ip+port). I hope to support read/write timeout settings in the future too. That functionality won't be available (outside of http-request; internally, it could bind the usocket, ofcourse). I hope that answers your question. bye, Erik. From edi at agharta.de Sun Mar 4 16:16:48 2007 From: edi at agharta.de (Edi Weitz) Date: Sun, 04 Mar 2007 17:16:48 +0100 Subject: [drakma-devel] Re: Portability of Drakma In-Reply-To: (Erik Huelsmann's message of "Sun, 4 Mar 2007 17:07:01 +0100") References: Message-ID: On Sun, 4 Mar 2007 17:07:01 +0100, "Erik Huelsmann" wrote: >> I'd prefer (3) because it means we don't have to change Drakma and >> I won't have to do any work (for which I don't have time right >> now). > > I figured so much, so I've been working to make the API support (3). > It looks like I'll be able to make it so. > >> But I can live with (1), if you provide a clean patch for that - >> including all the necessary updates to docstrings and the HTML >> documentation... > > I'll regard (3) as a minimal version. I can submit a patch for (1) > which will ofcourse change the docs and docstrings. For now, I'll > focus on (3) though. OK, fine with me. >> What is the additional usocket functionality we'd lose if we opt >> for (3)? > > It won't be possible to support any other function than > reading/writing content on the socket. Other functionality includes > retrieving the local or remote socket 'names' (ip+port). I hope to > support read/write timeout settings in the future too. That > functionality won't be available (outside of http-request; > internally, it could bind the usocket, ofcourse). > > I hope that answers your question. Yes, thanks. It looks like (1) should be the medium-term goal, especially as (user-defined) timeouts are currently only available for LispWorks. Thanks, Edi. From ehuels at gmail.com Mon Mar 5 07:54:57 2007 From: ehuels at gmail.com (Erik Huelsmann) Date: Mon, 5 Mar 2007 08:54:57 +0100 Subject: [drakma-devel] Re: Portability of Drakma In-Reply-To: References: Message-ID: On 2/25/07, Edi Weitz wrote: > On Sun, 25 Feb 2007 14:38:02 +0100, "Erik Huelsmann" wrote: > I think that at least LispWorks, SBCL, AllegroCL, CMUCL, CLISP, and > OpenMCL should be supported, everything else being a bonus. Operating > systems: Windows (where applicable), Linux, OS X. > > If you don't want to test all of this for yourself, how about offering > a tarball of Drakma which uses usocket for download? Send an > announcement to this mailing list including an overview of what you've > tested and what not. We can ask "interested parties" to try it out > and we'll switch to the new version in four weeks, say, unless someone > objects. Does that sound OK? I created a patch (and a full archive) for drakma 0.6.0. I did initial testing with usocket 0.3.2 and sbcl on linux 2.6. I'll put up a text file at the same location to log the already-tested combinations. The archive is at http://hix.nu/drakma/ (which is behind my dsl line at 1Mbit). Version 0.3.2 for usocket is required: I added the close-the-stream-means-closing-the-socket API guarantee in that version. Any problems with usocket can be reported directly to me, here (for reference for other testers), or at usocket-devel at common-lisp.net. I'll be testing a number of other combinations too, but I was out of time this weekend. Thanks in advance for any help provided! bye, Erik. From jianshi.huang at gmail.com Mon Mar 5 08:13:09 2007 From: jianshi.huang at gmail.com (Jianshi Huang) Date: Mon, 5 Mar 2007 17:13:09 +0900 Subject: [drakma-devel] [Bug Report] split-string behaves different in Lispworks and SBCL. Message-ID: <28b25c50703050013u21b4f3s4a2ba4e7d0273dae@mail.gmail.com> I updated drakma to 0.6.0 but found the cookie could not be parsed correctly in SBCL. So I traced down into the source code and found it's the problem of function split-string. But in Lispworks Personal, it works well. As the function split-string is a big extended loop macro. So I think it might be a bug in SBCL's loop facility. I also tested in Allegro CL and found it behaves the same as SBCL. So which is correct? Here's the function split-string in 0.6.0 (10defun split-string (11string &optional (separators " ,-")) "Splits STRING into substrings separated by the characters in the sequence SEPARATORS. Empty substrings aren't collected." (12loop for 13char across 14string 15when (16find 17char separators :test #'18char=) 19when collector collect (20coerce collector '21string) into result 22and 23do (24setq collector 25nil) end else collect 26char into collector finally (27return (28if collector (29append result (30list (31coerce collector '32string))) result)))) Here's the REPL history in Lispworks: CL-USER 2 > (33setf x "Tue, 06-Mar-2007 00:00:00 GMT") "Tue, 06-Mar-2007 00:00:00 GMT" CL-USER 4 > (split-string x) ("Tue" "06" "Mar" "2007" "00:00:00" "GMT") Here's in SBCL: CL-USER> (34setf x "Tue, 06-Mar-2007 00:00:00 GMT") "Tue, 06-Mar-2007 00:00:00 GMT" CL-USER> (split-string x) ("Tue" "Tue06" "Tue06Mar" "Tue06Mar2007" "Tue06Mar200700:00:00" "Tue06Mar200700:00:00GMT") From edi at agharta.de Tue Mar 6 19:09:49 2007 From: edi at agharta.de (Edi Weitz) Date: Tue, 06 Mar 2007 20:09:49 +0100 Subject: [drakma-devel] [Bug Report] split-string behaves different in Lispworks and SBCL. In-Reply-To: <28b25c50703050013u21b4f3s4a2ba4e7d0273dae@mail.gmail.com> (Jianshi Huang's message of "Mon, 5 Mar 2007 17:13:09 +0900") References: <28b25c50703050013u21b4f3s4a2ba4e7d0273dae@mail.gmail.com> Message-ID: On Mon, 5 Mar 2007 17:13:09 +0900, "Jianshi Huang" wrote: > I updated drakma to 0.6.0 but found the cookie could not be parsed > correctly in SBCL. So I traced down into the source code and found > it's the problem of function split-string. But in Lispworks > Personal, it works well. As the function split-string is a big > extended loop macro. So I think it might be a bug in SBCL's loop > facility. I also tested in Allegro CL and found it behaves the same > as SBCL. So which is correct? I don't know. Obviously, AllegroCL and SBCL both simply ignore the (SETQ COLLECTOR NIL) form. I took a brief look at the CLHS description of LOOP but couldn't find anything specific which supported either LispWorks or SBCL/AllegroCL except for the sentence "The VAR argument is bound as if by the construct WITH" which for me sounds as if LispWorks is right. Any language lawyers on this mailing list? Would you like to ask on the SBCL mailing list? > Here's the function split-string in 0.6.0 > > (10defun split-string (11string &optional (separators " ,-")) > "Splits STRING into substrings separated by the characters in the > sequence SEPARATORS. Empty substrings aren't collected." > (12loop for 13char across 14string How did you manage to add those numbers there? Cheers, Edi. From jianshi.huang at gmail.com Thu Mar 8 07:52:54 2007 From: jianshi.huang at gmail.com (Jianshi Huang) Date: Thu, 8 Mar 2007 16:52:54 +0900 Subject: [drakma-devel] Re: [Bug Report] split-string behaves different in Lispworks and SBCL. (Jianshi Huang) Message-ID: <28b25c50703072352p482423f3g55ae25695baba860@mail.gmail.com> > Would you like to ask on the SBCL mailing list? > Yes, I asked this question on #lisp and Andreas Fuchs has helped me posting it on SBCL. One of the reply said it should be a bug in SBCL because he gave an example showing the inconsistent behavior. Here's the example: Andreas Fuchs posted: (loop for i in '(1 2 3 a 4 5 6) if (not (symbolp i)) collect i into collector else do (setf collector nil) finally (return collector)) does not return (4 5 6), but (1 2 3 4 5 6). And Alastair Bridgewater said: Note using (1 2 3 a 4 5 6 b) as input will result in NIL. > > Here's the function split-string in 0.6.0 > > > > (10defun split-string (11string &optional (separators " ,-")) > > "Splits STRING into substrings separated by the characters in the > > sequence SEPARATORS. Empty substrings aren't collected." > > (12loop for 13char across 14string > > How did you manage to add those numbers there? Hmm.... these are all the magic numbers added by copying from paste.lisp.org. Here's the right one. (defun split-string (string &optional (separators " ,-")) "Splits STRING into substrings separated by the characters in the sequence SEPARATORS. Empty substrings aren't collected." (loop for char across string when (find char separators :test #'char=) when collector collect (coerce collector 'string) into result and do (setq collector nil) end else collect char into collector finally (return (if collector (append result (list (coerce collector 'string))) result)))) Cheers, pebblestone From edi at agharta.de Thu Mar 8 19:37:20 2007 From: edi at agharta.de (Edi Weitz) Date: Thu, 08 Mar 2007 20:37:20 +0100 Subject: [drakma-devel] New release 0.6.1 (was: [Bug Report] split-string behaves different in Lispworks and SBCL.) In-Reply-To: <28b25c50703050013u21b4f3s4a2ba4e7d0273dae@mail.gmail.com> (Jianshi Huang's message of "Mon, 5 Mar 2007 17:13:09 +0900") References: <28b25c50703050013u21b4f3s4a2ba4e7d0273dae@mail.gmail.com> Message-ID: On Mon, 5 Mar 2007 17:13:09 +0900, "Jianshi Huang" wrote: > I updated drakma to 0.6.0 but found the cookie could not be parsed > correctly in SBCL. So I traced down into the source code and found > it's the problem of function split-string. But in Lispworks > Personal, it works well. As the function split-string is a big > extended loop macro. So I think it might be a bug in SBCL's loop > facility. I also tested in Allegro CL and found it behaves the same > as SBCL. So which is correct? OK, with the help of comp.lang.lisp[1] I've convinced myself that SPLIT-STRING relied on unspecified (or too vaguely specified) behaviour. I've now released a new version which hopefully only uses documented LOOP features. Please test. Thanks for the report, Edi. [1] http://groups.google.com/group/comp.lang.lisp/browse_frm/thread/e4bf7cc93bb3c50a/fc5c10fed5382464#fc5c10fed5382464 From edi at agharta.de Thu Mar 8 23:38:47 2007 From: edi at agharta.de (Edi Weitz) Date: Fri, 09 Mar 2007 00:38:47 +0100 Subject: [drakma-devel] Re: Portability of Drakma In-Reply-To: (Erik Huelsmann's message of "Mon, 5 Mar 2007 08:54:57 +0100") References: Message-ID: On Mon, 5 Mar 2007 08:54:57 +0100, "Erik Huelsmann" wrote: > I created a patch (and a full archive) for drakma 0.6.0. I did > initial testing with usocket 0.3.2 and sbcl on linux 2.6. I'll put > up a text file at the same location to log the already-tested > combinations. The archive is at http://hix.nu/drakma/ (which is > behind my dsl line at 1Mbit). > > Version 0.3.2 for usocket is required: I added the > close-the-stream-means-closing-the-socket API guarantee in that > version. > > Any problems with usocket can be reported directly to me, here (for > reference for other testers), or at usocket-devel at common-lisp.net. > > I'll be testing a number of other combinations too, but I was out of > time this weekend. Did you get any more test results in the meantime? From nowhere.man at levallois.eu.org Fri Mar 9 01:23:47 2007 From: nowhere.man at levallois.eu.org (Pierre THIERRY) Date: Fri, 9 Mar 2007 02:23:47 +0100 Subject: [drakma-devel] Re: Portability of Drakma In-Reply-To: References: Message-ID: <20070309012347.GB8176@bateleur.arcanes.fr.eu.org> Scribit Erik Huelsmann dies 05/03/2007 hora 08:54: > Any problems with usocket can be reported directly to me, here (for > reference for other testers), or at usocket-devel at common-lisp.net. Testing a network system is far more difficult and error-prone than testing e.g. a calculus library, so maybe providing a test suite would be valuable here. I was about to test this usocket version of drakma when I realized that I wouldn't where to begin... Uncertainly, Pierre -- nowhere.man at levallois.eu.org OpenPGP 0xD9D50D8A -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From edi at agharta.de Fri Mar 9 01:31:45 2007 From: edi at agharta.de (Edi Weitz) Date: Fri, 09 Mar 2007 02:31:45 +0100 Subject: [drakma-devel] Re: Portability of Drakma In-Reply-To: <20070309012347.GB8176@bateleur.arcanes.fr.eu.org> (Pierre THIERRY's message of "Fri, 9 Mar 2007 02:23:47 +0100") References: <20070309012347.GB8176@bateleur.arcanes.fr.eu.org> Message-ID: On Fri, 9 Mar 2007 02:23:47 +0100, Pierre THIERRY wrote: > Testing a network system is far more difficult and error-prone than > testing e.g. a calculus library, so maybe providing a test suite > would be valuable here. I thought about a test suite for Drakma, but for that to make sense you'd need a reasonable set of very stable URLs (can always be reached, content never changes). These are hard to find, I'd say... > I was about to test this usocket version of drakma when I realized > that I wouldn't where to begin... Just visit a few URLs you're usually visiting with your web browser. Or follow the examples in the Drakma documentation. From jeffrey at cunningham.net Fri Mar 9 02:48:41 2007 From: jeffrey at cunningham.net (Jeffrey Cunningham) Date: Thu, 8 Mar 2007 18:48:41 -0800 Subject: [drakma-devel] Release data on Drakma In-Reply-To: References: <28b25c50703050013u21b4f3s4a2ba4e7d0273dae@mail.gmail.com> Message-ID: <20070309024841.GA24035@achilles.olympus.net> Edi, You might want to change the change year on your website for Drakma to 2007 (it says 2006). --Jeff From edi at agharta.de Fri Mar 9 08:42:30 2007 From: edi at agharta.de (Edi Weitz) Date: Fri, 09 Mar 2007 09:42:30 +0100 Subject: [drakma-devel] New release 0.6.2 (Was: Release data on Drakma) In-Reply-To: <20070309024841.GA24035@achilles.olympus.net> (Jeffrey Cunningham's message of "Thu, 8 Mar 2007 18:48:41 -0800") References: <28b25c50703050013u21b4f3s4a2ba4e7d0273dae@mail.gmail.com> <20070309024841.GA24035@achilles.olympus.net> Message-ID: On Thu, 8 Mar 2007 18:48:41 -0800, Jeffrey Cunningham wrote: > You might want to change the change year on your website for Drakma > to 2007 (it says 2006). Ah, yes, thanks. I've made a new release to clean up this mess - no code changes, though. From jianshi.huang at gmail.com Fri Mar 9 17:44:03 2007 From: jianshi.huang at gmail.com (Jianshi Huang) Date: Sat, 10 Mar 2007 02:44:03 +0900 Subject: [drakma-devel] Re: New release 0.6.1 (was: [Bug Report] split-string behaves different in Lispworks and SBCL.) (Jianshi Huang) Message-ID: <28b25c50703090944s5feb26fcwa899e5eb7a2372a4@mail.gmail.com> > OK, with the help of comp.lang.lisp[1] I've convinced myself that > SPLIT-STRING relied on unspecified (or too vaguely specified) > behaviour. I've now released a new version which hopefully only uses > documented LOOP features. Please test. > So it's unspecified. BTW I enjoyed reading the discussion.