From dl at znain.net Wed Sep 3 02:27:45 2008 From: dl at znain.net (Dimitre Liotev) Date: Tue, 02 Sep 2008 22:27:45 -0400 Subject: [drakma-devel] END-OF-FILE error Message-ID: <87iqtem09q.fsf@debo.znain.com> When trying this: (drakma:http-request "http://news.ycombinator.com/rss") I get and end-of-file error. I can pull this page with curl and firefox, and can't figure out the problem with Drakma. Any ideas? -- Dimitre Liotev From edi at agharta.de Wed Sep 3 06:44:39 2008 From: edi at agharta.de (Edi Weitz) Date: Wed, 03 Sep 2008 08:44:39 +0200 Subject: [drakma-devel] END-OF-FILE error In-Reply-To: <87iqtem09q.fsf@debo.znain.com> (Dimitre Liotev's message of "Tue, 02 Sep 2008 22:27:45 -0400") References: <87iqtem09q.fsf@debo.znain.com> Message-ID: On Tue, 02 Sep 2008 22:27:45 -0400, Dimitre Liotev
wrote: > When trying this: > > (drakma:http-request "http://news.ycombinator.com/rss") > > I get and end-of-file error. http://weitz.de/drakma/#prob From dl at znain.net Thu Sep 4 01:27:09 2008 From: dl at znain.net (Dimitre Liotev) Date: Wed, 03 Sep 2008 21:27:09 -0400 Subject: [drakma-devel] END-OF-FILE error In-Reply-To: (sfid-20080903_024448_361790_26B9C6EB) (Edi Weitz's message of "Wed, 03 Sep 2008 08:44:39 +0200") References: <87iqtem09q.fsf@debo.znain.com> Message-ID: <87wshslmz6.fsf@debo.znain.com> Edi Weitz writes: > On Tue, 02 Sep 2008 22:27:45 -0400, Dimitre Liotev
wrote: > >> When trying this: >> >> (drakma:http-request "http://news.ycombinator.com/rss") >> >> I get and end-of-file error. > > http://weitz.de/drakma/#prob Thanks! From milan at soci.ali.sm Tue Sep 9 15:35:44 2008 From: milan at soci.ali.sm (Milan Cermak) Date: Tue, 9 Sep 2008 17:35:44 +0200 Subject: [drakma-devel] digest authentication In-Reply-To: References: Message-ID: <1d530fcd0809090835o43746d3sc41cb1f2ff8d438c@mail.gmail.com> Hi, is anybody planning to add digest authentication do Drakma in the near future (say, 1 month)? If not, I would do it myself, because I need it. However it would be a poor's man implementation, not the whole RFC. I'd appreciate some tips how to integrate the functionality into the Drakma source code so I can send a clean patch, not just a dirty hack (I already know about http://weitz.de/patches.html). Milan -------------- next part -------------- An HTML attachment was scrubbed... URL: From milan at soci.ali.sm Tue Sep 9 15:25:54 2008 From: milan at soci.ali.sm (Milan Cermak) Date: Tue, 9 Sep 2008 17:25:54 +0200 Subject: [drakma-devel] digest authentication Message-ID: <1d530fcd0809090825h44e3508fx898fba943af2d405@mail.gmail.com> Hi, is anybody planning to add digest authentication do Drakma in the near future (say, 1 month)? If not, I would do it myself, because I need it. However it would be a poor's man implementation, not the whole RFC. I'd appreciate some tips how to integrate the functionality into the Drakma source code so I can send a clean patch, not just a dirty hack (I already know about http://weitz.de/patches.html). Milan -------------- next part -------------- An HTML attachment was scrubbed... URL: From edi at agharta.de Tue Sep 9 17:57:05 2008 From: edi at agharta.de (Edi Weitz) Date: Tue, 09 Sep 2008 19:57:05 +0200 Subject: [drakma-devel] digest authentication In-Reply-To: <1d530fcd0809090835o43746d3sc41cb1f2ff8d438c@mail.gmail.com> (Milan Cermak's message of "Tue, 9 Sep 2008 17:35:44 +0200") References: <1d530fcd0809090835o43746d3sc41cb1f2ff8d438c@mail.gmail.com> Message-ID: On Tue, 9 Sep 2008 17:35:44 +0200, "Milan Cermak" wrote: > However it would be a poor's man implementation, not the whole RFC. Hi! I tend to think that if it goes into the source it should implement the whole RFC or at least a very clearly defined subset which is of interest to more than one person. Thanks, Edi. From milan at soci.ali.sm Wed Sep 10 10:15:24 2008 From: milan at soci.ali.sm (Milan Cermak) Date: Wed, 10 Sep 2008 12:15:24 +0200 Subject: [drakma-devel] digest authentication In-Reply-To: References: <1d530fcd0809090835o43746d3sc41cb1f2ff8d438c@mail.gmail.com> Message-ID: <1d530fcd0809100315i25b79f28o51a572be290f1c51@mail.gmail.com> On Tue, Sep 9, 2008 at 7:57 PM, Edi Weitz wrote: > On Tue, 9 Sep 2008 17:35:44 +0200, "Milan Cermak" > wrote: > > > However it would be a poor's man implementation, not the whole RFC. > > Hi! > > I tend to think that if it goes into the source it should implement > the whole RFC or at least a very clearly defined subset which is > of interest to more than one person. > I can do just WWW-Authenticate. I don't need Proxy-Authenticate. -------------- next part -------------- An HTML attachment was scrubbed... URL: From xach at xach.com Wed Sep 10 13:20:30 2008 From: xach at xach.com (Zach Beane) Date: Wed, 10 Sep 2008 09:20:30 -0400 Subject: [drakma-devel] Sending content without loading it all in memory? Message-ID: <20080910132030.GZ28048@xach.com> Is there any way to send a non-chunked request without having the entire content in memory at once? I'm working on an Amazon S3 client based on Drakma, but sometimes I want to upload (via PUT) very large files and I'd prefer to determine the content-length via FILE-LENGTH and upload the body buffer-by-buffer instead of all at once. Zach From edi at agharta.de Wed Sep 10 15:03:33 2008 From: edi at agharta.de (Edi Weitz) Date: Wed, 10 Sep 2008 17:03:33 +0200 Subject: [drakma-devel] Sending content without loading it all in memory? In-Reply-To: <20080910132030.GZ28048@xach.com> (Zach Beane's message of "Wed, 10 Sep 2008 09:20:30 -0400") References: <20080910132030.GZ28048@xach.com> Message-ID: On Wed, 10 Sep 2008 09:20:30 -0400, Zach Beane wrote: > Is there any way to send a non-chunked request without having the > entire content in memory at once? You can set the content length header yourself. In that case Drakma shouldn't use chunked encoding IIRC. http://weitz.de/drakma/#content-length > I'm working on an Amazon S3 client based on Drakma, but sometimes I > want to upload (via PUT) very large files and I'd prefer to > determine the content-length via FILE-LENGTH and upload the body > buffer-by-buffer instead of all at once. That should be no problem if you provide a function as the :content keyword argument or use the "continuation" facility. Cheers, Edi. From xach at xach.com Wed Sep 10 15:07:49 2008 From: xach at xach.com (Zach Beane) Date: Wed, 10 Sep 2008 11:07:49 -0400 Subject: [drakma-devel] Sending content without loading it all in memory? In-Reply-To: References: <20080910132030.GZ28048@xach.com> Message-ID: <20080910150749.GB28048@xach.com> On Wed, Sep 10, 2008 at 05:03:33PM +0200, Edi Weitz wrote: > On Wed, 10 Sep 2008 09:20:30 -0400, Zach Beane wrote: > > > Is there any way to send a non-chunked request without having the > > entire content in memory at once? > > You can set the content length header yourself. In that case Drakma > shouldn't use chunked encoding IIRC. > > http://weitz.de/drakma/#content-length Unfortunately that's not the case, as the documentation says: | A non-NIL content-length argument means that Drakma must build the | request body in RAM and compute the content length even if it would | have otherwise used chunked encoding - for example in the case of | file uploads. The code says it too: (when (and content-length (not (or (arrayp content) (listp content) (eq content :continuation)))) ;; CONTENT-LENGTH forces us to compute request body ;; in RAM ...) > > I'm working on an Amazon S3 client based on Drakma, but sometimes I > > want to upload (via PUT) very large files and I'd prefer to > > determine the content-length via FILE-LENGTH and upload the body > > buffer-by-buffer instead of all at once. > > That should be no problem if you provide a function as the :content > keyword argument or use the "continuation" facility. I tried that too, but: (when (and (eq content :continuation) content-length) (error "CONTENT-LENGTH must be NIL if CONTENT is :CONTINUATION.")) I think I might be out of luck. Zach From xach at xach.com Wed Sep 10 15:11:15 2008 From: xach at xach.com (Zach Beane) Date: Wed, 10 Sep 2008 11:11:15 -0400 Subject: [drakma-devel] Sending content without loading it all in memory? In-Reply-To: <20080910150749.GB28048@xach.com> References: <20080910132030.GZ28048@xach.com> <20080910150749.GB28048@xach.com> Message-ID: <20080910151115.GC28048@xach.com> On Wed, Sep 10, 2008 at 11:07:49AM -0400, Zach Beane wrote: > On Wed, Sep 10, 2008 at 05:03:33PM +0200, Edi Weitz wrote: > > On Wed, 10 Sep 2008 09:20:30 -0400, Zach Beane wrote: > > > > > Is there any way to send a non-chunked request without having the > > > entire content in memory at once? > > > > You can set the content length header yourself. In that case Drakma > > shouldn't use chunked encoding IIRC. > > > > http://weitz.de/drakma/#content-length > > Unfortunately that's not the case, as the documentation says: [snip] Or rather, you're right, but not chunking is only half the battle. I'd like to avoid loading the whole content into memory too... Zach From lispercat at gmail.com Wed Sep 10 15:20:56 2008 From: lispercat at gmail.com (Andrei Stebakov) Date: Wed, 10 Sep 2008 11:20:56 -0400 Subject: [drakma-devel] Calculating the progress of the file sent through POST request Message-ID: I've always wanted to show the user some progress bar when I have to upload a large file to the remote site using http-request. My first attempts to figure out where to plug the progress indicators didn't bring my any success. I'll give it another try, I just would like to know if it's technically possible? Thank you, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From edi at agharta.de Wed Sep 10 16:02:37 2008 From: edi at agharta.de (Edi Weitz) Date: Wed, 10 Sep 2008 18:02:37 +0200 Subject: [drakma-devel] Calculating the progress of the file sent through POST request In-Reply-To: (Andrei Stebakov's message of "Wed, 10 Sep 2008 11:20:56 -0400") References: Message-ID: On Wed, 10 Sep 2008 11:20:56 -0400, "Andrei Stebakov" wrote: > I've always wanted to show the user some progress bar when I have to > upload a large file to the remote site using http-request. My first > attempts to figure out where to plug the progress indicators didn't > bring my any success. I'll give it another try, I just would like > to know if it's technically possible? It's possible but not actually the task of the web server. If you search the archives of this list, you'll find some previous discussions and solutions. It might also be worthwhile to look at existing solutions in, say, PHP or Java and to translate them. Edi. From edi at agharta.de Wed Sep 10 16:06:09 2008 From: edi at agharta.de (Edi Weitz) Date: Wed, 10 Sep 2008 18:06:09 +0200 Subject: [drakma-devel] Sending content without loading it all in memory? In-Reply-To: <20080910151115.GC28048@xach.com> (Zach Beane's message of "Wed, 10 Sep 2008 11:11:15 -0400") References: <20080910132030.GZ28048@xach.com> <20080910150749.GB28048@xach.com> <20080910151115.GC28048@xach.com> Message-ID: On Wed, 10 Sep 2008 11:11:15 -0400, Zach Beane wrote: > Or rather, you're right, but not chunking is only half the > battle. I'd like to avoid loading the whole content into memory > too... >From a brief look it seems to me that some of the tests in the Drakma code are unnecessary and could be removed so that what you want can actually be achieved. I don't have the time to look at it right now, though. (Actually, one of the many half-finished things I'm working on is a test suite for Drakma and Hunchentoot and I'd rather finish the test suite first before I modify the internals. RSN...) From lispercat at gmail.com Wed Sep 10 20:51:35 2008 From: lispercat at gmail.com (Andrei Stebakov) Date: Wed, 10 Sep 2008 16:51:35 -0400 Subject: [drakma-devel] Calculating the progress of the file sent through POST request In-Reply-To: References: Message-ID: I've found a lot of PHP implementations of file upload from the client to the sever via rfc2388. When you receive the stream on your own server you can calculate the progress. I am not so sure about calculating the progress when you as a client send it via POST http-request to some other server. The actual data writing to the stream using (write-sequence buf stream :end pos) may happen pretty fast, but you'll wait for much longer time till you receive the response with the status code from the server. BTW, do we have some search in the mailing list? The archivepage doesn't have the search field, how do you guys search it? Thank you, Andrew On Wed, Sep 10, 2008 at 12:02 PM, Edi Weitz wrote: > On Wed, 10 Sep 2008 11:20:56 -0400, "Andrei Stebakov" > wrote: > > > I've always wanted to show the user some progress bar when I have to > > upload a large file to the remote site using http-request. My first > > attempts to figure out where to plug the progress indicators didn't > > bring my any success. I'll give it another try, I just would like > > to know if it's technically possible? > > It's possible but not actually the task of the web server. If you > search the archives of this list, you'll find some previous > discussions and solutions. It might also be worthwhile to look at > existing solutions in, say, PHP or Java and to translate them. > > Edi. > _______________________________________________ > drakma-devel mailing list > drakma-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: