From xach at xach.com  Mon Oct 20 15:33:01 2008
From: xach at xach.com (Zach Beane)
Date: Mon, 20 Oct 2008 11:33:01 -0400
Subject: [drakma-devel] Sending content without loading it all in memory?
In-Reply-To: <20080910150749.GB28048@xach.com>
References: <20080910132030.GZ28048@xach.com> <uzlmg6o1m.fsf@agharta.de>
	<20080910150749.GB28048@xach.com>
Message-ID: <20081020153301.GB14051@xach.com>

On Wed, Sep 10, 2008 at 11:07:49AM -0400, Zach Beane wrote:
> On Wed, Sep 10, 2008 at 05:03:33PM +0200, Edi Weitz wrote:
> > On Wed, 10 Sep 2008 09:20:30 -0400, Zach Beane <xach at xach.com> wrote:
> > 
> > > Is there any way to send a non-chunked request without having the
> > > entire content in memory at once?
> > 
> > You can set the content length header yourself.  In that case Drakma
> > shouldn't use chunked encoding IIRC.
> > 
> >   http://weitz.de/drakma/#content-length

Attached is a small patch that updates Drakma so it doesn't reject a
content-length if content is :continuation. It has allowed me to
upload big files without loading them into memory first.

Zach
-------------- next part --------------
diff --git a/doc/index.html b/doc/index.html
index 1474aae..3e3057e 100755
--- a/doc/index.html
+++ b/doc/index.html
@@ -871,7 +871,9 @@ use <a href="http://weitz.de/chunga/">chunked encoding</a> to send the
 content body.  Note that this will not work with some older web
 servers.
 <p>
-<a class=none name="content-length2">A non-<code>NIL</code> <code><i>content-length</i></code> argument</a>
+When <code><i>content</i></code> is
+not <code>:CONTINUATION</code>, <a class=none name="content-length2">a
+non-<code>NIL</code> <code><i>content-length</i></code> argument</a>
 means that Drakma <em>must</em> build the request body in RAM and
 compute the content length even if it would have otherwise used
 chunked encoding - for example in the case of file uploads.  A special
diff --git a/request.lisp b/request.lisp
index f56a113..fac9cb1 100755
--- a/request.lisp
+++ b/request.lisp
@@ -386,8 +386,6 @@ LispWorks 5.0 or higher."
     (error "Don't know how to handle scheme ~S." (uri-scheme uri)))
   (when (and close keep-alive)
     (error "CLOSE and KEEP-ALIVE must not be both true."))
-  (when (and (eq content :continuation) content-length)
-    (error "CONTENT-LENGTH must be NIL if CONTENT is :CONTINUATION."))
   (when (and form-data (not (eq method :post)))
     (error "FORM-DATA makes only sense with POST requests."))
   ;; convert PROXY argument to canonical form

From matt.lamari at gmail.com  Sun Oct 26 05:04:18 2008
From: matt.lamari at gmail.com (Matthew Lamari)
Date: Sun, 26 Oct 2008 00:04:18 -0500
Subject: [drakma-devel] How to handle persistent sessions across
	http-request calls?
Message-ID: <dc28374b0810252204i5aabde8ufc42fbcf33e39412@mail.gmail.com>

I acknowledge that I may be doing this wrong.

I have a very trivial hunchentoot server which merely returns a string
containing the Hunchentoot SessionID.

>From browsers, this works - subsequent requests do not result in a new
session being created.

However, even (mis?) using the cookie-jar, I end up with new sessions being
created.


(let ((jar (make-instance 'cookie-jar)))
  (labels ((doit ()
             (print (multiple-value-list (http-request
http://mysite.com/:cookie-jar jar)))
             (print (cookie-jar-cookies jar))))
    (doit)
    (doit)
    (doit)
    (cookie-jar-cookies jar)))


The cookie seems to be replaced with each call, with a new session's
cookie.  I imagine hunchentoot uses cookies to track sessions - is the old
cookie not being sent back?

Is there something I am doing wrong?  Is there a way to use Drakma to track
a persistent session?

The set-cookie comes back as hunchentoot-session=xx%yyy - where xx is the
session number.

Thanks,
Matt.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/drakma-devel/attachments/20081026/ddbbef40/attachment.html>

From matt.lamari at gmail.com  Sun Oct 26 05:17:38 2008
From: matt.lamari at gmail.com (Matthew Lamari)
Date: Sun, 26 Oct 2008 00:17:38 -0500
Subject: [drakma-devel] How to handle persistent sessions across
	http-request calls?
In-Reply-To: <dc28374b0810252204i5aabde8ufc42fbcf33e39412@mail.gmail.com>
References: <dc28374b0810252204i5aabde8ufc42fbcf33e39412@mail.gmail.com>
Message-ID: <dc28374b0810252217g3fce971bvfe668c05785fd0c1@mail.gmail.com>

On investigating further (and turning on the diagnostics) it seems
that my problem is solved by having the URI refer to a file, e.g.
mysite.com/blah.html, rather than mysite.com or mysite.com/

I'm no expert on cookies, and I imagine this is probably an inherent
property of cookie usage semantics.  Either way, I am no longer facing
any difficulty in this area, sorry to bother you :)



From mail at chaitanyagupta.com  Tue Oct 28 18:41:23 2008
From: mail at chaitanyagupta.com (Chaitanya Gupta)
Date: Wed, 29 Oct 2008 00:11:23 +0530
Subject: [drakma-devel] Returning server protocol as the eighth value of
	HTTP-REQUEST
Message-ID: <49075CD3.7080205@chaitanyagupta.com>

Hi,

I have made a small change to HTTP-REQUEST wherein it returns the server 
protocol (as a keyword) as its eighth value. If its worth incorporating 
this change in the main drakma code, should I send a patch against the 
dev version (bknr svn) or the released version (0.11.5)?

Cheers,
Chaitanya




From pinterface at gmail.com  Tue Oct 28 20:53:15 2008
From: pinterface at gmail.com (Pixel // pinterface)
Date: Tue, 28 Oct 2008 20:53:15 -0000
Subject: [drakma-devel] Sending content without loading it all in memory?
References: <20080910132030.GZ28048@xach.com>
	<uzlmg6o1m.fsf@agharta.de><20080910150749.GB28048@xach.com>
	<20080910151115.GC28048@xach.com> <umyig6l5a.fsf@agharta.de>
Message-ID: <ge7u3a$8ec$1@ger.gmane.org>

"Edi Weitz" <edi at agharta.de> wrote in message
news:umyig6l5a.fsf at agharta.de...
> On Wed, 10 Sep 2008 11:11:15 -0400, Zach Beane <xach at xach.com> wrote:
>
> > Or rather, you're right, but not chunking is only half the
> > battle. I'd like to avoid loading the whole content into memory
> > too...
>
> From a brief look it seems to me that some of the tests in the Drakma
> code are unnecessary and could be removed so that what you want can
> actually be achieved.  I don't have the time to look at it right now,
> though.  [...]

Is the attached patch in the general direction of what you're getting at?
(patch avoids computing request-body in RAM if content-length is provided
and numeric.)

I'm fairly certain the behavior is correct:
* drakma::send-content already handles sending streams and files using a
  buffer (what Zach wanted)
* content-length is always trusted if provided, even when the request-body
  is computed in RAM--which makes the in-memory computation superfluous
  unless (eq content-length t)
The only change in behavior I am aware of is the error "Don't know how to
send content ~S to server." (e.g., for a non-octet stream) moves from being
before drakma sends the 'Content-length' header to after, when
content-length is numeric.

However, there is always the very real possibility I missed something and it
will blow up when somebody else uses it.

-pix


begin 666 content-length.patch
M+2TM(&]L9"UD<F%K;6$O<F5Q=65S="YL:7-P"3(P,#@M,3 M,C@@,#4Z,#8Z
M,#DN,# P,# P,# P("LP,# P"BLK*R!N97<M9')A:VUA+W)E<75E<W0N;&ES
M< DR,# X+3$P+3(X(# U.C V.C$P+C P,# P,# P," K,# P, I 0" M,C at Q
M+#$R("LR.#$L,3(@0$ *('-E;F0@=&AE(&-O;G1E;G0 at 8F]D>2X@($YO=&4@
M=&AA="!T:&ES('=I;&P@;F]T('=O<FL@=VET:"!O;&1E<B!W96(*('-E<G9E
M<G,N"B *+4$@;F]N+4Y)3"!#3TY414Y4+4Q%3D=42"!A<F=U;65N="!M96%N
M<R!T:&%T($1R86MM82 O;75S="\@8G5I;&0@=&AE"BUR97%U97-T(&)O9'D@
M:6X at 4D%-(&%N9"!C;VUP=71E('1H92!C;VYT96YT(&QE;F=T:"!E=F5N(&EF
M(&ET('=O=6QD"BUH879E(&]T:&5R=VES92!U<V5D(&-H=6YK960 at 96YC;V1I
M;F<L(&9O<B!E>&%M<&QE(&EN('1H92!C87-E(&]F(&9I;&4*+75P;&]A9',N
M("!!('-P96-I86P at 8V%S92!I<R!T:&4@=F%L=64 at 5"!F;W(@0T].5$5.5"U,
M14Y'5$@@=VAI8V@@;65A;G,*+71H870 at 1')A:VUA('-H;W5L9"!C;VUP=71E
M('1H92!C;VYT96YT(&QE;F=T:"!A9G1E<B!B=6EL9&EN9R!T:&4*+7)E<75E
M<W0 at 8F]D>2X**T-/3E1%3E0M3$5.1U1(+"!S970@=&\@5"P@;65A;G,@=&AA
M="!$<F%K;6$@+VUU<W0O(&)U:6QD('1H92!R97%U97-T"BMB;V1Y(&EN(%)!
M32!A;F0 at 8V]M<'5T92!T:&4 at 8V]N=&5N="!L96YG=&@@979E;B!I9B!I="!W
M;W5L9"!H879E"BMO=&AE<G=I<V4@=7-E9"!C:'5N:V5D(&5N8V]D:6YG+"!F
M;W(@97AA;7!L92!I;B!T:&4 at 8V%S92!O9B!F:6QE"BMU<&QO861S+B @4V5T
M=&EN9R!#3TY414Y4+4Q%3D=42"!T;R!A(&YU;65R:6,@=F%L=64@=VEL;"!B
M92!A<W-U;65D"BMC;W)R96-T+"!A;&QO=VEN9R!T:&4@<F5Q=65S="!B;V1Y
M('1O(&)E('-T<F5A;65D(&9R;VT at 0T].5$5.5"!W:&5R90HK87!P;&EC86)L
M92X*( H at 0T].5$5.5"U465!%(&ES('1H92!C;W)R97-P;VYD:6YG(&!#;VYT
M96YT+51Y<&4G(&AE861E<B!T;R!B92!S96YT(&%N9 H@=VEL;"!B92!I9VYO
M<F5D('5N;&5S<R!#3TY414Y4(&ES('!R;W9I9&5D(&%S('=E;&PN"D! ("TU
M,C at L-R K-3(X+#@@0$ *(" @(" @(" @(" @(" @(" H=VAE;B!C;VYT96YT
M+71Y<&4*(" @(" @(" @(" @(" @(" @("AW<FET92UH96%D97(@(D-O;G1E
M;G0M5'EP92(@(GY!(B!C;VYT96YT+71Y<&4I*0H@(" @(" @(" @(" @(" @
M("AW:&5N("AA;F0 at 8V]N=&5N="UL96YG=&@*+2 @(" @(" @(" @(" @(" @
M(" @(" @(" @("AN;W0@*&]R("AA<G)A>7 @8V]N=&5N="D**R @(" @(" @
M(" @(" @(" @(" @(" @(" @("AN;W0@*&]R("AN=6UB97)P(&-O;G1E;G0M
M;&5N9W1H*0HK(" @(" @(" @(" @(" @(" @(" @(" @(" @(" @(" @(" @
M*&%R<F%Y<"!C;VYT96YT*0H@(" @(" @(" @(" @(" @(" @(" @(" @(" @
M(" @(" @(" @*&QI<W1P(&-O;G1E;G0I"B @(" @(" @(" @(" @(" @(" @
M(" @(" @(" @(" @(" @(" H97$@8V]N=&5N=" Z8V]N=&EN=6%T:6]N*2DI
M*0H@(" @(" @(" @(" @(" @(" @.SL at 0T].5$5.5"U,14Y'5$@@9F]R8V5S
=('5S('1O(&-O;7!U=&4@<F5Q=65S="!B;V1Y"@H`
`
end