From raison at chatsubo.net  Wed Aug  6 03:25:11 2008
From: raison at chatsubo.net (Kevin Raison)
Date: Tue, 05 Aug 2008 20:25:11 -0700
Subject: [hunchentoot-devel] Is there a better way?
Message-ID: <48991997.8030403@chatsubo.net>

I am trying to determine the best way to accomplish the following task 
and could use some help.  I am writing a web-based (using hunchentoot, 
of course) application for managing voicemail that is stored in Cisco's 
Unity system.  I can access the voicemail box of a given use via IMAP. 
The mel project has helped make that part easy.  However, I am not sure 
the best way to send the audio stream to the user's browser.  The method 
below illustrates (inefficiently) what I need to do.  Can anyone suggest 
a more efficient method that does not use (send-headers)?  The 
hunchentoot docs say, "If your handlers return the full body as a string 
or as an array of octets, you should not call this function."  However, 
I cannot find an alternative way to do what I need to do.  I must fetch 
the voicemail as an email via IMAP and then decode the base64-encoded 
body, which contains the voicemail as a wav file.  Writing out a wav 
file and pointing the user's browser to it would introduce security 
issues, so I must do this entire process in memory.  Any help is 
appreciated.

(defmethod send-audio-stream ((user user) msg-id)
   "Convert the message body and send the wav content to the user."
   (handler-case
       (let ((msg (mel:find-message (mbox user) msg-id)))
         (setf (content-type) "audio/x-wav")
         (setf (header-out "Content-Disposition")
               (format t "inline; filename=~a.wav" msg-id))
         (setf (content-length) ;; FIXME!  Why do this twice?
               (length (base64-decode-msg-body-to-array msg)))
         (let ((stream (send-headers)))
           (base64-decode-msg-body-to-stream msg :stream stream)))
     (error (condition)
       (format nil "Unable to play message: ~a" condition))))

Cheers.
Kevin Raison


From hans at huebner.org  Wed Aug  6 08:50:06 2008
From: hans at huebner.org (=?ISO-8859-1?Q?Hans_H=FCbner?=)
Date: Wed, 6 Aug 2008 03:50:06 -0500
Subject: [hunchentoot-devel] Is there a better way?
In-Reply-To: <48991997.8030403@chatsubo.net>
References: <48991997.8030403@chatsubo.net>
Message-ID: <fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>

Kevin,

why do you feel that you need to set the content length explictly?  By
default, SEND-HEADERS returns a chunked stream, so setting the content
length is not required.  My suggestion is to keep using SEND-HEADERS
and let the base64 decoder write to the stream.  There is nothing
wrong with your approach, except for the content length setting, which
seems wasteful.

-Hans

On Tue, Aug 5, 2008 at 22:25, Kevin Raison <raison at chatsubo.net> wrote:
> I am trying to determine the best way to accomplish the following task and
> could use some help.  I am writing a web-based (using hunchentoot, of
> course) application for managing voicemail that is stored in Cisco's Unity
> system.  I can access the voicemail box of a given use via IMAP. The mel
> project has helped make that part easy.  However, I am not sure the best way
> to send the audio stream to the user's browser.  The method below
> illustrates (inefficiently) what I need to do.  Can anyone suggest a more
> efficient method that does not use (send-headers)?  The hunchentoot docs
> say, "If your handlers return the full body as a string or as an array of
> octets, you should not call this function."  However, I cannot find an
> alternative way to do what I need to do.  I must fetch the voicemail as an
> email via IMAP and then decode the base64-encoded body, which contains the
> voicemail as a wav file.  Writing out a wav file and pointing the user's
> browser to it would introduce security issues, so I must do this entire
> process in memory.  Any help is appreciated.
>
> (defmethod send-audio-stream ((user user) msg-id)
>  "Convert the message body and send the wav content to the user."
>  (handler-case
>      (let ((msg (mel:find-message (mbox user) msg-id)))
>        (setf (content-type) "audio/x-wav")
>        (setf (header-out "Content-Disposition")
>              (format t "inline; filename=~a.wav" msg-id))
>        (setf (content-length) ;; FIXME!  Why do this twice?
>              (length (base64-decode-msg-body-to-array msg)))
>        (let ((stream (send-headers)))
>          (base64-decode-msg-body-to-stream msg :stream stream)))
>    (error (condition)
>      (format nil "Unable to play message: ~a" condition))))
>
> Cheers.
> Kevin Raison
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>


From raison at chatsubo.net  Wed Aug  6 16:16:29 2008
From: raison at chatsubo.net (Kevin Raison)
Date: Wed, 06 Aug 2008 09:16:29 -0700
Subject: [hunchentoot-devel] Is there a better way?
In-Reply-To: <fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>
References: <48991997.8030403@chatsubo.net>
	<fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>
Message-ID: <4899CE5D.8010208@chatsubo.net>

Thanks for the response, Hans.  To explain, I added the explicit content 
length header after seeing some strange and inconsistent behavior with 
browsers across several platforms.  On the Mac using Firefox, the entire 
WAV stream would not reach the browser, so users would only hear part of 
their audio file.  With Safari, the stream would not play at all.  On 
Windows, IE behaved like Firefox on the Mac, but Firefox on Windows 
would not play the WAV stream at all.  On Linux using Firefox, the 
browser would generally crash mid-way through the stream downloading. 
Opera on Linux worked fine.  Explicitly adding the content length header 
solved all of these issues.  Any ideas?

Thanks,
Kevin


Hans H?bner wrote:
> Kevin,
> 
> why do you feel that you need to set the content length explictly?  By
> default, SEND-HEADERS returns a chunked stream, so setting the content
> length is not required.  My suggestion is to keep using SEND-HEADERS
> and let the base64 decoder write to the stream.  There is nothing
> wrong with your approach, except for the content length setting, which
> seems wasteful.
> 
> -Hans
> 
> On Tue, Aug 5, 2008 at 22:25, Kevin Raison <raison at chatsubo.net> wrote:
>> I am trying to determine the best way to accomplish the following task and
>> could use some help.  I am writing a web-based (using hunchentoot, of
>> course) application for managing voicemail that is stored in Cisco's Unity
>> system.  I can access the voicemail box of a given use via IMAP. The mel
>> project has helped make that part easy.  However, I am not sure the best way
>> to send the audio stream to the user's browser.  The method below
>> illustrates (inefficiently) what I need to do.  Can anyone suggest a more
>> efficient method that does not use (send-headers)?  The hunchentoot docs
>> say, "If your handlers return the full body as a string or as an array of
>> octets, you should not call this function."  However, I cannot find an
>> alternative way to do what I need to do.  I must fetch the voicemail as an
>> email via IMAP and then decode the base64-encoded body, which contains the
>> voicemail as a wav file.  Writing out a wav file and pointing the user's
>> browser to it would introduce security issues, so I must do this entire
>> process in memory.  Any help is appreciated.
>>
>> (defmethod send-audio-stream ((user user) msg-id)
>>  "Convert the message body and send the wav content to the user."
>>  (handler-case
>>      (let ((msg (mel:find-message (mbox user) msg-id)))
>>        (setf (content-type) "audio/x-wav")
>>        (setf (header-out "Content-Disposition")
>>              (format t "inline; filename=~a.wav" msg-id))
>>        (setf (content-length) ;; FIXME!  Why do this twice?
>>              (length (base64-decode-msg-body-to-array msg)))
>>        (let ((stream (send-headers)))
>>          (base64-decode-msg-body-to-stream msg :stream stream)))
>>    (error (condition)
>>      (format nil "Unable to play message: ~a" condition))))
>>
>> Cheers.
>> Kevin Raison
>> _______________________________________________
>> tbnl-devel site list
>> tbnl-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
> 


From raison at chatsubo.net  Wed Aug  6 16:27:53 2008
From: raison at chatsubo.net (Kevin Raison)
Date: Wed, 06 Aug 2008 09:27:53 -0700
Subject: [hunchentoot-devel] Is there a better way?
In-Reply-To: <4899CE5D.8010208@chatsubo.net>
References: <48991997.8030403@chatsubo.net>	<fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>
	<4899CE5D.8010208@chatsubo.net>
Message-ID: <4899D109.7090207@chatsubo.net>

Actually, let me add a correction to my last post  (too many browsers, 
hard to remember which one did what):

On Windows, without setting the content length header explicitly, IE 
would play the WAV stream properly using iTunes as its helper 
application, while Firefox would only play part of the stream using 
Quicktime as its helper.

Cheers.
Kevin


Kevin Raison wrote:
> Thanks for the response, Hans.  To explain, I added the explicit content 
> length header after seeing some strange and inconsistent behavior with 
> browsers across several platforms.  On the Mac using Firefox, the entire 
> WAV stream would not reach the browser, so users would only hear part of 
> their audio file.  With Safari, the stream would not play at all.  On 
> Windows, IE behaved like Firefox on the Mac, but Firefox on Windows 
> would not play the WAV stream at all.  On Linux using Firefox, the 
> browser would generally crash mid-way through the stream downloading. 
> Opera on Linux worked fine.  Explicitly adding the content length header 
> solved all of these issues.  Any ideas?
> 
> Thanks,
> Kevin
> 
> 
> Hans H?bner wrote:
>> Kevin,
>>
>> why do you feel that you need to set the content length explictly?  By
>> default, SEND-HEADERS returns a chunked stream, so setting the content
>> length is not required.  My suggestion is to keep using SEND-HEADERS
>> and let the base64 decoder write to the stream.  There is nothing
>> wrong with your approach, except for the content length setting, which
>> seems wasteful.
>>
>> -Hans
>>
>> On Tue, Aug 5, 2008 at 22:25, Kevin Raison <raison at chatsubo.net> wrote:
>>> I am trying to determine the best way to accomplish the following 
>>> task and
>>> could use some help.  I am writing a web-based (using hunchentoot, of
>>> course) application for managing voicemail that is stored in Cisco's 
>>> Unity
>>> system.  I can access the voicemail box of a given use via IMAP. The mel
>>> project has helped make that part easy.  However, I am not sure the 
>>> best way
>>> to send the audio stream to the user's browser.  The method below
>>> illustrates (inefficiently) what I need to do.  Can anyone suggest a 
>>> more
>>> efficient method that does not use (send-headers)?  The hunchentoot docs
>>> say, "If your handlers return the full body as a string or as an 
>>> array of
>>> octets, you should not call this function."  However, I cannot find an
>>> alternative way to do what I need to do.  I must fetch the voicemail 
>>> as an
>>> email via IMAP and then decode the base64-encoded body, which 
>>> contains the
>>> voicemail as a wav file.  Writing out a wav file and pointing the user's
>>> browser to it would introduce security issues, so I must do this entire
>>> process in memory.  Any help is appreciated.
>>>
>>> (defmethod send-audio-stream ((user user) msg-id)
>>>  "Convert the message body and send the wav content to the user."
>>>  (handler-case
>>>      (let ((msg (mel:find-message (mbox user) msg-id)))
>>>        (setf (content-type) "audio/x-wav")
>>>        (setf (header-out "Content-Disposition")
>>>              (format t "inline; filename=~a.wav" msg-id))
>>>        (setf (content-length) ;; FIXME!  Why do this twice?
>>>              (length (base64-decode-msg-body-to-array msg)))
>>>        (let ((stream (send-headers)))
>>>          (base64-decode-msg-body-to-stream msg :stream stream)))
>>>    (error (condition)
>>>      (format nil "Unable to play message: ~a" condition))))
>>>
>>> Cheers.
>>> Kevin Raison
>>> _______________________________________________
>>> tbnl-devel site list
>>> tbnl-devel at common-lisp.net
>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>
>> _______________________________________________
>> tbnl-devel site list
>> tbnl-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
> 


From hans at huebner.org  Wed Aug  6 16:56:59 2008
From: hans at huebner.org (=?ISO-8859-1?Q?Hans_H=FCbner?=)
Date: Wed, 6 Aug 2008 11:56:59 -0500
Subject: [hunchentoot-devel] Is there a better way?
In-Reply-To: <4899D109.7090207@chatsubo.net>
References: <48991997.8030403@chatsubo.net>
	<fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>
	<4899CE5D.8010208@chatsubo.net> <4899D109.7090207@chatsubo.net>
Message-ID: <fd2982320808060956v69a96de5q695b3c5ae9537f95@mail.gmail.com>

A cheap workaround would be to use HTTP/1.0 (no persistent connection,
non Content-Length header required).  I lack the time to try this out
myself, but what you write seems like being indicative of a problem in
the chunked encoding.

If you can afford the main memory footage of the WAV data encoded as
octets, you could also just return the vector from your handler and
let Hunchentoot deal with setting the headers properly.

-Hans

On Wed, Aug 6, 2008 at 11:27, Kevin Raison <raison at chatsubo.net> wrote:
> Actually, let me add a correction to my last post  (too many browsers, hard
> to remember which one did what):
>
> On Windows, without setting the content length header explicitly, IE would
> play the WAV stream properly using iTunes as its helper application, while
> Firefox would only play part of the stream using Quicktime as its helper.
>
> Cheers.
> Kevin
>
>
> Kevin Raison wrote:
>>
>> Thanks for the response, Hans.  To explain, I added the explicit content
>> length header after seeing some strange and inconsistent behavior with
>> browsers across several platforms.  On the Mac using Firefox, the entire WAV
>> stream would not reach the browser, so users would only hear part of their
>> audio file.  With Safari, the stream would not play at all.  On Windows, IE
>> behaved like Firefox on the Mac, but Firefox on Windows would not play the
>> WAV stream at all.  On Linux using Firefox, the browser would generally
>> crash mid-way through the stream downloading. Opera on Linux worked fine.
>>  Explicitly adding the content length header solved all of these issues.
>>  Any ideas?
>>
>> Thanks,
>> Kevin
>>
>>
>> Hans H?bner wrote:
>>>
>>> Kevin,
>>>
>>> why do you feel that you need to set the content length explictly?  By
>>> default, SEND-HEADERS returns a chunked stream, so setting the content
>>> length is not required.  My suggestion is to keep using SEND-HEADERS
>>> and let the base64 decoder write to the stream.  There is nothing
>>> wrong with your approach, except for the content length setting, which
>>> seems wasteful.
>>>
>>> -Hans
>>>
>>> On Tue, Aug 5, 2008 at 22:25, Kevin Raison <raison at chatsubo.net> wrote:
>>>>
>>>> I am trying to determine the best way to accomplish the following task
>>>> and
>>>> could use some help.  I am writing a web-based (using hunchentoot, of
>>>> course) application for managing voicemail that is stored in Cisco's
>>>> Unity
>>>> system.  I can access the voicemail box of a given use via IMAP. The mel
>>>> project has helped make that part easy.  However, I am not sure the best
>>>> way
>>>> to send the audio stream to the user's browser.  The method below
>>>> illustrates (inefficiently) what I need to do.  Can anyone suggest a
>>>> more
>>>> efficient method that does not use (send-headers)?  The hunchentoot docs
>>>> say, "If your handlers return the full body as a string or as an array
>>>> of
>>>> octets, you should not call this function."  However, I cannot find an
>>>> alternative way to do what I need to do.  I must fetch the voicemail as
>>>> an
>>>> email via IMAP and then decode the base64-encoded body, which contains
>>>> the
>>>> voicemail as a wav file.  Writing out a wav file and pointing the user's
>>>> browser to it would introduce security issues, so I must do this entire
>>>> process in memory.  Any help is appreciated.
>>>>
>>>> (defmethod send-audio-stream ((user user) msg-id)
>>>>  "Convert the message body and send the wav content to the user."
>>>>  (handler-case
>>>>     (let ((msg (mel:find-message (mbox user) msg-id)))
>>>>       (setf (content-type) "audio/x-wav")
>>>>       (setf (header-out "Content-Disposition")
>>>>             (format t "inline; filename=~a.wav" msg-id))
>>>>       (setf (content-length) ;; FIXME!  Why do this twice?
>>>>             (length (base64-decode-msg-body-to-array msg)))
>>>>       (let ((stream (send-headers)))
>>>>         (base64-decode-msg-body-to-stream msg :stream stream)))
>>>>   (error (condition)
>>>>     (format nil "Unable to play message: ~a" condition))))
>>>>
>>>> Cheers.
>>>> Kevin Raison
>>>> _______________________________________________
>>>> tbnl-devel site list
>>>> tbnl-devel at common-lisp.net
>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>>
>>> _______________________________________________
>>> tbnl-devel site list
>>> tbnl-devel at common-lisp.net
>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>
>> _______________________________________________
>> tbnl-devel site list
>> tbnl-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>


From edi at agharta.de  Wed Aug  6 17:13:44 2008
From: edi at agharta.de (Edi Weitz)
Date: Wed, 06 Aug 2008 19:13:44 +0200
Subject: [hunchentoot-devel] Is there a better way?
In-Reply-To: <4899CE5D.8010208@chatsubo.net> (Kevin Raison's message of
	"Wed, 06 Aug 2008 09:16:29 -0700")
References: <48991997.8030403@chatsubo.net>
	<fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>
	<4899CE5D.8010208@chatsubo.net>
Message-ID: <usktikqwn.fsf@agharta.de>

On Wed, 06 Aug 2008 09:16:29 -0700, Kevin Raison <raison at chatsubo.net> wrote:

> To explain, I added the explicit content length header after seeing
> some strange and inconsistent behavior with browsers across several
> platforms.  On the Mac using Firefox, the entire WAV stream would
> not reach the browser, so users would only hear part of their audio
> file.  With Safari, the stream would not play at all.  On Windows,
> IE behaved like Firefox on the Mac, but Firefox on Windows would not
> play the WAV stream at all.  On Linux using Firefox, the browser
> would generally crash mid-way through the stream downloading. Opera
> on Linux worked fine.  Explicitly adding the content length header
> solved all of these issues.  Any ideas?

Which version of Hunchentoot are you using, dev or release?  Have you
checked that it actually does the right thing when sending chunked
content?  I would hope that it does, but with the dev version I'm less
sure and even with the release version there's of course always a
chance that there are bugs.  Also, could it be that you get requests
for partial content somewhere that aren't handled as expected?

Edi (on vacation).


From raison at chatsubo.net  Wed Aug  6 18:15:08 2008
From: raison at chatsubo.net (Kevin Raison)
Date: Wed, 06 Aug 2008 11:15:08 -0700
Subject: [hunchentoot-devel] Is there a better way?
In-Reply-To: <usktikqwn.fsf@agharta.de>
References: <48991997.8030403@chatsubo.net>	<fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>	<4899CE5D.8010208@chatsubo.net>
	<usktikqwn.fsf@agharta.de>
Message-ID: <4899EA2C.6070603@chatsubo.net>

Thanks for the response, Edi.  I am using 0.15.7.  I tried Hans' 
recommendation of converting the WAV data into octets and then having 
the handler return those octets. This seems to have fixed the issue.

Further analysis of the Safari-related problems revealed something else: 
  it seems that when Safari calls its helper application (in this case, 
Quicktime), it hands the helper a URL and all relevant cookies. 
Quicktime then does a GET which does not succeed because it is sending 
the wrong User Agent string for its Hunchentoot session cookie.  <sigh> 
I guess I'll switch to IP-based sessions.

Thanks to both of you for your help.

Cheers.
Kevin

Edi Weitz wrote:
> On Wed, 06 Aug 2008 09:16:29 -0700, Kevin Raison <raison at chatsubo.net> wrote:
> 
>> To explain, I added the explicit content length header after seeing
>> some strange and inconsistent behavior with browsers across several
>> platforms.  On the Mac using Firefox, the entire WAV stream would
>> not reach the browser, so users would only hear part of their audio
>> file.  With Safari, the stream would not play at all.  On Windows,
>> IE behaved like Firefox on the Mac, but Firefox on Windows would not
>> play the WAV stream at all.  On Linux using Firefox, the browser
>> would generally crash mid-way through the stream downloading. Opera
>> on Linux worked fine.  Explicitly adding the content length header
>> solved all of these issues.  Any ideas?
> 
> Which version of Hunchentoot are you using, dev or release?  Have you
> checked that it actually does the right thing when sending chunked
> content?  I would hope that it does, but with the dev version I'm less
> sure and even with the release version there's of course always a
> chance that there are bugs.  Also, could it be that you get requests
> for partial content somewhere that aren't handled as expected?
> 
> Edi (on vacation).
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
> 


From edi at agharta.de  Thu Aug  7 21:15:48 2008
From: edi at agharta.de (Edi Weitz)
Date: Thu, 07 Aug 2008 23:15:48 +0200
Subject: [hunchentoot-devel] Is there a better way?
In-Reply-To: <4899EA2C.6070603@chatsubo.net> (Kevin Raison's message of
	"Wed, 06 Aug 2008 11:15:08 -0700")
References: <48991997.8030403@chatsubo.net>
	<fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>
	<4899CE5D.8010208@chatsubo.net> <usktikqwn.fsf@agharta.de>
	<4899EA2C.6070603@chatsubo.net>
Message-ID: <uzlnojzln.fsf@agharta.de>

On Wed, 06 Aug 2008 11:15:08 -0700, Kevin Raison <raison at chatsubo.net> wrote:

> Further analysis of the Safari-related problems revealed something
> else: it seems that when Safari calls its helper application (in
> this case, Quicktime), it hands the helper a URL and all relevant
> cookies. Quicktime then does a GET which does not succeed because it
> is sending the wrong User Agent string for its Hunchentoot session
> cookie.  <sigh> I guess I'll switch to IP-based sessions.

Set this one to NIL:

  http://weitz.de/hunchentoot/#*use-user-agent-for-sessions*

Maybe you can try this and report back to the list if this solves your
problems?  I'd be happy to hear that you didn't find bugs related to
chunked encoding... :)

Edi (still on vacation).


From raison at chatsubo.net  Fri Aug  8 17:21:31 2008
From: raison at chatsubo.net (Kevin Raison)
Date: Fri, 08 Aug 2008 10:21:31 -0700
Subject: [hunchentoot-devel] Is there a better way?
In-Reply-To: <uzlnojzln.fsf@agharta.de>
References: <48991997.8030403@chatsubo.net>	<fd2982320808060150t3524ebbv3af049a6a7639550@mail.gmail.com>	<4899CE5D.8010208@chatsubo.net>
	<usktikqwn.fsf@agharta.de>	<4899EA2C.6070603@chatsubo.net>
	<uzlnojzln.fsf@agharta.de>
Message-ID: <489C809B.1070401@chatsubo.net>

Edi, this solved the issues with Safari and a few other odd browser 
setups.  You can rest assured that I did not discover a chunked encoding 
bug!

Thanks again for the help.
Kevin


Edi Weitz wrote:
> On Wed, 06 Aug 2008 11:15:08 -0700, Kevin Raison <raison at chatsubo.net> wrote:
> 
>> Further analysis of the Safari-related problems revealed something
>> else: it seems that when Safari calls its helper application (in
>> this case, Quicktime), it hands the helper a URL and all relevant
>> cookies. Quicktime then does a GET which does not succeed because it
>> is sending the wrong User Agent string for its Hunchentoot session
>> cookie.  <sigh> I guess I'll switch to IP-based sessions.
> 
> Set this one to NIL:
> 
>   http://weitz.de/hunchentoot/#*use-user-agent-for-sessions*
> 
> Maybe you can try this and report back to the list if this solves your
> problems?  I'd be happy to hear that you didn't find bugs related to
> chunked encoding... :)
> 
> Edi (still on vacation).
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
> 


From rflug05 at gmail.com  Mon Aug 11 12:56:15 2008
From: rflug05 at gmail.com (Nick Allen)
Date: Mon, 11 Aug 2008 14:56:15 +0200
Subject: [hunchentoot-devel] re: thread pool benchmark results
Message-ID: <e3478c000808110556u10b0f80co4fcbf748516a1b42@mail.gmail.com>

on Friday, July 18, 2008, 7:34:27 AM Hans wrote:

> On Fri, Jul 18, 2008 at 03:00, Anton Vodonosov <avodonosov at yandex.ru> wrote:

>> I have set up a little benchmark to test ithow
>> thread pool may improve the hunchentoot performance.
>>
>> In two words it looks useful. Details are below.
> [...]
>> Patch for port-sbcl.lisp with the simple thread pool
>> implementation is also attached. It is against 0.15.7,
>> sorry it's not for the current development version of
>> hunchentoot, but I created this thead pool code about
>> half a year ago.

> I think we would accept performance patches to the development
> version.  The revised connection management should make it easier to
> integrate thread pooling in a portable fashion.

I used pcall

   http://marijn.haverbeke.nl/pcall/

to make a portable thread-pool-powered connection manager (note: I
don't have lispworks and I have no idea if pcall and lispworks work
together since it uses bordeaux threads)

   darcs get http://common-lisp.net/project/bpm/darcs/pcall-connection-manager

I think once the cl.net darcsweb script it might be browsable on the web at

   http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=bpm-pcall-connection-manager;a=summary

but I'm not 100% positive and I don't know how long it will take for
it to be visible...

note: in order to use it you have to add &ALLOW-OTHER-KEYS to the end
of the parameters to HUNCHENTOOT:START-SERVER so it will accept the
:CONNECTION-MANAGER-CLASS keyword arg

anyways I tried to benchmark it against the
one-thread-per-connection-manager on my laptop using siege. I used
both the default dispatcher (the default hunchentoot landing page) and
a dispatcher that returned a handler that slept for 3 seconds and
returned a page

   (defun wait-for-3-seconds (hunchentoot:*request*)
     (lambda () (sleep 3) "this is the page"))

with 3-5 threads in the pool the 1-thread-per-connection manager
performed a lot better with both dispatchers. with 20-50 threads both
connections managers had about the same performance with the
1-thread-per-connection manager being slightly faster.

I'm sure there are better ways to benchmark it, but I'm not in a
position to be playing with our server at the moment. I haven't taken
a look at Anton's tests yet.

Nick


From avodonosov at yandex.ru  Fri Aug 15 23:46:40 2008
From: avodonosov at yandex.ru (Anton Vodonosov)
Date: Sat, 16 Aug 2008 02:46:40 +0300
Subject: [hunchentoot-devel] re: thread pool benchmark results
In-Reply-To: <e3478c000808110556u10b0f80co4fcbf748516a1b42@mail.gmail.com>
References: <e3478c000808110556u10b0f80co4fcbf748516a1b42@mail.gmail.com>
Message-ID: <1371374298.20080816024640@yandex.ru>

on Monday, August 11, 2008, 3:56:15 PM Nick wrote:

> on Friday, July 18, 2008, 7:34:27 AM Hans wrote:

>> On Fri, Jul 18, 2008 at 03:00, Anton Vodonosov <avodonosov at yandex.ru> wrote:

>>> I have set up a little benchmark to test ithow
>>> thread pool may improve the hunchentoot performance.
>>>
>>> In two words it looks useful. Details are below.
[...]
>> I think we would accept performance patches to the development
>> version.  The revised connection management should make it easier to
>> integrate thread pooling in a portable fashion.

> I used pcall

>    http://marijn.haverbeke.nl/pcall/

> to make a portable thread-pool-powered connection manager
[...]
> anyways I tried to benchmark it against the
> one-thread-per-connection-manager on my laptop using siege. I used
> both the default dispatcher (the default hunchentoot landing page) and
> a dispatcher that returned a handler that slept for 3 seconds and
> returned a page

>    (defun wait-for-3-seconds (hunchentoot:*request*)
>      (lambda () (sleep 3) "this is the page"))

> with 3-5 threads in the pool the 1-thread-per-connection manager
> performed a lot better with both dispatchers. with 20-50 threads both
> connections managers had about the same performance with the
> 1-thread-per-connection manager being slightly faster.

> I'm sure there are better ways to benchmark it, but I'm not in a
> position to be playing with our server at the moment. I haven't taken
> a look at Anton's tests yet.

> Nick

It is easy to understand why the 3-seconds-sleeping handler
made the thread pool inefficient - while threads are sleeping during
handling of first few requests, other requests are suspended.

For the cases when so long-running handlers are present, maximum
thread count may be unbound - thread pool have number of threads
preallocated, but when new request arrives and there is no free
thread, new thread is created/added to pool temporary.

What do you mean when you say that you tried the default dispatcher
and the sleeping dispatcher? Did you tested them simultaneously,
so that sleeping handler blocked the threads?

Or you tested the default dispatcher separately, and thread pool
performance was worse? If yes - it is very strange. In this case
I'd be interested to know what is your platform, lisp implementation.
What is the siege parameters?

Best regards,
-Anton



From rflug05 at gmail.com  Sat Aug 16 00:40:40 2008
From: rflug05 at gmail.com (Nick Allen)
Date: Sat, 16 Aug 2008 02:40:40 +0200
Subject: [hunchentoot-devel] re: thread pool benchmark results
In-Reply-To: <1371374298.20080816024640@yandex.ru>
References: <e3478c000808110556u10b0f80co4fcbf748516a1b42@mail.gmail.com>
	<1371374298.20080816024640@yandex.ru>
Message-ID: <e3478c000808151740w271fab7fx4a751fbe3c3c32bf@mail.gmail.com>

> It is easy to understand why the 3-seconds-sleeping handler
> made the thread pool inefficient - while threads are sleeping during
> handling of first few requests, other requests are suspended.
>
> For the cases when so long-running handlers are present, maximum
> thread count may be unbound - thread pool have number of threads
> preallocated, but when new request arrives and there is no free
> thread, new thread is created/added to pool temporary.
>
> What do you mean when you say that you tried the default dispatcher
> and the sleeping dispatcher? Did you tested them simultaneously,
> so that sleeping handler blocked the threads?
>
> Or you tested the default dispatcher separately, and thread pool
> performance was worse? If yes - it is very strange. In this case
> I'd be interested to know what is your platform, lisp implementation.
> What is the siege parameters?

um... I mean that I sieged hunchentoot w/ a dispatcher that just
returned the default page twice, once with the thread-pool connection
manager then once with the one-thread-per-connection manager

then I sieged it twice more (once with the thread pool, once with the
one-thread-per..) with the dispatcher that returned a handler that
slept then returned the page

SBCL, compiled for multi-threads on Mac OS X 10.4.11 Intel

siege parameters were nothing special

I haven't had time to come back and look at this... am I wrong in
assuming that calling SLEEP is somehow fundamentally different than a
thread wasting time being blocked on IO?

Nick


From edi at agharta.de  Mon Aug 18 21:22:01 2008
From: edi at agharta.de (Edi Weitz)
Date: Mon, 18 Aug 2008 23:22:01 +0200
Subject: [hunchentoot-devel] Re: The old hunchentoot shared thread on
	openmcl bug
In-Reply-To: <AAAC7AA9-1229-490F-93FF-15CEC40A148F@mac.com> (mikel evins's
	message of "Sun, 17 Aug 2008 06:08:54 -0500")
References: <AAAC7AA9-1229-490F-93FF-15CEC40A148F@mac.com>
Message-ID: <uod3q81dy.fsf@agharta.de>

Hi Mikel,

Please send Hunchentoot question to the appropriate mailing list, see
Cc.

On Sun, 17 Aug 2008 06:08:54 -0500, mikel evins <mevins at mac.com> wrote:

> Sorry to bother you with an old problem, but I'm stumbling over a
> problem that is maybe an old one you're familiar with. I have a web
> application built on Hunchentoot and cl-prevalence. I got fairly far
> along and then ran into the problem where the lisp (Clozure CL)
> process dies trying to write to a stream:
>
> Error: Stream #<BASIC-FILE-CHARACTER-OUTPUT-STREAM ("/Users/mikel/
> Valise/guild/trunk/src/ccl-src/ymra.world.db/transaction-log.xml"/6
> ISO-8859-1) #x30004294610D> is private to #<PROCESS hunchentoot-
> worker-7(10) [Exhausted] #x300042944F8D>
>
> Presumably the cl-prevalence code that tries to write to the db is
> using a stream that was created private in a different thread. Do
> you remember where I should look for the code you used to fix this
> for other cases? I'll track down the culprit and fix it; I was just
> hoping to speed the process along by finding code that already did
> the right thing somewhere.
>
> Thanks for any help.
>
> In case it's of interest, I'm using Hunchentoot 0.15.6 on Clozure CL
> 1.2-r10477M-trunk with cl-prevalence 1.6.

This looks like a CCL-specific problem not really related to
Hunchentoot.  Look at the :SHARING :LOCK keyword arguments to OPEN in
log.lisp, maybe this'll help for your streams.

Cheers,
Edi.


From yarek.kowalik at gmail.com  Mon Aug 25 05:14:21 2008
From: yarek.kowalik at gmail.com (Yarek Kowalik)
Date: Sun, 24 Aug 2008 22:14:21 -0700
Subject: [hunchentoot-devel] Reading and writing additional cookies.
Message-ID: <ecb948b30808242214k7cfe4ddch779d90fdbf90769b@mail.gmail.com>

I'm in the process of writing an application based on Weblocks.  Weblocks
uses Hunchentoot - I have been using version 0.11.1.

I would like to set some application specific cookies.   I would I go about
doing that?  My initial guess would be to setup a handler in the dispatch
table and access (via special) the *reply* object to set the cookie using
setf on cookies-out, and similarly access the  *reqest* object to read the
cookie value.

Am I on the right track?  Suggestions please.

Thanks,

Yarek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/tbnl-devel/attachments/20080824/75f01620/attachment.html>

From larsnostdal at gmail.com  Mon Aug 25 06:23:53 2008
From: larsnostdal at gmail.com (Lars Rune =?ISO-8859-1?Q?N=F8stdal?=)
Date: Mon, 25 Aug 2008 08:23:53 +0200
Subject: [hunchentoot-devel] Reading and writing additional cookies.
In-Reply-To: <ecb948b30808242214k7cfe4ddch779d90fdbf90769b@mail.gmail.com>
References: <ecb948b30808242214k7cfe4ddch779d90fdbf90769b@mail.gmail.com>
Message-ID: <1219645433.5053.9.camel@blackbox>

On Sun, 2008-08-24 at 22:14 -0700, Yarek Kowalik wrote:
> I'm in the process of writing an application based on Weblocks.
> Weblocks uses Hunchentoot - I have been using version 0.11.1.   
> 
> I would like to set some application specific cookies.   I would I go
> about doing that?  My initial guess would be to setup a handler in the
> dispatch table and access (via special) the *reply* object to set the
> cookie using setf on cookies-out, and similarly access the  *reqest*
> object to read the cookie value.
> 
> Am I on the right track?  Suggestions please.
> 
> Thanks,
> 
> Yarek
> 

Hi,
SET-COOKIE and (COOKIE-IN <name>) works for me when setting "custom"
cookies.

-- 
Lars Rune N?stdal   || AJAX/Comet GUI type stuff for Common Lisp
http://nostdal.org/ || http://groups.google.com/group/symbolicweb




From yarek.kowalik at gmail.com  Tue Aug 26 00:34:39 2008
From: yarek.kowalik at gmail.com (Yarek Kowalik)
Date: Mon, 25 Aug 2008 17:34:39 -0700
Subject: [hunchentoot-devel] Reading and writing additional cookies.
In-Reply-To: <1219645433.5053.9.camel@blackbox>
References: <ecb948b30808242214k7cfe4ddch779d90fdbf90769b@mail.gmail.com>
	<1219645433.5053.9.camel@blackbox>
Message-ID: <ecb948b30808251734p6763450esd721653458df153e@mail.gmail.com>

Can you confirm if I should be doing this while the handler is processing a
request?

Much appreciated,

Yarek

On Sun, Aug 24, 2008 at 11:23 PM, Lars Rune N?stdal
<larsnostdal at gmail.com>wrote:

> On Sun, 2008-08-24 at 22:14 -0700, Yarek Kowalik wrote:
> > I'm in the process of writing an application based on Weblocks.
> > Weblocks uses Hunchentoot - I have been using version 0.11.1.
> >
> > I would like to set some application specific cookies.   I would I go
> > about doing that?  My initial guess would be to setup a handler in the
> > dispatch table and access (via special) the *reply* object to set the
> > cookie using setf on cookies-out, and similarly access the  *reqest*
> > object to read the cookie value.
> >
> > Am I on the right track?  Suggestions please.
> >
> > Thanks,
> >
> > Yarek
> >
>
> Hi,
> SET-COOKIE and (COOKIE-IN <name>) works for me when setting "custom"
> cookies.
>
> --
> Lars Rune N?stdal   || AJAX/Comet GUI type stuff for Common Lisp
> http://nostdal.org/ || http://groups.google.com/group/symbolicweb
>
>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/tbnl-devel/attachments/20080825/a74e8ae2/attachment.html>

From yarek.kowalik at gmail.com  Wed Aug 27 05:38:18 2008
From: yarek.kowalik at gmail.com (Yarek Kowalik)
Date: Tue, 26 Aug 2008 22:38:18 -0700
Subject: [hunchentoot-devel] Reading and writing additional cookies.
In-Reply-To: <ecb948b30808251734p6763450esd721653458df153e@mail.gmail.com>
References: <ecb948b30808242214k7cfe4ddch779d90fdbf90769b@mail.gmail.com>
	<1219645433.5053.9.camel@blackbox>
	<ecb948b30808251734p6763450esd721653458df153e@mail.gmail.com>
Message-ID: <ecb948b30808262238w5f1c9362g855c85be391bf91a@mail.gmail.com>

To answer myself: yes.

Yarek

On Mon, Aug 25, 2008 at 5:34 PM, Yarek Kowalik <yarek.kowalik at gmail.com>wrote:

> Can you confirm if I should be doing this while the handler is processing a
> request?
>
> Much appreciated,
>
> Yarek
>
>
> On Sun, Aug 24, 2008 at 11:23 PM, Lars Rune N?stdal <larsnostdal at gmail.com
> > wrote:
>
>> On Sun, 2008-08-24 at 22:14 -0700, Yarek Kowalik wrote:
>> > I'm in the process of writing an application based on Weblocks.
>> > Weblocks uses Hunchentoot - I have been using version 0.11.1.
>> >
>> > I would like to set some application specific cookies.   I would I go
>> > about doing that?  My initial guess would be to setup a handler in the
>> > dispatch table and access (via special) the *reply* object to set the
>> > cookie using setf on cookies-out, and similarly access the  *reqest*
>> > object to read the cookie value.
>> >
>> > Am I on the right track?  Suggestions please.
>> >
>> > Thanks,
>> >
>> > Yarek
>> >
>>
>> Hi,
>> SET-COOKIE and (COOKIE-IN <name>) works for me when setting "custom"
>> cookies.
>>
>> --
>> Lars Rune N?stdal   || AJAX/Comet GUI type stuff for Common Lisp
>> http://nostdal.org/ || http://groups.google.com/group/symbolicweb
>>
>>
>> _______________________________________________
>> tbnl-devel site list
>> tbnl-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/tbnl-devel/attachments/20080826/2c66f2aa/attachment.html>

From mail at chaitanyagupta.com  Sun Aug 31 05:10:01 2008
From: mail at chaitanyagupta.com (Chaitanya Gupta)
Date: Sun, 31 Aug 2008 10:40:01 +0530
Subject: [hunchentoot-devel] Request methods suffixed with * -- not
	backwards compatible?
Message-ID: <48BA27A9.6020508@chaitanyagupta.com>

Hi,

I was trying to use the svn branch of Hunchentoot, and found, to my 
surprise, that most of my old code didn't work anymore. The culprit 
mainly seems to be request.lisp, where a lot of old functions/methods 
have been renamed with a * suffixed to their name.

Are these changes permanent? Will these make it into the release 
version? Also, what is the reason behind this? I tried searching the 
mailing list for any discussion on this, but couldn't find anything.

One more thing, there were no such changes to reply.lisp i.e. no 
*-suffixed names there. Will the reply methods also undergo these changes?

Chaitanya