From fahree at gmail.com Thu Oct 27 20:41:41 2011 From: fahree at gmail.com (=?ISO-8859-1?Q?Far=E9?=) Date: Thu, 27 Oct 2011 16:41:41 -0400 Subject: [hunchentoot-devel] A few patches from ITA Message-ID: I have been merging the latest hunchentoot into our (Google/ITA) tree, and I have a few patches. Is this list still the place to exchange patches, or should I be sending github pull requests instead? My attached patches are: * some package tweaks (symbols not exported that should, symbols exported that are no more) * patch to make-cooked-message so it works for 503 and other messages not in the case shortlist. * patch for send-service-unavailable-reply (my "solution" is horrible, someone who understands hunchentoot internals better might know what the "right thing" is). Not included: * a patch for xcvb support, but I believe you're not interested. * some debugging code we use to track instabilities that we've observed in our farms. Of course, I'm willing to massage the code until it is satisfactory to you who maintain the upstream tree. Just tell me what I need to change so it's commit-worthy. ??? ? Fran?ois-Ren? ?VB Rideau ?Reflection&Cybernethics? http://fare.tunes.org -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fixup-some-symbol-exports.patch Type: application/octet-stream Size: 2088 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Make-sure-we-always-send-a-cooked-message-even-for-c.patch Type: application/octet-stream Size: 3696 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Quick-fix-to-allow-send-service-unavailable-reply-to.patch Type: application/octet-stream Size: 2200 bytes Desc: not available URL: From hans.huebner at gmail.com Thu Oct 27 21:02:19 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Thu, 27 Oct 2011 23:02:19 +0200 Subject: [hunchentoot-devel] A few patches from ITA In-Reply-To: References: Message-ID: Hi Far?, this is still the right place to discuss Hunchentoot, and thanks for your patches. The preferred way to submit patches is github, the official repository is https://github.com/edicl/hunchentoot - If you could submit them there, it will be easier to review and pull them. I will do so over the weekend. Thanks, Hans From farzadbekran at gmail.com Sat Oct 29 21:10:10 2011 From: farzadbekran at gmail.com (Farzad Bekran) Date: Sun, 30 Oct 2011 00:40:10 +0330 Subject: [hunchentoot-devel] problem with the hunchentoots handler for POST methods Message-ID: hi, i get this error when i send a page that has a form on it that uses POST method, if i change the method to GET it works fine tho. and the POST test on the hunchentoot's own test site gives me this too. here is the stack trace: # is not a binary input stream. [Condition of type SIMPLE-TYPE-ERROR] Restarts: 0: [TERMINATE-THREAD] Terminate this thread (#) Backtrace: 0: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGING-ENVIRONMENT) #) 1: (SWANK::DEBUG-IN-EMACS #>) Locals: SB-DEBUG::ARG-0 = # 2: (SWANK-BACKEND::CALL-WITH-BREAK-HOOK # #) 3: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK) # #) 4: (SWANK::CALL-WITH-BINDINGS ((*STANDARD-OUTPUT* . #) (*STANDARD-INPUT* . #) (*TRACE-OUTPUT* . #) (*ERROR-OUTPUT* . #) (*DEBUG-IO* . #) (*QUERY-IO* . #) ...) #) 5: (SWANK:INVOKE-SLIME-DEBUGGER #>) 6: (SWANK-BACKEND::CALL-WITH-BREAK-HOOK # #) 7: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK) # #) 8: (SWANK:SWANK-DEBUGGER-HOOK #> #) 9: (SB-DEBUG::RUN-HOOK *DEBUGGER-HOOK* #>) 10: (INVOKE-DEBUGGER #>) 11: ((SB-PCL::FAST-METHOD HUNCHENTOOT:MAYBE-INVOKE-DEBUGGER (T)) ..) 12: (SIGNAL #>) 13: (ERROR ..) 14: (SB-KERNEL:ILL-BIN #) 15: (READ-BYTE # NIL NIL) 16: (CHUNGA:READ-CHAR* # NIL NIL) 17: (CHUNGA:PEEK-CHAR* # NIL NIL) 18: (CHUNGA:READ-TOKEN #) 19: (HUNCHENTOOT::PARSE-CONTENT-TYPE "application/x-www-form-urlencoded" T) 20: ((SB-PCL::FAST-METHOD INITIALIZE-INSTANCE :AFTER (HUNCHENTOOT::REQUEST)) # # #) 21: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.| SB-PCL::|.P3.| SB-PCL::|.P4.|)) ..) 22: (HUNCHENTOOT::PROCESS-REQUEST ..) 23: (HUNCHENTOOT::PROCESS-CONNECTION # #) 24: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK405]410)) 25: ((FLET SB-THREAD::WITH-MUTEX-THUNK)) 26: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]309)) 27: (SB-THREAD::CALL-WITH-MUTEX ..) 28: (SB-THREAD::INITIAL-THREAD-FUNCTION) 29: ("foreign function: call_into_lisp") 30: ("foreign function: funcall0") 31: ("foreign function: new_thread_trampoline") 32: ("foreign function: #xB7FB6E99") -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans.huebner at gmail.com Sat Oct 29 21:27:43 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sat, 29 Oct 2011 23:27:43 +0200 Subject: [hunchentoot-devel] problem with the hunchentoots handler for POST methods In-Reply-To: References: Message-ID: Hi Farzad, I have just verified that the POST test passes for me on Linux with hunchentoot-1.1.1 on SBCL 1.0.51, installed with quicklisp. What operating system, Hunchentoot version and SBCL version do you use? Does the problem show when you evaluate (HUNCHENTOOT::PARSE-CONTENT-TYPE "application/x-www-form-urlencoded") in your repl? In fact, this part of the backtrace makes me wonder a bit: On Sat, Oct 29, 2011 at 11:10 PM, Farzad Bekran wrote: > ?19: (HUNCHENTOOT::PARSE-CONTENT-TYPE "application/x-www-form-urlencoded" T) There is an extra argument (T) which I can't relate to the argument list of PARSE-CONTENT-TYPE. Did you install Hunchentoot with quicklisp? If not, I'd suggest that you try that to make sure that you have a set of libraries that work together. Cheers, Hans From farzadbekran at gmail.com Sat Oct 29 23:11:39 2011 From: farzadbekran at gmail.com (Farzad Bekran) Date: Sun, 30 Oct 2011 02:41:39 +0330 Subject: [hunchentoot-devel] problem with the hunchentoots handler for POST methods In-Reply-To: References: Message-ID: Hi Hans Thanks for the quick reply. it sums up to this: somehow when I used quicklisp to load hunchentoot it used a really old version of the hunchentoot that was picked up by asdf and didn't update to the latest. it was fixed by deleting all the old packages and installing freshly using quicklisp. Cheers, Farzad -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans.huebner at gmail.com Sun Oct 30 20:55:39 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sun, 30 Oct 2011 21:55:39 +0100 Subject: [hunchentoot-devel] Hunchentoot v1.2.0 released Message-ID: After over a year of no releases, Hunchentoot v1.2.0 is out. It incorporates a large number of changes from various people and is the first release that I have made as the maintainer of Hunchentoot. There have been a fair number of changes, many of them contributed by Hunchentoot's users. Thank you all for that! The release tarball is available here: https://github.com/downloads/edicl/hunchentoot/hunchentoot-1.2.0.tar.gz This is my first release, and as such it is likely that I'm not yet up to the release standards that Edi has set forth. Please report any problems you may have with the new version to the Hunchentoot mailing list. I will try to resolve problems quickly and possible make a new minor release for each major problem found. The development repository for Hunchentoot is now hosted by github (https://github.com/edicl/hunchentoot). If you want to contribute, it would be appreciated if you could submit your patches through github, too. Here is a list of changes: Add www/ directory with default file tree that is being served Add error template mechanism and improve error reporting in general. Improve automatic testing, SBCL kludge to support asdf:test-op Allegro CL modern mode fixes Fix bugs in serving partial responses Limit maximum number of threads that Hunchentoot creates (Dan Weinreb, Scott McKay) Export fixes (Gordon Sims, Andrey Moskvitin) Factor out easy-handler logic into separate acceptor subclass Export two session functions (Nico de Jager) Allow no Content-Type header (Chaitanya Gupta) Patch for compilation with ECL (Sohail Somani) Fix DEFINE-EASY-HANDLER for multiple acceptors (Nicolas Neuss) Revived *SHOW-LISP-BACKTRACES-P* Made sure "100 Continue" is returned even if the client sends "Expect: 100-continue" twice (reported by Gordon Sims) Fixed typo in code which interprets transfer encodings Thank you for your patience! Hans From hans.huebner at gmail.com Sun Oct 30 21:21:01 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sun, 30 Oct 2011 22:21:01 +0100 Subject: [hunchentoot-devel] Garbage in REPL In-Reply-To: References: Message-ID: Hi Andrey, this suggestion did not make it to the release. Actually, I like the current default, but I have converted your suggestion into an issue and will look at it for the next release. Sorry that it took so long to respond at all. -Hans On Mon, Apr 4, 2011 at 5:59 PM, Andrey Moskvitin wrote: > Hi, > > For ACCESS-LOG-DESIGNATION default value is *ERROR-OUTPUT*. It is the cause of > littering REPL for SLIME users. Maybe better use NIL as default and for suppress > logging use a keyword :suppress? > > (defmethod acceptor-access-log-destination ((acceptor acceptor)) > ?(let ((destination (slot-value acceptor 'access-log-destination))) > ? ?(case destination > ? ? ?((nil) *error-output*) > ? ? ?(:suppress nil) > ? ? ?(otherwise destination)))) > > In this case, ?the logging will be sent to *inferior-lisp* and ?will not > interfere with. > > Andrey > > _______________________________________________ > tbnl-devel site list > tbnl-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/tbnl-devel > From hans.huebner at gmail.com Sun Oct 30 21:24:50 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sun, 30 Oct 2011 22:24:50 +0100 Subject: [hunchentoot-devel] Hunchentoot Errors (e.g. Couldn't write to #) In-Reply-To: References: Message-ID: Cyrus, sorry that it took so long to respond to this. The error handling behavior in the v1.2.0 release has been much improved, so if you could let me know if the problem still exists with the new version, it'd be great. Cheers, Hans On Tue, Apr 5, 2011 at 2:37 AM, Cyrus Harmon wrote: > > Hello tbnl-folk, > > So, I've been seeing transient problems with hunchentoot for as long as I can remember, and I haven't had the chance to try to track down the problems until now. It seems there are a couple of classes of problems. In order of severity, the are: > > 1. things that crash hunchentoot/SBCL (or, in some cases, the host OS) > > 2. things that throw "unexpected" errors > > 3. things that throw "expected" errors (e.g. 404) > > As for class 1, these are tough to track down as the server is usually crashed by the time I get to it, but some causes I consider plausible are: > > * running out of some limited resource such as RAM, threads, or network sockets > * possible SBCL bug in, say, signal handling or dealing with lots of threads > > I don't have any data to present on this class, at the moment, so let's move on to the next class. > > I often see errors such as this in the log: > > [2011-04-05 00:04:52 [ERROR]] Couldn't write to # ? ? ? ? ? ? ? ? ? ?for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" > ? ? ? ? ? ? ? ? ? ?{B6FB859}>: > ?Broken pipe > 0: (SB-DEBUG::MAP-BACKTRACE > ? ?# > ? ?:START > ? ?0 > ? ?:COUNT > ? ?536870911) > 1: (BACKTRACE 536870911 #) > 2: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE-TO-STREAM > ? ?#) > 3: (HUNCHENTOOT::GET-BACKTRACE) > 4: ((FLET #:LAMBDA271) > ? ?#" {B71EA51}>) > 5: (SIGNAL #" {B71EA51}>) > 6: (ERROR > ? ?SB-INT:SIMPLE-STREAM-ERROR > ? ?:STREAM > ? ?# ? ? ?{B6FB859}> > ? ?:FORMAT-CONTROL > ? ?"~@<~?: ~2I~_~A~:>" > ? ?:FORMAT-ARGUMENTS > ? ?("Couldn't write to ~S" > ? ? (# ? ? ? ?for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) > ? ? "Broken pipe")) > 7: (SB-IMPL::SIMPLE-STREAM-PERROR > ? ?"Couldn't write to ~S" > ? ?# ? ? ?{B6FB859}> > ? ?32) > 8: (SB-IMPL::WRITE-OUTPUT-FROM-QUEUE > ? ?# ? ? ?{B6FB859}>) > 9: (SB-IMPL::SUB-SUB-SERVE-EVENT NIL NIL) > 10: (SB-IMPL::SUB-SERVE-EVENT NIL NIL NIL) > 11: (SB-SYS:SERVE-ALL-EVENTS NIL) > 12: (SB-IMPL::FINISH-FD-STREAM-OUTPUT > ? ? # ? ? ? for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) > 13: (FINISH-OUTPUT > ? ? # ? ? ? for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) > 14: (HUNCHENTOOT:HANDLE-STATIC-FILE > ? ? #P"/home/sly/projects/www-cyrusharmon-org/static/images/fish-clusters.png" > ? ? NIL) > 15: ((SB-PCL::FAST-METHOD HUNCHENTOOT:HANDLE-REQUEST > ? ? ?(HUNCHENTOOT:ACCEPTOR HUNCHENTOOT:REQUEST)) > ? ? # > ? ? # > ? ? # > ? ? #) > 16: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-REQUEST (T)) > ? ? # > ? ? # > ? ? #) > 17: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION > ? ? ?(HUNCHENTOOT:ACCEPTOR T)) > ? ? # > ? ? # > ? ? # > ? ? #) > 18: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION :AROUND > ? ? ?(HUNCHENTOOT:ACCEPTOR T)) > ? ? # > ? ? #S(SB-PCL::FAST-METHOD-CALL > ? ? ? ?:FUNCTION # > ? ? ? ?:PV NIL > ? ? ? ?:NEXT-METHOD-CALL NIL > ? ? ? ?:ARG-INFO (2)) > ? ? # > ? ? #) > 19: ((LAMBDA ())) > 20: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK353]358)) > 21: ((FLET SB-THREAD::WITH-MUTEX-THUNK)) > 22: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]300)) > 23: (SB-THREAD::CALL-WITH-MUTEX > ? ? # > ? ? #S(SB-THREAD:MUTEX > ? ? ? ?:NAME "thread result lock" > ? ? ? ?:%OWNER # ? ? ? ? ? ? ? ? ?"Hunchentoot worker (client: 217.72.218.248:2741)" RUNNING > ? ? ? ? ? ? ? ? ?{B6FCE39}> > ? ? ? ?:STATE 1) > ? ? # ? ? ? "Hunchentoot worker (client: 217.72.218.248:2741)" RUNNING {B6FCE39}> > ? ? T) > 24: (SB-THREAD::INITIAL-THREAD-FUNCTION) > 25: ("foreign function: call_into_lisp") > 26: ("foreign function: funcall0") > 27: ("foreign function: new_thread_trampoline") > 28: ("foreign function: #xB7FC5506") > > What does this whole "broken pipe" mean? Is this more of an "expected error" or is something going wrong here? > > On a slightly different note, I'd like to be able to run a relatively reliable hunchentoot server that can serve a modest traffic load without having to resort to caching proxy servers and the like. I recall some discussion of thread pools, epoll, etc... Has the state of the art for high-reliability hunchentoot instances changed in the last couple years? Any suggestions on improving it? > > thanks! > > Cyrus > > > _______________________________________________ > tbnl-devel site list > tbnl-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/tbnl-devel > From hans.huebner at gmail.com Sun Oct 30 21:26:16 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sun, 30 Oct 2011 22:26:16 +0100 Subject: [hunchentoot-devel] Hunchentoot support for ECL In-Reply-To: References: Message-ID: Hi Olivier, sorry for the long delay. Can you try compiling Hunchentoot v1.2.0 with ECL and send any patches that may be required? GitHub pull requests prefered, but I'll take a standard patch, too. -Hans On Thu, Jun 9, 2011 at 8:37 PM, Olivier Dumont wrote: > Hi, > the attached patch allows hunchentoot to function properly with the latest > release of ECL (11.1.1). > best regards, > Olivier > _______________________________________________ > tbnl-devel site list > tbnl-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/tbnl-devel > From xach at xach.com Mon Oct 31 01:08:50 2011 From: xach at xach.com (Zach Beane) Date: Sun, 30 Oct 2011 21:08:50 -0400 Subject: [hunchentoot-devel] Hunchentoot v1.2.0 released In-Reply-To: ("Hans =?utf-8?Q?H=C3=BCbner=22's?= message of "Sun, 30 Oct 2011 21:55:39 +0100") References: Message-ID: <87aa8idl25.fsf@hangup.portland.xach.com> Hans H?bner writes: > After over a year of no releases, Hunchentoot v1.2.0 is out. It > incorporates a large number of changes from various people and is the > first release that I have made as the maintainer of Hunchentoot. > There have been a fair number of changes, many of them contributed by > Hunchentoot's users. Thank you all for that! > > The release tarball is available here: > https://github.com/downloads/edicl/hunchentoot/hunchentoot-1.2.0.tar.gz > > This is my first release, and as such it is likely that I'm not yet up > to the release standards that Edi has set forth. Please report any > problems you may have with the new version to the Hunchentoot mailing > list. I will try to resolve problems quickly and possible make a new > minor release for each major problem found. Congratulations on the new release! I've seen a number of projects affected by this release, but mostly for these reasons: - no more HUNCHENTOOT:*APPROVED-RETURN-CODES* (affects cl-oauth, cl-twitter) - no more HUNCHENTOOT:LOG-MESSAGE (affects hunchentoot-cgi) - no more HUNCHENTOOT:ACCEPTOR-REQUEST-DISPATCHER (affects webfunk) I didn't see these explicitly mentioned in your list of notable changes; is there any guidance for what should be used instead in each case? Thanks, Zach From hans.huebner at gmail.com Mon Oct 31 07:37:24 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Mon, 31 Oct 2011 08:37:24 +0100 Subject: [hunchentoot-devel] Hunchentoot v1.2.0 released In-Reply-To: <87aa8idl25.fsf@hangup.portland.xach.com> References: <87aa8idl25.fsf@hangup.portland.xach.com> Message-ID: Hi Zach, thank you for reporting the problems with quicklisp packages that depend on Hunchentoot. Unfortunately, due to the long time since the last release, there have been a number of small but incompatible changes that have accumulated and that would probably have received more prominent mention if they had been released timely. I will try to release quicker from now on. Here are responses to your findings in quicklisp packages: On Mon, Oct 31, 2011 at 2:08 AM, Zach Beane wrote: > ?- no more HUNCHENTOOT:*APPROVED-RETURN-CODES* (affects cl-oauth, > ? ?cl-twitter) The whole concept of "approved" HTTP status codes is gone. Handlers are free to return whatever status code they require. > ?- no more HUNCHENTOOT:LOG-MESSAGE (affects hunchentoot-cgi) ACCEPTOR-LOG-MESSAGE must be used instead. There is LOG-MESSAGE* which uses the current *ACCEPTOR*. > ?- no more HUNCHENTOOT:ACCEPTOR-REQUEST-DISPATCHER (affects webfunk) Request dispatching is now done by the way of the generic ACCEPTOR-DISPATCH-REQUEST function. The *DISPATCH-TABLE* is only used with EASY-ACCEPTOR instances. For web frameworks that do their own request dispatching should implement a method for ACCEPTOR-DISPATCH-REQUEST, possibly with a CALL-NEXT-METHOD in the end if the request could not be dispatched. Let me know if other issues come up or if I can provide more info. -Hans From hans.huebner at gmail.com Mon Oct 31 09:52:33 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Mon, 31 Oct 2011 10:52:33 +0100 Subject: [hunchentoot-devel] Hunchentoot v1.2.0 released In-Reply-To: References: <87aa8idl25.fsf@hangup.portland.xach.com> Message-ID: I put some work into updating the documentation to reflect how request processing is done. The documentation is now a bit more clear about the base functionality and the functionality that is provided by HUNCHENTOOT:EASY-ACCEPTOR. In a nutshell, if you have been using DEFINE-EASY-HANDLER and/or *DISPATCH-TABLE* in the past, you'll want to use EASY-ACCEPTOR instead of ACCEPTOR from now on. Currently, no EASY-SSL-ACCEPTOR class is defined. If you need it, you can simply add it ( (defclass easy-ssl-acceptor (easy-acceptor ssl-acceptor) ()) - If you do, please let me know so I'll add it to the next release. I will wait a few more days to collect your input and make corrections before I release v1.2.1 to include the documentation update and any hot fixes that might be required. Thanks! -Hans From desoi at pgedit.com Mon Oct 31 12:50:22 2011 From: desoi at pgedit.com (John DeSoi) Date: Mon, 31 Oct 2011 08:50:22 -0400 Subject: [hunchentoot-devel] [hunchentoot] wrong character set in parsing request (#1) In-Reply-To: References: Message-ID: <96A224E1-1373-4E74-9F63-F03FAF5976F5@pgedit.com> Since this issue has been closed on github, I'd like to repost to the list and get some feedback. It still seems like a bug to me. > An incoming request has > > Content-Type: application/json; charset=UTF-8 > > but the server treats it as latin-1 (the default). It seems the code correctly parses the character set from the content type, but ignores it unless the type is "text". See parse-content-type. On Oct 30, 2011, at 4:29 PM, Hans H?bner wrote: > application/json is always encoded as utf-8, at least that is what I understood so far. > So in order to correctly handle a application/json request, I have to explicitly control the request parsing, even though the request has already specified it is UTF-8. (defhandler (rpc-handler :uri "/rpc" :default-request-type :post) () (setf (tbnl:content-type*) "application/json") (setf (tbnl:reply-external-format*) +utf-8+) (invoke-rpc (tbnl:raw-post-data :external-format +utf-8+))) The parse-content-type function parses the "charset" part, but then completely ignores it if the mime type does not have "text" in it. If the content type had been "text/json", it would have worked correctly. John DeSoi, Ph.D. From hans.huebner at gmail.com Mon Oct 31 14:11:36 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Mon, 31 Oct 2011 15:11:36 +0100 Subject: [hunchentoot-devel] [hunchentoot] wrong character set in parsing request (#1) In-Reply-To: <96A224E1-1373-4E74-9F63-F03FAF5976F5@pgedit.com> References: <96A224E1-1373-4E74-9F63-F03FAF5976F5@pgedit.com> Message-ID: On Mon, Oct 31, 2011 at 1:50 PM, John DeSoi wrote: > Since this issue has been closed on github, I'd like to repost to the list and get some feedback. It still seems like a bug to me. I have continued the topic on github, even though the issue is closed. Hunchentoot makes no attempt in determining the content type for non-text bodies. I think that the piece of code that you've posted does the right thing. I also think that RAW-POST-DATA should return the body as octet vector rather than making a possibly false attempt at guessing the character set. If you have another suggestion, please let us know. -Hans From desoi at pgedit.com Mon Oct 31 14:44:15 2011 From: desoi at pgedit.com (John DeSoi) Date: Mon, 31 Oct 2011 10:44:15 -0400 Subject: [hunchentoot-devel] [hunchentoot] wrong character set in parsing request (#1) In-Reply-To: References: <96A224E1-1373-4E74-9F63-F03FAF5976F5@pgedit.com> Message-ID: <66B5B936-D72C-4BD3-8AFC-985E25845D16@pgedit.com> Hans, On Oct 31, 2011, at 10:11 AM, Hans H?bner wrote: > I have continued the topic on github, even though the issue is closed. > > Hunchentoot makes no attempt in determining the content type for > non-text bodies. I think that the piece of code that you've posted > does the right thing. I also think that RAW-POST-DATA should return > the body as octet vector rather than making a possibly false attempt > at guessing the character set. If you have another suggestion, please > let us know. Does not the fact that charset is provided, tell you that it is textual? There is nothing magic about the mime time having "text" in it. I think there are a lot of mime types that are textual that don't have "text" as the subtype. In all cases Hunchentoot is going to give you the wrong character type by default (unless it matches by luck). The handler I provided is not a complete solution because, I assumed UTF-8 which is incorrect. Other character sets are allowed. To handle any text request correctly (that does not have "text" in the mime type), the handler will need to parse the charset itself. This repeats what Hunchentoot has already executed, but ignored because "text" was not in the mime subtype. This is not a big deal and I can work around it. But I suspect it will come up again in the near future :). Thanks for your feedback, John DeSoi, Ph.D. From hans.huebner at gmail.com Mon Oct 31 14:58:09 2011 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Mon, 31 Oct 2011 15:58:09 +0100 Subject: [hunchentoot-devel] [hunchentoot] wrong character set in parsing request (#1) In-Reply-To: <66B5B936-D72C-4BD3-8AFC-985E25845D16@pgedit.com> References: <96A224E1-1373-4E74-9F63-F03FAF5976F5@pgedit.com> <66B5B936-D72C-4BD3-8AFC-985E25845D16@pgedit.com> Message-ID: I think it is silly to conduct a conversation in two channels. Anyway: The applicability of the charset attribute in content type specifications is restricted to text content types as per rfc2045. At least that is my reading of it. Now, there is nothing wrong with being liberal in accepting, but in this case I actually tend to make RAW-POST-DATA return an octet vector for non-text content types and have the application handle the desired decoding. -Hans On Mon, Oct 31, 2011 at 3:44 PM, John DeSoi wrote: > Hans, > > On Oct 31, 2011, at 10:11 AM, Hans H?bner wrote: > >> I have continued the topic on github, even though the issue is closed. >> >> Hunchentoot makes no attempt in determining the content type for >> non-text bodies. ?I think that the piece of code that you've posted >> does the right thing. ?I also think that RAW-POST-DATA should return >> the body as octet vector rather than making a possibly false attempt >> at guessing the character set. ?If you have another suggestion, please >> let us know. > > Does not the fact that charset is provided, tell you that it is textual? There is nothing magic about the mime time having "text" in it. I think there are a lot of mime types that are textual that don't have "text" as the subtype. In all cases Hunchentoot is going to give you the wrong character type by default (unless it matches by luck). The handler I provided is not a complete solution because, I assumed UTF-8 which is incorrect. Other character sets are allowed. To handle any text request correctly (that does not have "text" in the mime type), the handler will need to parse the charset itself. This repeats what Hunchentoot has already executed, but ignored because "text" was not in the mime subtype. > > This is not a big deal and I can work around it. But I suspect it will come up again in the near future :). > > Thanks for your feedback, > > > > John DeSoi, Ph.D. > > > > > > _______________________________________________ > tbnl-devel site list > tbnl-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/tbnl-devel > From desoi at pgedit.com Mon Oct 31 20:25:59 2011 From: desoi at pgedit.com (John DeSoi) Date: Mon, 31 Oct 2011 16:25:59 -0400 Subject: [hunchentoot-devel] [hunchentoot] wrong character set in parsing request (#1) In-Reply-To: References: <96A224E1-1373-4E74-9F63-F03FAF5976F5@pgedit.com> <66B5B936-D72C-4BD3-8AFC-985E25845D16@pgedit.com> Message-ID: <1ACDAE93-6A1F-4C7A-9BF5-CD9B96792209@pgedit.com> I'll put further comments here. Fun stuff for issue #1 :) https://github.com/edicl/hunchentoot/issues/1 John DeSoi, Ph.D. On Oct 31, 2011, at 10:58 AM, Hans H?bner wrote: > I think it is silly to conduct a conversation in two channels. > Anyway: The applicability of the charset attribute in content type > specifications is restricted to text content types as per rfc2045. At > least that is my reading of it. > > Now, there is nothing wrong with being liberal in accepting, but in > this case I actually tend to make RAW-POST-DATA return an octet vector > for non-text content types and have the application handle the desired > decoding.