[hunchentoot-devel] Encoding question

[Viljo Marrandi]

Hello,

Thanks for quick reply. Now I changed from UTF-8 keyword to this:
(defvar *utf-8* (flex:make-external-format :utf-8 :eol-style :lf))
(setq hunchentoot:*hunchentoot-default-external-format* *utf-8*)

My system:
Ubuntu Linux w 2.6.17 kernel
SBCL 1.0
hunchentoot-0.5.1
flexi-streams-0.9.1
Currently I run hunchentoot as standalone server, not behind mod_lisp

After testing one strange thing came up. The error I get from
uploading files is this:

[ERROR] While parsing multipart/form-data parameters: Unexpected value
#xFF at start of UTF-8 sequence.

But this is only for some files, some files don't give this at all.
Some files give this error, but upload anyway. Files that fail first
time succeed when I just hit reload on browser and submit same form
again.

Here is code that behaves like described (basically copied from test.lisp):

(defun upload-test ()
  (setf (content-type) "text/html; charset=utf-8")
  (when (and (post-parameter "upload-file")
	     (listp (post-parameter "upload-file")))
    (handle-file (post-parameter "upload-file")))
  (with-html
   (:html
    (:head (:title "Test"))
    (:body
     (:form :method :post :enctype "multipart/form-data"
	    (:input :type "file" :name "upload-file")(:br)
	    (:input :type :submit :name "Upload"))))))

(defun handle-file (post-parameter)
  (let ((result nil))
    (when (and post-parameter
	       (listp post-parameter))
      (destructuring-bind (path file-name content-type)
	  post-parameter
	(let ((new-path (concatenate 'string *upload-directory* file-name)))
	  (when (search "Windows" (user-agent) :test #'char-equal)
	    (setq file-name (cl-ppcre:regex-replace ".*\\\\" file-name "")))
	  (rename-file path (ensure-directories-exist new-path))
	  (setf result (list :file-path new-path
			     :file-name file-name
			     :file-length (ignore-errors (with-open-file (in new-path)
(file-length in)))
			     :content-type content-type)))))
    result))

Rgds,
Viljo

On 1/22/07, Edi Weitz <edi at agharta.de> wrote:
> On Mon, 22 Jan 2007 12:20:35 +0200, "Viljo Marrandi" <viltsik at gmail.com> wrote:
>
> > So I tried to switch everything to UTF-8 (which would be preferred
> > anyway).  (setq hunchentoot:*hunchentoot-default-external-format*
> > :utf-8)
>
> According to the docs the value of this variable should be a flexi
> streams external format.  ATM it might work with a keyword like
> :UTF-8, but there's no guarantee it will continue to work in the
> future.
>
> > Localized dates and text from database now displayed correctly, but
> > then I couldn't upload binary files from forms, I guess utf-8 and
> > binary don't mix well.
>
> Hmm, I just tried (0.5.1 with LWW 5.0.1) and file uploads work fine
> for me even if I set the default external format to UTF-8.
>
> What exactly doesn't work for you?  Which error message do you get?
> Can you provide a small, simple example to reproduce the error?  And a
> backtrace?  Which Lisp and which OS are you using?  Are you using the
> latest releases of Hunchentoot and the other libraries?  Is you
> website behind mod_lisp?  (You know, all this stuff that usually
> belongs to a bug report...)
>
> Cheers,
> Edi.
>