From dl at znain.net  Fri May 18 14:01:06 2007
From: dl at znain.net (Dimitre Liotev)
Date: Fri, 18 May 2007 10:01:06 -0400
Subject: [html-template-devel] problem with UTF characters in template
Message-ID: <81bqgigrj1.fsf@oh177.znain.com>


Hi,

when I have UTF characters in a template I get this error:

#\? is not a LATIN-1 character.
   [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]

Is there a way to fix this?

I use Allegro CL Express on Windows. 

-- 
Dimitre Liotev


From edi at agharta.de  Fri May 18 14:08:51 2007
From: edi at agharta.de (Edi Weitz)
Date: Fri, 18 May 2007 16:08:51 +0200
Subject: [html-template-devel] problem with UTF characters in template
In-Reply-To: <81bqgigrj1.fsf@oh177.znain.com> (Dimitre Liotev's message of
	"Fri, 18 May 2007 10:01:06 -0400")
References: <81bqgigrj1.fsf@oh177.znain.com>
Message-ID: <ud50y6x70.fsf@agharta.de>

On Fri, 18 May 2007 10:01:06 -0400, Dimitre Liotev <dl at znain.net> wrote:

> when I have UTF characters in a template I get this error:
>
> #\? is not a LATIN-1 character.
>    [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]

What external format did you specify?  How about showing us the code
you tried and providing a backtrace?

Cheers,
Edi.


From dl at znain.net  Fri May 18 15:24:15 2007
From: dl at znain.net (Dimitre Liotev)
Date: Fri, 18 May 2007 11:24:15 -0400
Subject: [html-template-devel] problem with UTF characters in template
In-Reply-To: <ud50y6x70.fsf@agharta.de> (Edi Weitz's message of "Fri, 18 May
	2007 16:08:51 +0200")
References: <81bqgigrj1.fsf@oh177.znain.com> <ud50y6x70.fsf@agharta.de>
Message-ID: <814pmagnog.fsf@oh177.znain.com>

Edi Weitz <edi at agharta.de> writes:

> On Fri, 18 May 2007 10:01:06 -0400, Dimitre Liotev <dl at znain.net> wrote:
>
>> when I have UTF characters in a template I get this error:
>>
>> #\? is not a LATIN-1 character.
>>    [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]
>
> What external format did you specify?  How about showing us the code
> you tried and providing a backtrace?
>
> Cheers,
> Edi.


Hi Edi,

I am using the "Simple example" from your page
http://weitz.de/html-template/, specifying :external-format "UTF-8" in
the call to create-template-printer:

(defparameter *template-dispatcher*
  (hunchentoot:create-prefix-dispatcher "/template-test/"
   (lambda ()
     (let* ((html-template:*string-modifier* #'identity)
           (rows (loop for i below 49 by 7
                   collect
                     (list :cols
                           (loop for j from i below (+ i 7)
                              for string = (format nil "~R" j)
                              collect (list :content string 
                                            :colorful-style (oddp j))))))
          (values (list :rows rows)))
       (with-output-to-string (html-template:*default-template-output*)
         (html-template:fill-and-print-template
          (html-template:create-template-printer #p"foo.tmpl" :external-format "UTF-8")
          values))))))                           


(push *template-dispatcher* hunchentoot:*dispatch-table*)


The template:

<table border=1>
  <!-- TMPL_LOOP rows -->
    <tr>
      <!-- TMPL_LOOP cols -->
        <!-- TMPL_IF colorful-style -->
          <td align="right" bgcolor="blue">some cyrullic chars: ????<!-- TMPL_VAR content --></td>
        <!-- TMPL_ELSE -->
          <td align="right" ><!-- TMPL_VAR content --></td>
        <!-- /TMPL_IF -->
      <!-- /TMPL_LOOP -->
    </tr>
  <!-- /TMPL_LOOP -->
</table>


Now if I point the browser to http://localhost:8082/template-test/ it
works fine if the UTF chars are not there, but if I put any cyrillic UTF
encoded character in the template I get the error:


#\? is not a LATIN-1 character.
   [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]

Restarts:
 0: [ABORT] Abort entirely from this (lisp) process.

Backtrace:
  0: ((:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0))
      Locals:
        CONDITION = :UNKNOWN
  1: ((:INTERNAL (:TOP-LEVEL-FORM "swank-backend.lisp" 20112) 0)
      #<Function SWANK-DEBUGGER-HOOK>
      #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK-BACKEND::HOOK = #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        SWANK-BACKEND::FUN = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
  2: (SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK
       #<Function SWANK-DEBUGGER-HOOK>
       #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK-BACKEND::HOOK = #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        SWANK-BACKEND::HOOK = #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
  3: ((:INTERNAL SWANK::CALL-WITH-CONNECTION 0))
      Locals:
        SWANK::FUN = #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
  4: (SWANK::CALL-WITH-REDIRECTED-IO #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>)
      Locals:
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        FUNCTION = #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>
        FUNCTION = #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>
        SWANK::IO = #<TWO-WAY-STREAM "" @ #x20726182>
        SWANK::IN = #<SWANK-BACKEND::SLIME-INPUT-STREAM @ #x20726192>
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SWANK::OUT = #<SWANK-BACKEND::SLIME-OUTPUT-STREAM @ #x207161ca>
  5: (SWANK::MAYBE-CALL-WITH-IO-REDIRECTION
       #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>)
      Locals:
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN = #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0)
  @ #x2127a6aa>
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN = #(#<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0)
  @ #x2127a67a>)
  6: (SWANK::CALL-WITH-CONNECTION #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN = #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SYSTEM::.LAMBDA-LEXICAL-ENV. = #(#<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
        SWANK::FUN = #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
  7: (SWANK:SWANK-DEBUGGER-HOOK
       #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @ #x2127a602>
       #<Function SWANK-DEBUGGER-HOOK>)
      Locals:
        CONDITION = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        SWANK::HOOK = #<Function SWANK-DEBUGGER-HOOK>
        SYSTEM::.LAMBDA-LEXICAL-ENV. = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        CONDITION = :UNKNOWN
  8: (INVOKE-DEBUGGER #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
                        #x2127a602>)
      Locals:
        CONDITION = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-0 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-1 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-2 = T
  9: (SIGNAL #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @ #x2127a602>)
      Locals:
        EXCL::DATUM = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::ARGUMENTS = NIL
        EXCL::LOCAL-0 = NIL
        EXCL::LOCAL-1 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-2 = ((ERROR . #<Function MAYBE-INVOKE-DEBUGGER>))
        EXCL::LOCAL-3 = (ERROR . #<Function MAYBE-INVOKE-DEBUGGER>)
        EXCL::LOCAL-4 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-5 = #\null
        EXCL::LOCAL-6 = #<Function MAYBE-INVOKE-DEBUGGER>
 10: (ERROR FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR :FORMAT-CONTROL
            "~S is not a LATIN-1 character." :FORMAT-ARGUMENTS (#\?)
            :STREAM
            #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>)
      Locals:
        EXCL::DATUM = FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR
        EXCL::ARGUMENTS = (:FORMAT-CONTROL "~S is not a LATIN-1
  character." :FORMAT-ARGUMENTS
                           (#\?) :STREAM
                           #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM
  @ #x21279e5a>)
        EXCL::LOCAL-0 = (:FORMAT-CONTROL "~S is not a LATIN-1
  character." :FORMAT-ARGUMENTS
                         (#\?) :STREAM
                         #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>)
        EXCL::LOCAL-1 = FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR
        EXCL::LOCAL-2 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-3 = 97
        EXCL::LOCAL-4 = 1
        EXCL::LOCAL-5 = #<non-lisp object @ #xfffffffd>
        EXCL::LOCAL-6 = #<non-lisp object @ #x63897f0f>
        :UNKNOWN = NIL
        :UNKNOWN = NIL
        :UNKNOWN = NIL
        :UNKNOWN = NIL
        :UNKNOWN = NIL
 11: (FLEXI-STREAMS::SIGNAL-ENCODING-ERROR
       #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
       "~S is not a LATIN-1 character." #\?)
      Locals:
        FLEXI-STREAMS:FLEXI-STREAM =
  #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        FLEXI-STREAMS::FORMAT-CONTROL = "~S is not a LATIN-1 character."
        FLEXI-STREAMS::FORMAT-ARGS = (#\?)
        FLEXI-STREAMS::FORMAT-ARGS = (#\?)
        FLEXI-STREAMS:FLEXI-STREAM =
  #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        FLEXI-STREAMS::FORMAT-CONTROL = "~S is not a LATIN-1 character."
 12: ((METHOD STREAM-WRITE-CHAR
       (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #\?)
      Locals:
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        CHAR = #\?
        CHAR = #\?
        FLEXI-STREAMS:OCTET = 1044
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
 13: ((:INTERNAL (:EFFECTIVE-METHOD 2 NIL NIL NIL NIL) 0)
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #\?)
      Locals:
        EXCL::METHARG0 = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        EXCL::METHARG1 = #\?
        EXCL::LOCAL-0 = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        EXCL::LOCAL-1 = #\?
        EXCL::LOCAL-2 = #<Closure (:INTERNAL (:EFFECTIVE-METHOD 2 NIL
  NIL NIL NIL) 0) @
                          #x20c1f10a>
        EXCL::LOCAL-3 = #<non-lisp object @ #x2>
        EXCL::LOCAL-4 = NIL
        EXCL::LOCAL-5 = 3
        EXCL::LOCAL-6 = 137387322
        EXCL::LOCAL-7 = #<Function (METHOD STREAM-WRITE-CHAR
                                    (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM
  T))>
        EXCL::LOCAL-8 = 2
 14: ((METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE
       (FLEXI-STREAMS:FLEXI-OUTPUT-STREAM T T T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
      "<table border=1>
       
         <tr>
           
             
               <td align=\"right\" >zero</td>
              ..)
      Locals:
        FLEXI-STREAMS:FLEXI-OUTPUT-STREAM =
              #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        SEQUENCE = "<table border=1>
                     
                       <tr>
                         
                           
                             <td align=\"right\" >zero</td>
                           
                         
                           
                             <td align=\"right\" bgcolor=\"blue\">some
                             cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::START = 173
        FLEXI-STREAMS::END = 4537
        SEQUENCE = "<table border=1>
                     
                       <tr>
                         
                           
                             <td align=\"right\" >zero</td>
                           
                         
                           
                             <td align=\"right\" bgcolor=\"blue\">some
                             cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS:FLEXI-OUTPUT-STREAM =
                             #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM
                             @ #x21279e5a>
        #:|g1798| = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
                             #x21279e5a>
 15: ((METHOD STREAM-WRITE-STRING
       (TRIVIAL-GRAY-STREAMS:TRIVIAL-GRAY-STREAM-MIXIN T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
      "<table border=1>
       
         <tr>
           
             
               <td align=\"right\" >zero</td>
              ..)
      Locals:
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
              #x21279e5a>
        TRIVIAL-GRAY-STREAMS::SEQ = "<table border=1>
                                      
                                        <tr>
                                          
                                            
                                              <td align=\"right\"
                                              >zero</td>
                                            
                                          
                                            
                                              <td align=\"right\"
                                              bgcolor=\"blue\">some
                                              cyrillic chars: ????
                                              one</td> ..
        TRIVIAL-GRAY-STREAMS::START = 0
        TRIVIAL-GRAY-STREAMS::END = NIL
        TRIVIAL-GRAY-STREAMS::START = 0
        TRIVIAL-GRAY-STREAMS::END = NIL
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
                                              #x21279e5a>
 16: (FLEXI-STREAMS:STRING-TO-OCTETS "<table border=1>
       
         <tr>
           
             
               <td align=\"right\" >zero</td>
             
           
             
               <td align=\"right\" bgcolor=\"blue\">some cyrillic chars:
               ???? one</td> ..)
      Locals:
        STRING = "<table border=1>
                   
                     <tr>
                       
                         
                           <td align=\"right\" >zero</td>
                         
                       
                         
                           <td align=\"right\" bgcolor=\"blue\">some
                           cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::EXTERNAL-FORMAT =
                           #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                           :EOL-STYLE :LF) @
                                           #x20760e02>
        FLEXI-STREAMS::START = :UNSUPPLIED
        FLEXI-STREAMS::END = :UNSUPPLIED
        FLEXI-STREAMS::OUT = #<FLEXI-STREAMS::VECTOR-OUTPUT-STREAM @
                           #x21279d72>
        NIL = 0
        STRING = "<table border=1>
                   
                     <tr>
                       
                         
                           <td align=\"right\" >zero</td>
                         
                       
                         
                           <td align=\"right\" bgcolor=\"blue\">some
                           cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::START = 0
        FLEXI-STREAMS::END = NIL
        FLEXI-STREAMS::EXTERNAL-FORMAT =
                           #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                           :EOL-STYLE :LF) @
                                           #x20760e02>
 17: (HUNCHENTOOT::COMPUTE-LENGTH "<table border=1>
       
         <tr>
           
             
               <td align=\"right\" >zero</td>
             
           
             
               <td align=\"right\" bgcolor=\"blue\">some cyrillic chars:
               ???? one</td> ..)
      Locals:
        NIL = "<table border=1>
                
                  <tr>
                    
                      
                        <td align=\"right\" >zero</td>
                      
                    
                      
                        <td align=\"right\" bgcolor=\"blue\">some
                        cyrillic chars: ???? one</td> ..
        EXCL::LOCAL-0 = "<table border=1>
                          
                            <tr>
                              
                                
                                  <td align=\"right\" >zero</td>
                                
                              
                                
                                  <td align=\"right\"
                                  bgcolor=\"blue\">some cyrillic chars:
                                  ???? one</td> ..
        EXCL::LOCAL-1 = #<HUNCHENTOOT::REPLY @ #x2126e772>
        EXCL::LOCAL-2 = #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                                  :EOL-STYLE :LF) @
                          #x20760e02>
 18: (HUNCHENTOOT::START-OUTPUT "<table border=1>
       
         <tr>
           
             
               <td align=\"right\" >zero</td>
             
           
             
               <td align=\"right\" bgcolor=\"blue\">some cyrillic chars:
               ???? one</td> ..)
 19: (HUNCHENTOOT::PROCESS-REQUEST
       ((:HOST . "localhost:8082")
        (:USER-AGENT
         . "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)")
        (:ACCEPT
         . "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5")
        (:ACCEPT-LANGUAGE . "en-us,en;q=0.5")
        (:ACCEPT-ENCODING . "gzip,deflate")
        (:ACCEPT-CHARSET . "ISO-8859-1,utf-8;q=0.7,*;q=0.7")
        (:KEEP-ALIVE . "300") (:CONNECTION . "keep-alive") ..))
 20: (HUNCHENTOOT::PROCESS-CONNECTION #<HUNCHENTOOT::SERVER @
               #x20c14f9a>
       #<MULTIVALENT #1=stream socket #1#connected from
               #2=localhost/8082
         to #2#/4142 @ #x211e0dba>)


From edi at agharta.de  Fri May 18 15:58:16 2007
From: edi at agharta.de (Edi Weitz)
Date: Fri, 18 May 2007 17:58:16 +0200
Subject: [html-template-devel] problem with UTF characters in template
In-Reply-To: <814pmagnog.fsf@oh177.znain.com> (Dimitre Liotev's message of
	"Fri, 18 May 2007 11:24:15 -0400")
References: <81bqgigrj1.fsf@oh177.znain.com> <ud50y6x70.fsf@agharta.de>
	<814pmagnog.fsf@oh177.znain.com>
Message-ID: <uveeq5dk7.fsf@agharta.de>

On Fri, 18 May 2007 11:24:15 -0400, Dimitre Liotev <dl at znain.net> wrote:

> I am using the "Simple example" from your page
> http://weitz.de/html-template/, specifying :external-format "UTF-8"
> in the call to create-template-printer:

Is the string "UTF-8" a valid external format specifier on AllegroCL?

> (defparameter *template-dispatcher*
>   (hunchentoot:create-prefix-dispatcher "/template-test/"
>    (lambda ()
>      (let* ((html-template:*string-modifier* #'identity)
>            (rows (loop for i below 49 by 7
>                    collect
>                      (list :cols
>                            (loop for j from i below (+ i 7)
>                               for string = (format nil "~R" j)
>                               collect (list :content string 
>                                             :colorful-style (oddp j))))))
>           (values (list :rows rows)))
>        (with-output-to-string (html-template:*default-template-output*)
>          (html-template:fill-and-print-template
>           (html-template:create-template-printer #p"foo.tmpl" :external-format "UTF-8")
>           values))))))
>
> [...]
>
>  12: ((METHOD STREAM-WRITE-CHAR
>        (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM T))
>       #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #\?)
>       Locals:
>         STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
>   #x21279e5a>
>         CHAR = #\?
>         CHAR = #\?
>         FLEXI-STREAMS:OCTET = 1044
>         STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
>   #x21279e5a>

The condition is signalled when Hunchentoot is trying to write the
page to the client.  You have to make sure that Hunchentoot knows that
it's supposed to output UTF-8.  See the examples that come with
Hunchentoot for how this can be done.  (And you should also set the
headers accordingly, so the browser knows what to expect.)


From dl at znain.net  Fri May 18 18:55:33 2007
From: dl at znain.net (Dimitre Liotev)
Date: Fri, 18 May 2007 14:55:33 -0400
Subject: [html-template-devel] problem with UTF characters in template
In-Reply-To: <uveeq5dk7.fsf@agharta.de> (Edi Weitz's message of "Fri, 18 May
	2007 17:58:16 +0200")
References: <81bqgigrj1.fsf@oh177.znain.com> <ud50y6x70.fsf@agharta.de>
	<814pmagnog.fsf@oh177.znain.com> <uveeq5dk7.fsf@agharta.de>
Message-ID: <81y7jmezbu.fsf@oh177.znain.com>

Edi Weitz <edi at agharta.de> writes:

> On Fri, 18 May 2007 11:24:15 -0400, Dimitre Liotev <dl at znain.net> wrote:
>
>> I am using the "Simple example" from your page
>> http://weitz.de/html-template/, specifying :external-format "UTF-8"
>> in the call to create-template-printer:
>
> Is the string "UTF-8" a valid external format specifier on AllegroCL?

I thought so, but it seems that the format must be specified as a keyword:

CL-USER> (describe (find-if (lambda (x)
                              (member :UTF-8 (excl:ef-nicknames x)))
                            (EXCL:ALL-EXTERNAL-FORMATS)))

#<EXTERNAL-FORMAT :UTF8 ['(:E-CRLF :UTF8-BASE)] @ #x20116f6a> is a
structure of type EXTERNAL-FORMAT.  It has these slots:
 NAME               (:E-CRLF :UTF8-BASE)
 NICKNAMES          (:UTF8 :UTF-8)
...


I thought the string "UTF-8" would also work because when I try WITH-OPEN-FILE  
with an invalid external format like "UTF-9", an error is thrown, while "UTF-8"
is accepted and no error is raised.


>> (defparameter *template-dispatcher*
>>   (hunchentoot:create-prefix-dispatcher "/template-test/"
>>    (lambda ()
>>      (let* ((html-template:*string-modifier* #'identity)
>>            (rows (loop for i below 49 by 7
>>                    collect
>>                      (list :cols
>>                            (loop for j from i below (+ i 7)
>>                               for string = (format nil "~R" j)
>>                               collect (list :content string 
>>                                             :colorful-style (oddp j))))))
>>           (values (list :rows rows)))
>>        (with-output-to-string (html-template:*default-template-output*)
>>          (html-template:fill-and-print-template
>>           (html-template:create-template-printer #p"foo.tmpl" :external-format "UTF-8")
>>           values))))))
>>
>> [...]
>>
>>  12: ((METHOD STREAM-WRITE-CHAR
>>        (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM T))
>>       #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #\?)
>>       Locals:
>>         STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
>>   #x21279e5a>
>>         CHAR = #\?
>>         CHAR = #\?
>>         FLEXI-STREAMS:OCTET = 1044
>>         STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
>>   #x21279e5a>
>
> The condition is signalled when Hunchentoot is trying to write the
> page to the client.  You have to make sure that Hunchentoot knows that
> it's supposed to output UTF-8.  See the examples that come with
> Hunchentoot for how this can be done.  (And you should also set the
> headers accordingly, so the browser knows what to expect.)


Thanks!

For the sake of those who might encounter the same problem:
I added this to my handler function:

  (setf (hunchentoot:reply-external-format) :UTF-8)

This will set the reply-external-format only for the given handler. To
set it globaly use this:

  (setf hunchentoot:*hunchentoot-default-external-format* :UTF-8)