[cl-wiki-devel] Encoding question and a patch

[Ian Clelland]

On 9/6/05, Emre Sevinç <emres at bilgi.edu.tr> wrote:
> But the one that comes with Debian SID (unstable) does support Unicode,
> as far as I know. I had similar
> problems with older versions of SBCL on my Debian GNU/Linux PC at home
> and the solution
> was to upgrade to the unstable package (which also needed an upgrade
> from linux 2.4 to 2.6.11).
> 
> And I think CLISP supports it too. So is it a problem only related to
> the Lisp compiler used
> for cl-wiki?

Well, the wiki pages on disk are not loaded in any special way, so I
believe that they default to 8-bit ascii mode. Because of this, the
escape-string function of cl-who is inappropriate for handling
utf-8-encoded data (it just encodes each byte as a separate
character).

If there was full unicode support in my version of sbcl, we might be
able to pass :external-format :utf-8 to #'with-open-file and get
multi-byte support internally. If that were the case, then
escape-string would probably work correctly, and would create the
proper sgml entities for unicode characters.

I don't know if that technique is portable at all across the different
implementations of common lisp.

The change that I made just makes cl-wiki ignore high-ascii
characters, and only escape <, >, ", ', and &. Since HTTP is an
8-bit-safe protocol, there is no problem sending unicode characters
directly.

(The only other change I made, which I forgot to mention earlier, is
that I added the line
<meta http-equiv="Content-type" content="text/html; charset=UTF-8" />
to the head section of my main template.)

> I hope it works for us, too. (Sinan will probably apply and try and
> inform everybody today) :)
> 
> Istanbul Bilgi University's Lisp User's Group trusts you:
> 
> http://church.cs.bilgi.edu.tr/lcg

Thanks; I'll try not to let you down :)


Regards,

Ian Clelland
<clelland at gmail.com>