[postmodern-devel] character encoding

[Roman Budzianowski]

Marijn,

I can retrieve some records. I am attaching a file with a record that  
failed in  lisp
but was retrieved fine in psql - no guarantee of course that the bytes  
are exactly the same.

It failed in lisp like so:

Invalid byte at start of character: 0xae
    [Condition of type TRIVIAL-UTF-8:UTF-8-DECODING-ERROR]

Restarts:
  0: [ABORT] Return to SLIME's top level.
  1: [ABORT] Abort entirely from this (lisp) process.

Backtrace:
   0: (SWANK:SWANK-DEBUGGER-HOOK #<TRIVIAL-UTF-8:UTF-8-DECODING-ERROR  
@ #x719c5ec2> #<Function SWANK-DEBUGGER-HOOK>)
   1: (ERROR TRIVIAL-UTF-8:UTF-8-DECODING-ERROR :BYTE 174 :MESSAGE  
"Invalid byte at start of character: 0x~X")
       Locals:
         EXCL::DATUM = TRIVIAL-UTF-8:UTF-8-DECODING-ERROR
         EXCL::ARGUMENTS = (:BYTE 174 :MESSAGE "Invalid byte at start  
of character: 0x~X")
         POSTMODERN::STATEMENT-ID = :STATEMENT-1
         #:G1332 = WEB-SOURCE-ITEMS
         PQ-RDB::ARGS = (25)
         #:G1333 = WEB-SOURCE-ITEMS-SELECT-BY-ID
         PQ-RDB::SELECT-QUERY = "(SELECT * FROM web_source_items WHERE  
(id = $1))"
         ID = NIL
         POSTMODERN::PARAMS = (22225)
         EXCL::LOCAL-0 = (:BYTE 174 :MESSAGE "Invalid byte at start of  
character: 0x~X")
         EXCL::LOCAL-1 = TRIVIAL-UTF-8:UTF-8-DECODING-ERROR
         EXCL::LOCAL-2 = #<TRIVIAL-UTF-8:UTF-8-DECODING-ERROR @  
#x719c5ec2>
         EXCL::LOCAL-3 = 97
         EXCL::LOCAL-4 = T
         EXCL::LOCAL-5 = 0
         EXCL::LOCAL-6 = 8
         :UNKNOWN = NIL
         :UNKNOWN = NIL
         :UNKNOWN = NIL
   2: (TRIVIAL-UTF-8:UTF-8-GROUP-SIZE 174)
       Locals:
         BYTE = 174
         POSTMODERN::STATEMENT-ID = :STATEMENT-1
         #:G1332 = WEB-SOURCE-ITEMS
         PQ-RDB::ARGS = (25)
         #:G1333 = WEB-SOURCE-ITEMS-SELECT-BY-ID
         PQ-RDB::SELECT-QUERY = "(SELECT * FROM web_source_items WHERE  
(id = $1))"
         ID = NIL
         POSTMODERN::PARAMS = (22225)
         EXCL::LOCAL-0 = 174
   3: (TRIVIAL-UTF-8:READ-UTF-8-STRING #<MULTIVALENT #1=stream socket  
#1#connected from #2=localhost/33896 to #2#/5435 @ #x719a7dba> :BYTE- 
LENGTH 1353)
   4: ((:INTERNAL (:TOP-LEVEL-FORM "interpret.lisp" 4106) 0)  
#<MULTIVALENT #1=stream socket #1#connected from #2=localhost/33896 to  
#2#/5435 @ #x719a7dba> 1353)
   5: ((FLET POSTMODERN::SYMBOL-ALIST-ROW-READER CL-POSTGRES:NEXT- 
FIELD) #<CL-POSTGRES::FIELD-DESCRIPTION @ #x719c4542>)
   6: (POSTMODERN::SYMBOL-ALIST-ROW-READER #<MULTIVALENT #1=stream  
socket #1#connected from #2=localhost/33896 to #2#/5435 @ #x719a7dba>  
#(#<CL-POSTGRES::FIELD-DESCRIPTION @ #x719c370a> #<CL-POSTGRES::FIELD- 
DESCRIPTION @ #x719c3912> #<CL-POSTGRES::FIELD-DESCRIPTION @  
#x719c3b1a> #<CL-POSTGRES::FIELD-DESCRIPTION @ #x719c3d22> #<CL- 
POSTGRES::FIELD-DESCRIPTION @ #x719c3f2a> #<CL-POSTGRES::FIELD- 
DESCRIPTION @ #x719c4132> ...))
   7: (CL-POSTGRES::SEND-EXECUTE #<MULTIVALENT #1=stream socket  
#1#connected from #2=localhost/33896 to #2#/5435 @ #x719a7dba>  
"STATEMENT-1" (22225) POSTMODERN::SYMBOL-ALIST-ROW-READER)
   8: ((LABELS CL-POSTGRES:EXEC-PREPARED #:G3648))
   9: (CL-POSTGRES:EXEC-PREPARED #<CL-POSTGRES:DATABASE-CONNECTION @  
#x719a7a9a> "STATEMENT-1" (22225) POSTMODERN::SYMBOL-ALIST-ROW-READER)
  10: ((:INTERNAL WEB-SOURCE-ITEMS-SELECT-BY-ID) 22225)
  11: (LET* ((PQ-RDB::SELECT-QUERY (GET # #)) (PQ-RDB::SELECT-FUNCTION  
(OR # #))) (APPLY PQ-RDB::SELECT-FUNCTION PQ-RDB::ARGS))
  12: (WEB-SOURCE-ITEMS-SELECT-BY-ID 22225)

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: 22225.txt
URL: <https://mailman.common-lisp.net/pipermail/postmodern-devel/attachments/20080910/fd5f4715/attachment.txt>
-------------- next part --------------


On Sep 10, 2008, at 8:51 AM, Marijn Haverbeke wrote:

> Hey,
>
>> it seems to imply that the problem might be on the lisp side.
>> E.g. a byte might be skipped or misinterpreted.
>
> Could be, but I can't imagine where. It would be useful if you could
> send me (some subset of) the DB that causes the error + the query you
> were trying, so I can try to reproduce the problem.
>
> Best,
> Marijn
> _______________________________________________
> postmodern-devel mailing list
> postmodern-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel