From fred at streamfocus.com Thu Jan 7 20:29:00 2010 From: fred at streamfocus.com (Fred Gibson) Date: Thu, 7 Jan 2010 12:29:00 -0800 Subject: [flexi-streams-devel] Flexi-streams problem with SBCL 1.0.34 Message-ID: <19290c161001071229o29d18655i14b8e3db91da9241@mail.gmail.com> I found a problem with the flexi-streams:string-to-octets function. I was getting the following back-trace when using it: (ironclad:make-hmac (string-octets key) :sha1) from zs3:make-digester gives: The value #(55 98 50 52 104 75 97 65 50 54 ...) is not of type (SIMPLE-ARRAY (UNSIGNED-BYTE 8) (*)). [Condition of type TYPE-ERROR] Restarts: 0: [DEFAULT-DEBUGGER] Use default debugger. 1: [ABORT] Return to SLIME's top level. 2: [TERMINATE-THREAD] Terminate this thread (#) Backtrace: 0: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER (IRONCLAD::HMAC T)) # # SB-DEBUG::ARG-2 = : SB-DEBUG::ARG-3 = # SB-DEBUG::ARG-4 = : 1: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.|)) # # SB-DEBUG::ARG-1 = : SB-DEBUG::ARG-2 = : zs3's string-octets uses flexi-streams:string-to-octets The workaround I'm using for now is instead going to acl-compat's string-to-octets function from acl-excl.lisp: (defun string-to-octets (string &key (null-terminate t) (start 0) end mb-vector make-mb-vector? (external-format :default)) "This function returns a lisp-usb8-vector and the number of bytes copied." (declare (ignore external-format)) ;; The end parameter is different in ACL's lambda list, but this ;; variant lets us give an argument :end nil explicitly, and the ;; right thing will happen (unless end (setf end (length string))) (let* ((number-of-octets (if null-terminate (1+ (- end start)) (- end start))) (mb-vector (cond ((and mb-vector (>= (length mb-vector) number-of-octets)) mb-vector) ((or (not mb-vector) make-mb-vector?) (make-array (list number-of-octets) :element-type '(unsigned-byte 8) :initial-element 0)) (t (error "Was given a vector of length ~A, ~ but needed at least length ~A." (length mb-vector) number-of-octets))))) (declare (type (simple-array (unsigned-byte 8) (*)) mb-vector)) (loop for from-index from start below end for to-index upfrom 0 do (progn (setf (aref mb-vector to-index) (char-code (aref string from-index))))) (when null-terminate (setf (aref mb-vector (1- number-of-octets)) 0)) (values mb-vector number-of-octets))) which seems to work fine. My best, Fred Gibson Founder / Software Developer http://www.streamfocus.com (c)2010 Organon Technologies LLC From edi at agharta.de Thu Jan 7 21:28:44 2010 From: edi at agharta.de (Edi Weitz) Date: Thu, 7 Jan 2010 22:28:44 +0100 Subject: [flexi-streams-devel] Flexi-streams problem with SBCL 1.0.34 In-Reply-To: <19290c161001071229o29d18655i14b8e3db91da9241@mail.gmail.com> References: <19290c161001071229o29d18655i14b8e3db91da9241@mail.gmail.com> Message-ID: I'm not familiar with ironclad or zs3. Could you maybe send a full backtrace and a simple way to reproduce the problem? Thanks, Edi. On Thu, Jan 7, 2010 at 9:29 PM, Fred Gibson wrote: > I found a problem with the flexi-streams:string-to-octets function. ?I > was getting the following back-trace when using it: > > (ironclad:make-hmac (string-octets key) :sha1) ?from zs3:make-digester gives: > > The value #(55 98 50 52 104 75 97 65 50 54 ...) > is not of type > ?(SIMPLE-ARRAY (UNSIGNED-BYTE 8) (*)). > ?[Condition of type TYPE-ERROR] > > Restarts: > ?0: [DEFAULT-DEBUGGER] Use default debugger. > ?1: [ABORT] Return to SLIME's top level. > ?2: [TERMINATE-THREAD] Terminate this thread (# RUNNING {ACA40D1}>) > > Backtrace: > ?0: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER (IRONCLAD::HMAC > T)) # # ? ?Locals: > ? ? ?SB-DEBUG::ARG-0 = 10 > ? ? ?SB-DEBUG::ARG-1 = : > ? ? ?SB-DEBUG::ARG-2 = : > ? ? ?SB-DEBUG::ARG-3 = # > ? ? ?SB-DEBUG::ARG-4 = : > ?1: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.|)) > # # ? ?Locals: > ? ? ?SB-DEBUG::ARG-0 = : > ? ? ?SB-DEBUG::ARG-1 = : > ? ? ?SB-DEBUG::ARG-2 = : > > zs3's string-octets uses flexi-streams:string-to-octets > > The workaround I'm using for now is instead going to acl-compat's > string-to-octets function from acl-excl.lisp: > > (defun string-to-octets (string &key (null-terminate t) (start 0) > ? ? ? ? ? ? ? ? ? ? ? end mb-vector make-mb-vector? > ? ? ? ? ? ? ? ? ? ? ? (external-format :default)) > ?"This function returns a lisp-usb8-vector and the number of bytes copied." > ?(declare (ignore external-format)) > ?;; The end parameter is different in ACL's lambda list, but this > ?;; variant lets us give an argument :end nil explicitly, and the > ?;; right thing will happen > ?(unless end (setf end (length string))) > ?(let* ((number-of-octets (if null-terminate (1+ (- end start)) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? (- end start))) > ? ? ? (mb-vector (cond > ? ? ? ? ? ? ? ? ? ?((and mb-vector (>= (length mb-vector) number-of-octets)) > ? ? ? ? ? ? ? ? ? ? mb-vector) > ? ? ? ? ? ? ? ? ? ?((or (not mb-vector) make-mb-vector?) > ? ? ? ? ? ? ? ? ? ? (make-array (list number-of-octets) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? :element-type '(unsigned-byte 8) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? :initial-element 0)) > ? ? ? ? ? ? ? ? ? ?(t (error "Was given a vector of length ~A, ~ > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? but needed at least length ~A." > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?(length mb-vector) number-of-octets))))) > ?(declare (type (simple-array (unsigned-byte 8) (*)) mb-vector)) > ?(loop for from-index from start below end > ? ? for to-index upfrom 0 > ? ? do (progn > ? ? ? ? ?(setf (aref mb-vector to-index) > ? ? ? ? ? ? ? ?(char-code (aref string from-index))))) > ?(when null-terminate > ? ?(setf (aref mb-vector (1- number-of-octets)) 0)) > ?(values mb-vector number-of-octets))) > > which seems to work fine. > > My best, > > Fred Gibson > > Founder / Software Developer > http://www.streamfocus.com > > (c)2010 Organon Technologies LLC > > _______________________________________________ > flexi-streams-devel mailing list > flexi-streams-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel > > From fred at streamfocus.com Thu Jan 7 22:42:44 2010 From: fred at streamfocus.com (Fred Gibson) Date: Thu, 7 Jan 2010 14:42:44 -0800 Subject: [flexi-streams-devel] Flexi-streams problem with SBCL 1.0.34 In-Reply-To: References: <19290c161001071229o29d18655i14b8e3db91da9241@mail.gmail.com> Message-ID: <19290c161001071442n4e254cc9md7e40a64773dd130@mail.gmail.com> It looks like the problem is with the resultant data type although the printed representation is identical in both results: ZS3> (flexi-streams:STRING-TO-OCTETS "XXiXWAxW93c" :EXTERNAL-FORMAT :UTF-8) #(88 88 105 88 87 65 120 87 57 51 99) ZS3> (describe *) #(88 88 105 88 87 65 120 87 57 51 99) [specialized vector] Element-type: (UNSIGNED-BYTE 8) Fill-pointer: 11 Size: 15 Adjustable: yes Displaced-to: NIL Displaced-offset: 0 Storage vector: #<(SIMPLE-ARRAY (UNSIGNED-BYTE 8) (15)) {D0760BF}> ; No value ZS3> (acl-compat.excl:string-to-octets "XXiXWAxW93c" :EXTERNAL-FORMAT :UTF-8) 0: (SB-EXT:STRING-TO-OCTETS "XXiXWAxW93c" :EXTERNAL-FORMAT :UTF-8) 0: SB-EXT:STRING-TO-OCTETS returned #(88 88 105 88 87 65 120 87 57 51 99) #(88 88 105 88 87 65 120 87 57 51 99) ZS3> (describe *) #(88 88 105 88 87 65 120 87 57 51 99) [simple specialized vector] Element-type: (UNSIGNED-BYTE 8) Length: 11 ; No value I'm not sure which type is right, only that the acl-compat type is what's needed for ironclad and zs3 to run. On Thu, Jan 7, 2010 at 1:28 PM, Edi Weitz wrote: > I'm not familiar with ironclad or zs3. ?Could you maybe send a full > backtrace and a simple way to reproduce the problem? > > Thanks, > Edi. > > > On Thu, Jan 7, 2010 at 9:29 PM, Fred Gibson wrote: >> I found a problem with the flexi-streams:string-to-octets function. ?I >> was getting the following back-trace when using it: >> >> (ironclad:make-hmac (string-octets key) :sha1) ?from zs3:make-digester gives: >> >> The value #(55 98 50 52 104 75 97 65 50 54 ...) >> is not of type >> ?(SIMPLE-ARRAY (UNSIGNED-BYTE 8) (*)). >> ?[Condition of type TYPE-ERROR] >> >> Restarts: >> ?0: [DEFAULT-DEBUGGER] Use default debugger. >> ?1: [ABORT] Return to SLIME's top level. >> ?2: [TERMINATE-THREAD] Terminate this thread (#> RUNNING {ACA40D1}>) >> >> Backtrace: >> ?0: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER (IRONCLAD::HMAC >> T)) # #> ? ?Locals: >> ? ? ?SB-DEBUG::ARG-0 = 10 >> ? ? ?SB-DEBUG::ARG-1 = : >> ? ? ?SB-DEBUG::ARG-2 = : >> ? ? ?SB-DEBUG::ARG-3 = # >> ? ? ?SB-DEBUG::ARG-4 = : >> ?1: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.|)) >> # #> ? ?Locals: >> ? ? ?SB-DEBUG::ARG-0 = : >> ? ? ?SB-DEBUG::ARG-1 = : >> ? ? ?SB-DEBUG::ARG-2 = : >> >> zs3's string-octets uses flexi-streams:string-to-octets >> >> The workaround I'm using for now is instead going to acl-compat's >> string-to-octets function from acl-excl.lisp: >> >> (defun string-to-octets (string &key (null-terminate t) (start 0) >> ? ? ? ? ? ? ? ? ? ? ? end mb-vector make-mb-vector? >> ? ? ? ? ? ? ? ? ? ? ? (external-format :default)) >> ?"This function returns a lisp-usb8-vector and the number of bytes copied." >> ?(declare (ignore external-format)) >> ?;; The end parameter is different in ACL's lambda list, but this >> ?;; variant lets us give an argument :end nil explicitly, and the >> ?;; right thing will happen >> ?(unless end (setf end (length string))) >> ?(let* ((number-of-octets (if null-terminate (1+ (- end start)) >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? (- end start))) >> ? ? ? (mb-vector (cond >> ? ? ? ? ? ? ? ? ? ?((and mb-vector (>= (length mb-vector) number-of-octets)) >> ? ? ? ? ? ? ? ? ? ? mb-vector) >> ? ? ? ? ? ? ? ? ? ?((or (not mb-vector) make-mb-vector?) >> ? ? ? ? ? ? ? ? ? ? (make-array (list number-of-octets) >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? :element-type '(unsigned-byte 8) >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? :initial-element 0)) >> ? ? ? ? ? ? ? ? ? ?(t (error "Was given a vector of length ~A, ~ >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? but needed at least length ~A." >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?(length mb-vector) number-of-octets))))) >> ?(declare (type (simple-array (unsigned-byte 8) (*)) mb-vector)) >> ?(loop for from-index from start below end >> ? ? for to-index upfrom 0 >> ? ? do (progn >> ? ? ? ? ?(setf (aref mb-vector to-index) >> ? ? ? ? ? ? ? ?(char-code (aref string from-index))))) >> ?(when null-terminate >> ? ?(setf (aref mb-vector (1- number-of-octets)) 0)) >> ?(values mb-vector number-of-octets))) >> >> which seems to work fine. >> >> My best, >> >> Fred Gibson >> >> Founder / Software Developer >> http://www.streamfocus.com >> >> (c)2010 Organon Technologies LLC >> >> _______________________________________________ >> flexi-streams-devel mailing list >> flexi-streams-devel at common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel >> >> > > _______________________________________________ > flexi-streams-devel mailing list > flexi-streams-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel > -- Fred Gibson Founder / Software Developer http://www.streamfocus.com (c)2010 Organon Technologies LLC From edi at agharta.de Thu Jan 7 23:17:00 2010 From: edi at agharta.de (Edi Weitz) Date: Fri, 8 Jan 2010 00:17:00 +0100 Subject: [flexi-streams-devel] Flexi-streams problem with SBCL 1.0.34 In-Reply-To: <19290c161001071442n4e254cc9md7e40a64773dd130@mail.gmail.com> References: <19290c161001071229o29d18655i14b8e3db91da9241@mail.gmail.com> <19290c161001071442n4e254cc9md7e40a64773dd130@mail.gmail.com> Message-ID: Yeah, flexi-streams might returned vectors with fill pointers. But it doesn't claim otherwise. Looks like this is a bug on the side of the caller. Cheers, Edi. On Thu, Jan 7, 2010 at 11:42 PM, Fred Gibson wrote: > It looks like the problem is with the resultant data type although the > printed representation is identical in both results: > > ZS3> (flexi-streams:STRING-TO-OCTETS "XXiXWAxW93c" :EXTERNAL-FORMAT :UTF-8) > #(88 88 105 88 87 65 120 87 57 51 99) > ZS3> (describe *) > #(88 88 105 88 87 65 120 87 57 51 99) > ?[specialized vector] > > Element-type: (UNSIGNED-BYTE 8) > Fill-pointer: 11 > Size: 15 > Adjustable: yes > Displaced-to: NIL > Displaced-offset: 0 > Storage vector: #<(SIMPLE-ARRAY (UNSIGNED-BYTE 8) (15)) {D0760BF}> > ; No value > ZS3> (acl-compat.excl:string-to-octets "XXiXWAxW93c" :EXTERNAL-FORMAT :UTF-8) > ?0: (SB-EXT:STRING-TO-OCTETS "XXiXWAxW93c" :EXTERNAL-FORMAT :UTF-8) > ?0: SB-EXT:STRING-TO-OCTETS returned #(88 88 105 88 87 65 120 87 57 51 99) > #(88 88 105 88 87 65 120 87 57 51 99) > ZS3> (describe *) > #(88 88 105 88 87 65 120 87 57 51 99) > ?[simple specialized vector] > > Element-type: (UNSIGNED-BYTE 8) > Length: 11 > ; No value > > I'm not sure which type is right, only that the acl-compat type is > what's needed for ironclad and zs3 to run. > > On Thu, Jan 7, 2010 at 1:28 PM, Edi Weitz wrote: >> I'm not familiar with ironclad or zs3. ?Could you maybe send a full >> backtrace and a simple way to reproduce the problem? >> >> Thanks, >> Edi. >> >> >> On Thu, Jan 7, 2010 at 9:29 PM, Fred Gibson wrote: >>> I found a problem with the flexi-streams:string-to-octets function. ?I >>> was getting the following back-trace when using it: >>> >>> (ironclad:make-hmac (string-octets key) :sha1) ?from zs3:make-digester gives: >>> >>> The value #(55 98 50 52 104 75 97 65 50 54 ...) >>> is not of type >>> ?(SIMPLE-ARRAY (UNSIGNED-BYTE 8) (*)). >>> ?[Condition of type TYPE-ERROR] >>> >>> Restarts: >>> ?0: [DEFAULT-DEBUGGER] Use default debugger. >>> ?1: [ABORT] Return to SLIME's top level. >>> ?2: [TERMINATE-THREAD] Terminate this thread (#>> RUNNING {ACA40D1}>) >>> >>> Backtrace: >>> ?0: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER (IRONCLAD::HMAC >>> T)) # #>> ? ?Locals: >>> ? ? ?SB-DEBUG::ARG-0 = 10 >>> ? ? ?SB-DEBUG::ARG-1 = : >>> ? ? ?SB-DEBUG::ARG-2 = : >>> ? ? ?SB-DEBUG::ARG-3 = # >>> ? ? ?SB-DEBUG::ARG-4 = : >>> ?1: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.|)) >>> # #>> ? ?Locals: >>> ? ? ?SB-DEBUG::ARG-0 = : >>> ? ? ?SB-DEBUG::ARG-1 = : >>> ? ? ?SB-DEBUG::ARG-2 = : >>> >>> zs3's string-octets uses flexi-streams:string-to-octets >>> >>> The workaround I'm using for now is instead going to acl-compat's >>> string-to-octets function from acl-excl.lisp: >>> >>> (defun string-to-octets (string &key (null-terminate t) (start 0) >>> ? ? ? ? ? ? ? ? ? ? ? end mb-vector make-mb-vector? >>> ? ? ? ? ? ? ? ? ? ? ? (external-format :default)) >>> ?"This function returns a lisp-usb8-vector and the number of bytes copied." >>> ?(declare (ignore external-format)) >>> ?;; The end parameter is different in ACL's lambda list, but this >>> ?;; variant lets us give an argument :end nil explicitly, and the >>> ?;; right thing will happen >>> ?(unless end (setf end (length string))) >>> ?(let* ((number-of-octets (if null-terminate (1+ (- end start)) >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? (- end start))) >>> ? ? ? (mb-vector (cond >>> ? ? ? ? ? ? ? ? ? ?((and mb-vector (>= (length mb-vector) number-of-octets)) >>> ? ? ? ? ? ? ? ? ? ? mb-vector) >>> ? ? ? ? ? ? ? ? ? ?((or (not mb-vector) make-mb-vector?) >>> ? ? ? ? ? ? ? ? ? ? (make-array (list number-of-octets) >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? :element-type '(unsigned-byte 8) >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? :initial-element 0)) >>> ? ? ? ? ? ? ? ? ? ?(t (error "Was given a vector of length ~A, ~ >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? but needed at least length ~A." >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?(length mb-vector) number-of-octets))))) >>> ?(declare (type (simple-array (unsigned-byte 8) (*)) mb-vector)) >>> ?(loop for from-index from start below end >>> ? ? for to-index upfrom 0 >>> ? ? do (progn >>> ? ? ? ? ?(setf (aref mb-vector to-index) >>> ? ? ? ? ? ? ? ?(char-code (aref string from-index))))) >>> ?(when null-terminate >>> ? ?(setf (aref mb-vector (1- number-of-octets)) 0)) >>> ?(values mb-vector number-of-octets))) >>> >>> which seems to work fine. >>> >>> My best, >>> >>> Fred Gibson >>> >>> Founder / Software Developer >>> http://www.streamfocus.com >>> >>> (c)2010 Organon Technologies LLC >>> >>> _______________________________________________ >>> flexi-streams-devel mailing list >>> flexi-streams-devel at common-lisp.net >>> http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel >>> >>> >> >> _______________________________________________ >> flexi-streams-devel mailing list >> flexi-streams-devel at common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel >> > > > > -- > Fred Gibson > > Founder / Software Developer > http://www.streamfocus.com > > (c)2010 Organon Technologies LLC > > _______________________________________________ > flexi-streams-devel mailing list > flexi-streams-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel > >