From quasilists at gmail.com  Thu Apr  5 13:37:40 2007
From: quasilists at gmail.com (quasi)
Date: Thu, 05 Apr 2007 19:07:40 +0530
Subject: [flexi-streams-devel] 0.11.1 trouble.
Message-ID: <4614FBA4.2030309@gmail.com>

Hey,

I am running Hunchentoot 8.1.  As soon as I upgraded to flexi-streams 
0.11.1 I have encountered a weird problem.  Hunchentoot (or anything 
else) threw no errors but the socket was getting closed unexpectedly I 
think, because links cribbed "error reading socket" or something.  I am 
running Allegro 8.0 Enterprise 64bit on Redhat Linux. (problem also 
persisted on Allegro 8.0 for the Intel Mac's)

As soon as I went back to 0.8.0 (my previous version) all worked as before.

Am I doing something wrong here ?  How can I help debug this more ?

thanks,
quasi


From edi at agharta.de  Thu Apr  5 13:49:50 2007
From: edi at agharta.de (Edi Weitz)
Date: Thu, 05 Apr 2007 15:49:50 +0200
Subject: [flexi-streams-devel] 0.11.1 trouble.
In-Reply-To: <4614FBA4.2030309@gmail.com> (quasi's message of "Thu, 05 Apr
	2007 19:07:40 +0530")
References: <4614FBA4.2030309@gmail.com>
Message-ID: <u4pnvylq9.fsf@agharta.de>

On Thu, 05 Apr 2007 19:07:40 +0530, quasi <quasilists at gmail.com> wrote:

> I am running Hunchentoot 8.1.

0.8.1 I hope... :)

> As soon as I upgraded to flexi-streams 0.11.1 I have encountered a
> weird problem.  Hunchentoot (or anything else) threw no errors but
> the socket was getting closed unexpectedly I think, because links
> cribbed "error reading socket" or something.  I am running Allegro
> 8.0 Enterprise 64bit on Redhat Linux. (problem also persisted on
> Allegro 8.0 for the Intel Mac's)
>
> As soon as I went back to 0.8.0 (my previous version) all worked as
> before.
>
> Am I doing something wrong here ?

I doubt it.  There were significant changes between 0.8.0 and 0.11.1
and it is likely that you're the first one to use the new version on
AllegroCL, especially on a 64-bit version (if that matters, don't
know).

There's also a chance that some of my recent Hunchentoot changes which
I hastily submitted in the wake of my ILC preparations broke something
which results in sockets being closed although they shouldn't.  That's
why I'm sending a copy of this to tbnl-devel as well.

> How can I help debug this more ?

Try to get a detailed backtrace of the error.  You might want to check
what the previous page (before the socket was closed) was supposed to
send (return code, size of content, headers, transfer encoding).

Also, delete the FASL files from FLEXI-STREAMS and Hunchentoot,
recompile, and check all the warnings you might get during
compilation.

Thanks,
Edi.


From quasilists at gmail.com  Thu Apr  5 15:30:03 2007
From: quasilists at gmail.com (quasi)
Date: Thu, 05 Apr 2007 21:00:03 +0530
Subject: [flexi-streams-devel] 0.11.1 trouble.
In-Reply-To: <u4pnvylq9.fsf@agharta.de>
References: <4614FBA4.2030309@gmail.com> <u4pnvylq9.fsf@agharta.de>
Message-ID: <461515FB.4060800@gmail.com>

Edi Weitz wrote:
> On Thu, 05 Apr 2007 19:07:40 +0530, quasi <quasilists at gmail.com> wrote:
>
>   
>> I am running Hunchentoot 8.1.
>>     
>
> 0.8.1 I hope... :)
>   
my bad. :) it is indeed 0.8.1
>   
>> As soon as I upgraded to flexi-streams 0.11.1 I have encountered a
>> weird problem.  Hunchentoot (or anything else) threw no errors but
>> the socket was getting closed unexpectedly I think, because links
>> cribbed "error reading socket" or something.  I am running Allegro
>> 8.0 Enterprise 64bit on Redhat Linux. (problem also persisted on
>> Allegro 8.0 for the Intel Mac's)
>>
>> As soon as I went back to 0.8.0 (my previous version) all worked as
>> before.
>>
>> Am I doing something wrong here ?
>>     
>
> I doubt it.  There were significant changes between 0.8.0 and 0.11.1
> and it is likely that you're the first one to use the new version on
> AllegroCL, especially on a 64-bit version (if that matters, don't
> know).
>   
I dont think it matters.  Observed on 32bit ACL 8.0 on the Intel Mac and 
64bit ACL 8.0 on RH Linux.
> There's also a chance that some of my recent Hunchentoot changes which
> I hastily submitted in the wake of my ILC preparations broke something
> which results in sockets being closed although they shouldn't.  That's
> why I'm sending a copy of this to tbnl-devel as well.
>
>   
>> How can I help debug this more ?
>>     
>
> Try to get a detailed backtrace of the error.  You might want to check
> what the previous page (before the socket was closed) was supposed to
> send (return code, size of content, headers, transfer encoding).
>
> Also, delete the FASL files from FLEXI-STREAMS and Hunchentoot,
> recompile, and check all the warnings you might get during
> compilation.
>   

Ya I tried a complete recompile. No change.  Also on another machine 
which had 0.9.1 and it worked.  With 0.11.1 compile gave only 2 warnings.

;;;   /Users/quasi/quasiLabs/lisp-libraries/flexi-streams-0.11.1/stream.lisp
; While compiling (METHOD SET-ENCODING-TABLE (T)):
Warning: Variable STREAM is never used.
; While compiling (METHOD SET-ENCODING-HASH (T)):
Warning: Variable STREAM is never used.


Problem is there are no errors thrown at all, so what do I do?
> Thanks,
> Edi.
>
>   



From edi at agharta.de  Thu Apr  5 16:41:16 2007
From: edi at agharta.de (Edi Weitz)
Date: Thu, 05 Apr 2007 18:41:16 +0200
Subject: [hunchentoot-devel] Re: [flexi-streams-devel] 0.11.1 trouble.
In-Reply-To: <461515FB.4060800@gmail.com> (quasi's message of "Thu, 05 Apr
	2007 21:00:03 +0530")
References: <4614FBA4.2030309@gmail.com> <u4pnvylq9.fsf@agharta.de>
	<461515FB.4060800@gmail.com>
Message-ID: <u3b3eydsj.fsf@agharta.de>

On Thu, 05 Apr 2007 21:00:03 +0530, quasi <quasilists at gmail.com> wrote:

> Problem is there are no errors thrown at all, so what do I do?

Did you try with *CATCH-ERRORS-P* set to NIL?


From quasilists at gmail.com  Thu Apr  5 16:58:00 2007
From: quasilists at gmail.com (quasi)
Date: Thu, 05 Apr 2007 22:28:00 +0530
Subject: [hunchentoot-devel] Re: [flexi-streams-devel] 0.11.1 trouble.
In-Reply-To: <u3b3eydsj.fsf@agharta.de>
References: <4614FBA4.2030309@gmail.com>
	<u4pnvylq9.fsf@agharta.de>	<461515FB.4060800@gmail.com>
	<u3b3eydsj.fsf@agharta.de>
Message-ID: <46152A98.7090303@gmail.com>

Edi Weitz wrote:
> On Thu, 05 Apr 2007 21:00:03 +0530, quasi <quasilists at gmail.com> wrote:
>
>   
>> Problem is there are no errors thrown at all, so what do I do?
>>     
>
> Did you try with *CATCH-ERRORS-P* set to NIL?
>
>   
OOPS.  Did'nt know about that.  I was dependent on the 
*show-lisp-errors-p*. :( Have to find time to RTF[M|D].

Well the backtrace is here.

`NIL' is not of the expected type `REAL'
   [Condition of type TYPE-ERROR]

Restarts:
  0: [ABORT] Abort entirely from this (lisp) process.

Backtrace:
  0: (SWANK::DEBUG-IN-EMACS #<TYPE-ERROR @ #x1111bdba>)
...
  8: (INVOKE-DEBUGGER #<TYPE-ERROR @ #x1111bdba>)
  9: (SIGNAL #<TYPE-ERROR @ #x1111bdba>)
 10: (ERROR TYPE-ERROR :DATUM NIL :EXPECTED-TYPE REAL :FORMAT-CONTROL 
"~@<`~s' is not of the expected type `~s'~:@>" :FORMAT-ARGUMENTS (NIL REAL))
 11: ((METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE 
(FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM STRING T T)) 
#<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x111199ca> 
"<html><head><title>Hunchentoot</title></head><body>...</body></html>" 0 
NIL)
      Locals:
        #:STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ 
#x111199ca>
        #:SEQUENCE919 = 
"<html><head><title>Hunchentoot</title></head><body>...</body></html>"
        #:START920 = 0
        #:END921 = NIL
        #:AMPERSAND-ARGS = NIL
        #:AMPERSAND-ARGS = NIL
        #:|g919| = 
"<html><head><title>Hunchentoot</title></head><body>...</p></body></html>"
        #:|g921| = NIL
        #:|g920| = 0
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x111199ca>
        #:|g922| = #(153 156 17 17 133 6 0 16 153 156 ...)
        #:|g923| = 0
        #:|g926| = NIL
        #:|g927| = #<non-lisp object @ #x5>
        #:DEST-PTR925 = -268442126
        #:SRC-PTR924 = #\null
        #:|g932| = ((EXCL:PIPE-STREAM . :STREAMP))
        #:STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ 
#x111199ca>
        #:|g919| = 
"<html><head><title>Hunchentoot</title></head><body><h2>H...</p></body></html>"
        #:|g961| = (8192)
        #:|g928| = -2
        EXCL::.NEXT-METHOD. = #<Function (METHOD 
TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE 
(FLEXI-STREAMS::FLEXI-8-BIT-OUTPUT-STREAM STRING T T))>
        #:|g929| = #<non-lisp object @ #x1>
        OCTET = #<STANDARD-GENERIC-FUNCTION INITIALIZE-INSTANCE>
        NIL = #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE 
:LF) @ #x10f07d32>
        #:|g963| = :FLEXI-STREAM-EXTERNAL-FORMAT
        CHAR = #<FLEXI-STREAMS::VECTOR-OUTPUT-STREAM @ #x111198e2>
        NIL = :STREAM
        NIL = #<STANDARD-CLASS FLEXI-OUTPUT-STREAM>
        #:|g964| = #<non-lisp object @ #x459ca1>
        EXCL::.NEXT-METHODS. = (#<Function (METHOD 
TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE (FLEXI-OUTPUT-STREAM T T T))>)


From edi at agharta.de  Thu Apr  5 23:57:37 2007
From: edi at agharta.de (Edi Weitz)
Date: Fri, 06 Apr 2007 01:57:37 +0200
Subject: [flexi-streams-devel] New release 0.11.2   (Was: 0.11.1 trouble)
In-Reply-To: <46152A98.7090303@gmail.com> (quasi's message of "Thu, 05 Apr
	2007 22:28:00 +0530")
References: <4614FBA4.2030309@gmail.com> <u4pnvylq9.fsf@agharta.de>
	<461515FB.4060800@gmail.com> <u3b3eydsj.fsf@agharta.de>
	<46152A98.7090303@gmail.com>
Message-ID: <u1wiyv0ge.fsf_-_@agharta.de>

On Thu, 05 Apr 2007 22:28:00 +0530, quasi <quasilists at gmail.com> wrote:

>  10: (ERROR TYPE-ERROR :DATUM NIL :EXPECTED-TYPE REAL :FORMAT-CONTROL
>  "~@<`~s' is not of the expected type `~s'~:@>" :FORMAT-ARGUMENTS (NIL
>  REAL))

Ah, OK, that was a bug only AllegroCL complained about it seems.
Should be fixed in 0.11.2.

Thanks,
Edi.


From michaelw+flexi at foldr.org  Sat Apr 21 10:20:50 2007
From: michaelw+flexi at foldr.org (Michael Weber)
Date: Sat, 21 Apr 2007 12:20:50 +0200
Subject: [flexi-streams-devel] Programmatic classes scrap boilerplate code
Message-ID: <B345172F-86E9-4600-9842-D643885D38AE@foldr.org>

Hi Edi,

I looked at flexi-streams' stream.lisp some time ago, and there seems  
to be a lot of boilerplate code to combine various mixins to specific  
stream classes.  Then, there seems to be code for parsing and  
combining what basically amounts to mixin designators, which are then  
again mapped on the specific stream classes.

The underlying issue is not new.  AMOP describes a nice way around it  
which creates classes composed from several mixins programmatically.   
About a year or so ago,  Christophe Rhodes fixed SBCL so that it  
accepts generalized class names.  The support code is little, you can  
find it here (usage example at the end of this mail):
http://www.foldr.org/~michaelw/lisp/amop-programmatic-class.lisp

I can't tell how well this is supported in Lispworks and other  
implementations, but it might be worth opening trouble tickets if  
it's not. :)

OTOH, I understand the cost that comes with maintaining special-cased  
code for non-supporting implementations.  It might not be worth it.

Still, I felt like sharing this, before I forget again. :)


Cheers,
Michael

;;; Example code:
(progn
   (defclass shape () ())
   (defclass circle (shape) ())

   (defclass color () ())
   (defclass orange (color) ())
   (defclass mangenta (color) ())

   (defclass label-type () ())
   (defclass top-labeled (label-type) ())
   (defclass bottom-labeled (label-type) ()))
#||
===> (class-direct-subclasses (find-class 'circle))
()
===> (defparameter *i1* (make-programmatic-instance '(circle orange  
top-labeled)))
===> (defparameter *i2* (make-programmatic-instance '(circle mangenta  
bottom-labeled)))
===> (defparameter *i3* (make-programmatic-instance '(circle orange  
top-labeled)))
===> (class-direct-subclasses (find-class 'circle))
(#<Standard-Class (CIRCLE MAGENTA BOTTOM-LABELED) >
#<Standard-Class (CIRCLE ORANGE TOP-LABELED) >)
||#


From vodonosov at mail.ru  Sat Apr 21 18:33:02 2007
From: vodonosov at mail.ru (Vodonosov Anton)
Date: Sat, 21 Apr 2007 22:33:02 +0400
Subject: =?koi8-r?Q?Re=3A_[flexi-streams-devel]_New_release_0.11.1?=
In-Reply-To: <u8xdphrek.fsf@agharta.de>
References: <u8xdphrek.fsf@agharta.de>
Message-ID: <E1HfKOM-0009nS-00.vodonosov-mail-ru@f66.mail.ru>

Hi!

Edi, as far as I understand external format 
with multibyte encodings and cr-lf style newlines
are not optimized because its difficult to predict
number of characters that will fit into buffer.

We can solve it if we will always have few reserved
bytes in buffer. 20 will be sufficient for any encoding.
I.e. loop while we have at least 20 free bytes in buffer.

This solution is implemented in the attached patch (against 0.11.2).

The patch also contains some additions in tests. As for
separate tests for READ-SEQUENCE/WRITE-SEQUENCE,
maybe thay are useless - I noticed that, at least in CLISP, 
WRITE-LINE is implemented using WRITE-SEQUENCE. So in 
case of errors in WRITE-SEQUENCE both WRITE-LINE and 
WRITE-SEQUENCE tests fail.

In regard to the way I reused existing WRITE-CHAR code
in STREAM-WRITE-SEQUENCE. I do not like mach working 
throught temporary stream, for example we have redundant 
slot access in WRITE-BYTE*. I've tried to keep changes small 
and not disturb other code. Maybe with some refactoring 
it will be possible to have more clean and efficient code.

For example WRITE-CHAR code will not use WRITE-BYTE* directly
but use instead some BYTE-WRITER-FUN passed as a parameter.
Also external format may be distinguished as a separate 
entity responsible for byte/character conversions.

BTW, I've started changing STREAM-WRITE-SEQUENCE with the 
version provided below. It is more efficient, but it isn't
thread safe.

(defmacro dyna-let-f-global (symbol func-to-bind &body body)
  "Something similar to FLET for global functions but with dynamic extent"
  `(let ((old-fdef (fdefinition ,symbol)))
    (unwind-protect (progn 
              (setf (fdefinition ,symbol) ,func-to-bind)
              , at body)
      (setf (fdefinition ,symbol) old-fdef))))

(defmethod stream-write-sequence ((stream flexi-output-stream) (sequence string) start end &key)
  (declare (optimize speed) (type (integer 0 *) start end fill-pointer src-ptr))
; (declare (optimize speed) (type (integer 0 *) start end))
  (let ((buffer (make-array (+ +buffer-size+ 20) :element-type 'octet)))
    ;; use repeated calls to WRITE-SEQUENCE for arrays of octets
    (loop with src-ptr = start 
      while (< src-ptr end)
      do
      (let ((fill-pointer 0))
        (dyna-let-f-global 'write-byte* (lambda (byte stream) 
                          (declare (ignore stream) (type (integer 0 *) fill-pointer) (type octet byte)) 
                          (setf (aref buffer fill-pointer) byte) (incf fill-pointer))
          (loop while (and (< src-ptr end) (< fill-pointer +buffer-size+))
            do (stream-write-char stream (aref sequence src-ptr)) (incf src-ptr)))
        (write-sequence buffer (flexi-stream-stream stream)
                :start 0 :end fill-pointer))))
  sequence)

Here are some performance tests. 
Tested in CLISP on Windows XP.

(asdf:operate 'asdf:load-op :flexi-streams)
(asdf:oos 'asdf:test-op :flexi-streams)

(defparameter long-str "")
;; populate it with some Russian characters
(dotimes (i 10)  
  (setf long-str 
	(concatenate 'string 
		     long-str 
		     "?????????? ??????? ?????? ?? ??????? ?????. 
?? ????? ?????????? ??????? ??? ????????")))

(defun time-test(external-format)
  (with-open-file (stream "/cygdrive/c/usr/projects/flexi-dev/test-output.txt" 
			  :direction :output 
			  :element-type '(unsigned-byte 8) 
			  :buffered nil) ; :buffered is a CLISP extension
    (with-open-stream (fstream (flex:make-flexi-stream stream 
						       :external-format external-format))
      (loop for i from 0 below 200 do
	    (stream-write-sequence long-str fstream)))))


;; original flexi-streams-0.11.2 
;; =============================

(time (time-test :utf-8))
;; Real time: 12.178 sec.
;; Run time: 12.093 sec.
;; Space: 430660 Bytes
;; GC: 1, GC time: 0.015 sec.

(time (time-test :koi8-r))
;; Real time: 0.246 sec.
;; Run time: 0.25 sec.
;; Space: 1820584 Bytes
;; GC: 2, GC time: 0.048 sec.

;; with temp flexi stream
;; ======================

(time (time-test :utf-8))
;; Real time: 1.08 sec.
;; Run time: 1.079 sec.
;; Space: 1791632 Bytes
;; GC: 2, GC time: 0.063 sec.

(time (time-test :koi8-r))
;; Real time: 0.861 sec.
;; Run time: 0.875 sec.
;; Space: 1795636 Bytes
;; GC: 2, GC time: 0.048 sec.

;; with dyna-let-f-global
;; ======================

(time (time-test :utf-8))
;; Real time: 0.639 sec.
;; Run time: 0.641 sec.
;; Space: 1734832 Bytes
;; GC: 2, GC time: 0.062 sec.

(time (time-test :koi8-r))
;; Real time: 0.567 sec.
;; Run time: 0.563 sec.
;; Space: 1734836 Bytes
;; GC: 2, GC time: 0.062 sec.

Best regards,
-Anton

-----Original Message-----
From: Edi Weitz <edi at agharta.de>
To: flexi-streams-devel at common-lisp.net
Date: Thu, 22 Mar 2007 22:58:27 +0100
Subject: [flexi-streams-devel] New release 0.11.1

> 
> ChangeLog:
> 
>   Version 0.11.1
>   2007-03-22
>   More ugliness for a bit of output performance in special cases
> 
> Download:
> 
>   http://weitz.de/files/flexi-streams.tar.gz
> 
> Cheers,
> Edi.
> _______________________________________________
> flexi-streams-devel mailing list
> flexi-streams-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: flexi-patch.diff
Type: application/octet-stream
Size: 15340 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/flexi-streams-devel/attachments/20070421/d1572c25/attachment.obj>

From edi at agharta.de  Tue Apr 24 08:50:05 2007
From: edi at agharta.de (Edi Weitz)
Date: Tue, 24 Apr 2007 10:50:05 +0200
Subject: [flexi-streams-devel] Programmatic classes scrap boilerplate code
In-Reply-To: <B345172F-86E9-4600-9842-D643885D38AE@foldr.org> (Michael
	Weber's message of "Sat, 21 Apr 2007 12:20:50 +0200")
References: <B345172F-86E9-4600-9842-D643885D38AE@foldr.org>
Message-ID: <uk5w22m2q.fsf@agharta.de>

Hi Michael,

On Sat, 21 Apr 2007 12:20:50 +0200, Michael Weber <michaelw+flexi at foldr.org> wrote:

> I looked at flexi-streams' stream.lisp some time ago, and there
> seems to be a lot of boilerplate code to combine various mixins to
> specific stream classes.  Then, there seems to be code for parsing
> and combining what basically amounts to mixin designators, which are
> then again mapped on the specific stream classes.

Yes, unfortunately... :(

If you look at older versions[1], you'll see that FLEXI-STREAMS' code
was quite clear and elegant until 0.8.0, at least compared to the mess
it is now.  But, alas, it was simply too slow for several things and
the changes in 0.9.0 and afterwards have made it significantly faster
in many important areas (although of course it still can't compare to
direct stream access).

> The underlying issue is not new.  AMOP describes a nice way around
> it which creates classes composed from several mixins
> programmatically.  About a year or so ago, Christophe Rhodes fixed
> SBCL so that it accepts generalized class names.  The support code
> is little, you can find it here (usage example at the end of this
> mail):
> http://www.foldr.org/~michaelw/lisp/amop-programmatic-class.lisp
>
> I can't tell how well this is supported in Lispworks and other
> implementations, but it might be worth opening trouble tickets if
> it's not. :)
>
> OTOH, I understand the cost that comes with maintaining
> special-cased code for non-supporting implementations.  It might not
> be worth it.
>
> Still, I felt like sharing this, before I forget again. :)

Thanks for the info.  That's interesting, but certainly not something
I'll lose sleep about in the near future.  Adding something like this
before at least all major implementations support it, looks like a
nightmare to me.  I think I'm too lazy and pragmatic for that... :)

Cheers,
Edi.


[1] http://arcanes.fr.eu.org/~pierre/2007/02/weitz/


From edi at agharta.de  Fri Apr 27 10:20:23 2007
From: edi at agharta.de (Edi Weitz)
Date: Fri, 27 Apr 2007 12:20:23 +0200
Subject: [flexi-streams-devel] New release 0.11.1
In-Reply-To: <E1HfKOM-0009nS-00.vodonosov-mail-ru@f66.mail.ru> (Vodonosov
	Anton's message of "Sat, 21 Apr 2007 22:33:02 +0400")
References: <u8xdphrek.fsf@agharta.de>
	<E1HfKOM-0009nS-00.vodonosov-mail-ru@f66.mail.ru>
Message-ID: <utzv2p194.fsf@agharta.de>

Hi Anton,

sorry for the delay.  I have to think about this a bit more, but
here's some preliminary feedback:

On Sat, 21 Apr 2007 22:33:02 +0400, Vodonosov Anton <vodonosov at mail.ru> wrote:

> Edi, as far as I understand external format with multibyte encodings
> and cr-lf style newlines are not optimized because its difficult to
> predict number of characters that will fit into buffer.
>
> We can solve it if we will always have few reserved bytes in
> buffer. 20 will be sufficient for any encoding.  I.e. loop while we
> have at least 20 free bytes in buffer.
>
> This solution is implemented in the attached patch (against 0.11.2).

Thanks.  The win for UTF-8 is impressive, but I'm a bit concerned that
you'll lose a lot of performance for 8-bit encodings.  I think it'd be
better to leave the 8-bit version in there and only use your code for
other strings.

> The patch also contains some additions in tests. As for separate
> tests for READ-SEQUENCE/WRITE-SEQUENCE, maybe thay are useless - I
> noticed that, at least in CLISP, WRITE-LINE is implemented using
> WRITE-SEQUENCE. So in case of errors in WRITE-SEQUENCE both
> WRITE-LINE and WRITE-SEQUENCE tests fail.

Thanks, more tests are always good.

> In regard to the way I reused existing WRITE-CHAR code in
> STREAM-WRITE-SEQUENCE. I do not like mach working throught temporary
> stream, for example we have redundant slot access in
> WRITE-BYTE*. I've tried to keep changes small and not disturb other
> code. Maybe with some refactoring it will be possible to have more
> clean and efficient code.

Yes, the more we optimize for performance, the more of a mess it
becomes... :(

> BTW, I've started changing STREAM-WRITE-SEQUENCE with the version
> provided below. It is more efficient, but it isn't thread safe.

Hmmm...

A I said, I have to ponder this a bit more, but unfortunately I'm too
busy right now.  More later.

Cheers,
Edi.


From vodonosov at mail.ru  Sun Apr 29 18:43:44 2007
From: vodonosov at mail.ru (Vodonosov Anton)
Date: Sun, 29 Apr 2007 22:43:44 +0400
Subject: =?koi8-r?Q?Re[2]=3A_[flexi-streams-devel]_New_release_0.11.1?=
In-Reply-To: <utzv2p194.fsf@agharta.de>
References: <utzv2p194.fsf@agharta.de>
Message-ID: <E1HiEN6-000GT2-00.vodonosov-mail-ru@f49.mail.ru>

Hello Edi

> sorry for the delay.  
> ... i'm too busy right now.  

No problem, I'm not hurry.

I have a new version, with better performance (see attached 
diff of output.lisp against 0.11.2).

Results for the same test:

;; original flexi-streams-0.11.2 
;; =============================

(time (time-test :utf-8))
;; Real time: 12.178 sec.
;; Run time: 12.093 sec.
;; Space: 430660 Bytes
;; GC: 1, GC time: 0.015 sec.

(time (time-test :koi8-r))
;; Real time: 0.246 sec.
;; Run time: 0.25 sec.
;; Space: 1820584 Bytes
;; GC: 2, GC time: 0.048 sec.

;; with char-to-octets
;; ===================
 
(time (time-test :utf-8))
;; Real time: 0.328 sec.
;; Run time: 0.328 sec.
;; Space: 1728432 Bytes
;; GC: 2, GC time: 0.047 sec.

(time (time-test :koi8-r))
;; Real time: 0.323 sec.
;; Run time: 0.297 sec.
;; Space: 1728436 Bytes
;; GC: 2, GC time: 0.063 sec.

Surprising is that utf-8 works with almost equal efficiency as 
8-bit encoding. 

I think the difference in performance for 8-bit between this version 
and the original 0.11.2 is caused by two function calls instead of inlined 
character to bytes conversion (CLOS dispatch of CHAR-TO-OCTETS which in 
turn calls OCTET-SINK passed to it).

> I have to think about this a bit more 
> ...
> The win for UTF-8 is impressive, but I'm a bit concerned that
> you'll lose a lot of performance for 8-bit encodings. I think it'd be
> better to leave the 8-bit version in there and only use your code for
> other strings. 

It's up to you. I hope it will be possible to inline code as you do in 0.11.2, 
maybe redefining CHAR-TO-OCTETS methods as macros. 

Maybe I'll try it when I have some spare time.

Regards,
-Anton

-------------- next part --------------
A non-text attachment was scrubbed...
Name: output.lisp.diff
Type: application/octet-stream
Size: 12928 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/flexi-streams-devel/attachments/20070429/de1c2ea2/attachment.obj>