From marc.battyani at fractalconcept.com Fri Apr 2 20:04:59 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Fri, 2 Apr 2004 22:04:59 +0200 Subject: [cl-typesetting-devel] cl-pdf and cl-typesetting repositories merged Message-ID: <03f401c418ed$c7cdb700$0303a8c0@marc2> Hello, As cl-typesetting depends heavily on cl-pdf, I merged the cl-pdf and cl-typesetting repositories into a single repository. The full repository is here: http://www.fractalconcept.com:8000/open-source/ The cl-pdf part: http://www.fractalconcept.com:8000/open-source/cl-pdf/ The cl-typesetting part: http://www.fractalconcept.com:8000/open-source/cl-typesetting/ I hope to be able to make a new release of cl-typesetting next week with all the stuff added by Dmitri Ivanov(col-span and splitable tables, document layout, etc.). Though all this is already in the repository. Marc From david.cooper at genworks.com Sat Apr 3 06:14:34 2004 From: david.cooper at genworks.com (David J Cooper Jr) Date: Sat, 3 Apr 2004 01:14:34 -0500 Subject: [cl-typesetting-devel] memory not getting released Message-ID: <200404030614.i336EYks007163@mccarthy.genworks.com> Hello, I'm not sure why I didn't notice this before, but it looks like compile-text (or something) is allocating memory and keeping some kind of pointer to it alive so that even a global GC will not free it. Before I dive in and try tracking it down i tought i would ask if i'm missing something obvious... This is in Allegro CL, so maybe it is some issue specific to Allegro. Anyway here is a sample little function in the TT package: (defun pdf-text (&key (columns 5) (rows 10)) (pdf:with-document () (pdf:with-page (:bounds (make-array 4 :initial-contents (list 0 0 800 7000))) (let ((content (compile-text () (paragraph () (dotimes (row rows) (with-style () :eol) (dotimes (column columns) (put-string "babababababababa"))))))) (draw-block content 20 6900 545 6700 0))) (pdf:write-document #p"/tmp/try.pdf"))) and below is a trace of running it (in the room commands, i've edited the output to show just total oldspace). Note that before running pdf-text 100 times, there is 15604616 of total used other bytes in oldspace. After running it and doing a couple of global GC's, there is 27760472 used. So i'm trying to figure out what is taking up the extra 12,155,856 bytes here... (as well as an extra 230,799 cons cells). Trace: ================================================================== TT(4): (room) area area address(bytes) cons other bytes # type 8 bytes each (free:used) (free:used) OTot(Old Areas) 859:395532 18464:15604616 TT(5): (dotimes (n 100) (pdf-text)) 9310456 bytes have been tenured, next gc will be global. See the documentation for variable *GLOBAL-GC-BEHAVIOR* for more information. 8640608 bytes have been tenured, next gc will be global. See the documentation for variable *GLOBAL-GC-BEHAVIOR* for more information. NIL TT(6): (room) area area address(bytes) cons other bytes # type 8 bytes each (free:used) (free:used) OTot(Old Areas) 201:621389 837440:27384152 TT(7): (excl:gc t) TT(8): (excl:gc t) TT(9): (room) area area address(bytes) cons other bytes # type 8 bytes each (free:used) (free:used) OTot(Old Areas) 354:626331 420160:27760472 ================================================================== For more complicated documents with tables etc the problem seems to grow accordingly... Thanks to everyone for all the great additions lately, and again sorry if i'm missing something obvious here... -dave David J Cooper Jr, Genworks International david.cooper at genworks.com +1 248-737-8835(o), +1 248-330-2979(c) From david.cooper at genworks.com Sat Apr 3 07:51:36 2004 From: david.cooper at genworks.com (David J Cooper Jr) Date: Sat, 3 Apr 2004 02:51:36 -0500 Subject: [cl-typesetting-devel] false alarm, debug *boxes* is the culprit (was: memory not getting released) Message-ID: <16494.27912.806206.352035@mccarthy.genworks.com> Soon after I sent that last question the problem practically jumped out of the screen and bit me on the nose. And now that I think of it I remember Marc mentioning something about this a while back. The draw-block function in test.lisp pushes each box onto *boxes* which is for debug purposes. Comment out that line and we no longer have the strange growth of oldspace. Incidentally, is there a reason the draw-block function is still in test.lisp and apparently not yet the "real" version? Thanks and sorry for the noise. -dave David J Cooper Jr, Genworks International david.cooper at genworks.com +1 248-737-8835(o), +1 248-330-2979(c) From marc.battyani at fractalconcept.com Sat Apr 3 08:22:17 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sat, 3 Apr 2004 10:22:17 +0200 Subject: [cl-typesetting-devel] false alarm, debug *boxes* is the culprit (was: memory not getting released) References: <16494.27912.806206.352035@mccarthy.genworks.com> Message-ID: <066301c41954$c884cc30$0303a8c0@marc2> "David J Cooper Jr" writes: > Soon after I sent that last question the problem practically jumped > out of the screen and bit me on the nose. > > And now that I think of it I remember Marc mentioning something about > this a while back. > > The draw-block function in test.lisp pushes each box onto *boxes* > which is for debug purposes. Comment out that line and we no longer > have the strange growth of oldspace. Incidentally, is there a reason > the draw-block function is still in test.lisp and apparently not yet > the "real" version? Hi dave, Your version is probably out of date. The draw-block function in test.lisp is commented out as it is replaced by the one in top-level.lisp. In that one the (push box *boxes*) is also commented out. ;-) Marc From klaus at atsec.com Sun Apr 18 10:04:08 2004 From: klaus at atsec.com (Klaus Weidner) Date: Sun, 18 Apr 2004 05:04:08 -0500 Subject: [cl-typesetting-devel] HTML rendering Message-ID: <20040418100408.GA17606@w-m-p.com> Hello, after an all-day hacking session, I now have a proof of concept for XHTML rendering via cl-typesetting working, see attachment. It's still missing many tags, but that's mostly a matter of patience, I think the hard work is largely done. The following article had inspired me to do this: http://www-106.ibm.com/developerworks/library/x-xslfo2app/ It covers using XSL-FO stylesheet transforms on XHTML documents. There you can also find the input document "everything.html" I used, along with a FOP-rendered PDF version for comparison. Instead of XSL-FO, I'm using Lisp-based tree transform from XML (read from a file via the XMLS library) to cl-typesetting commands. I think this is an important step towards having cl-typesetting behave more like a normal program with input and output files rather than exclusively as a library. And all it took was an "eval" in the right place :-) I've extended cl-typesetting a bit, to support cross-referencing (needed for the table of contents and intra-document page references), left/right page header alternation (containing chapter titles) and some other minor stuff. The only change to the core code was adding a :fresh-page token. It works like :eop, but does not start a new page if the current one is already empty. (This makes automated generation easier.) All the other extensions were in client code. I've attached the :fresh-page patch, and sample code to demonstrate the new features. The latter is still a bit messy and undocumented, it's a work in progress. The XHTML transformer is still too unfinished for distribution, but please let me know if you're interested in that and I'll work on cleaning it up. But I'm surprised at how well it works already, considering that most of my time had been spent figuring out how things work. Some miscellaneous things I noticed: - there are a couple of rather grating spelling errors in the source code, which I'd suggest fixing before the code is in more widespread use (later changes would be harder) - I can make a patch for those but wanted to check first if that's okay. trimable => trimmable splitable => splittable ponctuation => punctuation Also, I'm not sure what a "trie" is (used by hyphenation). And I think that keyword arguments should not use a "-p" suffix. - This would also be a good opportunity to document the style parameters supported by the public interface, since grepping through the source is suboptimal. - I don't know how leaders (the dots joining table-of-content entries to the page numbers) should be implemented. Note that the dots should line up even if the TOC entries have different lengths, so it's not a simple matter of sticking periods in boxes. - PDF hyperlinks aren't handled yet by the typesetting code - how about a link-box class which acts as a clickable area for the bounding box of the content? - The W3 FO specification is a surprisingly good fit to the cl-typesetting internals, and since XML is isomorphic to s-exprs, I think it would make sense to stay close to that model where appropriate. That would make it much easier to reuse existing work, i.e. for DocBook or OpenOffice rendering. Comments and feedback would be much appreciated. -Klaus -------------- next part -------------- A non-text attachment was scrubbed... Name: output.pdf Type: application/pdf Size: 14871 bytes Desc: not available URL: -------------- next part -------------- diff -urN -b orig/cl-typesetting/layout.lisp cl-typesetting/layout.lisp --- orig/cl-typesetting/layout.lisp Mon Mar 22 03:09:13 2004 +++ cl-typesetting/layout.lisp Sun Apr 18 01:45:08 2004 @@ -221,6 +221,15 @@ (when (eq *h-align* :justified) (push (make-hfill-glue) line-boxes)) (next-line line-boxes)) + ((eq box :fresh-page) + (unless (or (null text-lines) + (and (null (rest text-lines)) + (every (lambda (box) (or (style-p box) + (typep box 'h-spacing) + (typep box 'v-spacing))) + (boxes (first text-lines))))) + (next-line line-boxes) + (return-lines 0))) ((eq box :eop) (next-line line-boxes) (return-lines 0)) diff -urN -b orig/cl-typesetting/typo.lisp cl-typesetting/typo.lisp --- orig/cl-typesetting/typo.lisp Wed Mar 10 13:12:49 2004 +++ cl-typesetting/typo.lisp Sun Apr 18 01:36:16 2004 @@ -231,6 +231,9 @@ (defun new-line () (add-box :eol)) +(defun fresh-page () + (add-box :fresh-page)) + (defun new-page () (add-box :eop)) @@ -242,6 +245,9 @@ (defmethod insert-stuff ((obj (eql :eol))) '(new-line)) + +(defmethod insert-stuff ((obj (eql :fresh-page))) + '(fresh-page)) (defmethod insert-stuff ((obj (eql :eop))) '(new-page)) -------------- next part -------------- (in-package typeset) ;; reference handling (defvar *ref-table* (make-hash-table :test #'equal)) (defvar *ref-counter* 0) (defvar *bad-reference* nil) (defclass ref-mark () ((id :accessor ref-id :initform nil :initarg :id) (value :accessor ref-mark-value :initform nil :initarg :value) (page :accessor ref-mark-page :initform nil) (x :accessor ref-x :initform nil) (y :accessor ref-y :initform nil))) (defmethod stroke ((mark ref-mark) x y) (setf (ref-mark-page mark) (length (pages pdf:*document*)) (ref-x mark) x (ref-y mark) y)) (defmacro ref-get (id) `(gethash ,id *ref-table*)) (defun make-ref-mark (id &optional value) (let ((mark (or (ref-get id) (make-instance 'ref-mark :id id)))) (setf (ref-get id) mark) (setf (ref-mark-value mark) value) (add-box mark))) (defun ref-page (id) (let* ((ref (ref-get id)) (page (if ref (ref-mark-page ref)))) (cond (page page) (t (push id *bad-reference*) 999)))) (defun put-ref-page (id) (put-string (format nil "~D" (ref-page id)))) (defgeneric ref-value (ref)) (defmethod ref-value ((ref ref-mark)) (if ref (ref-mark-value ref))) (defmethod ref-value ((id t)) (let ((ref (ref-get id))) (if ref (ref-mark-value ref)))) (defun put-ref-value (id) (put-string (ref-value id))) (defun this-page-number () (length (pages pdf:*document*))) (defun make-ref-page-mark (reftype value) (make-ref-mark (cons reftype (incf *ref-counter*)) value)) (defun get-latest-ref-to (reftype for-page) (let ((refs nil)) ;; Find all references of a type, store unsorted ;; (ordinal page ref) lists in "refs". (maphash (lambda (key ref) (if (and (consp key) (equal reftype (car key))) (push (list (cdr key) (or (ref-mark-page ref) most-positive-fixnum) ref) refs))) *ref-table*) ;; Now walk through the reverse sorted references, ;; and get the last matching one on or before the ;; current page. (third (find-if (lambda (page) (<= page for-page)) (sort refs #'> :key #'car) :key #'cadr)))) (defun current-ref-value (reftype) (ref-value (get-latest-ref-to reftype (this-page-number)))) ;; higher-level layout (defun safe-eval (expr) (let ((*package* (find-package "TYPESET")) (*read-eval* nil)) (eval expr))) ;; Note that the tree argument to render-document is a dead list of ;; symbols and strings. This is a prerequisite for being to handle ;; documents that are completely generated at runtime. (defun render-document (tree &key (file #P"/tmp/stuff.pdf") (twosided t) (paper-size :letter)) "Render the document specified by tree, which is a s-exp containing recursive typesetting commands. It gets eval'ed here to typeset it." (do ((*ref-table* (make-hash-table :test #'equal)) (*ref-counter* 0) (*bad-reference* nil) (pass 0 (1+ pass))) ((or (> pass 1) (and (> pass 0) (not *bad-reference*))) *bad-reference*) (setq *bad-reference* nil) (format t "Pass ~d~%" pass) (with-document () (let ((margins '(72 72 72 50)) (header (lambda (pdf:*page*) (if (current-ref-value :header-enabled) (let ((inside (or (current-ref-value :title) "Untitled Document")) (outside (current-ref-value :chapter))) (if (and twosided (evenp (this-page-number))) (compile-text (:font "Times-Roman" :font-size 10) (hbox (:align :center :adjustable-p t) (put-string outside) :hfill (with-style (:font "Times-Italic") (put-string inside)))) (compile-text (:font "Times-Roman" :font-size 10) (hbox (:align :center :adjustable-p t) (with-style (:font "Times-Italic") (put-string inside)) :hfill (put-string outside)))))))) (footer (lambda (pdf:*page*) (if (current-ref-value :footer-enabled) (let ((inside (or (current-ref-value :version) "")) (outside (format nil "Page ~d of ~d" (this-page-number) (ref-page "DocumentEnd")))) (if (and twosided (evenp (this-page-number))) (compile-text (:font "Times-Roman" :font-size 10) (hbox (:align :center :adjustable-p t) (put-string outside) :hfill (put-string inside))) (compile-text (:font "Times-Roman" :font-size 10) (hbox (:align :center :adjustable-p t) (put-string inside) :hfill (put-string outside))))))))) (draw-pages (safe-eval (list 'compile-text () tree)) :margins margins :header header :footer footer :size paper-size) (when pdf:*page* (finalize-page pdf:*page*)) (pdf:write-document file))))) ;; Example follows. (defun document-test () (render-document '(with-style () (make-ref-page-mark :title "Titled Document") (make-ref-page-mark :version "Version 1.x") (make-ref-page-mark :header-enabled nil) (make-ref-page-mark :footer-enabled nil) :vfill (paragraph (:font "Helvetica-Bold" :font-size 24 :h-align :center :bottom-margin 20) "This is the Document Title") (paragraph (:font "Helvetica-Bold" :font-size 16 :h-align :center) "A. N. Author") :vfill :eop (make-ref-page-mark :header-enabled t) (make-ref-page-mark :footer-enabled t) (make-ref-mark '(:chapter . 0) "Table of Contents") (with-style (:font "Helvetica" :font-size 10) (hbox (:align :center :adjustable-p t) (put-ref-value '(:chapter . 1)) :hfill (put-ref-page '(:chapter . 1))) (hbox (:align :center :adjustable-p t) (put-ref-value '(:chapter . 2)) :hfill (put-ref-page '(:chapter . 2)))) :eop (make-ref-mark '(:chapter . 1) "Introduction") (paragraph (:font "Times-Roman" :font-size 10 :h-align :left :bottom-margin 7) "Test with " (with-style (:font "Times-Bold") "bold") " and " (with-style (:font "Times-Italic") "italic") " text.") (paragraph (:font "Times-Roman" :font-size 10 :h-align :left :bottom-margin 7) (make-ref-mark "link-from") "See also stuff on page " (put-ref-page "stuff") ".") :eop (make-ref-mark '(:chapter . 2) "Interesting Stuff") (paragraph (:font "Courier" :font-size 10 :bottom-margin 7) (make-ref-mark "stuff") "Some" :eol "more" :eol "Text." ) (paragraph (:font "Times-Roman" :font-size 10 :h-align :left :bottom-margin 7) "This is linked to from page " (put-ref-page "link-from") ".") (make-ref-mark "DocumentEnd")))) From marc.battyani at fractalconcept.com Sun Apr 18 12:46:04 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 18 Apr 2004 14:46:04 +0200 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com> Message-ID: <034e01c42543$1e5e9c90$0a02a8c0@marcxp> "Klaus Weidner" wrote: > after an all-day hacking session, I now have a proof of concept for XHTML > rendering via cl-typesetting working, see attachment. It's still missing > many tags, but that's mostly a matter of patience, I think the hard work > is largely done. Great! The ouput is already rather nice. > The following article had inspired me to do this: > > http://www-106.ibm.com/developerworks/library/x-xslfo2app/ > > It covers using XSL-FO stylesheet transforms on XHTML documents. There > you can also find the input document "everything.html" I used, along with > a FOP-rendered PDF version for comparison. The comparison is not bad. Especially for only one day. > Instead of XSL-FO, I'm using Lisp-based tree transform from XML (read > from a file via the XMLS library) to cl-typesetting commands. > > I think this is an important step towards having cl-typesetting behave > more like a normal program with input and output files rather than > exclusively as a library. And all it took was an "eval" in the right > place :-) Lisp Rulez! :) > I've extended cl-typesetting a bit, to support cross-referencing (needed > for the table of contents and intra-document page references), left/right > page header alternation (containing chapter titles) and some other minor > stuff. Good. I will add this to cl-typesetting. > The only change to the core code was adding a :fresh-page token. It works > like :eop, but does not start a new page if the current one is already > empty. (This makes automated generation easier.) All the other extensions > were in client code. > > I've attached the :fresh-page patch, and sample code to demonstrate the > new features. The latter is still a bit messy and undocumented, it's a > work in progress. It will probably match the cl-typesetting doc anyway. > The XHTML transformer is still too unfinished for distribution, but please > let me know if you're interested in that and I'll work on cleaning it up. > But I'm surprised at how well it works already, considering that most of > my time had been spent figuring out how things work. Of course I'm interested! Please send it as soon as you feel it's clean enough. > Some miscellaneous things I noticed: > > - there are a couple of rather grating spelling errors in the source > code, which I'd suggest fixing before the code is in more widespread > use (later changes would be harder) - I can make a patch for those but > wanted to check first if that's okay. > > trimable => trimmable > splitable => splittable > ponctuation => punctuation Ahem... ok I will change them. > Also, I'm not sure what a "trie" is (used by hyphenation). And I think > that keyword arguments should not use a "-p" suffix. A trie is a kind of search tree (prefix). This code is a contribution, but you can fix it if you want. What keyword arg are you talking about ? > - This would also be a good opportunity to document the style parameters > supported by the public interface, since grepping through the source is > suboptimal. > > - I don't know how leaders (the dots joining table-of-content entries to > the page numbers) should be implemented. Note that the dots should line > up even if the TOC entries have different lengths, so it's not a simple > matter of sticking periods in boxes. IMO it's only a matter of sticking the correct number of dots with a hfill between the last word and the dots. The only trouble is to compute the correct number of dots ;-) > - PDF hyperlinks aren't handled yet by the typesetting code - how about a > link-box class which acts as a clickable area for the bounding box of > the content? I wanted to do this but it's not so simple. I have a few ideas on how to do it. I will try to find some time to implement them. > - The W3 FO specification is a surprisingly good fit to the > cl-typesetting internals, and since XML is isomorphic to s-exprs, I > think it would make sense to stay close to that model where > appropriate. That would make it much easier to reuse existing work, > i.e. for DocBook or OpenOffice rendering. It's a good idea. I wil look at this. > Comments and feedback would be much appreciated. Good work, go on! Marc BTW what about the CMUCL problems ? From marc.battyani at fractalconcept.com Sun Apr 18 13:16:59 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 18 Apr 2004 15:16:59 +0200 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com> Message-ID: <038a01c42547$6eef78b0$0a02a8c0@marcxp> "Klaus Weidner" wrote: More comments: [...] > I think this is an important step towards having cl-typesetting behave > more like a normal program with input and output files rather than > exclusively as a library. Yes, I even think it will be a good thing to deliver cl-typesetting stand alone applications in the future. [...] > - This would also be a good opportunity to document the style parameters > supported by the public interface, since grepping through the source is > suboptimal. Sure, of course the doc should be written with cl-typesetting :) [...] > - The W3 FO specification is a surprisingly good fit to the > cl-typesetting internals, and since XML is isomorphic to s-exprs, I > think it would make sense to stay close to that model where > appropriate. That would make it much easier to reuse existing work, > i.e. for DocBook or OpenOffice rendering. Is the W3 FO something considered a good thing and used in a lot of applications ? Marc Marc From klaus at atsec.com Sun Apr 18 16:30:14 2004 From: klaus at atsec.com (Klaus Weidner) Date: Sun, 18 Apr 2004 11:30:14 -0500 Subject: [cl-typesetting-devel] HTML rendering In-Reply-To: <038a01c42547$6eef78b0$0a02a8c0@marcxp> References: <20040418100408.GA17606@w-m-p.com> <038a01c42547$6eef78b0$0a02a8c0@marcxp> Message-ID: <20040418163014.GA20748@w-m-p.com> On Sun, Apr 18, 2004 at 03:16:59PM +0200, Marc Battyani wrote: > "Klaus Weidner" wrote: > > I think this is an important step towards having cl-typesetting behave > > more like a normal program with input and output files rather than > > exclusively as a library. > > Yes, I even think it will be a good thing to deliver cl-typesetting stand > alone applications in the future. The major pain regarding this is (you guessed it) zlib, but other than that it should be fairly easy to build a standalone image that can do this. I'll have a shot with clisp, since that has by far the smallest core sizes. > > - This would also be a good opportunity to document the style parameters > > supported by the public interface, since grepping through the source is > > suboptimal. > > Sure, of course the doc should be written with cl-typesetting :) I disagree there :-) I think that it should be written in some kind of standard format that can be rendered by cl-typesetting, which isn't the same thing. I'd suggest using XHTML, and personally I'd even prefer POD (Perl "Plain Old Documentation"), which has the advantage of having minimally intrusive markup and therefore is unpainful to type. Compare: (with-style (:font "Times-Italic") "italic") " and " (with-style (:font "Times-Bold") "bold") "text. \texit{italic} and \texbb{bold} text. italic and bold text. I and B text. Of course the Lisp version could be abbreviated, but even the minimal (I "italic") " and " (B "bold") " text." is still way too long, and I'd hate having to put all the text I'm writing into double quotes, and having to worry about escaping backslashes and such things. I even have a POD emacs mode that displays the font changes inline while editing and can hide the tags from view... > > - The W3 FO specification is a surprisingly good fit to the > > cl-typesetting internals, and since XML is isomorphic to s-exprs, I > > think it would make sense to stay close to that model where > > appropriate. That would make it much easier to reuse existing work, > > i.e. for DocBook or OpenOffice rendering. > > Is the W3 FO something considered a good thing and used in a lot of > applications ? There's two parts of this. XSL style sheet transformations are a descendant of CSS (cascading style sheets), and can be used to do transformations on XML document. The FO (formatting objects) XML specification is basically a low-level way to describe page layout. I think it would be doable to read XSL transform descriptions and run them as Lisp code, which would instantly make the fairly large existing number of transformations that were worked out (with huge effort) by the XML-heads available for Lisp code. But that only makes sense if the target typesetting engine can handle FO, in the sense of being able to render FO objects in the expected way with isomorphic transformations. I'm of course not talking about rewriting parts of cl-typesetting in XML, what I mean is that it would be useful to add support for FO features that cl-typesetting doesn't have yet, and try to make the existing ones have the same semantics. Another potential advantage is that FO has well-documented style and layout keywords and parameters that could be adapted directly, instead of inventing our own incompatible set. The XSL-FO route is being used by DocBook documentation, and this is currently hindered substantially by the rather clunky toolchain that's being used for this. I'm sure that an easily installed and useful tool would be greatly appreciated by many people. In general, I think there's a huge opportunity for Lisp here - XML is for the most part extremely easy to read and write with Lisp (except for character encoding issues), and since apparently many people love XML syntax (and are alarmed by Lisp parentheses), I think that's a great way to interface with the rest of the world. -Klaus From klaus at atsec.com Sun Apr 18 17:05:52 2004 From: klaus at atsec.com (Klaus Weidner) Date: Sun, 18 Apr 2004 12:05:52 -0500 Subject: [cl-typesetting-devel] HTML rendering In-Reply-To: <034e01c42543$1e5e9c90$0a02a8c0@marcxp> References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> Message-ID: <20040418170552.GB20748@w-m-p.com> On Sun, Apr 18, 2004 at 02:46:04PM +0200, Marc Battyani wrote: > "Klaus Weidner" wrote: > > I've extended cl-typesetting a bit, to support cross-referencing (needed > > for the table of contents and intra-document page references), left/right > > page header alternation (containing chapter titles) and some other minor > > stuff. > > Good. I will add this to cl-typesetting. I'm still planning to clean this up and add some docstrings, but I guess updating this later should not be a problem. BTW, I'm currently handling the forward references by completely re-running the layout engine if unresolved dependencies were detected, similar to the way LaTeX does it. Any remaining unresolved ones get reported (well, printed to stdout) after the second pass. There may be more direct ways to handle this, but the current approach works well enough and has the advantage of being very simple. There may be rare cases where replacing the "999" currently used in the first pass for forward references would cause a change in pagination (and therefore wrong references) when replaced by a different number of digits on the second run. Currently the code just ignores this, but it would be possible to add checks to see if the marks moved. > > I've attached the :fresh-page patch, and sample code to demonstrate the > > new features. The latter is still a bit messy and undocumented, it's a > > work in progress. > > It will probably match the cl-typesetting doc anyway. ;-) > > The XHTML transformer is still too unfinished for distribution, but please > > let me know if you're interested in that and I'll work on cleaning it up. > > But I'm surprised at how well it works already, considering that most of > > my time had been spent figuring out how things work. > > Of course I'm interested! Please send it as soon as you feel it's clean > enough. That depends on when I can find time to do more hacking on it - I'd like to have it support the "everything.html" file completely. But if it turns out that I can't find the time, I'll post what I have and let others worry about it. > > Also, I'm not sure what a "trie" is (used by hyphenation). And I think > > that keyword arguments should not use a "-p" suffix. > > A trie is a kind of search tree (prefix). Thanks, I guess I should have asked Google right away... > What keyword arg are you talking about ? I've found the following initargs and function keyword args: :adjustable-p :splitable-p :trimable-p In the CL libraries, the -p suffix is used just for predicate functions and not for boolean keyword arguments. I think that (... :adjustable t) is more readable. > > - I don't know how leaders (the dots joining table-of-content entries to > > the page numbers) should be implemented. Note that the dots should line > > up even if the TOC entries have different lengths, so it's not a simple > > matter of sticking periods in boxes. > > IMO it's only a matter of sticking the correct number of dots with a hfill > between the last word and the dots. The only trouble is to compute the > correct number of dots ;-) I don't think it's that simple - once you have a box containing the right number of dots, you still need to position it exactly by inserting glue in the front to handle the width variations of the preceding text - it looks extremely ugly if they don't line up vertically on different lines. Also, this may just be a lack of understanding on my part, but what would the best way be to handle the case where the title is too long to fit in a single line? Something like this: Chapter one . . . . . . . 3 Chapter two with a really long title . . . . . . . 5 Note that the first line should not run into the numbers column, and the last line needs to have the dot fill. Hmmm, seems to be a candidate for a paragraph that has a different right margin for the last line, kind of a reflection of the way the first-line-indent works. Also, there should be a way to completely control the additional glue added during line breaking - I think at the moment there's no way to prevent the last line from getting an additional hfill added which would break the layout. > > - PDF hyperlinks aren't handled yet by the typesetting code - how about a > > link-box class which acts as a clickable area for the bounding box of > > the content? > > I wanted to do this but it's not so simple. I have a few ideas on how to do > it. I will try to find some time to implement them. A problematic area is matching up the references the typesetting layer knows about with the PDF cross-references. Note that my referencing code currently saves the page number and the x/y position for the mark, does that help? > BTW what about the CMUCL problems ? The only way I can get the code loaded on CMUCL is by moving the def-function into a separate file (i.e. zlib.lisp), which is loaded after the load-foreign-library was run. No changes within init.lisp worked, since it doesn't even load if there's an unresolved reference to "compress". -Klaus From divanov at aha.ru Mon Apr 19 07:41:09 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Mon, 19 Apr 2004 11:41:09 +0400 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com><038a01c42547$6eef78b0$0a02a8c0@marcxp> <20040418163014.GA20748@w-m-p.com> Message-ID: <000001c425e1$e3587130$134302c3@digo> Hello Klaus, | On Sun, Apr 18, 2004 at 03:16:59PM +0200, Marc Battyani wrote: |> "Klaus Weidner" wrote: | |> Sure, of course the doc should be written with cl-typesetting :) | | I disagree there :-) | | I think that it should be written in some kind of standard format that | can be rendered by cl-typesetting, which isn't the same thing. I'd | suggest using XHTML, and personally I'd even prefer POD (Perl "Plain | Old Documentation"), which has the advantage of having minimally | intrusive markup and therefore is unpainful to type. | | Compare: | | (with-style (:font "Times-Italic") "italic") | " and " | (with-style (:font "Times-Bold") "bold") | "text. | | \texit{italic} and \texbb{bold} text. | | italic and bold text. | | I and B text. | | Of course the Lisp version could be abbreviated, but even the minimal | | (I "italic") " and " (B "bold") " text." | | is still way too long, and I'd hate having to put all the text I'm | writing into double quotes, and having to worry about escaping | backslashes and such things. We do need something like this to deal with font variants in cl-typesetting. IMHO, that should not use strings in double quotes, but keywords instead. To achive that, we can split font names into :font-family and :font-style (or :font-variant) components. (with-style (:font-family "Times") ... (with-style (:font-style :bold) ; or (:font-style :b) ...)) The require-cyr-font function (see the attachment) could be a starting sample. |...snip...| | In general, I think there's a huge opportunity for Lisp here - XML is | for the most part extremely easy to read and write with Lisp (except | for character encoding issues), and since apparently many people love | XML syntax (and are alarmed by Lisp parentheses), I think that's a | great way to interface with the rest of the world. I am not an XML expert, but many respectful lispniks (IIRC Eric Naggum, Erann Gat to name a few) do not share your love with XML :-) It would be well having all the above as optional modules. But I am not for making cl-typesetting depend entirely on X-. And I agree with Marc about what cl-typesetting doc should be written with. -- Sincerely, Dmitri Ivanov lisp.ystok.ru From divanov at aha.ru Mon Apr 19 07:44:57 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Mon, 19 Apr 2004 11:44:57 +0400 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> Message-ID: <002501c425e3$55148ab0$134302c3@digo> Sorry, here is the attachment. -- Sincerely, Dmitri Ivanov lisp.ystok.ru -------------- next part -------------- A non-text attachment was scrubbed... Name: di-pdf-utils.lisp Type: application/octet-stream Size: 2212 bytes Desc: not available URL: From klaus at atsec.com Mon Apr 19 13:28:08 2004 From: klaus at atsec.com (Klaus Weidner) Date: Mon, 19 Apr 2004 08:28:08 -0500 Subject: [cl-typesetting-devel] HTML rendering In-Reply-To: <000001c425e1$e3587130$134302c3@digo> References: <20040418163014.GA20748@w-m-p.com> <000001c425e1$e3587130$134302c3@digo> Message-ID: <20040419132808.GB28281@w-m-p.com> On Mon, Apr 19, 2004 at 11:41:09AM +0400, Dmitri Ivanov wrote: > We do need something like this to deal with font variants in cl-typesetting. > IMHO, that should not use strings in double quotes, but keywords instead. To > achive that, we can split font names into :font-family and :font-style (or > :font-variant) components. > > (with-style (:font-family "Times") > ... > (with-style (:font-style :bold) ; or (:font-style :b) > ...)) > > The require-cyr-font function (see the attachment) could be a starting > sample. That's one example where the FO classification is IMHO useful - remember that many fonts have more weights than just :bold and :normal. It defines the following attributes, cf. http://www.w3.org/TR/xsl/slice7.html#font-selection-strategy 7.8.9 "font-weight" CSS2 Definition: Value: normal | bold | bolder | lighter | 100 | 200 | 300 | 400 | 500 | 600 | 700 | 800 | 900 | inherit 7.8.7 "font-style" CSS2 Definition: Value: normal | italic | oblique | backslant | inherit > I am not an XML expert, but many respectful lispniks (IIRC Eric Naggum, > Erann Gat to name a few) do not share your love with XML :-) Well, to each his opinion I guess. But I get the impression that some people get personally affronted by people embracing XML when Lisp forms could do the same thing 40 years ago. At least XML is something Lisp can easily deal with - interfacing with existing programs such as OpenOffice is entirely possible, while the various binary gunk formats XML is largely replacing were impossible to deal with by anything other than the program that wrote them. Remember that I'm not talking about using XML anywhere *in* your code - it's just two simple function calls when reading and writing the data, and everything done internally uses s-exprs. > It would be well having all the above as optional modules. But I am not for > making cl-typesetting depend entirely on X-. I agree with that - I'm not talking about touching the way it works internally, and writing typesetting instructions in Lisp forms should always be supported. What I mean regarding FO is just that some *semantic* compatibility would be nice - the font property selection is IMHO something that this standard got right. > And I agree with Marc about what cl-typesetting doc should be written > with. Well, I guess we'll just have to agree to disagree there. But please imagine for a minute how confusing the input text will look for a document that explains how to use cl-typesetting and is itself written in that language (despite verbatim). When writing text, I want to be able to concentrate on that and don't want to think about markup while doing it. The POD extra characters are the limit of what I personally can tolerate while typing. -Klaus From divanov at aha.ru Mon Apr 19 15:21:40 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Mon, 19 Apr 2004 19:21:40 +0400 Subject: [cl-typesetting-devel] HTML rendering References: <20040418163014.GA20748@w-m-p.com> <000001c425e1$e3587130$134302c3@digo> <20040419132808.GB28281@w-m-p.com> Message-ID: <003e01c42628$093322b0$134302c3@digo> Hello Klaus, | That's one example where the FO classification is IMHO useful - | remember that many fonts have more weights than just :bold and :normal. | | It defines the following attributes, cf. | http://www.w3.org/TR/xsl/slice7.html#font-selection-strategy | | 7.8.9 "font-weight" | | CSS2 Definition: | Value: normal | bold | bolder | lighter | 100 | 200 | 300 | 400 | | 500 | 600 | 700 | 800 | 900 | inherit | | 7.8.7 "font-style" | | CSS2 Definition: | Value: normal | italic | oblique | backslant | inherit |...snip...| What is the point to name this CSS2-originated staff by "FO classification" ? (BTW I have not said I disregard CSS :-)) -- Sincerely, Dmitri Ivanov www.ystok.ru From klaus at atsec.com Mon Apr 19 16:41:05 2004 From: klaus at atsec.com (Klaus Weidner) Date: Mon, 19 Apr 2004 11:41:05 -0500 Subject: [cl-typesetting-devel] HTML rendering In-Reply-To: <003e01c42628$093322b0$134302c3@digo> References: <20040418163014.GA20748@w-m-p.com> <000001c425e1$e3587130$134302c3@digo> <20040419132808.GB28281@w-m-p.com> <003e01c42628$093322b0$134302c3@digo> Message-ID: <20040419164105.GB30467@w-m-p.com> On Mon, Apr 19, 2004 at 07:21:40PM +0400, Dmitri Ivanov wrote: > What is the point to name this CSS2-originated staff by "FO classification" > ? (BTW I have not said I disregard CSS :-)) It's the same thing, just renamed. CSS/CSS2 is deprecated and replaced by XSL-FO, which contains the CSS2 spec, a lot of extensions (i.e. much improved print media handling, which is relevant for us), and uses XML syntax rather than the rather ad-hoc CSS stylesheets, which makes it trivial to parse. -Klaus From marc.battyani at fractalconcept.com Mon Apr 19 23:19:45 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 20 Apr 2004 01:19:45 +0200 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> Message-ID: <091401c42664$ce6b4c60$0a02a8c0@marcxp> "Klaus Weidner" wrote: > > Good. I will add this to cl-typesetting. > > I'm still planning to clean this up and add some docstrings, but I guess > updating this later should not be a problem. OK > BTW, I'm currently handling the forward references by completely > re-running the layout engine if unresolved dependencies were detected, > similar to the way LaTeX does it. Any remaining unresolved ones get > reported (well, printed to stdout) after the second pass. There may be > more direct ways to handle this, but the current approach works > well enough and has the advantage of being very simple. It's OK for me. Anyway there could be degenerate cases where there is no convergence. > There may be rare cases where replacing the "999" currently used in the > first pass for forward references would cause a change in pagination (and > therefore wrong references) when replaced by a different number of digits > on the second run. Currently the code just ignores this, but it would be > possible to add checks to see if the marks moved. OK > > > The XHTML transformer is still too unfinished for distribution, but ... > > Of course I'm interested! Please send it as soon as you feel it's clean > > enough. > That depends on when I can find time to do more hacking on it - I'd like > to have it support the "everything.html" file completely. But if it turns > out that I can't find the time, I'll post what I have and let others > worry about it. The fact that time comes in so limited amounts is one of the best reasons to write in Lisp ;-) > > > Also, I'm not sure what a "trie" is (used by hyphenation). And I think > > > that keyword arguments should not use a "-p" suffix. > > > > A trie is a kind of search tree (prefix). > > Thanks, I guess I should have asked Google right away... > > > What keyword arg are you talking about ? > > I've found the following initargs and function keyword args: > > :adjustable-p > :splitable-p > :trimable-p I see, I thought you were speaking of the trie code. I generally use the same initarg as the accessor in CLOS slots. > In the CL libraries, the -p suffix is used just for predicate functions > and not for boolean keyword arguments. I think that (... :adjustable t) > is more readable. At least it's more consistent with the standard functions. Anybody against this change ? > > > - I don't know how leaders (the dots joining table-of-content entries to > > > the page numbers) should be implemented. Note that the dots should line > > > up even if the TOC entries have different lengths, so it's not a simple > > > matter of sticking periods in boxes. > > > > IMO it's only a matter of sticking the correct number of dots with a hfill > > between the last word and the dots. The only trouble is to compute the > > correct number of dots ;-) > > I don't think it's that simple - once you have a box containing the right > number of dots, you still need to position it exactly by inserting glue > in the front to handle the width variations of the preceding text - it > looks extremely ugly if they don't line up vertically on different lines. > > Also, this may just be a lack of understanding on my part, but what would > the best way be to handle the case where the title is too long to fit in > a single line? Something like this: > > Chapter one . . . . . . . 3 > > Chapter two with a really > long title . . . . . . . 5 > > Note that the first line should not run into the numbers column, and the > last line needs to have the dot fill. Hmmm, seems to be a candidate for a > paragraph that has a different right margin for the last line, kind of a > reflection of the way the first-line-indent works. I was rather thinking about a fill-with-dots-and-put-a-number-box special box that would act like a hfill (and prevent the right margin space to be inserted) > Also, there should be a way to completely control the additional glue > added during line breaking - I think at the moment there's no way to > prevent the last line from getting an additional hfill added which would > break the layout. Same process here. > > > - PDF hyperlinks aren't handled yet by the typesetting code - how about a > > > link-box class which acts as a clickable area for the bounding box of > > > the content? > > > > I wanted to do this but it's not so simple. I have a few ideas on how to do > > it. I will try to find some time to implement them. > > A problematic area is matching up the references the typesetting layer > knows about with the PDF cross-references. Note that my referencing code > currently saves the page number and the x/y position for the mark, does > that help? Yes, there is stuff for this (references) in cl-pdf. > > BTW what about the CMUCL problems ? > > The only way I can get the code loaded on CMUCL is by moving the > def-function into a separate file (i.e. zlib.lisp), which is loaded after > the load-foreign-library was run. No changes within init.lisp worked, > since it doesn't even load if there's an unresolved reference to > "compress". It's strange how something so simple can be the major source of problems. Marc From marc.battyani at fractalconcept.com Mon Apr 19 23:20:01 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 20 Apr 2004 01:20:01 +0200 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com><038a01c42547$6eef78b0$0a02a8c0@marcxp> <20040418163014.GA20748@w-m-p.com> <000001c425e1$e3587130$134302c3@digo> Message-ID: <091501c42664$d78a32c0$0a02a8c0@marcxp> Marc> Sure, of course the doc should be written with cl-typesetting :) Klaus>| I disagree there :-) > | I think that it should be written in some kind of standard format that > | can be rendered by cl-typesetting, which isn't the same thing. I'd > | suggest using XHTML, and personally I'd even prefer POD (Perl "Plain > | Old Documentation"), which has the advantage of having minimally > | intrusive markup and therefore is unpainful to type. > | > | Compare: > | > | (with-style (:font "Times-Italic") "italic") > | " and " > | (with-style (:font "Times-Bold") "bold") > | "text. > | > | \texit{italic} and \texbb{bold} text. > | > | italic and bold text. > | > | I and B text. > | > | Of course the Lisp version could be abbreviated, but even the minimal > | > | (I "italic") " and " (B "bold") " text." > | > | is still way too long, and I'd hate having to put all the text I'm > | writing into double quotes, and having to worry about escaping > | backslashes and such things. There is also the TeX syntax which is rather popular. Dmitri> We do need something like this to deal with font variants in cl-typesetting. > IMHO, that should not use strings in double quotes, but keywords instead. To > achive that, we can split font names into :font-family and :font-style (or > :font-variant) components. > > (with-style (:font-family "Times") > ... > (with-style (:font-style :bold) ; or (:font-style :b) > ...)) > > The require-cyr-font function (see the attachment) could be a starting > sample. I agree for the principle I will look at it tomorrow. (thought it's already tomorow ;-) > |...snip...| > | In general, I think there's a huge opportunity for Lisp here - XML is > | for the most part extremely easy to read and write with Lisp (except > | for character encoding issues), and since apparently many people love > | XML syntax (and are alarmed by Lisp parentheses), I think that's a > | great way to interface with the rest of the world. > > I am not an XML expert, but many respectful lispniks (IIRC Eric Naggum, > Erann Gat to name a few) do not share your love with XML :-) Ahem, I don't think Klaus wants to work with XML, just import some. > It would be well having all the above as optional modules. But I am not for > making cl-typesetting depend entirely on X-. And I agree with > Marc about what cl-typesetting doc should be written with. Klaus>Well, I guess we'll just have to agree to disagree there. But please >imagine for a minute how confusing the input text will look for a >document that explains how to use cl-typesetting and is itself written in >that language (despite verbatim). The syntax is a subject that was debated a lot on c.l.l My position is that the intermediate syntax layer must be s-expr based otherwise we would loose a lot of power. But though some (like Dmitri) prefer to work directly in this syntax, it seems that people generally prefer to work on a more user friendly syntax. I think both are needed. When I write a document, A user friendly syntax is better IMO. But when I write code to generate a document then I prefer to work on the s-expr syntax. A lot of people proposed some syntaxes that could be cool but so far nobody has published one. :( So all syntaxes are welcomed as long as there is some lisp code to convert it to s-expr. Klaus>What I mean regarding FO is just that some *semantic* compatibility would >be nice - the font property selection is IMHO something that this >standard got right. Sure, in fact the current cl-typesetting lisp syntax is loosely inspired by HTML/CSS so a semantic compatilibity with FO is a good idea. BTW, I looked at some FO typesetting systems and it seems that the price of a server licence can be of more than 80Keuros ! Marc From klaus at atsec.com Mon Apr 19 23:46:12 2004 From: klaus at atsec.com (Klaus Weidner) Date: Mon, 19 Apr 2004 18:46:12 -0500 Subject: [cl-typesetting-devel] HTML rendering In-Reply-To: <091501c42664$d78a32c0$0a02a8c0@marcxp> References: <20040418163014.GA20748@w-m-p.com> <000001c425e1$e3587130$134302c3@digo> <091501c42664$d78a32c0$0a02a8c0@marcxp> Message-ID: <20040419234612.GD7767@w-m-p.com> On Tue, Apr 20, 2004 at 01:20:01AM +0200, Marc Battyani wrote: > There is also the TeX syntax which is rather popular. You mean the old-style {\bf bold} and {\it italic}? Sure, I still use that myself on occasion if I have to write LaTeX and I don't have my emacs macros available. I think that parsing TeX would be harder than XML (unless you restrict yourself to a very small subset), and you'd have to be careful not to inherit the limitations. The TeX/LaTeX table model is not a good thing to emulate. If somebody wants to do this, more power to them, but it isn't going to be me. > > I am not an XML expert, but many respectful lispniks (IIRC Eric Naggum, > > Erann Gat to name a few) do not share your love with XML :-) > > Ahem, I don't think Klaus wants to work with XML, just import some. Exactly. Just a few lines of code to be able to read and (if needed) write it. Lisp is unfortunately a niche language, and beside that it's much easier to deal with XML in Lisp than to convince people to import s-exps in their favourite language. > The syntax is a subject that was debated a lot on c.l.l My position is that > the intermediate syntax layer must be s-expr based otherwise we would loose > a lot of power. But though some (like Dmitri) prefer to work directly in > this syntax, it seems that people generally prefer to work on a more user > friendly syntax. > > I think both are needed. When I write a document, A user friendly syntax is > better IMO. But when I write code to generate a document then I prefer to > work on the s-expr syntax. I fully agree here. It's great to have the full power of Lisp at your fingertips if you want to do something complex, but most of the time I just want to write text and let the engine handle the formatting. It's not an either/or choice, you can simply pick the right tool for the job at hand. > A lot of people proposed some syntaxes that could be cool but so far nobody > has published one. :( > So all syntaxes are welcomed as long as there is some lisp code to convert > it to s-expr. That's exactly what I'm working on. Here's an extract from my XHTML tree rewriter (which corresponds to an XSL-FO stylesheet expressed in Lisp): (case elem ((:h1) `(with-style () :fresh-page (paragraph (:font "Helvetica-Bold" :font-size 20 :top-margin 12 :bottom-margin 12) (tt::make-ref-mark (cons :chapter ,(incf *chapter-num*)) ,(xml-extract-text node)) , at clst))) ((:pre) `(with-style (:font "Courier" :font-size 12 :bottom-margin 0) (verbatim , at clst))) ((:nobr) `(with-style () (hbox () , at clst))) ((:br) :eol) ((:i :em :var) `(with-style (:font "Times-Italic") , at clst)) ((:b :strong) `(with-style (:font "Times-Bold") , at clst)) ((:tt :kbd :samp :code :address) `(with-style (:font "Courier") , at clst)) ((:big) `(with-style (:font-size 14) , at clst)) > BTW, I looked at some FO typesetting systems and it seems that the price of > a server licence can be of more than 80Keuros ! Yes, the commercial systems are horrendously expensive. There are some free alternatives - FOP is a Java-based solution developed by the Apache team (which I couldn't get working on my machine), and there's passivetex a.k.a. xmltex, which appears to be rather fragile due to the choice of a TeX backend. -Klaus From kw at w-m-p.com Mon Apr 19 23:59:05 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Mon, 19 Apr 2004 18:59:05 -0500 Subject: [cl-typesetting-devel] HTML rendering In-Reply-To: <091401c42664$ce6b4c60$0a02a8c0@marcxp> References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> Message-ID: <20040419235905.GA8493@w-m-p.com> On Tue, Apr 20, 2004 at 01:19:45AM +0200, Marc Battyani wrote: > "Klaus Weidner" wrote: > > Also, this may just be a lack of understanding on my part, but what would > > the best way be to handle the case where the title is too long to fit in > > a single line? Something like this: > > > > Chapter one . . . . . . . 3 > > > > Chapter two with a really > > long title . . . . . . . 5 > > > > Note that the first line should not run into the numbers column, and the > > last line needs to have the dot fill. Hmmm, seems to be a candidate for a > > paragraph that has a different right margin for the last line, kind of a > > reflection of the way the first-line-indent works. > > I was rather thinking about a fill-with-dots-and-put-a-number-box special > box that would act like a hfill (and prevent the right margin space to be > inserted) Could you give me an example of how a box that can extend beyond the right margin needs to be defined? I thought the engine was supposed to stack them inside it. > > A problematic area is matching up the references the typesetting layer > > knows about with the PDF cross-references. Note that my referencing code > > currently saves the page number and the x/y position for the mark, does > > that help? > > Yes, there is stuff for this (references) in cl-pdf. I'll take a look to see if I can figure this out, time permitting (har!). > > The only way I can get the code loaded on CMUCL is by moving the > > def-function into a separate file (i.e. zlib.lisp), which is loaded after > > the load-foreign-library was run. No changes within init.lisp worked, > > since it doesn't even load if there's an unresolved reference to > > "compress". > > It's strange how something so simple can be the major source of problems. Hmm, the cl-png Debian package contains a pure-Lisp implementation of deflate compression, but it appears a bit unfinished. Pity. -Klaus From divanov at aha.ru Tue Apr 20 06:42:06 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Tue, 20 Apr 2004 10:42:06 +0400 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> Message-ID: <001201c426a2$b8b19620$e95702c3@digo> Hello Klaus, | Hmm, the cl-png Debian package contains a pure-Lisp implementation of | deflate compression, but it appears a bit unfinished. Pity. FYG, http://opensource.franz.com presents inflate.cl, which looks quite portable, except for the if* macro and the excl package. Those could be borrowed from the acl-compat module. -- Sincerely, Dmitri Ivanov lisp.ystok.ru From marc.battyani at fractalconcept.com Tue Apr 20 07:37:33 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 20 Apr 2004 09:37:33 +0200 Subject: [cl-typesetting-devel] HTML rendering References: <20040418163014.GA20748@w-m-p.com> <000001c425e1$e3587130$134302c3@digo> <091501c42664$d78a32c0$0a02a8c0@marcxp> <20040419234612.GD7767@w-m-p.com> Message-ID: <09c001c426aa$59762fb0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Tue, Apr 20, 2004 at 01:20:01AM +0200, Marc Battyani wrote: > > I think that parsing TeX would be harder than XML (unless you restrict > yourself to a very small subset), and you'd have to be careful not to > inherit the limitations. The TeX/LaTeX table model is not a good thing to > emulate. If somebody wants to do this, more power to them, but it isn't > going to be me. Trying to layout a table in TeX/LaTeX/ConTeX is what made me write cl-typesetting. :) > > I think both are needed. When I write a document, A user friendly syntax is > > better IMO. But when I write code to generate a document then I prefer to > > work on the s-expr syntax. > > I fully agree here. It's great to have the full power of Lisp at your > fingertips if you want to do something complex, but most of the time I > just want to write text and let the engine handle the formatting. > > It's not an either/or choice, you can simply pick the right tool for the > job at hand. > > > A lot of people proposed some syntaxes that could be cool but so far nobody > > has published one. :( > > So all syntaxes are welcomed as long as there is some lisp code to convert > > it to s-expr. > > That's exactly what I'm working on. Here's an extract from my XHTML tree > rewriter (which corresponds to an XSL-FO stylesheet expressed in Lisp): > > (case elem > ((:h1) `(with-style () > :fresh-page > (paragraph (:font "Helvetica-Bold" :font-size 20 > :top-margin 12 :bottom-margin 12) > (tt::make-ref-mark (cons :chapter ,(incf *chapter-num*)) > ,(xml-extract-text node)) > , at clst))) > ((:pre) `(with-style (:font "Courier" :font-size 12 :bottom-margin 0) > (verbatim , at clst))) > ((:nobr) `(with-style () (hbox () , at clst))) > ((:br) :eol) > ((:i :em :var) `(with-style (:font "Times-Italic") , at clst)) > ((:b :strong) `(with-style (:font "Times-Bold") , at clst)) > ((:tt :kbd :samp :code :address) `(with-style (:font "Courier") , at clst)) > ((:big) `(with-style (:font-size 14) , at clst)) Cool! > > BTW, I looked at some FO typesetting systems and it seems that the price of > > a server licence can be of more than 80Keuros ! > > Yes, the commercial systems are horrendously expensive. > > There are some free alternatives - FOP is a Java-based solution developed > by the Apache team (which I couldn't get working on my machine), and > there's passivetex a.k.a. xmltex, which appears to be rather fragile due > to the choice of a TeX backend. Yes, if we find the time to finish this, it will be a good Java/Lisp test on a real complex application. Marc From marc.battyani at fractalconcept.com Tue Apr 20 07:39:48 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 20 Apr 2004 09:39:48 +0200 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com> <001201c426a2$b8b19620$e95702c3@digo> Message-ID: <09cd01c426aa$aa7310e0$0a02a8c0@marcxp> "Dmitri Ivanov" wrote: > | Hmm, the cl-png Debian package contains a pure-Lisp implementation of > | deflate compression, but it appears a bit unfinished. Pity. > > FYG, http://opensource.franz.com presents inflate.cl, which looks quite > portable, except for the if* macro and the excl package. Those could be > borrowed from the acl-compat module. IIRC there was only the decompression not the compression in this package. Marc From marc.battyani at fractalconcept.com Tue Apr 20 07:44:19 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 20 Apr 2004 09:44:19 +0200 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> Message-ID: <09ce01c426ab$4a7ac740$0a02a8c0@marcxp> "Klaus Weidner" wrote: > Could you give me an example of how a box that can extend beyond the > right margin needs to be defined? I thought the engine was supposed to > stack them inside it. I will try to look at this today. > > > The only way I can get the code loaded on CMUCL is by moving the > > > def-function into a separate file (i.e. zlib.lisp), which is loaded after > > > the load-foreign-library was run. No changes within init.lisp worked, > > > since it doesn't even load if there's an unresolved reference to > > > "compress". > > > > It's strange how something so simple can be the major source of problems. > > Hmm, the cl-png Debian package contains a pure-Lisp implementation of > deflate compression, but it appears a bit unfinished. Pity. A Lisp only compression could be a good alternative. Another one would be to provide a set of different files for zlib (i.e. a zlib-cmucl like in the first versions of cl-pdf) Marc From edi at agharta.de Tue Apr 20 08:15:45 2004 From: edi at agharta.de (Edi Weitz) Date: Tue, 20 Apr 2004 10:15:45 +0200 Subject: [cl-typesetting-devel] HTML rendering In-Reply-To: <09ce01c426ab$4a7ac740$0a02a8c0@marcxp> (Marc Battyani's message of "Tue, 20 Apr 2004 09:44:19 +0200") References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <09ce01c426ab$4a7ac740$0a02a8c0@marcxp> Message-ID: On Tue, 20 Apr 2004 09:44:19 +0200, "Marc Battyani" wrote: > A Lisp only compression could be a good alternative. Another one > would be to provide a set of different files for zlib (i.e. a > zlib-cmucl like in the first versions of cl-pdf) Did someone try the approach I sent last week? From marc.battyani at fractalconcept.com Tue Apr 20 09:04:28 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 20 Apr 2004 11:04:28 +0200 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com><09ce01c426ab$4a7ac740$0a02a8c0@marcxp> Message-ID: <0a4f01c426b6$7d576780$0a02a8c0@marcxp> "Edi Weitz" wrote: > On Tue, 20 Apr 2004 09:44:19 +0200, "Marc Battyani" wrote: > > > A Lisp only compression could be a good alternative. Another one > > would be to provide a set of different files for zlib (i.e. a > > zlib-cmucl like in the first versions of cl-pdf) > > Did someone try the approach I sent last week? No, I don't have CMUCL. What do you think about going back to separate zlib-xxx.lisp files like in the early cl-pdf versions ? Marc From edi at agharta.de Tue Apr 20 09:09:06 2004 From: edi at agharta.de (Edi Weitz) Date: Tue, 20 Apr 2004 11:09:06 +0200 Subject: [cl-typesetting-devel] HTML rendering In-Reply-To: <0a4f01c426b6$7d576780$0a02a8c0@marcxp> (Marc Battyani's message of "Tue, 20 Apr 2004 11:04:28 +0200") References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <09ce01c426ab$4a7ac740$0a02a8c0@marcxp> <0a4f01c426b6$7d576780$0a02a8c0@marcxp> Message-ID: On Tue, 20 Apr 2004 11:04:28 +0200, "Marc Battyani" wrote: > What do you think about going back to separate zlib-xxx.lisp files > like in the early cl-pdf versions ? I'd prefer the UFFI approach (as a matter of policy). In this case it's just about load order and I think the version I proposed should work. (Maybe it also has to be conditionalized for SBCL.) I'm currently quite busy so I don't have the time to test this myself, sorry. Cheers, Edi. From divanov at aha.ru Tue Apr 20 08:36:56 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Tue, 20 Apr 2004 12:36:56 +0400 Subject: [cl-typesetting-devel] HTML rendering References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com> <001201c426a2$b8b19620$e95702c3@digo> <09cd01c426aa$aa7310e0$0a02a8c0@marcxp> Message-ID: <001401c426b9$400c7520$e95702c3@digo> Hello Marc, | "Dmitri Ivanov" wrote: | | >> Hmm, the cl-png Debian package contains a pure-Lisp implementation | >> of deflate compression, but it appears a bit unfinished. Pity. |> |> FYG, http://opensource.franz.com presents inflate.cl, which looks |> quite portable, except for the if* macro and the excl package. Those |> could be borrowed from the acl-compat module. | | IIRC there was only the decompression not the compression in this | package. You are right, sorry for bothering. -- Sincerely, Dmitri Ivanov lisp.ystok.ru From klaus at atsec.com Tue Apr 20 16:17:13 2004 From: klaus at atsec.com (Klaus Weidner) Date: Tue, 20 Apr 2004 11:17:13 -0500 Subject: [cl-typesetting-devel] zlib and CMUCL In-Reply-To: References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <09ce01c426ab$4a7ac740$0a02a8c0@marcxp> <0a4f01c426b6$7d576780$0a02a8c0@marcxp> Message-ID: <20040420161713.GA7196@w-m-p.com> On Tue, Apr 20, 2004 at 11:09:06AM +0200, Edi Weitz wrote: > I'd prefer the UFFI approach (as a matter of policy). In this case > it's just about load order and I think the version I proposed should > work. (Maybe it also has to be conditionalized for SBCL.) > > I'm currently quite busy so I don't have the time to test this myself, > sorry. Yes, your approach does work, and that's what my original patch did - all that's needed is moving the def-function to zlib.c. See attached patch. That definitely works on CMUCL and SBCL, and if it doesn't break one of the other supported platforms I'd suggest making this change. I can't get cl-pdf to load on clisp at all, since that doesn't implement uffi. Is there some way to conditionalize the .asd file on a feature, or to recover gracefully from a missing component? Or maybe a separate cl-pdf-zlib.asd file, and keep all the zlib stuff out of the default cl-pdf.asd? A pure Lisp zlib would really be a good thing... -Klaus -------------- next part -------------- diff -urN orig/cl-pdf/init.lisp cl-pdf/init.lisp --- orig/cl-pdf/init.lisp Mon Feb 9 14:39:42 2004 +++ cl-pdf/init.lisp Sat Apr 17 13:33:55 2004 @@ -23,13 +23,6 @@ (uffi:load-foreign-library zlib-path :module "zlib" :supporting-libraries '("c")) - (uffi:def-function ("compress" c-compress) - ((dest (* :unsigned-char)) - (destlen (* :long)) - (source :cstring) - (source-len :long)) - :returning :int - :module "zlib") (setf *zlib-loaded* t *compress-streams* t)) (progn (warn "Unable to load zlib. Disabling compression.") diff -urN orig/cl-pdf/zlib.lisp cl-pdf/zlib.lisp --- orig/cl-pdf/zlib.lisp Fri Jan 30 10:31:48 2004 +++ cl-pdf/zlib.lisp Sat Apr 17 13:35:01 2004 @@ -6,6 +6,14 @@ ;Adapted from an UFFI example +(uffi:def-function ("compress" c-compress) + ((dest (* :unsigned-char)) + (destlen (* :long)) + (source :cstring) + (source-len :long)) + :returning :int + :module "zlib") + (defun compress-string (source) "Returns two values: array of bytes containing the compressed data and the numbe of compressed bytes" From marc.battyani at fractalconcept.com Tue Apr 20 22:28:39 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 21 Apr 2004 00:28:39 +0200 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> Message-ID: <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > - I don't know how leaders (the dots joining table-of-content entries to > the page numbers) should be implemented. Note that the dots should line > up even if the TOC entries have different lengths, so it's not a simple > matter of sticking periods in boxes. I've added a dotted-hfill (can fill with any pattern not only dots, see the attached example) Only 20 lines of code :) (at the end of graphics.lisp) It's on the repository. Tomorrow I will try to find some time to look at the margin change for multiline entries. Marc -------------- next part -------------- A non-text attachment was scrubbed... Name: hello.pdf Type: application/pdf Size: 3575 bytes Desc: not available URL: From marc.battyani at fractalconcept.com Tue Apr 20 22:31:01 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 21 Apr 2004 00:31:01 +0200 Subject: [cl-typesetting-devel] New commits References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <09ce01c426ab$4a7ac740$0a02a8c0@marcxp> <0a4f01c426b6$7d576780$0a02a8c0@marcxp> <20040420161713.GA7196@w-m-p.com> Message-ID: <0e0801c42727$296acf10$0a02a8c0@marcxp> Some new commits to the repository: Rev 39: Added dotted-hfill for tables of content. see the #'hello example for use. Added :fresh-page to insert a new page when not already on a blank page. [Thanks to Klaus Weidner] Corrected some spelling errors (splittable, trimmable, punctuation)[Thanks to Klaus Weidner] Rev 35: Some layout fixes Rev 34: Portability fix (Thanks Peter Seibel) Marc From kw at w-m-p.com Wed Apr 21 00:07:48 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Tue, 20 Apr 2004 19:07:48 -0500 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) In-Reply-To: <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> Message-ID: <20040421000748.GA13780@w-m-p.com> On Wed, Apr 21, 2004 at 12:28:39AM +0200, Marc Battyani wrote: > I've added a dotted-hfill (can fill with any pattern not only dots, see the > attached example) > Only 20 lines of code :) > (at the end of graphics.lisp) Great, thanks! I still need to do some hacking on the paragraphs, the hfill always being added by layout.pl at :eol breaks what I want to do... > Tomorrow I will try to find some time to look at the margin change for > multiline entries. I'm starting to think that the existing model can handle it, if there's a way to completely control where extra glue gets added. I'll experiment a bit. -Klaus From kw at w-m-p.com Wed Apr 21 05:25:48 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Wed, 21 Apr 2004 00:25:48 -0500 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) In-Reply-To: <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> Message-ID: <20040421052548.GA15678@w-m-p.com> On Wed, Apr 21, 2004 at 12:28:39AM +0200, Marc Battyani wrote: > Tomorrow I will try to find some time to look at the margin change for > multiline entries. I've only gotten it to do what I want with a really ugly hack - the hfill added at the end of each left-justified line needed to be left out on the last line, otherwise the leader at that place didn't reach to the right margin. I didn't see an obvious way to prevent the hfill from being attached, so I got violent and added an extraction routine ;-) Also, I've hacked the enumerate function to be a bit smarter, so that it lines up the margins properly. To support that, I added a put-filled-string function which aligns a string within a predefined horizontal area. That also took some hacking - I may have missed some obvious solution. Putting real boxes inline messed up the formatting... Lastly, I've experimented with tables a bit - is it currently really required to explicitly specify column widths? If so, making that dynamic would be a fairly important project. I've attached the current output of the XHTML converter, which is starting to look promising. Still missing are strikethrough and underline, which I'm too tired to figure out how to implement. One more thing, is there some easy way to handle widow/orphan prevention automatically, i.e. by specifying a 'badness' in the interline vspace? -Klaus -------------- next part -------------- diff -urN orig/cl-typesetting/layout.lisp cl-typesetting/layout.lisp --- orig/cl-typesetting/layout.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/layout.lisp Tue Apr 20 19:42:51 2004 @@ -90,7 +90,7 @@ (if line-boxes (let ((text-line (make-instance 'text-line :dx dx :adjustable-p t))) (setf line-boxes (boxes-left-trim line-boxes)) - (when (member *h-align* '(:center :left)) + (when (member *h-align* '(:center :left :left-not-last)) (push (make-hfill-glue) line-boxes)) (unless (zerop *right-margin*) (push (make-instance 'h-spacing :dx *right-margin*) line-boxes)) @@ -130,7 +130,15 @@ (push box text-lines)) ((and trimming (trimmable-p box)) nil) ((eq box :eol) - #+nil(when (eq *h-align* :justified) + (when (eq *h-align* :left-not-last) + ;; incredibly ugly hack - need to get rid of the + ;; superfluous box added on the last line. + (let ((fbox (find-if (lambda (box) + (and (typep box 'h-spacing) + (eql (expansibility box) +huge-number+))) + line-boxes))) + (if fbox (setq line-boxes (remove fbox line-boxes))))) + (when (eq *h-align* :justified) (push (make-hfill-glue) line-boxes)) (next-line line-boxes)) ((eq box :fresh-page) diff -urN orig/cl-typesetting/typo.lisp cl-typesetting/typo.lisp --- orig/cl-typesetting/typo.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/typo.lisp Tue Apr 20 23:20:28 2004 @@ -216,8 +216,8 @@ (add-box (make-char-box char)) (add-box (make-inter-char-glue)))))))) -;;; put a string in a 'verbatim' way: no kerning, no hyphenation, significant whitespaces, significant newlines (defun verbatim (string) + "put a string in a 'verbatim' way: no kerning, no hyphenation, significant whitespaces, significant newlines" (when (stringp string) (loop for char across string for i from 0 @@ -228,6 +228,18 @@ (t (add-box (make-char-box char)) (add-box (make-inter-char-glue))))))) +(defun put-filled-string (string width &key (align :left)) + "place aligned string in fixed-width space" + (let* ((string-width + (loop for char across string + summing (pdf:get-char-width char *font* *font-size*))) + (blank (- width string-width))) + (case align + ((:left) (verbatim string) (hspace blank)) + ((:center) + (hspace (* 0.5 blank)) (verbatim string) (hspace (* 0.5 blank))) + ((:right) (hspace blank) (verbatim string))))) + (defun new-line () (add-box :eol)) @@ -363,13 +375,19 @@ (defvar %enumerate-indents% nil) -(defmacro enumerate ((&key (indent 20) (item-fmt "~D. ")) +(defmacro enumerate ((&key (indent 20) + text-style + (item-fmt "~D. ") + (start-from 1) + item-style) &body body) `(let ((%enumerate-indents% (cons ,indent %enumerate-indents%))) ,@(loop for item in body - for i from 1 collect + for i from start-from collect `(paragraph (:left-margin (reduce #'+ %enumerate-indents%) - :first-line-indent (- 4) - ,@(cadr item)) - ,(format nil item-fmt i) - ,@(cddr item))))) + :first-line-indent (- ,indent) + , at text-style) + (with-style ,item-style + (put-filled-string ,(format nil item-fmt i) + ,indent :align :right)) + ,item)))) -------------- next part -------------- A non-text attachment was scrubbed... Name: output.pdf Type: application/pdf Size: 19499 bytes Desc: not available URL: From divanov at aha.ru Wed Apr 21 06:52:58 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Wed, 21 Apr 2004 10:52:58 +0400 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com><0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> <20040421052548.GA15678@w-m-p.com> Message-ID: <002f01c4276d$6e46d1b0$474302c3@digo> Hello Klaus, | Lastly, I've experimented with tables a bit - is it currently really | required to explicitly specify column widths? If so, making that | dynamic would be a fairly important project. Yes, you are. Frankly, I've never seen any valuable result of fully dynamic width distribution. A powerful constraint engine is needed to cope with this, and none of the modern browsers has it. Introducing percentage and other measurements (similar to the DefDoc project) would be much easier and also useful. -- Sincerely, Dmitri Ivanov lisp.ystok.ru From marc.battyani at fractalconcept.com Wed Apr 21 06:58:06 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 21 Apr 2004 08:58:06 +0200 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com><0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp><20040421052548.GA15678@w-m-p.com> <002f01c4276d$6e46d1b0$474302c3@digo> Message-ID: <0f2401c4276e$00e63830$0a02a8c0@marcxp> "Dmitri Ivanov" wrote: > Yes, you are. Frankly, I've never seen any valuable result of fully dynamic > width distribution. A powerful constraint engine is needed to cope with > this, and none of the modern browsers has it. > > Introducing percentage and other measurements (similar to the DefDoc > project) would be much easier and also useful. I wanted to use the cl-typesetting box model. I'ts more powerful (min, max, compression, expansion) than only percentage . Marc From marc.battyani at fractalconcept.com Wed Apr 21 08:59:44 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 21 Apr 2004 10:59:44 +0200 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> <20040421052548.GA15678@w-m-p.com> Message-ID: <0fa401c4277e$fed019b0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > I've only gotten it to do what I want with a really ugly hack - the hfill > added at the end of each left-justified line needed to be left out on the > last line, otherwise the leader at that place didn't reach to the right > margin. I didn't see an obvious way to prevent the hfill from being > attached, so I got violent and added an extraction routine ;-) I think it will be easier to just change the alignement/margins settings. I will try this tonight. > Also, I've hacked the enumerate function to be a bit smarter, so that it > lines up the margins properly. To support that, I added a > put-filled-string function which aligns a string within a predefined > horizontal area. That also took some hacking - I may have missed some > obvious solution. Putting real boxes inline messed up the formatting... You should use a hbox with hfills IMO. I will also have a look at this. The best solution would be to add tab stops ;-) > Lastly, I've experimented with tables a bit - is it currently really > required to explicitly specify column widths? If so, making that dynamic > would be a fairly important project. Computing table column widths is not an easy problem. An intermediate improvement over fixed column sizes would be to use the box model for this. > I've attached the current output of the XHTML converter, which is > starting to look promising. Still missing are strikethrough and > underline, which I'm too tired to figure out how to implement. Very good. When do you think it will be publishable ? > One more thing, is there some easy way to handle widow/orphan prevention > automatically, i.e. by specifying a 'badness' in the interline vspace? No, not yet. Marc From kw at w-m-p.com Wed Apr 21 13:35:03 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Wed, 21 Apr 2004 08:35:03 -0500 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) In-Reply-To: <0fa401c4277e$fed019b0$0a02a8c0@marcxp> References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> <20040421052548.GA15678@w-m-p.com> <0fa401c4277e$fed019b0$0a02a8c0@marcxp> Message-ID: <20040421133503.GA17472@w-m-p.com> On Wed, Apr 21, 2004 at 10:59:44AM +0200, Marc Battyani wrote: > "Klaus Weidner" wrote: > > Also, I've hacked the enumerate function to be a bit smarter, so that it > > lines up the margins properly. To support that, I added a > > put-filled-string function which aligns a string within a predefined > > horizontal area. That also took some hacking - I may have missed some > > obvious solution. Putting real boxes inline messed up the formatting... > > You should use a hbox with hfills IMO. I will also have a look at this. I tried that, and it did not work at all the way I expected it to. Hboxes intermixed with text were placed on their own lines, not inline. Stacking them inside vboxes kept them in-line, but with completely screwed-up baselines, and the :align flag just changed which way it was screwed up. > The best solution would be to add tab stops ;-) I think that way lies madness... Making a specialized box that expands to a fixed x position wouldn't be hard, but my gut feeling is that this will get the layout model too close to the Word ruler-based one, which is broken. > Computing table column widths is not an easy problem. An intermediate > improvement over fixed column sizes would be to use the box model for this. I'd just need to find some type of ad-hoc solution, since HTML files don't always supply widths, or not for all columns. In fact one of the reasons I want this is because there's currently no reasonable way to print complex HTML tables in browsers. The only tool I know that does a good job is html2ps, but unfortunately that's an unholy combination of obfuscated Perl with obfuscated PostScript. I'll try to figure out how its table model works (the layout calculations are done in PostScript), maybe that will provide some pointers. Or is anyone willing to concede defeat and admit that PostScript can do something Lisp can't? ;-) > > I've attached the current output of the XHTML converter, which is > > starting to look promising. Still missing are strikethrough and > > underline, which I'm too tired to figure out how to implement. > > Very good. When do you think it will be publishable ? It's getting close. It's already starting to be useful, and the highest priority changes are a proper table of contents (currently it just contains the

level), clickable hyperlinks, and a few more missing markup items. What I personally need most urgently is change marking including change bars, and I already have a plan for that - just insert (changebar-start) and (changebar-end) special boxes, and add a special-fn to the output routine to draw lines in the margin at the corresponding y coordinates. Is there an easy way to handle strikethrough and underline? Also, being able to set the background color for character boxes would be a nice way to mark the changes. Should be doable with a special type of box, but it needs to be a style inherited automatically by whatever layout commands it affects - just a put-bg-colored-string isn't useful for automation. -Klaus From alemmens at xs4all.nl Wed Apr 21 18:40:40 2004 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Wed, 21 Apr 2004 20:40:40 +0200 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) In-Reply-To: <0f2401c4276e$00e63830$0a02a8c0@marcxp> References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com><0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp><20040421052548.GA15678@w-m-p.com> <002f01c4276d$6e46d1b0$474302c3@digo> <0f2401c4276e$00e63830$0a02a8c0@marcxp> Message-ID: Marc Battyani wrote: > I wanted to use the cl-typesetting box model. I'ts more powerful (min, max, > compression, expansion) than only percentage. If you haven't done that already, you could take a look at CLIM's layout protocol (including the space-requirement stuff, see chapter 29 of the CLIM 2 spec). It combines a box-like model (which I assume is similar to yours, but I haven't had the time to take a serious look at cl-typesetting) with the possibility to specify fixed widths or percentages. If you're interested, I have written an (untested) implementation of (90 percent of) CLIM's layout protocol. So have the McClim guys, I suppose, but I don't like their license. (I hope this is relevant to the discussion, but I know almost nothing about cl-typesetting so I may be barking up the wrong tree.) Regards, Arthur Lemmens From marc.battyani at fractalconcept.com Wed Apr 21 20:27:45 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 21 Apr 2004 22:27:45 +0200 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> <20040421052548.GA15678@w-m-p.com> <0fa401c4277e$fed019b0$0a02a8c0@marcxp> <20040421133503.GA17472@w-m-p.com> Message-ID: <124f01c427df$1bc6a510$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Wed, Apr 21, 2004 at 10:59:44AM +0200, Marc Battyani wrote: > > > > You should use a hbox with hfills IMO. I will also have a look at this. > > I tried that, and it did not work at all the way I expected it to. > Hboxes intermixed with text were placed on their own lines, not inline. > Stacking them inside vboxes kept them in-line, but with completely > screwed-up baselines, and the :align flag just changed which way it > was screwed up. Sorry I was wrong, in fact the hbox is used in the math mode and is a v-mode-mixin box. > > The best solution would be to add tab stops ;-) > > I think that way lies madness... Making a specialized box that expands to > a fixed x position wouldn't be hard, but my gut feeling is that this will > get the layout model too close to the Word ruler-based one, which is > broken. Hum, in Word all is broken anyway. ;-) The typesetting is just ugly. > > Computing table column widths is not an easy problem. An intermediate > > improvement over fixed column sizes would be to use the box model for this. > > I'd just need to find some type of ad-hoc solution, since HTML files > don't always supply widths, or not for all columns. In fact one of the > reasons I want this is because there's currently no reasonable way to > print complex HTML tables in browsers. > > The only tool I know that does a good job is html2ps, but unfortunately > that's an unholy combination of obfuscated Perl with obfuscated > PostScript. I'll try to figure out how its table model works (the layout > calculations are done in PostScript), maybe that will provide some > pointers. > > Or is anyone willing to concede defeat and admit that PostScript can do > something Lisp can't? ;-) Heh! Are you really sure it works well ? There are several papers on tables layout but none has a magic recipe... > > Very good. When do you think it will be publishable ? > > It's getting close. It's already starting to be useful, and the highest > priority changes are a proper table of contents (currently it just > contains the

level), clickable hyperlinks, and a few more missing > markup items. > > What I personally need most urgently is change marking including change > bars, and I already have a plan for that - just insert (changebar-start) > and (changebar-end) special boxes, and add a special-fn to the output > routine to draw lines in the margin at the corresponding y coordinates. Yes, this one is easy. Just do it after the layout pass. You can look at the example where I draw circles (and join them) under each #\a and #\Space of the text. > Is there an easy way to handle strikethrough and underline? Also, being > able to set the background color for character boxes would be a nice way > to mark the changes. Should be doable with a special type of box, but it > needs to be a style inherited automatically by whatever layout commands > it affects - just a put-bg-colored-string isn't useful for automation. Strikethrough and underline? Well this should be added to text-style. Drawing them is probably not difficult I will add this to my "to do soon" list ;-) I have variables and styles for background color but it's not used yet. If I find the time to do the underline stuff I will do that as well as it's probably almost the same code. Marc From marc.battyani at fractalconcept.com Wed Apr 21 20:28:02 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 21 Apr 2004 22:28:02 +0200 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com><0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp><20040421052548.GA15678@w-m-p.com> <002f01c4276d$6e46d1b0$474302c3@digo> <0f2401c4276e$00e63830$0a02a8c0@marcxp> Message-ID: <125201c427df$260664c0$0a02a8c0@marcxp> "Arthur Lemmens" wrote: > Marc Battyani wrote: > > > I wanted to use the cl-typesetting box model. I'ts more powerful (min, max, > > compression, expansion) than only percentage. > > If you haven't done that already, you could take a look at CLIM's > layout protocol (including the space-requirement stuff, see chapter 29 > of the CLIM 2 spec). It combines a box-like model (which I assume is > similar to yours, but I haven't had the time to take a serious look > at cl-typesetting) with the possibility to specify fixed widths or > percentages. > > If you're interested, I have written an (untested) implementation of > (90 percent of) CLIM's layout protocol. So have the McClim guys, I > suppose, but I don't like their license. > > (I hope this is relevant to the discussion, but I know almost nothing > about cl-typesetting so I may be barking up the wrong tree.) We were talking about computing the widths of the columns from the content of the cells in tables. Is this what you have ? Marc From kw at w-m-p.com Wed Apr 21 20:50:57 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Wed, 21 Apr 2004 15:50:57 -0500 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) In-Reply-To: <124f01c427df$1bc6a510$0a02a8c0@marcxp> References: <20040418100408.GA17606@w-m-p.com> <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> <20040421052548.GA15678@w-m-p.com> <0fa401c4277e$fed019b0$0a02a8c0@marcxp> <20040421133503.GA17472@w-m-p.com> <124f01c427df$1bc6a510$0a02a8c0@marcxp> Message-ID: <20040421205057.GA21965@w-m-p.com> On Wed, Apr 21, 2004 at 10:27:45PM +0200, Marc Battyani wrote: > "Klaus Weidner" wrote: > > The only tool I know that does a good job is html2ps, but unfortunately > > that's an unholy combination of obfuscated Perl with obfuscated > > PostScript. I'll try to figure out how its table model works (the layout > > calculations are done in PostScript), maybe that will provide some > > pointers. > > > > Or is anyone willing to concede defeat and admit that PostScript can do > > something Lisp can't? ;-) > > Heh! Are you really sure it works well ? There are several papers on tables > layout but none has a magic recipe... I'm not saying that html2ps works perfectly, but it *is* far better than any other tools I tried, including all popular web browsers. At least from the point of view of people who want to actually have the content of the table be on the page when printed, rather than arbitrarily truncated somewhere. One trick it does is to shrink the font size if there's no other way to fit the table on the page. But that's only a last resort, the default layout algorithm is good enough that this is only needed in extreme cases. Let me know if you're interested in examples, and I'll make some. > > What I personally need most urgently is change marking including change > > bars, and I already have a plan for that - just insert (changebar-start) > > and (changebar-end) special boxes, and add a special-fn to the output > > routine to draw lines in the margin at the corresponding y coordinates. > > Yes, this one is easy. Just do it after the layout pass. You can look at the > example where I draw circles (and join them) under each #\a and #\Space of > the text. That's where I got the idea ;-) It's great that this is so easy, compared to how the pain involved in altering the LaTeX output routines even slightly. > Strikethrough and underline? Well this should be added to text-style. > Drawing them is probably not difficult I will add this to my "to do soon" > list ;-) > > I have variables and styles for background color but it's not used yet. If I > find the time to do the underline stuff I will do that as well as it's > probably almost the same code. A text style makes the most sense. The obvious implementation would do this character by character - an optimization might be to use longer lines (or boxes for the background) for multiple characters, but I'm not sure if that would be worth the effort. Hmmm, one approach would be to support a function-valued style parameter, which is called when the character is stroked and can add whatever decoration it wants, based on the box geometry. Ideally this would be stackable, so that for example colored backgrounds don't get turned off when an underline is added in a subsidiary span. If I have some spare time I may look at it myself, but no promises... -Klaus From kw at w-m-p.com Thu Apr 22 05:38:32 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Thu, 22 Apr 2004 00:38:32 -0500 Subject: [cl-typesetting-devel] Decorations and change bars In-Reply-To: <20040421205057.GA21965@w-m-p.com> References: <034e01c42543$1e5e9c90$0a02a8c0@marcxp> <20040418170552.GB20748@w-m-p.com> <091401c42664$ce6b4c60$0a02a8c0@marcxp> <20040419235905.GA8493@w-m-p.com> <0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp> <20040421052548.GA15678@w-m-p.com> <0fa401c4277e$fed019b0$0a02a8c0@marcxp> <20040421133503.GA17472@w-m-p.com> <124f01c427df$1bc6a510$0a02a8c0@marcxp> <20040421205057.GA21965@w-m-p.com> Message-ID: <20040422053832.GA25451@w-m-p.com> On Wed, Apr 21, 2004 at 03:50:57PM -0500, Klaus Weidner wrote: > Hmmm, one approach would be to support a function-valued style parameter, > which is called when the character is stroked and can add whatever > decoration it wants, based on the box geometry. Ideally this would be > stackable, so that for example colored backgrounds don't get turned off > when an underline is added in a subsidiary span. Done, see attached patch. It adds pre-decoration and post-decoration styles, which can be keywords (not used at this time) or functions that are called before or after drawing characters. One thing that kept me busy debugging for some time was the effect that whenever a style property is NIL, any value assigned to it will be permanent since the NIL will not be restored. I've added a comment about that in specials.lisp. A small change to top-level.lisp was needed to support a :finalize-fn argument for a function that is called on each complete page. Implementing change bars was almost trivial after that, including left/right margin alteration for even/odd pages. And that's something that apparently isn't even possible in LaTeX... With these two changes, I got the fancy change markings I wanted :-) See attached PDF and tt-render.lisp. The latter is just a demo and too messy for inclusion, I'll try to find the time to clean it up when I'm less tired. This is addictive. -Klaus -------------- next part -------------- diff -urN orig/cl-typesetting/layout.lisp cl-typesetting/layout.lisp --- orig/cl-typesetting/layout.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/layout.lisp Wed Apr 21 16:50:35 2004 @@ -38,7 +38,9 @@ (background-color style) (h-align style) (left-margin style) - (right-margin style)))) + (right-margin style) + (pre-decoration style) + (post-decoration style)))) ;;This would need a complete rewrite... (defmethod v-split ((content text-content) dx dy &optional (v-align :top)) @@ -90,7 +92,7 @@ (if line-boxes (let ((text-line (make-instance 'text-line :dx dx :adjustable-p t))) (setf line-boxes (boxes-left-trim line-boxes)) - (when (member *h-align* '(:center :left)) + (when (member *h-align* '(:center :left :left-not-last)) (push (make-hfill-glue) line-boxes)) (unless (zerop *right-margin*) (push (make-instance 'h-spacing :dx *right-margin*) line-boxes)) @@ -130,7 +132,15 @@ (push box text-lines)) ((and trimming (trimmable-p box)) nil) ((eq box :eol) - #+nil(when (eq *h-align* :justified) + (when (eq *h-align* :left-not-last) + ;; incredibly ugly hack - need to get rid of the + ;; superfluous box added on the last line. + (let ((fbox (find-if (lambda (box) + (and (typep box 'h-spacing) + (eql (expansibility box) +huge-number+))) + line-boxes))) + (if fbox (setq line-boxes (remove fbox line-boxes))))) + (when (eq *h-align* :justified) (push (make-hfill-glue) line-boxes)) (next-line line-boxes)) ((eq box :fresh-page) diff -urN orig/cl-typesetting/specials.lisp cl-typesetting/specials.lisp --- orig/cl-typesetting/specials.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/specials.lisp Wed Apr 21 18:44:17 2004 @@ -11,6 +11,8 @@ ;; ;; FLAG -- collect all these in *default-text-style* and *current-text-style* ;; djc +;; Note: Don't let any of these variables become NIL, otherwise +;; that style won't be restored after a change. cf. typo.lisp ;; (defvar *default-font* (pdf:get-font)) (defvar *default-font-size* 12.0) @@ -21,12 +23,16 @@ (defvar *default-v-align* :top) (defvar *default-left-margin* 0) (defvar *default-right-margin* 0) +(defvar *default-pre-decoration* :none) +(defvar *default-post-decoration* :none) (defvar *font* *default-font*) (defvar *font-size* *default-font-size*) (defvar *text-x-scale* *default-text-x-scale*) (defvar *color* *default-color*) (defvar *background-color* *default-background-color*) +(defvar *pre-decoration* *default-pre-decoration*) +(defvar *post-decoration* *default-post-decoration*) (defvar *h-align* *default-h-align*) (defvar *v-align* *default-v-align*) (defvar *left-margin* *default-left-margin*) diff -urN orig/cl-typesetting/stroke.lisp cl-typesetting/stroke.lisp --- orig/cl-typesetting/stroke.lisp Wed Mar 10 13:12:49 2004 +++ cl-typesetting/stroke.lisp Wed Apr 21 23:02:55 2004 @@ -7,6 +7,24 @@ (defmethod stroke (box x y) ) +(defmethod stroke :before ((box box) x y) + (if (and (functionp *pre-decoration*) + (or (typep box 'char-box) + (typep box 'white-char-box))) + (funcall *pre-decoration* + box + x (+ y (baseline box) (offset box)) + (dx box) (- (dy box))))) + +(defmethod stroke :after ((box box) x y) + (if (and (functionp *post-decoration*) + (or (typep box 'char-box) + (typep box 'white-char-box))) + (funcall *post-decoration* + box + x (+ y (baseline box) (offset box)) + (dx box) (- (dy box))))) + (defmethod stroke ((hbox hbox) x y) (decf x (baseline hbox)) (decf x (offset hbox)) @@ -73,6 +91,10 @@ for size = (+ (dx box)(delta-size box)) do (cond + ((or (functionp *pre-decoration*) + (functionp *post-decoration*)) + (end-text-chunk) + (stroke box x y)) ((char-box-p box)(add-char box)) ((white-space-p box) (add-spacing size)) (t (end-text-chunk)(stroke box x y))) @@ -88,4 +110,8 @@ (setf *text-x-scale* (text-x-scale style))) (when (color style) (setf *color* (color style)) - (pdf::set-color-fill *color*))) + (pdf::set-color-fill *color*)) + (when (pre-decoration style) + (setf *pre-decoration* (pre-decoration style))) + (when (post-decoration style) + (setf *post-decoration* (post-decoration style)))) diff -urN orig/cl-typesetting/top-level.lisp cl-typesetting/top-level.lisp --- orig/cl-typesetting/top-level.lisp Mon Mar 22 03:09:13 2004 +++ cl-typesetting/top-level.lisp Wed Apr 21 23:24:03 2004 @@ -62,6 +62,7 @@ (header-top *default-page-header-footer-margin*) (footer-bottom *default-page-header-footer-margin*) break + finalize-fn &allow-other-keys) ;;; Args: ;; content Text content, multi-page-table, or other content. @@ -82,6 +83,7 @@ :footer-bottom footer-bottom ;; Move room-left into initialize-instance :after? :room-left (- height top-margin bottom-margin) + :finalize-fn finalize-fn (remove-properties args '(:size :orientation :bounds :header-top :footer-bottom :break)))))) diff -urN orig/cl-typesetting/typo.lisp cl-typesetting/typo.lisp --- orig/cl-typesetting/typo.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/typo.lisp Thu Apr 22 00:27:57 2004 @@ -24,7 +24,9 @@ (background-color :accessor background-color :initarg :background-color :initform nil) (h-align :accessor h-align :initarg :h-align :initform nil) (left-margin :accessor left-margin :initarg :left-margin :initform nil) - (right-margin :accessor right-margin :initarg :right-margin :initform nil))) + (right-margin :accessor right-margin :initarg :right-margin :initform nil) + (pre-decoration :accessor pre-decoration :initarg :pre-decoration :initform nil) + (post-decoration :accessor post-decoration :initarg :post-decoration :initform nil))) (defmethod initialize-instance :after ((obj text-style) &rest args &key font &allow-other-keys) (when font (setf (font obj) font))) @@ -75,7 +77,9 @@ (%use-style% background-color *background-color*) (%use-style% left-margin *left-margin*) (%use-style% right-margin *right-margin*) - (%use-style% h-align *h-align*)) + (%use-style% h-align *h-align*) + (%use-style% pre-decoration *pre-decoration*) + (%use-style% post-decoration *post-decoration*)) (when (or (font style)(font-size style)) (setf *leading* (* *font-size* *leading-ratio*)))) @@ -87,7 +91,9 @@ (setf (background-color style) *background-color*) (setf (left-margin style) *left-margin*) (setf (right-margin style) *right-margin*) - (setf (h-align style) *h-align*)) + (setf (h-align style) *h-align*) + (setf (pre-decoration style) *pre-decoration*) + (setf (post-decoration style) *post-decoration*)) (defmethod restore-default-style ((style text-style)) (setf (font style) *default-font*) @@ -97,7 +103,9 @@ (setf (background-color style) *default-background-color*) (setf (left-margin style) *default-left-margin*) (setf (right-margin style) *default-right-margin*) - (setf (h-align style) *default-h-align*)) + (setf (h-align style) *default-h-align*) + (setf (pre-decoration style) *default-pre-decoration*) + (setf (post-decoration style) *default-post-decoration*)) (defmethod make-restore-style ((style text-style)) (let ((new-style (make-instance 'text-style))) @@ -111,7 +119,9 @@ (%use-style% background-color *background-color*) (%use-style% left-margin *left-margin*) (%use-style% right-margin *right-margin*) - (%use-style% h-align *h-align*)) + (%use-style% h-align *h-align*) + (%use-style% pre-decoration *pre-decoration*) + (%use-style% post-decoration *post-decoration*)) new-style)) (defmethod copy-style ((style text-style)) @@ -121,7 +131,9 @@ :background-color (background-color style) :left-margin (left-margin style) :right-margin (right-margin style) - :h-align (h-align style))) + :h-align (h-align style) + :pre-decoration (pre-decoration style) + :post-decoration (post-decoration style))) (defclass text-line (hbox) ()) @@ -216,8 +228,8 @@ (add-box (make-char-box char)) (add-box (make-inter-char-glue)))))))) -;;; put a string in a 'verbatim' way: no kerning, no hyphenation, significant whitespaces, significant newlines (defun verbatim (string) + "put a string in a 'verbatim' way: no kerning, no hyphenation, significant whitespaces, significant newlines" (when (stringp string) (loop for char across string for i from 0 @@ -273,7 +285,9 @@ (*h-align* *default-h-align*) (*v-align* *default-v-align*) (*left-margin* *default-left-margin*) - (*right-margin* *default-right-margin*)) + (*right-margin* *default-right-margin*) + (*pre-decoration* *default-pre-decoration*) + (*post-decoration* *default-post-decoration*)) (progn , at body)))) (defmacro compile-text ((&rest args) &body body) -------------- next part -------------- A non-text attachment was scrubbed... Name: stuff.pdf Type: application/pdf Size: 5153 bytes Desc: not available URL: -------------- next part -------------- (in-package typeset) ;; reference handling (defvar *ref-table* (make-hash-table :test #'equal)) (defvar *ref-counter* 0) (defvar *bad-reference* nil) (defclass ref-mark () ((id :accessor ref-id :initform nil :initarg :id) (value :accessor ref-mark-value :initform nil :initarg :value) (page :accessor ref-mark-page :initform nil) (x :accessor ref-x :initform nil) (y :accessor ref-y :initform nil))) (defmethod stroke ((mark ref-mark) x y) (setf (ref-mark-page mark) (length (pages pdf:*document*)) (ref-x mark) x (ref-y mark) y)) (defmacro ref-get (id) `(gethash ,id *ref-table*)) (defun make-ref-mark (id &optional value) (let ((mark (or (ref-get id) (make-instance 'ref-mark :id id)))) (setf (ref-get id) mark) (setf (ref-mark-value mark) value) (add-box mark))) (defun ref-page (id) (let* ((ref (ref-get id)) (page (if ref (ref-mark-page ref)))) (cond (page page) (t (push id *bad-reference*) 999)))) (defun put-ref-page (id) (put-string (format nil "~D" (ref-page id)))) (defgeneric ref-value (ref)) (defmethod ref-value ((ref ref-mark)) (if ref (ref-mark-value ref))) (defmethod ref-value ((id t)) (let ((ref (ref-get id))) (if ref (ref-mark-value ref)))) (defun put-ref-value (id) (put-string (ref-value id))) (defun this-page-number () (length (pages pdf:*document*))) (defun make-ref-page-mark (reftype value) (make-ref-mark (cons reftype (incf *ref-counter*)) value)) (defun get-latest-ref-to (reftype for-page) (let ((refs nil)) ;; Find all references of a type, store unsorted ;; (ordinal page ref) lists in "refs". (maphash (lambda (key ref) (if (and (consp key) (equal reftype (car key))) (push (list (cdr key) (or (ref-mark-page ref) most-positive-fixnum) ref) refs))) *ref-table*) ;; Now walk through the reverse sorted references, ;; and get the last matching one on or before the ;; current page. (third (find-if (lambda (page) (<= page for-page)) (sort refs #'> :key #'car) :key #'cadr)))) (defun current-ref-value (reftype) (ref-value (get-latest-ref-to reftype (this-page-number)))) (defmacro itemize ((&key (indent 20) text-style (item-fmt "~D. ") (start-from 1) item-style) &body body) `(let ((%enumerate-indents% (cons ,indent %enumerate-indents%))) ,@(loop for item in body for i from start-from collect `(paragraph (:left-margin (reduce #'+ %enumerate-indents%) :first-line-indent (- ,indent) , at text-style) (with-style ,item-style (put-filled-string ,(format nil item-fmt i) ,indent :align :right)) ,item)))) (defmacro item ((&rest style) &body body) `(with-style ,style , at body)) (defun put-filled-string (string width &key (align :left)) "place aligned string in fixed-width space" (let* ((string-width (loop for char across string summing (pdf:get-char-width char *font* *font-size*))) (blank (- width string-width))) (case align ((:left) (verbatim string) (hspace blank)) ((:center) (hspace (* 0.5 blank)) (verbatim string) (hspace (* 0.5 blank))) ((:right) (hspace blank) (verbatim string))))) ;; higher-level layout (defun safe-read (stream) (let ((*package* (find-package "TYPESET")) (*read-eval* nil)) (read stream))) ;; change bars (defvar *change-bar-start* nil) (defvar *change-bar-end* nil) (defclass change-mark () ((type :accessor mark-type :initform nil :initarg :type))) (defmethod stroke ((mark change-mark) x y) (cond ((eq :start-insert (mark-type mark)) (push (cons (+ y *font-size*) :insert) *change-bar-start*)) ((eq :start-delete (mark-type mark)) (push (cons (+ y *font-size*) :delete) *change-bar-start*)) (t (push y *change-bar-end*)))) (defun change-start-insert () (add-box (make-instance 'change-mark :type :start-insert))) (defun change-start-delete () (add-box (make-instance 'change-mark :type :start-delete))) (defun change-end () (add-box (make-instance 'change-mark :type :end))) (defun page-decorations (page) (pdf:with-saved-state (pdf:set-line-width 2.0) (let ((xm (if (oddp (this-page-number)) (* 0.95 (aref (pdf::bounds page) 2)) (* 0.05 (aref (pdf::bounds page) 2))))) (loop for y0c in *change-bar-start* for y1 in *change-bar-end* do (let* ((y0 (car y0c)) (type (cdr y0c)) (color (if (eq type :insert) #x33aa33 #xaa3333)) (x (if (eq type :insert) xm (- xm 4)))) (pdf:set-color-stroke color) (pdf:move-to x y0) (pdf:line-to x y1) (pdf:stroke))))) (setq *change-bar-start* nil *change-bar-end* nil)) ;; Note that the tree argument to render-document is a dead list of ;; symbols and strings. This is a prerequisite for being to handle ;; documents that are completely generated at runtime. (defun render-document (tree &key (file #P"/tmp/stuff.pdf") (twosided t) (paper-size :letter)) "Render the document specified by tree, which is a s-exp containing recursive typesetting commands. It gets eval'ed here to typeset it." (do ((*ref-table* (make-hash-table :test #'equal)) (*ref-counter* 0) (*bad-reference* nil) (pass 0 (1+ pass))) ((or (> pass 1) (and (> pass 0) (not *bad-reference*))) *bad-reference*) (setq *bad-reference* nil) (format t "Pass ~d~%" pass) (with-document () (let ((margins '(72 72 72 50)) (header (lambda (pdf:*page*) (if (current-ref-value :header-enabled) (let ((inside (or (current-ref-value :title) "Untitled Document")) (outside (current-ref-value :chapter))) (if (and twosided (evenp (this-page-number))) (compile-text (:font "Times-Roman" :font-size 10) (hbox (:align :center :adjustable-p t) (put-string outside) :hfill (with-style (:font "Times-Italic") (put-string inside)))) (compile-text (:font "Times-Roman" :font-size 10) (hbox (:align :center :adjustable-p t) (with-style (:font "Times-Italic") (put-string inside)) :hfill (put-string outside)))))))) (footer (lambda (pdf:*page*) (if (current-ref-value :footer-enabled) (let ((inside (or (current-ref-value :version) "")) (outside (format nil "Page ~d of ~d" (this-page-number) (ref-page "DocumentEnd")))) (if (and twosided (evenp (this-page-number))) (compile-text (:font "Times-Roman" :font-size 10) (hbox (:align :center :adjustable-p t) (put-string outside) :hfill (put-string inside))) (compile-text (:font "Times-Roman" :font-size 10) (hbox (:align :center :adjustable-p t) (put-string inside) :hfill (put-string outside))))))))) (draw-pages (eval (list 'compile-text () tree)) :margins margins :header header :footer footer :size paper-size :finalize-fn #'page-decorations) (when pdf:*page* (finalize-page pdf:*page*)) (pdf:write-document file))))) ;; Example follows. (defun decoration-random-background (box x y dx dy) (pdf:with-saved-state (pdf:set-rgb-fill (random 1.0) (random 1.0) (random 1.0)) (pdf:basic-rect x y dx dy) (pdf:fill-path))) (defun decoration-green-background (box x y dx dy) (pdf:with-saved-state (pdf:set-rgb-fill 0.7 1.0 0.7) (pdf:basic-rect x y dx dy) (pdf:fill-path))) (defun decoration-gray-box (box x y dx dy) (pdf:with-saved-state (pdf:set-gray-stroke 0.5) (pdf:set-line-width 0.5) (pdf:basic-rect x y dx dy) (pdf:stroke))) (defun decoration-underline (box x y dx dy) (pdf:with-saved-state (pdf:set-gray-stroke 0) (pdf:set-line-width 0.5) (pdf:move-to x (+ y (* 0.9 dy))) (pdf:line-to (+ x dx) (+ y (* 0.9 dy))) (pdf:stroke))) (defun decoration-strikethrough (box x y dx dy) (pdf:with-saved-state (pdf:set-color-stroke :red) (pdf:set-line-width 0.5) (pdf:move-to x (+ y (* 0.66 dy))) (pdf:line-to (+ x dx) (+ y (* 0.66 dy))) (pdf:stroke))) (defun decoration-crosshatch (box x y dx dy) (pdf:with-saved-state (pdf:set-color-stroke :black) (pdf:set-line-width 0.5) (pdf:move-to x (+ y (* 0.3 dy))) (pdf:line-to (+ x dx) (+ y (* 0.9 dy))) (pdf:stroke))) (defun decoration-nil (box x y dx dy) (print "Called nil decoration.") nil) (defun document-test () (render-document '(with-style (:font "Times-Roman" :font-size 12 :top-margin 3 :bottom-margin 4) (make-ref-page-mark :title "Titled Document") (make-ref-page-mark :version "Version 1.x") (make-ref-page-mark :header-enabled nil) (make-ref-page-mark :footer-enabled nil) #|| :vfill (paragraph (:font "Helvetica-Bold" :font-size 24 :h-align :center :bottom-margin 20) "This is the Document Title") (paragraph (:font "Helvetica-Bold" :font-size 16 :h-align :center) "A. N. Author") :vfill :eop ||# (make-ref-page-mark :header-enabled t) (make-ref-page-mark :footer-enabled t) (make-ref-mark '(:chapter . 0) "Table of Contents") (with-style (:font "Helvetica") (paragraph (:h-align :left-but-last :top-margin 3 :bottom-margin 4) (put-ref-value '(:chapter . 1)) (dotted-hfill) (put-ref-page '(:chapter . 1))) (paragraph (:h-align :left-but-last :top-margin 3 :bottom-margin 4) (put-ref-value '(:chapter . 2)) "This is a chapter with an insanely long title, to verify if the leader dots at the end of the line will be printed properly" (dotted-hfill) (put-ref-page '(:chapter . 2)))) #|| :eop ||# (make-ref-mark '(:chapter . 1) "Introduction") (paragraph (:h-align :left :top-margin 3 :bottom-margin 4) "Test with " (with-style (:font "Times-Bold") "bold") " and " (with-style (:font "Times-Italic") "italic") " text.") #|| (paragraph (:h-align :left :top-margin 3 :bottom-margin 4) (make-ref-mark "link-from") "See also stuff on page " (put-ref-page "stuff") ".") ||# (paragraph (:top-margin 3 :bottom-margin 4) "Inline alignment test: [" (put-filled-string "L" 30) "][" (put-filled-string "C" 30 :align :center) "][" (put-filled-string "R" 30 :align :right) "]") (paragraph (:top-margin 3 :bottom-margin 4) "This is just normal text. " (with-style (:pre-decoration #'decoration-random-background) "This should look different.") " Back to normal. There's more; " (with-style (:post-decoration #'decoration-underline) "multi word underline") " and " (with-style (:pre-decoration #'decoration-gray-box) "visible boxes mode") " and " (with-style (:post-decoration #'decoration-crosshatch) "crosshatch.")) (paragraph (:top-margin 3 :bottom-margin 4) "This paragraph is not interesting.") (paragraph (:h-align :left :top-margin 3 :bottom-margin 4) "This paragraph has some " (change-start-insert) (with-style (:pre-decoration #'decoration-green-background) "inserted words") (change-end) " in it. Here's some filler to move to the next line. The now following line has both " (change-start-insert) (with-style (:pre-decoration #'decoration-green-background) "inserted words") (change-end) " and " (change-start-delete) (with-style (:post-decoration #'decoration-strikethrough) "deleted ones") "." (change-end) " Now here's even more filler text to again move to the next line, to demonstrate having just the following word " (change-start-delete) (with-style (:post-decoration #'decoration-strikethrough) "deleted") (change-end) ".") #|| (itemize (:text-style (:h-align :left :top-margin 3 :bottom-margin 4)) (item () "This is the first item, and it's rather long-winded. wjr aireg iureahg iureahg iureahg iureahg lrea hlieahg eliurhg eliurhg eliurhg liureahglueairhg liurea hliure hgliueahg liureahg liurea hgliureahg liureahg liureahg liureag realih." (itemize (:text-style (:top-margin 3 :bottom-margin 4) :item-fmt "- ") (item () "a" "1") (item () "b" "2") (item () "c") (item () "d"))) (item () "This is the second item, and it's rather long-winded. wjr aireg iureahg iureahg iureahg iureahg lrea hlieahg eliurhg eliurhg eliurhg liureahglueairhg liurea hliure hgliueahg liureahg liurea hgliureahg liureahg liureahg liureag realih.")) :eop (make-ref-mark '(:chapter . 2) "Interesting Stuff") (paragraph (:font "Courier" :top-margin 3 :bottom-margin 4) (make-ref-mark "stuff") "Some" :eol "more" :eol "Text." ) (paragraph (:h-align :left :top-margin 3 :bottom-margin 4) "This is linked to from page " (put-ref-page "link-from") ".") ||# (make-ref-mark "DocumentEnd")))) From marc.battyani at fractalconcept.com Thu Apr 22 10:57:07 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Thu, 22 Apr 2004 12:57:07 +0200 Subject: [cl-typesetting-devel] Decorations and change bars References: <034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com><0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp><20040421052548.GA15678@w-m-p.com><0fa401c4277e$fed019b0$0a02a8c0@marcxp><20040421133503.GA17472@w-m-p.com><124f01c427df$1bc6a510$0a02a8c0@marcxp><20040421205057.GA21965@w-m-p.com> <20040422053832.GA25451@w-m-p.com> Message-ID: <14bc01c42858$8fe1ceb0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Wed, Apr 21, 2004 at 03:50:57PM -0500, Klaus Weidner wrote: > > Hmmm, one approach would be to support a function-valued style parameter, > > which is called when the character is stroked and can add whatever > > decoration it wants, based on the box geometry. Ideally this would be > > stackable, so that for example colored backgrounds don't get turned off > > when an underline is added in a subsidiary span. > > Done, see attached patch. It adds pre-decoration and post-decoration > styles, which can be keywords (not used at this time) or functions that > are called before or after drawing characters. > > One thing that kept me busy debugging for some time was the effect that > whenever a style property is NIL, any value assigned to it will be > permanent since the NIL will not be restored. I've added a comment about > that in specials.lisp. > > A small change to top-level.lisp was needed to support a :finalize-fn > argument for a function that is called on each complete page. > Implementing change bars was almost trivial after that, including > left/right margin alteration for even/odd pages. And that's something > that apparently isn't even possible in LaTeX... Yeah I like that! (Stuff not possible with LaTeX) > With these two changes, I got the fancy change markings I wanted :-) See > attached PDF and tt-render.lisp. The latter is just a demo and too messy > for inclusion, I'll try to find the time to clean it up when I'm less > tired. This is addictive. Great! I know it's messy but I feel it's better to publish it anyway so that others can try and test it. So for now I left all the new code in a kw-extensions.lisp file but I will dispatch it to better locations later. I will also change the test example to show the new possibilities. (and move your tests functions to there) I've merged all this in the repository and the tarball. Marc From marc.battyani at fractalconcept.com Thu Apr 22 11:17:06 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Thu, 22 Apr 2004 13:17:06 +0200 Subject: dotted hfill (was: [cl-typesetting-devel] HTML rendering) References: <20040418100408.GA17606@w-m-p.com><034e01c42543$1e5e9c90$0a02a8c0@marcxp><20040418170552.GB20748@w-m-p.com><091401c42664$ce6b4c60$0a02a8c0@marcxp><20040419235905.GA8493@w-m-p.com><0dfc01c42726$d4e3b5b0$0a02a8c0@marcxp><20040421052548.GA15678@w-m-p.com> <002f01c4276d$6e46d1b0$474302c3@digo> <0f2401c4276e$00e63830$0a02a8c0@marcxp> <125201c427df$260664c0$0a02a8c0@marcxp> Message-ID: <14cf01c4285b$59d89e40$0a02a8c0@marcxp> "Arthur Lemmens" wrote: > Marc Battyani wrote: > > > We were talking about computing the widths of the columns from the content > > of the cells in tables. > > Is this what you have ? > > That depends on your definition of 'tables' ;-) The real context here is printing (X)HTML documents so it's HTML like tables. > The CLIM model is basically a nested hierarchy of boxes ('panes' in > CLIM's terminology), pretty much like TeX. Each box has a space requirement, > indicating its minimum, preferred and maximum width/height. COMPOSE-SPACE > computes the space requirements for an entire 'tree' of boxes by combining > bottom-up requirements from leaf panes with top-down requirements that > result from the call to COMPOSE-SPACE. ALLOCATE-SPACE gives actual positions > and dimensions to all boxes; it distributes all 'slack space'. The box model is popular! In cl-typesetting I've also added compression and expansion coefficients. But the problem here is to compute the columns boxes parameters. From these the layout is already done. > I'm not sure if this has some relationship to the way you handle tables. > (I don't have time right now to look at CL-TYPESETTING. I must prepare > my talk for the Benelux Lisp meeting next Sunday. You wouldn't be in > the neighbourhood of Amsterdam on Sunday, by any chance?) No, sorry. I think we will all meet at the ILC2005 in Paris :) > Anyway, I've included my code. You can use it any way you like. OK I will have a look at it. Thanks Marc From kw at w-m-p.com Sat Apr 24 03:00:14 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Fri, 23 Apr 2004 22:00:14 -0500 Subject: [cl-typesetting-devel] Hyphenation config Message-ID: <20040424030014.GA19869@w-m-p.com> Hello, just two minor patches today: - make the hyphenation table selectable via nix:*active-hyphenation-table* instead of always using the american one. (Oops, I forgot to add it to the export list. Make that nix::*active-hyphenation-table*. Or maybe it should be a tt:* variable anyway for consistency.) Setting it to NIL turns hyphenation off completely. Adding a "with-hyphenation-table" macro would make it easy to support multilanguage text... - the char decorations produced fairly ugly results on whitespace, since that doesn't have proper boxes. The patch restricts those functions to just characters. Next, I plan to take a look at handling at least simple CSS2/FO font handling - not being able to individually configure weight and slant is a bit restrictive and would require extra bookkeeping in client code. Also, the table model is turning into a bit of a nuisance, and I have to keep table support switched off for almost all HTML pages to avoid the dreaded "can't fit anything" fatal error. I consider this as a challenge ;-) Another challenge is that the code uses around 300 MB working memory for a simple 60-page document; this will need some tuning to be able to handle more complex or longer ones on affordable hardware. But that fits the Lisp philosophy, first get it working and then worry about efficiency... -Klaus -------------- next part -------------- diff -burN q/cl-typesetting/hyphenation-fp.lisp cl-typesetting/hyphenation-fp.lisp --- q/cl-typesetting/hyphenation-fp.lisp Sun Mar 7 16:30:35 2004 +++ cl-typesetting/hyphenation-fp.lisp Thu Apr 22 12:29:19 2004 @@ -298,5 +298,7 @@ (read-hyphen-file *american-hyphen-trie*) (read-hyphen-file *french-hyphen-trie*) +(defvar *active-hyphenation-table* *american-hyphen-trie*) + ;;(trace compile-hyphen-patterns) (setq *print-level* nil *print-length* nil) diff -burN q/cl-typesetting/hyphenation.lisp cl-typesetting/hyphenation.lisp --- q/cl-typesetting/hyphenation.lisp Fri Dec 19 15:37:16 2003 +++ cl-typesetting/hyphenation.lisp Thu Apr 22 12:28:29 2004 @@ -4,7 +4,7 @@ (in-package typeset) (defun hyphenate-string (string) -(when (> (length string) 4) +(when (and nix::*active-hyphenation-table* (> (length string) 4)) (loop with hyphen-points for start = 0 then (position-if #'alpha-char-p string :start (1+ end)) for end = (when start (position-if-not #'alpha-char-p string :start (1+ start))) @@ -12,4 +12,4 @@ when (> (- end start) 4) nconc (mapcar #'(lambda (n) (+ start n)) (nix::hyphen-find-hyphen-points - nix::*american-hyphen-trie* (subseq string start end)))))) + nix::*active-hyphenation-table* (subseq string start end)))))) -------------- next part -------------- diff -burN q/cl-typesetting/stroke.lisp cl-typesetting/stroke.lisp --- q/cl-typesetting/stroke.lisp Thu Apr 22 05:23:18 2004 +++ cl-typesetting/stroke.lisp Thu Apr 22 00:41:40 2004 @@ -7,10 +7,10 @@ (defmethod stroke (box x y) ) +;; Run these just on char boxes, the box geometry is different on whitespace. (defmethod stroke :before ((box box) x y) (if (and (functionp *pre-decoration*) - (or (typep box 'char-box) - (typep box 'white-char-box))) + (typep box 'char-box)) (funcall *pre-decoration* box x (+ y (baseline box) (offset box)) @@ -18,13 +18,11 @@ (defmethod stroke :after ((box box) x y) (if (and (functionp *post-decoration*) - (or (typep box 'char-box) - (typep box 'white-char-box))) + (typep box 'char-box)) (funcall *post-decoration* box x (+ y (baseline box) (offset box)) (dx box) (- (dy box))))) - (defmethod stroke ((hbox hbox) x y) (decf x (baseline hbox)) From kw at w-m-p.com Sat Apr 24 20:30:05 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sat, 24 Apr 2004 15:30:05 -0500 Subject: [cl-typesetting-devel] Final zlib fix, and CLISP support Message-ID: <20040424203005.GA5617@w-m-p.com> Hello, yet another attempt at fixing the annoying zlib issue... My new approach is the following: - in cl-pdf.asd, attempt to load the uffi library in an ignore-errors clause. If successful, add :uffi to *features*. - conditionalize the zlib dependent code using #+uffi - if uffi is not present or zlib is not loaded successfully, automatically set *compress-streams* to NIL. I have verified that this works with CMUCL and SBCL. But in addition, this change makes cl-pdf work on CLISP, and probably any other common lisp as well. cl-typesetting mostly works on CLISP, including my XHTML renderer. There are still a few portability issues in the table code to resolve that I haven't tracked down yet, nil values end up in max calls somehow. It's also complaining about floating point contagion, which may be a useful pointer towards potential optimizations for other platforms. BTW, I also removed the #+(or ...) from cl-typesetting.asd, which I haven't included in this patch. I think it is inappropriate anyway since that code should run on any common lisp. Marc, can you please verify if this patch breaks compatibility for LispWorks? If it works, I strongly suggest adding it. I assume most people would rather have a working system without compression rather than obscure error messages... -Klaus -------------- next part -------------- diff -urN -x *.fas -x *.lib orig/cl-pdf/cl-pdf.asd cl-pdf/cl-pdf.asd --- orig/cl-pdf/cl-pdf.asd Tue Apr 13 08:44:12 2004 +++ cl-pdf/cl-pdf.asd Sat Apr 24 14:45:11 2004 @@ -6,7 +6,13 @@ (in-package asdf) -#+(or allegro lispworks cmu sbcl openmcl mcl scl) +#-uffi +(ignore-errors + (print "Trying to load UFFI:") + (asdf:operate 'asdf:load-op :uffi) + (pushnew :uffi cl:*features*) + (print "UFFI loaded.")) + (defsystem :cl-pdf :name "cl-pdf" :author "Marc Battyani " @@ -32,5 +38,5 @@ (:file "text" :depends-on ("pdf-base")) (:file "bar-codes" :depends-on ("pdf-geom")) (:file "chart" :depends-on ("text" "pdf-geom"))) - :depends-on (:uffi) ) + diff -urN -x *.fas -x *.lib orig/cl-pdf/init.lisp cl-pdf/init.lisp --- orig/cl-pdf/init.lisp Mon Feb 9 14:39:42 2004 +++ cl-pdf/init.lisp Sat Apr 24 12:32:45 2004 @@ -6,6 +6,10 @@ (defvar *zlib-loaded* nil) +#-uffi +(setf *compress-streams* nil) + +#+uffi (defun find-zlib-path () (uffi:find-foreign-library "libz" @@ -13,6 +17,7 @@ :drive-letters '("C" "D" "E") :types '("so" "a" "dll" "dylib"))) +#+uffi (defun load-zlib (&optional force) (when force (setf *zlib-loaded* nil)) (unless *zlib-loaded* @@ -23,16 +28,10 @@ (uffi:load-foreign-library zlib-path :module "zlib" :supporting-libraries '("c")) - (uffi:def-function ("compress" c-compress) - ((dest (* :unsigned-char)) - (destlen (* :long)) - (source :cstring) - (source-len :long)) - :returning :int - :module "zlib") (setf *zlib-loaded* t *compress-streams* t)) (progn (warn "Unable to load zlib. Disabling compression.") (setf *compress-streams* nil)))))) +#+uffi (load-zlib) diff -urN -x *.fas -x *.lib orig/cl-pdf/zlib.lisp cl-pdf/zlib.lisp --- orig/cl-pdf/zlib.lisp Fri Jan 30 10:31:48 2004 +++ cl-pdf/zlib.lisp Sat Apr 24 14:37:15 2004 @@ -6,6 +6,16 @@ ;Adapted from an UFFI example +#+uffi +(uffi:def-function ("compress" c-compress) + ((dest (* :unsigned-char)) + (destlen (* :long)) + (source :cstring) + (source-len :long)) + :returning :int + :module "zlib") + +#+uffi (defun compress-string (source) "Returns two values: array of bytes containing the compressed data and the numbe of compressed bytes" From marc.battyani at fractalconcept.com Sat Apr 24 20:43:03 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sat, 24 Apr 2004 22:43:03 +0200 Subject: [cl-typesetting-devel] Hyphenation config References: <20040424030014.GA19869@w-m-p.com> Message-ID: <07cf01c42a3c$bf0c1160$0a02a8c0@marcxp> "Klaus Weidner" wrote: > just two minor patches today: Only! ;-) > - make the hyphenation table selectable via nix:*active-hyphenation-table* > instead of always using the american one. (Oops, I forgot to add it > to the export list. Make that nix::*active-hyphenation-table*. Or > maybe it should be a tt:* variable anyway for consistency.) > Setting it to NIL turns hyphenation off completely. Adding > a "with-hyphenation-table" macro would make it easy to support > multilanguage text... > > - the char decorations produced fairly ugly results on whitespace, since > that doesn't have proper boxes. The patch restricts those functions to > just characters. OK though I prefer this: (defmethod stroke :after ((box char-box) x y) (when (functionp *post-decoration*) (funcall *post-decoration* Rather than: (defmethod stroke :before ((box box) x y) (if (and (functionp *pre-decoration*) (typep box 'char-box)) > Next, I plan to take a look at handling at least simple CSS2/FO font > handling - not being able to individually configure weight and slant > is a bit restrictive and would require extra bookkeeping in client code. Good. > Also, the table model is turning into a bit of a nuisance, and I have to > keep table support switched off for almost all HTML pages to avoid the > dreaded "can't fit anything" fatal error. I consider this as a challenge ;-) Yes, this is a real point. I will look at it. (But I'm still very busy with Real Work (tm)) :( > Another challenge is that the code uses around 300 MB working memory for > a simple 60-page document; this will need some tuning to be able to > handle more complex or longer ones on affordable hardware. But that fits > the Lisp philosophy, first get it working and then worry about > efficiency... Yes, this is the normal way of doing stuff in Lisp. For instance there are lots of floating point computations and no declaration yet. Marc From marc.battyani at fractalconcept.com Sat Apr 24 20:46:02 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sat, 24 Apr 2004 22:46:02 +0200 Subject: [cl-typesetting-devel] Final zlib fix, and CLISP support References: <20040424203005.GA5617@w-m-p.com> Message-ID: <07d001c42a3d$288234d0$0a02a8c0@marcxp> "Klaus Weidner" wrote: Hi Klaus, > yet another attempt at fixing the annoying zlib issue... > > My new approach is the following: > > - in cl-pdf.asd, attempt to load the uffi library in an ignore-errors > clause. If successful, add :uffi to *features*. > > - conditionalize the zlib dependent code using #+uffi > > - if uffi is not present or zlib is not loaded successfully, > automatically set *compress-streams* to NIL. > > I have verified that this works with CMUCL and SBCL. > > But in addition, this change makes cl-pdf work on CLISP, and probably any > other common lisp as well. > > cl-typesetting mostly works on CLISP, including my XHTML renderer. There > are still a few portability issues in the table code to resolve that I > haven't tracked down yet, nil values end up in max calls somehow. It's > also complaining about floating point contagion, which may be a useful > pointer towards potential optimizations for other platforms. > > BTW, I also removed the #+(or ...) from cl-typesetting.asd, which I > haven't included in this patch. I think it is inappropriate anyway since > that code should run on any common lisp. > > Marc, can you please verify if this patch breaks compatibility for > LispWorks? If it works, I strongly suggest adding it. I assume most > people would rather have a working system without compression rather than > obscure error messages... The problem here is with delivered images or executables. By putting a compile time conditionnal (a :uffi feature) in cl-typesetting.asd you will decide at compile time if you have or not zlib (and where it is). This will break if you tranfert the Lisp software to another PC and zlib is not at the same place. The way it is now is to be completely dynamic. A (load-zlib) at run time would load it if it finds it. Marc From marc.battyani at fractalconcept.com Sat Apr 24 20:55:46 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sat, 24 Apr 2004 22:55:46 +0200 Subject: [cl-typesetting-devel] Portability was: Final zlib fix, and CLISP support References: <20040424203005.GA5617@w-m-p.com> Message-ID: <082901c42a3e$856193c0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > I have verified that this works with CMUCL and SBCL. > > But in addition, this change makes cl-pdf work on CLISP, and probably any > other common lisp as well. > > cl-typesetting mostly works on CLISP, including my XHTML renderer. There > are still a few portability issues in the table code to resolve that I > haven't tracked down yet, nil values end up in max calls somehow. It's > also complaining about floating point contagion, which may be a useful > pointer towards potential optimizations for other platforms. BTW I'm interested to know on which implementations cl-pdf and/or cl-typesetting works. It works on Lispworks, SBCL, CMUCL, ACL, almost CLISP now. Others ? [sent to the cl-pdf mailing list also] Marc From kw at w-m-p.com Sat Apr 24 21:03:48 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sat, 24 Apr 2004 16:03:48 -0500 Subject: [cl-typesetting-devel] Final zlib fix, and CLISP support In-Reply-To: <07d001c42a3d$288234d0$0a02a8c0@marcxp> References: <20040424203005.GA5617@w-m-p.com> <07d001c42a3d$288234d0$0a02a8c0@marcxp> Message-ID: <20040424210348.GA6970@w-m-p.com> On Sat, Apr 24, 2004 at 10:46:02PM +0200, Marc Battyani wrote: > The problem here is with delivered images or executables. > By putting a compile time conditionnal (a :uffi feature) in > cl-typesetting.asd you will decide at compile time if you have or not zlib > (and where it is). This will break if you tranfert the Lisp software to > another PC and zlib is not at the same place. The way it is now is to be > completely dynamic. A (load-zlib) at run time would load it if it finds it. I can't follow you here - the :uffi feature declares if the foreign function interface is available for this Lisp implementation or not. It doesn't have anything to do with the existence or location of a zlib library. So if you compile on a system with UFFI support, the resulting delivered image will include all the #+uffi code. It will locate and load zlib if it can, or fall back to no compression if that fails. Or am I completely misunderstanding something here? -Klaus From marc.battyani at fractalconcept.com Sat Apr 24 21:13:38 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sat, 24 Apr 2004 23:13:38 +0200 Subject: [cl-typesetting-devel] Final zlib fix, and CLISP support References: <20040424203005.GA5617@w-m-p.com> <07d001c42a3d$288234d0$0a02a8c0@marcxp> <20040424210348.GA6970@w-m-p.com> Message-ID: <083d01c42a41$03d30b60$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Sat, Apr 24, 2004 at 10:46:02PM +0200, Marc Battyani wrote: > > The problem here is with delivered images or executables. > > By putting a compile time conditionnal (a :uffi feature) in > > cl-typesetting.asd you will decide at compile time if you have or not zlib > > (and where it is). This will break if you tranfert the Lisp software to > > another PC and zlib is not at the same place. The way it is now is to be > > completely dynamic. A (load-zlib) at run time would load it if it finds it. > > I can't follow you here - the :uffi feature declares if the foreign > function interface is available for this Lisp implementation or not. It > doesn't have anything to do with the existence or location of a zlib > library. > > So if you compile on a system with UFFI support, the resulting delivered > image will include all the #+uffi code. It will locate and load zlib if > it can, or fall back to no compression if that fails. > > Or am I completely misunderstanding something here? No you are right, I didn't noticed that UFFI could be missing on some implementations and I thought (wrongly) that it was to conditionnalize zlib. Marc From kw at w-m-p.com Sun Apr 25 00:38:58 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sat, 24 Apr 2004 19:38:58 -0500 Subject: [cl-typesetting-devel] Re: Final zlib fix, and CLISP support In-Reply-To: <089a01c42a4a$f62a99b0$0a02a8c0@marcxp> References: <20040424203005.GA5617@w-m-p.com> <089a01c42a4a$f62a99b0$0a02a8c0@marcxp> Message-ID: <20040425003858.GB6970@w-m-p.com> On Sun, Apr 25, 2004 at 12:24:50AM +0200, Marc Battyani wrote: > "Klaus Weidner" > > - if uffi is not present or zlib is not loaded successfully, > > automatically set *compress-streams* to NIL. > > ok but I've rather set *compress-streams* to nil in the defvar in > config.lisp. it will be set to t only if zlib loads successfully Good point, that's cleaner. > Have you tried Edi's modification with CMUCL ? He kept the dynamic nature of > #'load-zlib so this will also work in delivered images/exe. This is lost > with your patch because you put the (uffi:def-function ("compress" > c-compress) at the top level. Almost - sbcl also needs the workaround, see attached patch. With the additional change, cl-pdf works on CMUCL, SBCL and CLISP, with compression being active for the first two and off for CLISP. The attached cl-typesetting patch makes the complex single-page-example work on clisp, including the table. It needs some settings to ensure the expected floating point behavior and charset, which I've put in specials.lisp for now. One error reported by CLISP was a floating point underflow in the wavelet rule, which may actually be conforming behavior. The exp value is 1.9d-41, which cannot be expressed in a single-float. The other implementations apparently return zero here. What still isn't working on CLISP are the splittable tables. I've fiddled around with the code a bit, but the complex loop macros are terrible to debug. -Klaus -------------- next part -------------- diff -urN -x *.fas -x *.lib orig/cl-pdf/init.lisp cl-pdf/init.lisp --- orig/cl-pdf/init.lisp Sat Apr 24 17:18:54 2004 +++ cl-pdf/init.lisp Sat Apr 24 18:47:51 2004 @@ -15,7 +15,7 @@ :drive-letters '("C" "D" "E") :types '("so" "a" "dll" "dylib")))) -#+(and cmu uffi) +#+(and uffi (or cmu sbcl)) (eval-when (:compile-toplevel :load-toplevel :execute) (let ((zlib-path (find-zlib-path))) (when zlib-path -------------- next part -------------- diff -urN -x *.fas -x *.lib orig/cl-typesetting/cl-typesetting.asd cl-typesetting/cl-typesetting.asd --- orig/cl-typesetting/cl-typesetting.asd Thu Apr 22 05:23:18 2004 +++ cl-typesetting/cl-typesetting.asd Sat Apr 24 18:32:45 2004 @@ -6,7 +6,6 @@ (in-package asdf) -#+(or allegro lispworks cmu sbcl openmcl mcl scl) (defsystem :cl-typesetting :name "cl-typesetting" :author "Marc Battyani " diff -urN -x *.fas -x *.lib orig/cl-typesetting/specials.lisp cl-typesetting/specials.lisp --- orig/cl-typesetting/specials.lisp Thu Apr 22 05:23:18 2004 +++ cl-typesetting/specials.lisp Sat Apr 24 18:44:12 2004 @@ -4,6 +4,11 @@ (in-package typeset) +#+clisp +(setq custom:*floating-point-contagion-ansi* t + custom:*warn-on-floating-point-contagion* nil + custom:*default-file-encoding* (ext:encoding-charset "iso-8859-1")) + (defconstant +huge-number+ (truncate most-positive-fixnum 10)) (defconstant +epsilon+ 0.0001) diff -urN -x *.fas -x *.lib orig/cl-typesetting/test.lisp cl-typesetting/test.lisp --- orig/cl-typesetting/test.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/test.lisp Sat Apr 24 18:34:20 2004 @@ -128,7 +128,7 @@ (pdf:set-color-stroke (color box)) (pdf:move-to (- dx/2) 0) (loop for x from (- dx/2) by 0.2 - for y = (* dy/2 (cos (* x 0.8)) (exp (* x x -0.006))) + for y = (* dy/2 (cos (* x 0.8)) (exp (* x x -0.006d0))) while (< x dx/2) do (pdf:line-to x y)) (pdf:stroke)))) From kw at w-m-p.com Sun Apr 25 04:18:52 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sat, 24 Apr 2004 23:18:52 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <20040425003858.GB6970@w-m-p.com> References: <20040424203005.GA5617@w-m-p.com> <089a01c42a4a$f62a99b0$0a02a8c0@marcxp> <20040425003858.GB6970@w-m-p.com> Message-ID: <20040425041852.GA9367@w-m-p.com> Hi, I got the table handling working on CLISP. This was *not* fun to track down, it turned out to be due to semantic differences in the way the different Lisps implement the LOOP macro. This has cost me a couple of hours which I'd rather have spent on something more productive :-( See below for results of my experiments. If someone can point me to the place in the standard that proves that one of these behaviors is not compliant, I'll gladly file a bug report for the affected implementation. Section 6.1.2.1 says that variables are stepped "at" each iteration (when exactly?), and that the difference between parallel AND and sequential FOR binding is analogous to the difference between DO and DO* (which behave identically as far as loop termination is concerned). I personally find the CLISP behavior more logical... The appended patch has workarounds, but IMHO the code should really be rewritten to be less loopy. Apparently having more than a single exit clause in a loop is asking for trouble, and the implementations can't even agree on the simple case of what an "for X on LIST" should return. -Klaus ;; LOOP test results. CMUCL and SBCL have identical behavior, which ;; apparently also matches that of LispWorks. (loop for r in '(42) finally (return r)) all => 42 (loop for r on '(42) finally (return r)) sbcl => nil clisp => (42) (loop for i from 0 below 2 for j from 0 below 1 finally (return (list i j))) all => (1 1) (loop for i from 0 below 2 for j from 0 below 2 finally (return (list i j))) sbcl => (2 1) clisp => (2 2) (loop for i from 0 below 2 and j from 0 below 2 finally (return (list i j))) all => (2 2) (loop for i from 0 below 2 for j from 0 below 3 finally (return (list i j))) sbcl => (2 1) clisp => (2 2) (loop for i from 0 below 3 for j from 0 below 2 finally (return (list i j))) all => (2 2) -------------- next part -------------- diff -urN -x *.fas -x *.lib orig/cl-typesetting/tables.lisp cl-typesetting/tables.lisp --- orig/cl-typesetting/tables.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/tables.lisp Sat Apr 24 22:27:12 2004 @@ -92,9 +92,9 @@ do (setf (splittable-p row) nil) do (loop for j = 0 then (+ j (col-span c)) - and tail on (cells row) + for tail = (cells row) then (cdr tail) for c = (first tail) ; j is the column number of c - while (< j col-number) + while (and tail (< j col-number)) collect (first tail) into head finally ; insert cell between head and tail (setf (cells row) (nconc head (list cell) tail))) @@ -131,7 +131,7 @@ :end (1- (length row-span)) :initial-value (* (1- (length row-span)) full-size-offset))) into max-height - finally (setf height (+ (max (or (height row) 0) max-height) +epsilon+))) + finally (setf height (+ (max (or (height row) 0) (or max-height 0)) +epsilon+))) (setf (height row) height) (loop for cell in (cells row) for row-span = (row-span cell) @@ -184,15 +184,22 @@ for row = (first rows) for y = (+ padding border (height row) full-size-offset) then (+ y (height row) full-size-offset) - while (<= y max-height) + while (and rows (<= y max-height)) ;do (setf (dy row) (+ (height row) full-size-offset)) do (push row boxes) finally + ;; screwed by loop semantic differences :-( + ;; (loop for r in '(42) finally (return r)) + ;; all => 42 + ;; (loop for r on '(42) finally (return r)) + ;; sbcl => nil + ;; clisp => (42) + #+clisp (unless (> y max-height) (pop rows)) (when (and boxes ;; Trim unsplitalbe rows and reverse the list of accumulated boxes - (setf boxes (loop for tail on boxes + (setf boxes (loop for tail = boxes then (cdr tail) for row = (first tail) - until (splittable-p row) + until (or (not row) (splittable-p row)) do (decf prev-y (+ (height row) full-size-offset)) finally (return (nreverse tail))))) (setq boxes (append header boxes footer)) From kw at w-m-p.com Sun Apr 25 04:37:52 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sat, 24 Apr 2004 23:37:52 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <20040425041852.GA9367@w-m-p.com> References: <20040424203005.GA5617@w-m-p.com> <089a01c42a4a$f62a99b0$0a02a8c0@marcxp> <20040425003858.GB6970@w-m-p.com> <20040425041852.GA9367@w-m-p.com> Message-ID: <20040425043752.GA9843@w-m-p.com> On Sat, Apr 24, 2004 at 11:18:52PM -0500, Klaus Weidner wrote: > The appended patch has workarounds, but IMHO the code should really be > rewritten to be less loopy. Apparently having more than a single exit > clause in a loop is asking for trouble, and the implementations can't > even agree on the simple case of what an "for X on LIST" should return. Some more notes on the patch... CLISP died at (max ... max-height) due to max-height being NIL in the loop. Replacing it with (or max-height 0) fixed that: > - finally (setf height (+ (max (or (height row) 0) max-height) +epsilon+))) > + finally (setf height (+ (max (or (height row) 0) (or max-height 0)) +epsilon+))) The following single change is not needed after all, it's left over from my experiments: > - while (<= y max-height) > + while (and rows (<= y max-height)) All the others *are* needed to make it work. I've appended a fixed full patch including the older floating point handling patch. -Klaus -------------- next part -------------- diff -urN -x *.fas -x *.lib orig/cl-typesetting/cl-typesetting.asd cl-typesetting/cl-typesetting.asd --- orig/cl-typesetting/cl-typesetting.asd Thu Apr 22 05:23:18 2004 +++ cl-typesetting/cl-typesetting.asd Sat Apr 24 18:32:45 2004 @@ -6,7 +6,6 @@ (in-package asdf) -#+(or allegro lispworks cmu sbcl openmcl mcl scl) (defsystem :cl-typesetting :name "cl-typesetting" :author "Marc Battyani " diff -urN -x *.fas -x *.lib orig/cl-typesetting/specials.lisp cl-typesetting/specials.lisp --- orig/cl-typesetting/specials.lisp Thu Apr 22 05:23:18 2004 +++ cl-typesetting/specials.lisp Sat Apr 24 18:44:12 2004 @@ -4,6 +4,11 @@ (in-package typeset) +#+clisp +(setq custom:*floating-point-contagion-ansi* t + custom:*warn-on-floating-point-contagion* nil + custom:*default-file-encoding* (ext:encoding-charset "iso-8859-1")) + (defconstant +huge-number+ (truncate most-positive-fixnum 10)) (defconstant +epsilon+ 0.0001) diff -urN -x *.fas -x *.lib orig/cl-typesetting/tables.lisp cl-typesetting/tables.lisp --- orig/cl-typesetting/tables.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/tables.lisp Sat Apr 24 23:34:58 2004 @@ -92,9 +92,9 @@ do (setf (splittable-p row) nil) do (loop for j = 0 then (+ j (col-span c)) - and tail on (cells row) + for tail = (cells row) then (cdr tail) for c = (first tail) ; j is the column number of c - while (< j col-number) + while (and tail (< j col-number)) collect (first tail) into head finally ; insert cell between head and tail (setf (cells row) (nconc head (list cell) tail))) @@ -131,7 +131,7 @@ :end (1- (length row-span)) :initial-value (* (1- (length row-span)) full-size-offset))) into max-height - finally (setf height (+ (max (or (height row) 0) max-height) +epsilon+))) + finally (setf height (+ (max (or (height row) 0) (or max-height 0)) +epsilon+))) (setf (height row) height) (loop for cell in (cells row) for row-span = (row-span cell) @@ -188,11 +188,18 @@ ;do (setf (dy row) (+ (height row) full-size-offset)) do (push row boxes) finally + ;; screwed by loop semantic differences :-( + ;; (loop for r in '(42) finally (return r)) + ;; all => 42 + ;; (loop for r on '(42) finally (return r)) + ;; sbcl => nil + ;; clisp => (42) + #+clisp (unless (> y max-height) (pop rows)) (when (and boxes ;; Trim unsplitalbe rows and reverse the list of accumulated boxes - (setf boxes (loop for tail on boxes + (setf boxes (loop for tail = boxes then (cdr tail) for row = (first tail) - until (splittable-p row) + until (or (not row) (splittable-p row)) do (decf prev-y (+ (height row) full-size-offset)) finally (return (nreverse tail))))) (setq boxes (append header boxes footer)) diff -urN -x *.fas -x *.lib orig/cl-typesetting/test.lisp cl-typesetting/test.lisp --- orig/cl-typesetting/test.lisp Tue Apr 20 17:16:58 2004 +++ cl-typesetting/test.lisp Sat Apr 24 23:36:59 2004 @@ -128,7 +128,7 @@ (pdf:set-color-stroke (color box)) (pdf:move-to (- dx/2) 0) (loop for x from (- dx/2) by 0.2 - for y = (* dy/2 (cos (* x 0.8)) (exp (* x x -0.006))) + for y = (* dy/2 (cos (* x 0.8)) (exp (* x x -0.006d0))) while (< x dx/2) do (pdf:line-to x y)) (pdf:stroke)))) From kw at w-m-p.com Sun Apr 25 06:40:07 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sun, 25 Apr 2004 01:40:07 -0500 Subject: [cl-typesetting-devel] Hyphenation config In-Reply-To: <07cf01c42a3c$bf0c1160$0a02a8c0@marcxp> References: <20040424030014.GA19869@w-m-p.com> <07cf01c42a3c$bf0c1160$0a02a8c0@marcxp> Message-ID: <20040425064007.GA10783@w-m-p.com> On Sat, Apr 24, 2004 at 10:43:03PM +0200, Marc Battyani wrote: > OK though I prefer this: > (defmethod stroke :after ((box char-box) x y) > (when (functionp *post-decoration*) > (funcall *post-decoration* > > Rather than: > (defmethod stroke :before ((box box) x y) > (if (and (functionp *pre-decoration*) > (typep box 'char-box)) I agree, I had overlooked that way of doing it. Have you made that change already? If not I'll add it to my next patch (whenever that is). > > Another challenge is that the code uses around 300 MB working memory for > > a simple 60-page document; this will need some tuning to be able to > > handle more complex or longer ones on affordable hardware. But that fits > > the Lisp philosophy, first get it working and then worry about > > efficiency... > > Yes, this is the normal way of doing stuff in Lisp. For instance there are > lots of floating point computations and no declaration yet. Surprisingly CLISP seems to be *faster* than the native compilers - probably because the simpler non-optimizing byte code compiler can eval the huge typesetting term more easily. One area that I think is a source of inefficiency is the style handling - each style change is handled by a large number of individual assignments and conditionals based on the style object for the various property variables. I think it would be faster to have a single object (class or structure) represent the entire current style, and support reuse of style objects rather than creating new ones all the time. For example, the higher-level code could create one style object each for every style combination it needs, and then each style change would be one unconditional assignment. The lookup of styles would be a bit slower, i.e. a (style-font-size style) indirect lookup instead of directly using *font-size*, but I think on balance it would be a lot faster. This would be most efficient for completely specified style objects (with no NULL fields). Handling incompletely filled style objects could still be supported via a stack of graphics contents. It may be faster to implement the low-level style objects as structures rather than classes for efficiency, once the format is fixed. Something for benchmarking... -Klaus From marc.battyani at fractalconcept.com Sun Apr 25 14:01:55 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 25 Apr 2004 16:01:55 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040424203005.GA5617@w-m-p.com> <089a01c42a4a$f62a99b0$0a02a8c0@marcxp> <20040425003858.GB6970@w-m-p.com> <20040425041852.GA9367@w-m-p.com> Message-ID: <0a4401c42acd$df0697b0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > I got the table handling working on CLISP. This was *not* fun to track > down, it turned out to be due to semantic differences in the way the > different Lisps implement the LOOP macro. This has cost me a couple of > hours which I'd rather have spent on something more productive :-( Sure and it's contagious as I took some time to look at it too. :( But at least for the maximize into, the culprit is clearly CLISP. ;-) >From 6.1.3 Value Accumulation Clauses: "The var argument is bound as if by the construct with. [...] The default type is implementation-dependent; but it must be a supertype of type real." >From 6.1.2.2 Local Variable Initializations: The with construct initializes variables that are local to a loop. [...] For example, for the types t, number, and float, the default values are nil, 0, and 0.0 respectively." So max-height should be 0.0 > See below for results of my experiments. If someone can point me to the > place in the standard that proves that one of these behaviors is not > compliant, I'll gladly file a bug report for the affected implementation. > > Section 6.1.2.1 says that variables are stepped "at" each iteration (when > exactly?), and that the difference between parallel AND and sequential > FOR binding is analogous to the difference between DO and DO* (which > behave identically as far as loop termination is concerned). I personally > find the CLISP behavior more logical... You have to look at the standard for each loop construct to be sure. For instance "6.1.2.1.2 The for-as-in-list subclause" says: "The variable var is bound to the successive elements of the list in form1 before each iteration." So (loop for r in '(42) finally (return r)) => 42 is ok. In "6.1.2.1.3 The for-as-on-list subclause" you have The variable var is bound to the successive tails of the list in form1. There is no "before" here so nil is the tail but it's not a list (atom nil) => t so the clause terminates. (loop for r on '(42) finally (return r)) should return nil. Don't forget that nil is an atom (atom nil) => t (loop for r on '(42 . 43) do finally (return r)) should return 43 So here also clisp is wrong. > The appended patch has workarounds, but IMHO the code should really be > rewritten to be less loopy. Apparently having more than a single exit > clause in a loop is asking for trouble, and the implementations can't > even agree on the simple case of what an "for X on LIST" should return. I was also prejudicied against loop earlier, but I took the time to learn it and now I rather like it. It fits well in the Common Lisp way of doing it (create a domain specfic language) the only bad thing is the syntax which is not lispy. (PG does not like loop but he likes scheme more than CL.) May be we should switch to iterate which is a more lispy loop? I have an iterate version and IIRC Andreas Fuchs has revived and packaged it also. > ;; LOOP test results. CMUCL and SBCL have identical behavior, which > ;; apparently also matches that of LispWorks. > > (loop for r in '(42) finally (return r)) > > all => 42 > > (loop for r on '(42) finally (return r)) > > sbcl => nil > clisp => (42) all is correct see above. > (loop for i from 0 below 2 > for j from 0 below 1 > finally (return (list i j))) > > all => (1 1) > > (loop for i from 0 below 2 > for j from 0 below 2 > finally (return (list i j))) > > sbcl => (2 1) > clisp => (2 2) all is correct: the clauses are executed sequentially. (6.1.2.1 Iteration Control) > (loop for i from 0 below 2 > and j from 0 below 2 > finally (return (list i j))) > > all => (2 2) > > (loop for i from 0 below 2 > for j from 0 below 3 > finally (return (list i j))) > > sbcl => (2 1) > clisp => (2 2) idem. So for the patches, it's not a problem of semantic differences but a problem of non conformance of clisp. I will put your paches but with #+/-clisp conditionnals. Do you send an email to the clisp mailing list (clisp-devel) or do I do it ? ;-) Marc From marc.battyani at fractalconcept.com Sun Apr 25 14:27:19 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 25 Apr 2004 16:27:19 +0200 Subject: [cl-typesetting-devel] Re: [cl-pdf-devel] Re: Final zlib fix, and CLISP support References: <20040424203005.GA5617@w-m-p.com><089a01c42a4a$f62a99b0$0a02a8c0@marcxp> <20040425003858.GB6970@w-m-p.com> Message-ID: <0a4501c42ad1$6b0ca9e0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > > Have you tried Edi's modification with CMUCL ? He kept the dynamic nature of > > #'load-zlib so this will also work in delivered images/exe. This is lost > > with your patch because you put the (uffi:def-function ("compress" > > c-compress) at the top level. > > Almost - sbcl also needs the workaround, see attached patch. > > With the additional change, cl-pdf works on CMUCL, SBCL and CLISP, > with compression being active for the first two and off for CLISP. OK done. Let's hope it's the final "final version" ;-) Marc From marc.battyani at fractalconcept.com Sun Apr 25 15:06:50 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 25 Apr 2004 17:06:50 +0200 Subject: [cl-typesetting-devel] Hyphenation config References: <20040424030014.GA19869@w-m-p.com><07cf01c42a3c$bf0c1160$0a02a8c0@marcxp> <20040425064007.GA10783@w-m-p.com> Message-ID: <0aaf01c42ad6$f07bf540$0a02a8c0@marcxp> "Klaus Weidner" wrote: > Surprisingly CLISP seems to be *faster* than the native compilers - > probably because the simpler non-optimizing byte code compiler can eval > the huge typesetting term more easily. Probably. When optimizing it will probably be useful to parse the typesetting document with a special reader and relying on the Lisp reader/evaluator when there is read code. > One area that I think is a source of inefficiency is the style handling - > each style change is handled by a large number of individual assignments > and conditionals based on the style object for the various property > variables. I think it would be faster to have a single object (class or > structure) represent the entire current style, and support reuse of style > objects rather than creating new ones all the time. > > For example, the higher-level code could create one style object each for > every style combination it needs, and then each style change would be one > unconditional assignment. The lookup of styles would be a bit slower, > i.e. a (style-font-size style) indirect lookup instead of directly using > *font-size*, but I think on balance it would be a lot faster. > > This would be most efficient for completely specified style objects (with > no NULL fields). Handling incompletely filled style objects could still > be supported via a stack of graphics contents. Yes this point (text styles) have been discussed earlier. I think it would be better to store the text-styles stack in a *style-stack* special variable rather than distributed in lots of separate variables. Or maybe a mixed system with a few specials for heavilly used styles like *font*. > It may be faster to implement the low-level style objects as structures > rather than classes for efficiency, once the format is fixed. Something > for benchmarking... At least on Lispworks and maybe ACL this will not give a big improvement IMO. Anyway optimisation should be done later. For now there are more urgent problems like the table colum sizes. ;-) Marc From marc.battyani at fractalconcept.com Sun Apr 25 16:46:58 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 25 Apr 2004 18:46:58 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040424203005.GA5617@w-m-p.com><089a01c42a4a$f62a99b0$0a02a8c0@marcxp><20040425003858.GB6970@w-m-p.com> <20040425041852.GA9367@w-m-p.com> <20040425043752.GA9843@w-m-p.com> Message-ID: <0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > Some more notes on the patch... > > CLISP died at (max ... max-height) due to max-height being NIL in the > loop. Replacing it with (or max-height 0) fixed that: > > > - finally (setf height (+ (max (or (height row) 0) max-height) +epsilon+))) > > + finally (setf height (+ (max (or (height row) 0) (or max-height 0)) +epsilon+))) > > > The following single change is not needed after all, it's left over from > my experiments: > > > - while (<= y max-height) > > + while (and rows (<= y max-height)) > > All the others *are* needed to make it work. I've appended a fixed full > patch including the older floating point handling patch. All this is in the repository and tarball now. With #+/-clisp so that we can remove them when CLISP loop is compliant. ;-) Marc From kw at w-m-p.com Sun Apr 25 17:24:24 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sun, 25 Apr 2004 12:24:24 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <0a4401c42acd$df0697b0$0a02a8c0@marcxp> References: <20040424203005.GA5617@w-m-p.com> <089a01c42a4a$f62a99b0$0a02a8c0@marcxp> <20040425003858.GB6970@w-m-p.com> <20040425041852.GA9367@w-m-p.com> <0a4401c42acd$df0697b0$0a02a8c0@marcxp> Message-ID: <20040425172424.GA13487@w-m-p.com> On Sun, Apr 25, 2004 at 04:01:55PM +0200, Marc Battyani wrote: > Sure and it's contagious as I took some time to look at it too. :( > But at least for the maximize into, the culprit is clearly CLISP. ;-) I'm not entirely convinced of that, see below. I tend to blame the rather imprecise standard here... > >From 6.1.3 Value Accumulation Clauses: > "The var argument is bound as if by the construct with. > [...] > The default type is implementation-dependent; but it must be a supertype of > type real." CLISP is correct here, "t" is a supertype of real. > >From 6.1.2.2 Local Variable Initializations: > The with construct initializes variables that are local to a loop. > [...] > For example, for the types t, number, and float, the default values are nil, > 0, and 0.0 respectively." > > So max-height should be 0.0 Why? It's not being declared as a float anywhere. The section you quote is specifically for the "with" declaration and states that the default value is supplied "If the optional type-spec argument is supplied". There is no type-spec argument here. The accumulation section for maximize specifically says "If the maximize or minimize clause is never executed, the accumulated value is unspecified". And that's what is happening here. Other compilers apparently use more complex type inference to figure out that it's a number (I can't see how they are supposed to deduce that it's a float given the available information), but that's beyond what the standard requires. > You have to look at the standard for each loop construct to be sure. > For instance "6.1.2.1.2 The for-as-in-list subclause" says: > "The variable var is bound to the successive elements of the list in form1 > before each iteration." > So (loop for r in '(42) finally (return r)) => 42 is ok. > > In "6.1.2.1.3 The for-as-on-list subclause" you have > The variable var is bound to the successive tails of the list in form1. > There is no "before" here so nil is the tail but it's not a list (atom nil) > => t so the clause terminates. > (loop for r on '(42) finally (return r)) should return nil. > Don't forget that nil is an atom (atom nil) => t > (loop for r on '(42 . 43) do finally (return r)) should return 43 > > So here also clisp is wrong. That I agree with, due to the "before each iteration" phrase that IN has and ON does not. > I was also prejudicied against loop earlier, but I took the time to learn it > and now I rather like it. It fits well in the Common Lisp way of doing it > (create a domain specfic language) the only bad thing is the syntax which is > not lispy. (PG does not like loop but he likes scheme more than CL.) May be > we should switch to iterate which is a more lispy loop? I have an iterate > version and IIRC Andreas Fuchs has revived and packaged it also. That may be an alternative, I have not looked at "iterate" yet. One reason I dislike LOOP is that it doesn't play well with my editor's s-exp manipulation commands. But another disadvantage is that it apparently encourages long complex functions with unobvious semantics which are painful to debug, which I'm not sure that iterate would be better with. > > (loop for i from 0 below 2 > > for j from 0 below 1 > > finally (return (list i j))) > > > > all => (1 1) > > > > (loop for i from 0 below 2 > > for j from 0 below 2 > > finally (return (list i j))) > > > > sbcl => (2 1) > > clisp => (2 2) > > all is correct: the clauses are executed sequentially. (6.1.2.1 Iteration > Control) I don't agree - the standard specifies sequential *binding* as in DO vs. DO*, but it does not mention anywhere that this has anything to do with the semantics of the end tests. My understanding of loop (such as it is) would say that the FOR and AND variants would be equivalent to DO*/DO loops like this: (do ((i 0 (incf i)) (j 0 (incf j))) ((or (>= i 2) (>= j 2)) (list i j))) (do* ((i 0 (incf i)) (j 0 (incf j))) ((or (>= i 2) (>= j 2)) (list i j))) which of course behave identically, since the i and j binding/stepping clauses are independent. > So for the patches, it's not a problem of semantic differences but a > problem of non conformance of clisp. I will put your paches but with > #+/-clisp conditionnals. Do you send an email to the clisp mailing > list (clisp-devel) or do I do it ? ;-) I'll do that at least to clarify what's going on. But except for the "on list" example I'm not sure if the standard offers much support. Is "iterate" specified less ambiguously? That alone would be a reason to choose it. -Klaus From marc.battyani at fractalconcept.com Sun Apr 25 19:39:58 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 25 Apr 2004 21:39:58 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040424203005.GA5617@w-m-p.com><089a01c42a4a$f62a99b0$0a02a8c0@marcxp><20040425003858.GB6970@w-m-p.com> <20040425041852.GA9367@w-m-p.com><0a4401c42acd$df0697b0$0a02a8c0@marcxp> <20040425172424.GA13487@w-m-p.com> Message-ID: <0b5001c42afd$18883a50$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Sun, Apr 25, 2004 at 04:01:55PM +0200, Marc Battyani wrote: > The accumulation section for maximize specifically says "If the maximize > or minimize clause is never executed, the accumulated value is > unspecified". And that's what is happening here. OK I missed this one and it's the good point. So CLISP is OK here. > Other compilers apparently use more complex type inference to figure out > that it's a number (I can't see how they are supposed to deduce that it's > a float given the available information), but that's beyond what the > standard requires. LW returns 0 not 0.0. It's better IMO as number variables are to be initialized with 0 (or 0.0 for a float) though we should not rely on it based on the standard. Anyway nil is also an arbitrary value so (or max-height 0) is not compliant either. > > I was also prejudicied against loop earlier, but I took the time to learn it > > and now I rather like it. It fits well in the Common Lisp way of doing it > > (create a domain specfic language) the only bad thing is the syntax which is > > not lispy. (PG does not like loop but he likes scheme more than CL.) May be > > we should switch to iterate which is a more lispy loop? I have an iterate > > version and IIRC Andreas Fuchs has revived and packaged it also. > > That may be an alternative, I have not looked at "iterate" yet. One > reason I dislike LOOP is that it doesn't play well with my editor's s-exp > manipulation commands. > > But another disadvantage is that it apparently encourages long complex > functions with unobvious semantics which are painful to debug, which I'm > not sure that iterate would be better with. The major advantage of iterate is that it is AFAIK there is only one version and the source is public :) > > > (loop for i from 0 below 2 > > > for j from 0 below 1 > > > finally (return (list i j))) > > > > > > all => (1 1) > > > > > > (loop for i from 0 below 2 > > > for j from 0 below 2 > > > finally (return (list i j))) > > > > > > sbcl => (2 1) > > > clisp => (2 2) > > > > all is correct: the clauses are executed sequentially. (6.1.2.1 Iteration > > Control) > > I don't agree - the standard specifies sequential *binding* as in DO vs. > DO*, but it does not mention anywhere that this has anything to do with > the semantics of the end tests. In 6.1.2.1 Iteration Control: "When iteration control clauses are used in a loop, the corresponding termination tests in the loop body are evaluated before any other loop body code is executed." For me "any other" applies to other clauses as well. The reference to do and do* is unfortunate because they have only one termination form. > My understanding of loop (such as it is) would say that the FOR and AND > variants would be equivalent to DO*/DO loops like this: > > (do ((i 0 (incf i)) > (j 0 (incf j))) > ((or (>= i 2) (>= j 2)) (list i j))) > > (do* ((i 0 (incf i)) > (j 0 (incf j))) > ((or (>= i 2) (>= j 2)) (list i j))) > > which of course behave identically, since the i and j binding/stepping > clauses are independent. > > > So for the patches, it's not a problem of semantic differences but a > > problem of non conformance of clisp. I will put your paches but with > > #+/-clisp conditionnals. Do you send an email to the clisp mailing > > list (clisp-devel) or do I do it ? ;-) > > I'll do that at least to clarify what's going on. But except for the "on > list" example I'm not sure if the standard offers much support. Unfortunally the standard is ambiguous on several points. Some time ago there was a problem with CMUCL and parse-integer. Again the standard could be interpreted in 2 ways. Finally they decided to go back to the interpretation of all the other implementations. Maybe the CLISP developpers will agree to do the same. (ie interpret the standard like all the other implementations) > Is "iterate" specified less ambiguously? That alone would be a reason to > choose it. see above. Marc From kw at w-m-p.com Sun Apr 25 20:05:10 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sun, 25 Apr 2004 15:05:10 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <0b5001c42afd$18883a50$0a02a8c0@marcxp> References: <20040425172424.GA13487@w-m-p.com> <0b5001c42afd$18883a50$0a02a8c0@marcxp> Message-ID: <20040425200510.GB14211@w-m-p.com> On Sun, Apr 25, 2004 at 09:39:58PM +0200, Marc Battyani wrote: > LW returns 0 not 0.0. It's better IMO as number variables are to be > initialized with 0 (or 0.0 for a float) though we should not rely on it > based on the standard. Anyway nil is also an arbitrary value so (or > max-height 0) is not compliant either. I'll have a shot at rewriting this in a compatible way. The variable needs to be initialized for all code paths. An easy way would be adding a simple "maximize 0 into max-height" somewhere, but there should be something cleaner. > The major advantage of iterate is that it is AFAIK there is only one version > and the source is public :) Good point ;-) > > I don't agree - the standard specifies sequential *binding* as in DO vs. > > DO*, but it does not mention anywhere that this has anything to do with > > the semantics of the end tests. > > In 6.1.2.1 Iteration Control: > "When iteration control clauses are used in a loop, the corresponding > termination tests in the loop body are evaluated before any other loop body > code is executed." > For me "any other" applies to other clauses as well. The reference to do and > do* is unfortunate because they have only one termination form. I think that interpretation goes beyond what the standard actually says. and I would interpret "loop body" to exclude the stepping statements, My mental picture of LOOP construct is this: prologue unless (end tests) loop body stepping statements epilogue That's basically equivalent to how the reference section in PG's "ANSI Common Lisp" defines it. There's no way to deduce from the standard that end tests and stepping statements are supposed to be intermingled with each other. > Unfortunally the standard is ambiguous on several points. Some time ago > there was a problem with CMUCL and parse-integer. Again the standard could > be interpreted in 2 ways. Finally they decided to go back to the > interpretation of all the other implementations. Maybe the CLISP developpers > will agree to do the same. (ie interpret the standard like all the other > implementations) Maybe it would be better if you send the note to the CLISP people, since I'm mostly convinced of their point of view by now (except for on-list). -Klaus From marc.battyani at fractalconcept.com Sun Apr 25 20:18:38 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 25 Apr 2004 22:18:38 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040425172424.GA13487@w-m-p.com><0b5001c42afd$18883a50$0a02a8c0@marcxp> <20040425200510.GB14211@w-m-p.com> Message-ID: <0ba601c42b02$7f2a8b00$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Sun, Apr 25, 2004 at 09:39:58PM +0200, Marc Battyani wrote: > > In 6.1.2.1 Iteration Control: > > "When iteration control clauses are used in a loop, the corresponding > > termination tests in the loop body are evaluated before any other loop body > > code is executed." > > For me "any other" applies to other clauses as well. The reference to do and > > do* is unfortunate because they have only one termination form. > > I think that interpretation goes beyond what the standard actually says. > and I would interpret "loop body" to exclude the stepping statements, It's an English problem here. I'm not a native speaker. ;-) For me the "other" word means it applies to other clauses otherwise they would have said "before the loop body". By saying "before the other loop body code" they imply that the clause is part of that body so it should terminate before the next one. > > Maybe it would be better if you send the note to the CLISP people, since > I'm mostly convinced of their point of view by now (except for on-list). Ok I will do it. Marc From kw at w-m-p.com Sun Apr 25 20:15:08 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sun, 25 Apr 2004 15:15:08 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp> References: <20040425041852.GA9367@w-m-p.com> <20040425043752.GA9843@w-m-p.com> <0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp> Message-ID: <20040425201508.GC14211@w-m-p.com> On Sun, Apr 25, 2004 at 06:46:58PM +0200, Marc Battyani wrote: > All this is in the repository and tarball now. With #+/-clisp so that we can > remove them when CLISP loop is compliant. ;-) I'm really unhappy with this approach. I had changed the code to work the same way on *all* three platforms I could test on by avoiding the ill-defined loop behavior, and having the platform-independent version conditionalized with #+clisp and keeping the previous platform-dependent version is unnecessary and makes the code even harder to read than it currently is. The code should work in a compatible way on all common platforms, and even if the CLISP people were to agree to change their LOOP implementation to fit with other people's interpretation of the standard (which I would not count on), we can't expect people to instantly upgrade once that change is made. So the conditionalized code would need to stay for a long time. I don't mind keeping the one #+clisp for the weird on-list effect, but even there I'd prefer working around the problem in a platform-independent way. -Klaus From marc.battyani at fractalconcept.com Sun Apr 25 19:56:24 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 25 Apr 2004 21:56:24 +0200 Subject: [cl-typesetting-devel] Hyphenation config References: <20040424030014.GA19869@w-m-p.com> Message-ID: <0b7801c42aff$644f6b50$0a02a8c0@marcxp> "Klaus Weidner" wrote: > > Also, the table model is turning into a bit of a nuisance, and I have to > keep table support switched off for almost all HTML pages to avoid the > dreaded "can't fit anything" fatal error. I consider this as a challenge ;-) The challenge is NP complete even for simple tables :( http://www.cs.princeton.edu/~sobti/papers/cg99/ The good news is that they give an heuristic that "will perform fairly well on most tables" Marc From marc.battyani at fractalconcept.com Sun Apr 25 21:09:04 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Sun, 25 Apr 2004 23:09:04 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040425041852.GA9367@w-m-p.com> <20040425043752.GA9843@w-m-p.com> <0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp> <20040425201508.GC14211@w-m-p.com> Message-ID: <0bd101c42b09$8d3705f0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Sun, Apr 25, 2004 at 06:46:58PM +0200, Marc Battyani wrote: > > All this is in the repository and tarball now. With #+/-clisp so that we can > > remove them when CLISP loop is compliant. ;-) > > I'm really unhappy with this approach. I had changed the code to work the > same way on *all* three platforms I could test on by avoiding the > ill-defined loop behavior, and having the platform-independent version > conditionalized with #+clisp and keeping the previous platform-dependent > version is unnecessary and makes the code even harder to read than it > currently is. > > The code should work in a compatible way on all common platforms, and > even if the CLISP people were to agree to change their LOOP > implementation to fit with other people's interpretation of the standard > (which I would not count on), we can't expect people to instantly > upgrade once that change is made. So the conditionalized code would need > to stay for a long time. Hum, I don't buy this argument as nobody uses cl-typesetting with clisp yet. I remind you that you made the port yesterday ;-) So it's not a problem to say that cl-typesetting works with clisp version > 42. > I don't mind keeping the one #+clisp for the weird on-list effect, but > even there I'd prefer working around the problem in a > platform-independent way. ?? The #+clisp is only related to the on-list and max-height problems. For the on-list one I will ask the clisp developpers it's not ill-defined as you say IMO, and for the max-height one the patch works for clisp but is not guaranted to work with every implantation as it is unspecified according to the standard. (Though I don't see why there would be anything else than nil or 0) Marc From kw at w-m-p.com Sun Apr 25 23:00:25 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sun, 25 Apr 2004 18:00:25 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <0bd101c42b09$8d3705f0$0a02a8c0@marcxp> References: <20040425041852.GA9367@w-m-p.com> <20040425043752.GA9843@w-m-p.com> <0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp> <20040425201508.GC14211@w-m-p.com> <0bd101c42b09$8d3705f0$0a02a8c0@marcxp> Message-ID: <20040425230025.GD14211@w-m-p.com> On Sun, Apr 25, 2004 at 11:09:04PM +0200, Marc Battyani wrote: > Hum, I don't buy this argument as nobody uses cl-typesetting with clisp yet. > I remind you that you made the port yesterday ;-) So it's not a problem to > say that cl-typesetting works with clisp version > 42. Yes it is a problem, and I don't see why we should make life unnecessarily difficult for people if the problem is so easy to work around. A reason why I worked on the port is because I *need* it to work with existing CLISP installations. Keep in mind that this is not an extremely major brokenness in CLISP that would make people upgrade in any case. It's a case of different interpretation of a badly-defined part of the standard, and I would not be surprised if yet other Lisp implementations would again have different behavior. And the workaround is painless. FYI, I'm not all that opposed to loop, and I think it's clearer than DO in many cases. But the unclear loop termination rules are not something I'd want to depend on. Keep in mind that if even the compiler implementors have a hard time figuring out what the behavior is supposed to be, the risk that a blurry-eyed programmer working with the code will get the wrong mental model is very high. > ?? The #+clisp is only related to the on-list and max-height problems. > For the on-list one I will ask the clisp developpers it's not ill-defined as > you say IMO, and for the max-height one the patch works for clisp but is not > guaranted to work with every implantation as it is unspecified according to > the standard. (Though I don't see why there would be anything else than nil > or 0) I fixed the max-height problem for all platforms in the attached patch, so I think this point is moot. For the on-list bug, I refactored v-split. See below for the full code, since it's harder to see what's going on in the patch. I think this is a cleaner solution than the old code. One more thing, please remove the #+(or ...) in cl-typesetting.asd, it should not be needed anymore. -Klaus (defmethod v-split ((table multi-page-table) dx dy &optional v-align) "Factor out rows that fit and return as a first value." ;; Treat unsplittable rows as a single unit - for this purpose, ;; group the rows-left list into the following form: ;; ;; ( (group1-height row1 row2 ...) ;; (group2-height row7) ;; (group3-height row8 row9 ...) ) ;; (with-slots (header footer border padding cell-padding) table (loop with boxes = () and current-height = (+ border padding (reduce #'+ header :key #'dy) (reduce #'+ footer :key #'dy)) and row-groups = (loop with height = 0 and rows = () for row in (rows-left table) do (incf height (+ (height row) (* 2 cell-padding) border)) (push row rows) when (splittable-p row) collect (cons height (nreverse rows)) and do (setf height 0 rows nil)) and rows-remaining = (rows-left table) for (group-height . rows) in row-groups while (<= (+ current-height group-height) dy) do (dolist (r rows) (push r boxes) (pop rows-remaining)) (incf current-height group-height) finally (when boxes (setq boxes (append header (nreverse boxes) footer)) (setf (rows-left table) rows-remaining) (decf (slot-value table 'dy) current-height) (let ((first (first boxes)) (last (first (last boxes)))) (setf (slot-value first 'position) :first (slot-value last 'position) (if (eq first last) :single :last))) (return (values boxes rows-remaining (- dy current-height)))) (return (values nil rows-remaining dy))))) From kw at w-m-p.com Sun Apr 25 23:10:55 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Sun, 25 Apr 2004 18:10:55 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <20040425230025.GD14211@w-m-p.com> References: <20040425041852.GA9367@w-m-p.com> <20040425043752.GA9843@w-m-p.com> <0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp> <20040425201508.GC14211@w-m-p.com> <0bd101c42b09$8d3705f0$0a02a8c0@marcxp> <20040425230025.GD14211@w-m-p.com> Message-ID: <20040425231055.GG14211@w-m-p.com> On Sun, Apr 25, 2004 at 06:00:25PM -0500, Klaus Weidner wrote: > I fixed the max-height problem for all platforms in the attached patch, ... which I forgot to attach. Here it is. -Klaus -------------- next part -------------- diff -urN -x *.fas -x *.lib orig/cl-typesetting/cl-typesetting.asd cl-typesetting/cl-typesetting.asd --- orig/cl-typesetting/cl-typesetting.asd Thu Apr 22 05:23:18 2004 +++ cl-typesetting/cl-typesetting.asd Sun Apr 25 17:07:06 2004 @@ -6,7 +6,6 @@ (in-package asdf) -#+(or allegro lispworks cmu sbcl openmcl mcl scl) (defsystem :cl-typesetting :name "cl-typesetting" :author "Marc Battyani " diff -urN -x *.fas -x *.lib orig/cl-typesetting/tables.lisp cl-typesetting/tables.lisp --- orig/cl-typesetting/tables.lisp Sun Apr 25 11:40:38 2004 +++ cl-typesetting/tables.lisp Sun Apr 25 18:09:17 2004 @@ -91,19 +91,10 @@ when (> i 1) ; set all but last rows unsplittable do (setf (splittable-p row) nil) do - #-clisp - (loop for j = 0 then (+ j (col-span c)) - and tail on (cells row) - for c = (first tail) ; j is the column number of c - while (< j col-number) - collect (first tail) into head - finally ; insert cell between head and tail - (setf (cells row) (nconc head (list cell) tail))) - #+clisp (loop for j = 0 then (+ j (col-span c)) for tail = (cells row) then (cdr tail) for c = (first tail) ; j is the column number of c - while (and tail (< j col-number)) + while (and c (< j col-number)) collect (first tail) into head finally ; insert cell between head and tail (setf (cells row) (nconc head (list cell) tail))) @@ -114,34 +105,49 @@ (let ((full-size-offset (+ (border table) (* 2 (cell-padding table)))) (height (or (height row) +huge-number+))) (loop with next-widths = (col-widths table) - for width = (or (pop next-widths) 0) ; in case less elements specified + for cell in (cells row) - for col-number = 0 then (+ col-number col-span 1) - and col-span = (1- (col-span cell)) + and width = (or (pop next-widths) 0) ; in case less elements specified + and col-number = 0 then (+ col-number col-span 1) + and cell-height = 0.0 + + for col-span = (1- (col-span cell)) and row-span = (row-span cell) + + ;; Adjust cell width for cells spanning multiple columns unless (zerop col-span) do (incf width (+ (* col-span full-size-offset) (reduce #'+ next-widths :end col-span))) (setf next-widths (nthcdr col-span next-widths)) + + ;; Fill cell with content if required when (cell-start-row-p cell row) do (setf (box cell) (make-filled-vbox (content cell) width height) (width cell) width) + ;; A cell spanning several rows participates only in height calculation ;; of the last row if (and (numberp row-span) (> row-span 1)) do (span-cell rows cell col-number) else unless (height row) if (eql row-span 1) - maximize (compute-boxes-natural-size (boxes (box cell)) #'dy) into max-height + do (setq cell-height + (compute-boxes-natural-size (boxes (box cell)) #'dy)) else if (cell-end-row-p cell row) - maximize (- (compute-boxes-natural-size (boxes (box cell)) #'dy) - (reduce #'+ row-span - :key #'height - :end (1- (length row-span)) - :initial-value (* (1- (length row-span)) - full-size-offset))) into max-height - finally (setf height (+ (max (or (height row) 0) - #-clisp max-height #+clisp (or max-height 0)) +epsilon+))) + do (setq cell-height + (- (compute-boxes-natural-size (boxes (box cell)) #'dy) + (reduce #'+ row-span + :key #'height + :end (1- (length row-span)) + :initial-value (* (1- (length row-span)) + full-size-offset)))) + + maximize cell-height into max-height + + finally (setf height (+ (max (or (height row) 0.0) + max-height) + +epsilon+))) + (setf (height row) height) (loop for cell in (cells row) for row-span = (row-span cell) @@ -181,48 +187,60 @@ (dolist (row footer) (compute-row-size table row footer)))) (defmethod v-split ((table multi-page-table) dx dy &optional v-align) - ;;; Factor out rows that fit and return as a first value. - (with-slots (header footer rows-left) table + "Factor out rows that fit and return as a first value." + ;; Treat unsplittable rows as a single unit - for this purpose, + ;; group the rows-left list into the following form: + ;; + ;; ( (group1-height row1 row2 ...) + ;; (group2-height row7) + ;; (group3-height row8 row9 ...) ) + ;; + (with-slots (header footer border padding cell-padding) table (loop with boxes = () - with border = (border table) - and padding = (padding table) - and cell-padding = (cell-padding table) - with full-size-offset = (+ cell-padding cell-padding border) - with max-height = (- dy (reduce #'+ header :key #'dy) (reduce #'+ footer :key #'dy)) - for rows on rows-left - and prev-y = 0 then y ; vertical space that has been output - for row = (first rows) - for y = (+ padding border (height row) full-size-offset) - then (+ y (height row) full-size-offset) - while (<= y max-height) - ;do (setf (dy row) (+ (height row) full-size-offset)) - do (push row boxes) - finally - #+clisp (unless (> y max-height) (pop rows)) - (when (and boxes - ;; Trim unsplitalbe rows and reverse the list of accumulated boxes - (setf boxes - #-clisp - (loop for tail on boxes - for row = (first tail) - until (splittable-p row) - do (decf prev-y (+ (height row) full-size-offset)) - finally (return (nreverse tail))) - #+clisp - (loop for tail = boxes then (cdr tail) - for row = (first tail) - until (or (not row) (splittable-p row)) - do (decf prev-y (+ (height row) full-size-offset)) - finally (return (nreverse tail))))) - (setq boxes (append header boxes footer)) + and current-height = (+ border + padding + (reduce #'+ header :key #'dy) + (reduce #'+ footer :key #'dy)) + and row-groups = (loop with height = 0 + and rows = () + + for row in (rows-left table) + + do + (incf height (+ (height row) + (* 2 cell-padding) + border)) + (push row rows) + + when (splittable-p row) + collect (cons height (nreverse rows)) + and do (setf height 0 rows nil)) + and rows-remaining = (rows-left table) + + for (group-height . rows) in row-groups + while (<= (+ current-height group-height) dy) + + do (dolist (r rows) + (push r boxes) + (pop rows-remaining)) + (incf current-height group-height) + + finally + (when boxes + (setq boxes (append header (nreverse boxes) footer)) + ;; reduce rows to output + (setf (rows-left table) rows-remaining) + ;; reduce space required by table + ;; (FIXME: need to subtract header/footer?) + (decf (slot-value table 'dy) current-height) (let ((first (first boxes)) (last (first (last boxes)))) (setf (slot-value first 'position) :first (slot-value last 'position) (if (eq first last) :single :last))) - (setf rows-left rows) ; reduce rows to output - (decf (slot-value table 'dy) prev-y) ; and space required by table - (return (values boxes rows (- max-height prev-y)))) - (return (values nil rows-left dy))))) + (return (values boxes + rows-remaining + (- dy current-height)))) + (return (values nil rows-remaining dy))))) (defmethod dy :around ((table multi-page-table)) (with-slots (header footer) table From marc.battyani at fractalconcept.com Mon Apr 26 15:29:21 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Mon, 26 Apr 2004 17:29:21 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040425041852.GA9367@w-m-p.com> <20040425043752.GA9843@w-m-p.com><0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp><20040425201508.GC14211@w-m-p.com><0bd101c42b09$8d3705f0$0a02a8c0@marcxp><20040425230025.GD14211@w-m-p.com> <20040425231055.GG14211@w-m-p.com> Message-ID: <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Sun, Apr 25, 2004 at 06:00:25PM -0500, Klaus Weidner wrote: > > I fixed the max-height problem for all platforms in the attached patch, > > ... which I forgot to attach. Here it is. OK but according to the clisp developpers. Your workarounds are broken also: To sum up the discussion: Sam was ok but not Bruno so it's no. "Sam Steingold" wrote: > As I said (and Paul confirmed), using iteration variables in FINALLY is > broken. Just don't do it. Your workarounds are just as broken as the > original code. > > There are many options: MAP, DO, REDUCE &c &c &c. > > What you want to do - use iteration variables in FINALLY - is not an > option for portable code. "Sam Steingold" wrote: > > * Bruno Haible [2004-04-26 13:30:49 +0200]: > > > >> You can try the appended patch though, I don't think it will hurt. > >> Bruno, what do you think? > > > > It's futile to change the implementation to match some reading of an > > ambiguous standard. Next time someone will come and prefer the reverse > > reading. We have to stop here and acknowledge that the standard is > > ambiguous. [snipped patch...] > based on this discussion, I will add a note to impnotes that we consider > iteration variables in FINALLY to be unportable code. In clisp these 2 forms are clearly not equivalents though IMO they should: [3] > (loop for i on '(1 2 3 4 5) for j from 0 while (< j 5) finally (return i)) (5) [3] > (loop for i = '(1 2 3 4 5) then (cdr i) for j from 0 while (< j 5) finally (return i)) NIL And in any case the clisp developpers say that even the second one is not supported though it works. The standard is not clearly specified here. So do I put your current patches or do you want to look at them again? BTW the multi-page table support has been written by Dmitri. Maybe we should put him in copy? Let's close this case anyway. We have already spent way too much time on it. Just give me your final patches and let's go back to more productive and interesting code. Marc From kw at w-m-p.com Mon Apr 26 15:52:31 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Mon, 26 Apr 2004 10:52:31 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> References: <20040425041852.GA9367@w-m-p.com> <20040425231055.GG14211@w-m-p.com> <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> Message-ID: <20040426155231.GA22907@w-m-p.com> On Mon, Apr 26, 2004 at 05:29:21PM +0200, Marc Battyani wrote: > So do I put your current patches or do you want to look at them again? > BTW the multi-page table support has been written by Dmitri. Maybe we should > put him in copy? > > Let's close this case anyway. We have already spent way too much time on it. > Just give me your final patches and let's go back to more productive and > interesting code. I've looked at the iterate macro this weekend, and I very much like the idea of switching to that - it's conceptually very close to LOOP, but more powerful and cleaner. It's X11-style-licensed, so could even be included in the source package (contrib/ ?) for people who don't have it. So any rework should probably be based on that. -Klaus From marc.battyani at fractalconcept.com Mon Apr 26 16:12:31 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Mon, 26 Apr 2004 18:12:31 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040425041852.GA9367@w-m-p.com> <20040425231055.GG14211@w-m-p.com> <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <20040426155231.GA22907@w-m-p.com> Message-ID: <0fc101c42ba9$47910410$0a02a8c0@marcxp> "Klaus Weidner" wrote: > I've looked at the iterate macro this weekend, and I very much like the > idea of switching to that - it's conceptually very close to LOOP, but > more powerful and cleaner. It's X11-style-licensed, so could even be > included in the source package (contrib/ ?) for people who don't have it. As it's not standard, the best way is to include it in the sources. > So any rework should probably be based on that. Just verify that it works on CLISP before ;-) Marc From marc.battyani at fractalconcept.com Mon Apr 26 16:48:28 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Mon, 26 Apr 2004 18:48:28 +0200 Subject: CLISP developers attitude. Was: [cl-typesetting-devel] CLISP support References: <20040425041852.GA9367@w-m-p.com> <20040425231055.GG14211@w-m-p.com> <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <20040426155231.GA22907@w-m-p.com> Message-ID: <100201c42bae$4d4e2f90$0a02a8c0@marcxp> "Klaus Weidner" > On Mon, Apr 26, 2004 at 05:29:21PM +0200, Marc Battyani wrote: > > I've looked at the iterate macro this weekend, and I very much like the > idea of switching to that - it's conceptually very close to LOOP, but > more powerful and cleaner. It's X11-style-licensed, so could even be > included in the source package (contrib/ ?) for people who don't have it. > > So any rework should probably be based on that. So do I wait for your new patches or do I put your last ones. Just note that your workaround will not work on the next release of CLISP as they are busy to patch it to be sure that it won't work!! This is truly amazing! I would have understood if they did nothing, but they are actually spending time to be sure that it won't work. When I think at the time we spent on this CLISP issue, I'm rather disgusted. Are you sure you need to use CLISP ? Otherwise I would put #+clisp (error "You should use another implementation") Ok, I'm rather upset but look at this: "Sam Steingold" > > * Bruno Haible [2004-04-26 17:28:10 +0200]: [...] > > Can we add a warning about it? Let's say, expand > > > > (loop for r in '(42) finally (return r)) > > > > into something where the loop epilogue looks like this: > > > > system::end-loop > > (let ((#:G172 r)) > > (macrolet ((loop-finish () (system::loop-finish-warn) '(go system::end-loop)) > > (#:G173 () (system::loop-finally-abuse-warn 'r) '#:G172)) > > (symbol-macrolet ((r (#:G173))) > > (return r)))) > > > > The system::loop-finally-abuse-warn function would signal a > > portability warning at macroexpansion time. > > (loop for r in '(42) finally (setq r 123) (return r)) > > will give 2 unwarranted warnings. > > I would prefer that this usage just be banned, via an ANSI action. And this one is even worse: "Sam Steingold" > The interesting thing is just starting. > IIUC, Paul's test suite uncovered some bugs in this MIT LOOP (which, > again IIUC, was something like _a_ reference implementation for the > standard - not _THE_ reference implementation, of course). > This means that some people who are using this implementation will be > fixing it. Incompatibly. Introducing different new bugs. Cool! He is happy to see that it will be the mess! [All this is public it's on the clisp-devel mailing list] :( Marc From kw at w-m-p.com Mon Apr 26 17:32:52 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Mon, 26 Apr 2004 12:32:52 -0500 Subject: CLISP developers attitude. Was: [cl-typesetting-devel] CLISP support In-Reply-To: <100201c42bae$4d4e2f90$0a02a8c0@marcxp> References: <20040425041852.GA9367@w-m-p.com> <20040425231055.GG14211@w-m-p.com> <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <20040426155231.GA22907@w-m-p.com> <100201c42bae$4d4e2f90$0a02a8c0@marcxp> Message-ID: <20040426173252.GB23515@w-m-p.com> On Mon, Apr 26, 2004 at 06:48:28PM +0200, Marc Battyani wrote: > So do I wait for your new patches or do I put your last ones. Please use the attached one, I slightly changed v-split to be closer to the old code. I'm still not convinced that either this one or the old one is correct regarding the slot-value dy update - this one assumes that nobody cares about the table dy if the table fit completely, same as the old one. > Just note that your workaround will not work on the next release of CLISP as > they are busy to patch it to be sure that it won't work!! This is truly > amazing! I would have understood if they did nothing, but they are actually > spending time to be sure that it won't work. When I think at the time we > spent on this CLISP issue, I'm rather disgusted. Are you sure you need to > use CLISP ? Otherwise I would put #+clisp (error "You should use another > implementation") It should work on the next clisp - the point of my most recent changes was to remove the dependency on the platform-specific differences. If not that needs to be fixed - hopefully we'll have moved to iterate by then. Note that the new patch doesn't have any +clisp in it. > He is happy to see that it will be the mess! > > [All this is public it's on the clisp-devel mailing list] One thing that I do agree with him on is that it's a bad idea to depend on the values of autostepped iteration values in the finally clause, since that's not explicitly defined. The CLISP behavior IMHO actually makes more sense than that of the MIT loop. -Klaus -------------- next part -------------- diff -urN -x *.fas -x *.lib orig/cl-typesetting/cl-typesetting.asd cl-typesetting/cl-typesetting.asd --- orig/cl-typesetting/cl-typesetting.asd Thu Apr 22 05:23:18 2004 +++ cl-typesetting/cl-typesetting.asd Sun Apr 25 17:07:06 2004 @@ -6,7 +6,6 @@ (in-package asdf) -#+(or allegro lispworks cmu sbcl openmcl mcl scl) (defsystem :cl-typesetting :name "cl-typesetting" :author "Marc Battyani " diff -urN -x *.fas -x *.lib orig/cl-typesetting/tables.lisp cl-typesetting/tables.lisp --- orig/cl-typesetting/tables.lisp Sun Apr 25 11:40:38 2004 +++ cl-typesetting/tables.lisp Mon Apr 26 12:25:14 2004 @@ -91,19 +91,10 @@ when (> i 1) ; set all but last rows unsplittable do (setf (splittable-p row) nil) do - #-clisp - (loop for j = 0 then (+ j (col-span c)) - and tail on (cells row) - for c = (first tail) ; j is the column number of c - while (< j col-number) - collect (first tail) into head - finally ; insert cell between head and tail - (setf (cells row) (nconc head (list cell) tail))) - #+clisp (loop for j = 0 then (+ j (col-span c)) for tail = (cells row) then (cdr tail) for c = (first tail) ; j is the column number of c - while (and tail (< j col-number)) + while (and c (< j col-number)) collect (first tail) into head finally ; insert cell between head and tail (setf (cells row) (nconc head (list cell) tail))) @@ -114,34 +105,49 @@ (let ((full-size-offset (+ (border table) (* 2 (cell-padding table)))) (height (or (height row) +huge-number+))) (loop with next-widths = (col-widths table) - for width = (or (pop next-widths) 0) ; in case less elements specified + for cell in (cells row) - for col-number = 0 then (+ col-number col-span 1) - and col-span = (1- (col-span cell)) + and width = (or (pop next-widths) 0) ; in case less elements specified + and col-number = 0 then (+ col-number col-span 1) + and cell-height = 0.0 + + for col-span = (1- (col-span cell)) and row-span = (row-span cell) + + ;; Adjust cell width for cells spanning multiple columns unless (zerop col-span) do (incf width (+ (* col-span full-size-offset) (reduce #'+ next-widths :end col-span))) (setf next-widths (nthcdr col-span next-widths)) + + ;; Fill cell with content if required when (cell-start-row-p cell row) do (setf (box cell) (make-filled-vbox (content cell) width height) (width cell) width) + ;; A cell spanning several rows participates only in height calculation ;; of the last row if (and (numberp row-span) (> row-span 1)) do (span-cell rows cell col-number) else unless (height row) if (eql row-span 1) - maximize (compute-boxes-natural-size (boxes (box cell)) #'dy) into max-height + do (setq cell-height + (compute-boxes-natural-size (boxes (box cell)) #'dy)) else if (cell-end-row-p cell row) - maximize (- (compute-boxes-natural-size (boxes (box cell)) #'dy) - (reduce #'+ row-span - :key #'height - :end (1- (length row-span)) - :initial-value (* (1- (length row-span)) - full-size-offset))) into max-height - finally (setf height (+ (max (or (height row) 0) - #-clisp max-height #+clisp (or max-height 0)) +epsilon+))) + do (setq cell-height + (- (compute-boxes-natural-size (boxes (box cell)) #'dy) + (reduce #'+ row-span + :key #'height + :end (1- (length row-span)) + :initial-value (* (1- (length row-span)) + full-size-offset)))) + + maximize cell-height into max-height + + finally (setf height (+ (max (or (height row) 0.0) + max-height) + +epsilon+))) + (setf (height row) height) (loop for cell in (cells row) for row-span = (row-span cell) @@ -181,48 +187,59 @@ (dolist (row footer) (compute-row-size table row footer)))) (defmethod v-split ((table multi-page-table) dx dy &optional v-align) - ;;; Factor out rows that fit and return as a first value. - (with-slots (header footer rows-left) table + "Factor out rows that fit and return as a first value." + ;; Treat unsplittable rows as a single unit - for this purpose, + ;; group the rows-left list into the following form: + ;; + ;; ( (group1-height row1 row2 ...) + ;; (group2-height row7) + ;; (group3-height row8 row9 ...) ) + ;; + (with-slots (header footer border padding cell-padding) table (loop with boxes = () - with border = (border table) - and padding = (padding table) - and cell-padding = (cell-padding table) - with full-size-offset = (+ cell-padding cell-padding border) - with max-height = (- dy (reduce #'+ header :key #'dy) (reduce #'+ footer :key #'dy)) - for rows on rows-left - and prev-y = 0 then y ; vertical space that has been output - for row = (first rows) - for y = (+ padding border (height row) full-size-offset) - then (+ y (height row) full-size-offset) - while (<= y max-height) - ;do (setf (dy row) (+ (height row) full-size-offset)) - do (push row boxes) - finally - #+clisp (unless (> y max-height) (pop rows)) - (when (and boxes - ;; Trim unsplitalbe rows and reverse the list of accumulated boxes - (setf boxes - #-clisp - (loop for tail on boxes - for row = (first tail) - until (splittable-p row) - do (decf prev-y (+ (height row) full-size-offset)) - finally (return (nreverse tail))) - #+clisp - (loop for tail = boxes then (cdr tail) - for row = (first tail) - until (or (not row) (splittable-p row)) - do (decf prev-y (+ (height row) full-size-offset)) - finally (return (nreverse tail))))) - (setq boxes (append header boxes footer)) + with header+footer-height = (+ (reduce #'+ header :key #'dy) + (reduce #'+ footer :key #'dy)) + with current-height = (+ border padding) + with available-height = (- dy header+footer-height) + with row-groups = (loop with height = 0 + and rows = () + + for row in (rows-left table) + + do + (incf height (+ (height row) + (* 2 cell-padding) + border)) + (push row rows) + + when (splittable-p row) + collect (cons height (nreverse rows)) + and do (setf height 0 rows nil)) + with rows-remaining = (rows-left table) + + for (group-height . rows) in row-groups + while (<= (+ current-height group-height) available-height) + + do (dolist (r rows) + (push r boxes) + (pop rows-remaining)) + (incf current-height group-height) + + finally + (when boxes + (setq boxes (append header (nreverse boxes) footer)) + ;; reduce rows to output + (setf (rows-left table) rows-remaining) + ;; reduce space required by table (don't subtract header/footer) + (decf (slot-value table 'dy) current-height) (let ((first (first boxes)) (last (first (last boxes)))) (setf (slot-value first 'position) :first (slot-value last 'position) (if (eq first last) :single :last))) - (setf rows-left rows) ; reduce rows to output - (decf (slot-value table 'dy) prev-y) ; and space required by table - (return (values boxes rows (- max-height prev-y)))) - (return (values nil rows-left dy))))) + (return (values boxes + rows-remaining + (- dy current-height header+footer-height)))) + (return (values nil rows-remaining dy))))) (defmethod dy :around ((table multi-page-table)) (with-slots (header footer) table From marc.battyani at fractalconcept.com Mon Apr 26 20:56:41 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Mon, 26 Apr 2004 22:56:41 +0200 Subject: CLISP developers attitude. Was: [cl-typesetting-devel] CLISPsupport References: <20040425041852.GA9367@w-m-p.com><20040425231055.GG14211@w-m-p.com><0f6701c42ba3$4064fbc0$0a02a8c0@marcxp><20040426155231.GA22907@w-m-p.com><100201c42bae$4d4e2f90$0a02a8c0@marcxp> <20040426173252.GB23515@w-m-p.com> Message-ID: <111401c42bd0$fad017b0$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Mon, Apr 26, 2004 at 06:48:28PM +0200, Marc Battyani wrote: > > So do I wait for your new patches or do I put your last ones. > > Please use the attached one, I slightly changed v-split to be closer to > the old code. I'm still not convinced that either this one or the old one > is correct regarding the slot-value dy update - this one assumes that > nobody cares about the table dy if the table fit completely, same as the > old one. I tested with the examples with the patches and they still work. :) > It should work on the next clisp - the point of my most recent changes > was to remove the dependency on the platform-specific differences. If not > that needs to be fixed - hopefully we'll have moved to iterate by then. No it's not ok for CLISP, you still use an iteration variable in the finally clause and they don't want that at all. Makes them sick. They want the ANSI to ban such an associal behavior. Now they even have to modify the compiler to be able to tell you that you are wrong. :( Bruno Haible wrote: >Sam wrote: >> I would prefer that this usage just be banned, via an ANSI action. >An ANSI action can take a long time to happen. Whereas a portability warning >can be implemented in clisp today. >> (loop for r in '(42) finally (setq r 123) (return r)) >> >> will give 2 unwarranted warnings. >... and an error about (SETF #:G173) being an undefined function. Argh. >Seems we need some extra compiler support to get this right. Klaus: > One thing that I do agree with him on is that it's a bad idea to depend > on the values of autostepped iteration values in the finally clause, > since that's not explicitly defined. The CLISP behavior IMHO actually > makes more sense than that of the MIT loop. Yes, but that's not the point. It's a handy feature when you like and use LOOP and it works on every implementation but CLISP. The standard says nothing clear on that point so the CLISP interpretation is permitted as well as the other. BTW LOOP is never used in the clisp sources while it is used in the SBCL sources for instance. Well enough with that.... It's on the repository and the tarball. Marc From divanov at aha.ru Tue Apr 27 04:37:21 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Tue, 27 Apr 2004 08:37:21 +0400 Subject: [cl-typesetting-devel] CLISP support References: <20040424203005.GA5617@w-m-p.com><089a01c42a4a$f62a99b0$0a02a8c0@marcxp><20040425003858.GB6970@w-m-p.com> <20040425041852.GA9367@w-m-p.com><0a4401c42acd$df0697b0$0a02a8c0@marcxp> <20040425172424.GA13487@w-m-p.com> Message-ID: <000001c42c13$c82d9dd0$cb4d02c3@digo> Hello Klaus, | >> From 6.1.3 Value Accumulation Clauses: |> "The var argument is bound as if by the construct with. |> [...] |> The default type is implementation-dependent; but it must be a |> supertype of type real." | | CLISP is correct here, "t" is a supertype of real. I am afraid, HyperSpec has a typo in here. For example, 0 is a fixnum, which is a _subtype_, not _supertype_ of real. Moreover, CLtL2 specifies: The default type is implementation-dependent, but it must be a subtype of type (or integer float). -- Sincerely, Dmitri Ivanov lisp.ystok.ru From divanov at aha.ru Tue Apr 27 05:52:56 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Tue, 27 Apr 2004 09:52:56 +0400 Subject: [cl-typesetting-devel] CLISP support References: <20040425041852.GA9367@w-m-p.com><20040425043752.GA9843@w-m-p.com><0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp><20040425201508.GC14211@w-m-p.com><0bd101c42b09$8d3705f0$0a02a8c0@marcxp><20040425230025.GD14211@w-m-p.com><20040425231055.GG14211@w-m-p.com> <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> Message-ID: <007601c42c20$bcffe8c0$cb4d02c3@digo> Hello, | So do I put your current patches or do you want to look at them again? | BTW the multi-page table support has been written by Dmitri. Maybe we | should put him in copy? Klaus's rewrite of this method on v-split (defmethod v-split ((table multi-page-table)...) looks quite improving. I have no possibility to test by myself now, but if it reportedly works, I agree. |> But I agree |> with Klaus that ITERATE is a very nice package. I've been using it |> from time to time for my own programs and would like to see it used |> more often for Open Source stuff. | | Yes, I also like ITERATE this is why I proposed it to Klaus. If | everybody is OK I will integrate it to cl-pdf/cl-typesetting. By using | it we will avoid the problem of the fuzzy loop semantics on the | anti-loop implementation. I respect Klaus's goal of platform independency, but could it be achieved with less effort? After the great battle against LOOP, we all have found out too many details about how to use it properly, to code cautiously, and to conditionalize implementations differences. Do you suggest throwing out all the knowledge?! I propose gather it into cl-typesetting-devel Style Guide :-) Increasing the total size of source code without real necessity is also not good IMHO. Adding ITERATE would blow up the project, let alone rewriting. Personally, I hate learning new things when the old work almost fine:-) -- Sincerely, Dmitri Ivanov lisp.ystok.ru From kw at w-m-p.com Tue Apr 27 16:13:11 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Tue, 27 Apr 2004 11:13:11 -0500 Subject: CLISP developers attitude. Was: [cl-typesetting-devel] CLISP support In-Reply-To: <100201c42bae$4d4e2f90$0a02a8c0@marcxp> References: <20040425041852.GA9367@w-m-p.com> <20040425231055.GG14211@w-m-p.com> <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <20040426155231.GA22907@w-m-p.com> <100201c42bae$4d4e2f90$0a02a8c0@marcxp> Message-ID: <20040427161311.GH28833@w-m-p.com> On Mon, Apr 26, 2004 at 06:48:28PM +0200, Marc Battyani wrote: > Ok, I'm rather upset but look at this: I hadn't had a chance until recently to look at the discussion... > "Sam Steingold" > > I would prefer that this usage just be banned, via an ANSI action. > > And this one is even worse: > "Sam Steingold" > > The interesting thing is just starting. > > IIUC, Paul's test suite uncovered some bugs in this MIT LOOP (which, > > again IIUC, was something like _a_ reference implementation for the > > standard - not _THE_ reference implementation, of course). > > This means that some people who are using this implementation will be > > fixing it. Incompatibly. Introducing different new bugs. Cool! > > He is happy to see that it will be the mess! My interpretation of what Sam said is apparently very different from yours - I'm certain that the "Cool!" was meant ironically. If the MIT LOOP is indeed being changed in ways that make it incompatible with older versions of itself, there would be no point in attempting to have CLISP emulate its behavior. The only sane conclusion is that compatible code can't depend on anything regarding iteration variables in a FINALLY clause. The change CLISP is making will *not* gratuitously change the loop semantics for the next version, they are adding a compatibility note and warning, which would have saved me a lot of grief this weekend by directly pointing out the problem. The current cl-typesetting code would not need to be changed for their next version after all. FYI, I just tested the CL extensions in Emacs, and its non-MIT loop implementation also happens to agree with CLISP. So they are not the only ones who do things that way. -Klaus From kw at w-m-p.com Tue Apr 27 16:00:21 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Tue, 27 Apr 2004 11:00:21 -0500 Subject: [cl-typesetting-devel] CLISP support In-Reply-To: <007601c42c20$bcffe8c0$cb4d02c3@digo> References: <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <007601c42c20$bcffe8c0$cb4d02c3@digo> Message-ID: <20040427160021.GG28833@w-m-p.com> On Tue, Apr 27, 2004 at 09:52:56AM +0400, Dmitri Ivanov wrote: > I respect Klaus's goal of platform independency, but could it be achieved > with less effort? After the great battle against LOOP, we all have found out > too many details about how to use it properly, to code cautiously, and to > conditionalize implementations differences.s I don't like conditionalizations unless they're unavoidable, and even then they should if at all possible be hidden in a separate file that provides a system-independent wrapper. > Do you suggest throwing out all the knowledge?! I propose gather it > into cl-typesetting-devel Style Guide :-) The LOOP chapter of the Style Guide could be quite short - don't use iteration variables in the FINALLY section ;-) The implementations seem to agree on everything other than that. > Increasing the total size of source code without real necessity is also not > good IMHO. Adding ITERATE would blow up the project, let alone rewriting. > Personally, I hate learning new things when the old work almost fine:-) ITERATE has a couple of very nice features that LOOP doesn't have, for example better nested loops - you can COLLECT data into a variable of the surrounding loop from an inner loop. Also, it's easily extensible. And the rewriting would be fairly straightforward, since the semantics are essentially equivalent to LOOP. It's not nearly as big a change as SERIES would be. It's Marc's decision - I can live with either one. I'd suggest reducing the complexity of loops of either kind if at all possible, similar in spirit to the v-split change I made. -Klaus From marc.battyani at fractalconcept.com Tue Apr 27 17:52:21 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 27 Apr 2004 19:52:21 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040425041852.GA9367@w-m-p.com><20040425043752.GA9843@w-m-p.com><0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp><20040425201508.GC14211@w-m-p.com><0bd101c42b09$8d3705f0$0a02a8c0@marcxp><20040425230025.GD14211@w-m-p.com><20040425231055.GG14211@w-m-p.com><0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <007601c42c20$bcffe8c0$cb4d02c3@digo> Message-ID: <015501c42c80$64f30260$0a02a8c0@marcxp> "Dmitri Ivanov" wrote: Marc: > | Yes, I also like ITERATE this is why I proposed it to Klaus. If > | everybody is OK I will integrate it to cl-pdf/cl-typesetting. By using > | it we will avoid the problem of the fuzzy loop semantics on the > | anti-loop implementation. > > I respect Klaus's goal of platform independency, but could it be achieved > with less effort? After the great battle against LOOP, we all have found out > too many details about how to use it properly, to code cautiously, and to > conditionalize implementations differences. Do you suggest throwing out all > the knowledge?! I propose gather it into cl-typesetting-devel Style Guide > :-) Ahem... I remind you that the non conforming code is yours... ;-) > Increasing the total size of source code without real necessity is also not > good IMHO. Adding ITERATE would blow up the project, let alone rewriting. > Personally, I hate learning new things when the old work almost fine:-) Personally I really love to learn new things when they are clever and useful, but I also hate learning useless and stupid things YMMV. Marc From marc.battyani at fractalconcept.com Tue Apr 27 18:34:35 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 27 Apr 2004 20:34:35 +0200 Subject: ITERATE use (was Re: [cl-typesetting-devel] CLISP support) References: <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <007601c42c20$bcffe8c0$cb4d02c3@digo> <20040427160021.GG28833@w-m-p.com> Message-ID: <01b501c42c86$4b84a300$0a02a8c0@marcxp> "Klaus Weidner" wrote: > > The LOOP chapter of the Style Guide could be quite short - don't use > iteration variables in the FINALLY section ;-) He, he... ;-) > The implementations seem to agree on everything other than that. No, there is also the sequential vs global tests. Where CLisp differs from the other ones. > > Increasing the total size of source code without real necessity is also not > > good IMHO. Adding ITERATE would blow up the project, let alone rewriting. > > Personally, I hate learning new things when the old work almost fine:-) > > ITERATE has a couple of very nice features that LOOP doesn't have, for > example better nested loops - you can COLLECT data into a variable of the > surrounding loop from an inner loop. Also, it's easily extensible. > > And the rewriting would be fairly straightforward, since the semantics > are essentially equivalent to LOOP. It's not nearly as big a change as > SERIES would be. Yes, ITERATE is a superset of LOOP. > It's Marc's decision - I can live with either one. I'd suggest reducing > the complexity of loops of either kind if at all possible, similar in > spirit to the v-split change I made. Still don't like LOOP heh... ;-) But I agree that loops should keep a reasonable size. A good point in favor of ITERATE is that the semantics can be precisely defined and implemented. I discussed with Andreas and he is open to modify it and the associated doc so that it is really usable. A bad point is that it is not standard. So this will make the sources less readable for others. Opinions on the subject are welcomed! BTW Andreas has put a new version of ITERATE with a postscript manual. I will try to look at it this evening. Marc [I've put Andreas Fuchs in copy] From kw at w-m-p.com Tue Apr 27 20:01:01 2004 From: kw at w-m-p.com (Klaus Weidner) Date: Tue, 27 Apr 2004 15:01:01 -0500 Subject: ITERATE use (was Re: [cl-typesetting-devel] CLISP support) In-Reply-To: <01b501c42c86$4b84a300$0a02a8c0@marcxp> References: <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <007601c42c20$bcffe8c0$cb4d02c3@digo> <20040427160021.GG28833@w-m-p.com> <01b501c42c86$4b84a300$0a02a8c0@marcxp> Message-ID: <20040427200101.GE29424@w-m-p.com> On Tue, Apr 27, 2004 at 08:34:35PM +0200, Marc Battyani wrote: > "Klaus Weidner" wrote: > > The implementations seem to agree on everything other than that. > > No, there is also the sequential vs global tests. Where CLisp differs from > the other ones. I'm not sure what you mean by that - the changes I did regarding in-loop and on-loop etc. were all related to getting the expected iteration variable value into the FINALLY clause. If the code had not cared about those, there would have been no need for a change - the values always matched while inside the loop body itself. The maximized NIL value was a separate issue and a real bug in the code that depended on the implicit 0 initialization done by MIT LOOP. > > It's Marc's decision - I can live with either one. I'd suggest reducing > > the complexity of loops of either kind if at all possible, similar in > > spirit to the v-split change I made. > > Still don't like LOOP heh... ;-) You caught me... This entire episode didn't exactly endear it to me. I learned more about it than I originally wanted to, and I'm not sure if that was a good thing. > But I agree that loops should keep a reasonable size. Thanks, then I don't think it'll be a problem. The complex ones are just IMHO hard to debug and experiment with. > A good point in favor of ITERATE is that the semantics can be precisely > defined and implemented. I discussed with Andreas and he is open to modify > it and the associated doc so that it is really usable. Note that the FINALLY values are also the rather weird MIT LOOP ones in ITERATE: (iter (for i :from 0 :below 2) (for j :from 0 :below 2) (finally (return (list i j)))) => (2 1) So even if this behavior is documented and dependable, I still would not consider it to be good style to write code that uses those values. > A bad point is that it is not standard. So this will make the sources less > readable for others. I don't think that it's such a big deal - if you mentally remove some parentheses it looks much like LOOP, with some standard Lisp expressions intermingled. I think it's far cleaner to be able to use standard if/when inside ITER rather than the weird pseudo-Lisp LOOP variants. > BTW Andreas has put a new version of ITERATE with a postscript manual. I > will try to look at it this evening. I had read the included "Iterate Manual" MIT report this weekend, and I like it a lot. As far as I can tell it also doesn't specifically say anything about values of iterate vars in FINALLY clauses ;-) -Klaus From marc.battyani at fractalconcept.com Tue Apr 27 20:54:43 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Tue, 27 Apr 2004 22:54:43 +0200 Subject: ITERATE use (was Re: [cl-typesetting-devel] CLISP support) References: <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp><007601c42c20$bcffe8c0$cb4d02c3@digo><20040427160021.GG28833@w-m-p.com><01b501c42c86$4b84a300$0a02a8c0@marcxp> <20040427200101.GE29424@w-m-p.com> Message-ID: <022a01c42c99$de734050$0a02a8c0@marcxp> "Klaus Weidner" wrote: > On Tue, Apr 27, 2004 at 08:34:35PM +0200, Marc Battyani wrote: > > "Klaus Weidner" wrote: > > > The implementations seem to agree on everything other than that. > > > > No, there is also the sequential vs global tests. Where CLisp differs from > > the other ones. > > I'm not sure what you mean by that - the changes I did regarding in-loop > and on-loop etc. were all related to getting the expected iteration > variable value into the FINALLY clause. If the code had not cared about > those, there would have been no need for a change - the values always > matched while inside the loop body itself. No, you still don't grok loop. ;-) (The CLISP guys neither) Loop is simple though: You execute the clauses in sequence. For each close you set the variable, execute the test, if it fails you go to the epilog. That's all. > The maximized NIL value was a separate issue and a real bug in the code > that depended on the implicit 0 initialization done by MIT LOOP. We agree on that point. :) > > > It's Marc's decision - I can live with either one. I'd suggest reducing > > > the complexity of loops of either kind if at all possible, similar in > > > spirit to the v-split change I made. > > > > Still don't like LOOP heh... ;-) > > You caught me... This entire episode didn't exactly endear it to me. I > learned more about it than I originally wanted to, and I'm not sure if > that was a good thing. > > > But I agree that loops should keep a reasonable size. > > Thanks, then I don't think it'll be a problem. The complex ones are just > IMHO hard to debug and experiment with. > > > A good point in favor of ITERATE is that the semantics can be precisely > > defined and implemented. I discussed with Andreas and he is open to modify > > it and the associated doc so that it is really usable. > > Note that the FINALLY values are also the rather weird MIT LOOP ones in > ITERATE: > > (iter (for i :from 0 :below 2) > (for j :from 0 :below 2) > (finally (return (list i j)))) > => (2 1) Sure nice isn't it ? I discussed with Andreas and he is ok to keep and document this behavior :) > So even if this behavior is documented and dependable, I still would not > consider it to be good style to write code that uses those values. It's not a really good style but it's natural. The clause variables don't vanish during the epilog. > > A bad point is that it is not standard. So this will make the sources less > > readable for others. > > I don't think that it's such a big deal - if you mentally remove some > parentheses it looks much like LOOP, with some standard Lisp expressions > intermingled. I think it's far cleaner to be able to use standard if/when > inside ITER rather than the weird pseudo-Lisp LOOP variants. Yes, the conditionals and the grouping of LOOP clauses by AND are ugly and not lispy. (At least one point of agreement ;-) And the simple ITERATE clauses are the same as the LOOP ones so they are understandable. Also there is a mean to get the previous iteration value (I'm sure you won't like that one) and that alone is great! > > BTW Andreas has put a new version of ITERATE with a postscript manual. I > > will try to look at it this evening. > > I had read the included "Iterate Manual" MIT report this weekend, and I > like it a lot. As far as I can tell it also doesn't specifically say > anything about values of iterate vars in FINALLY clauses ;-) Next release ;-) I've also quickly read the manual and I am rather in favor of ITERATE. Anybody else has an opinion on the subject ? Marc From marc.battyani at fractalconcept.com Tue Apr 27 22:09:11 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 28 Apr 2004 00:09:11 +0200 Subject: [cl-typesetting-devel] ITERATE added to cl-pdf and cl-typesetting References: <0f6701c42ba3$4064fbc0$0a02a8c0@marcxp><007601c42c20$bcffe8c0$cb4d02c3@digo><20040427160021.GG28833@w-m-p.com><01b501c42c86$4b84a300$0a02a8c0@marcxp><20040427200101.GE29424@w-m-p.com> <022a01c42c99$de734050$0a02a8c0@marcxp> Message-ID: <027601c42ca4$4571ef90$0a02a8c0@marcxp> Following the discussions about the need to have a looping construct with a clearly defined semantics and with a coherent and predictable behavior on different implementations, I've decided to integrate ITERATE to cl-pdf. ITERATE is a superset of LOOP. I've also integrated the ITERATE manual to the repository so that everybody can read it. This has been added to the cl-pdf and cl-typesetting repositories and tarballs. Comments welcomed! Marc From divanov at aha.ru Wed Apr 28 04:48:04 2004 From: divanov at aha.ru (Dmitri Ivanov) Date: Wed, 28 Apr 2004 08:48:04 +0400 Subject: [cl-typesetting-devel] CLISP support References: <20040425041852.GA9367@w-m-p.com><20040425043752.GA9843@w-m-p.com><0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp><20040425201508.GC14211@w-m-p.com><0bd101c42b09$8d3705f0$0a02a8c0@marcxp><20040425230025.GD14211@w-m-p.com><20040425231055.GG14211@w-m-p.com><0f6701c42ba3$4064fbc0$0a02a8c0@marcxp> <007601c42c20$bcffe8c0$cb4d02c3@digo> <015501c42c80$64f30260$0a02a8c0@marcxp> Message-ID: <002c01c42cde$5c024130$7e4302c3@digo> Hello Marc, | "Dmitri Ivanov" wrote: |> I respect Klaus's goal of platform independency, but could it be |> achieved with less effort? After the great battle against LOOP, we all |> have found out |> too many details about how to use it properly, to code cautiously, and |> to conditionalize implementations differences. Do you suggest throwing |> out all |> the knowledge?! I propose gather it into cl-typesetting-devel Style |> Guide :-) | | Ahem... I remind you that the non conforming code is yours... ;-) May be, but I have only mimicked yours :-) -- Sincerely, Dmitri Ivanov lisp.ystok.ru From marc.battyani at fractalconcept.com Wed Apr 28 19:59:25 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 28 Apr 2004 21:59:25 +0200 Subject: [cl-typesetting-devel] CLISP support References: <20040425041852.GA9367@w-m-p.com><20040425043752.GA9843@w-m-p.com><0b0301c42ae4$ed9a29b0$0a02a8c0@marcxp><20040425201508.GC14211@w-m-p.com><0bd101c42b09$8d3705f0$0a02a8c0@marcxp><20040425230025.GD14211@w-m-p.com><20040425231055.GG14211@w-m-p.com><0f6701c42ba3$4064fbc0$0a02a8c0@marcxp><007601c42c20$bcffe8c0$cb4d02c3@digo><015501c42c80$64f30260$0a02a8c0@marcxp> <002c01c42cde$5c024130$7e4302c3@digo> Message-ID: <05d801c42d5b$4f632cf0$0a02a8c0@marcxp> "Dmitri Ivanov" wrote: Marc: > | Ahem... I remind you that the non conforming code is yours... ;-) > > May be, but I have only mimicked yours :-) Heh, good point... :-) Marc