From dherring at tentpost.com Tue Jan 23 05:48:05 2007 From: dherring at tentpost.com (Daniel Herring) Date: Tue, 23 Jan 2007 00:48:05 -0500 (EST) Subject: [fetter-devel] Verrazano Speedup Message-ID: In file, frontent/utility.lisp:47, change (defmacro push-end (value lst) `(setf ,lst (append ,lst (list ,value)))) to (defmacro push-end (value lst) `(nconc ,lst (list ,value))) for a massive speedup (on my machine, parsing Qt/QtCore dropped from 90s to 9s). - Daniel From rayiner at gmail.com Tue Jan 23 19:55:16 2007 From: rayiner at gmail.com (Rayiner Hashem) Date: Tue, 23 Jan 2007 14:55:16 -0500 Subject: [fetter-devel] Verrazano Speedup In-Reply-To: References: Message-ID: Awesome! I'll try to make the change and commit it to the tree. On 1/23/07, Daniel Herring wrote: > In file, frontent/utility.lisp:47, change > > (defmacro push-end (value lst) > `(setf ,lst (append ,lst (list ,value)))) > > to > (defmacro push-end (value lst) > `(nconc ,lst (list ,value))) > > for a massive speedup (on my machine, parsing Qt/QtCore dropped from 90s > to 9s). > > - Daniel > _______________________________________________ > fetter-devel mailing list > fetter-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/fetter-devel > From christian.lynbech at ericsson.com Wed Jan 24 11:29:19 2007 From: christian.lynbech at ericsson.com (Christian Lynbech) Date: Wed, 24 Jan 2007 12:29:19 +0100 Subject: [fetter-devel] Verrazano Speedup In-Reply-To: (Rayiner Hashem's message of "Tue, 23 Jan 2007 14:55:16 -0500") References: Message-ID: I just discovered that the fix needs a fix, like the following version: (defmacro push-end (value lst) `(setf ,lst (nconc ,lst (list ,value)))) Without the explicit setf, the value change may get lost as nconc is a destructive function. At least with SBCL, I suddenly had no definitions derived from macros after applying the fix. -- Christian From dherring at tentpost.com Fri Jan 26 05:57:58 2007 From: dherring at tentpost.com (Daniel Herring) Date: Fri, 26 Jan 2007 00:57:58 -0500 (EST) Subject: [fetter-devel] Verrazano Speedup In-Reply-To: References: Message-ID: On Wed, 24 Jan 2007, Christian Lynbech wrote: > I just discovered that the fix needs a fix, like the following > version: > > (defmacro push-end (value lst) > `(setf ,lst (nconc ,lst (list ,value)))) > > Without the explicit setf, the value change may get lost as nconc is a > destructive function. Mea culpa. Please ignore my "speedup". As Christian found, it is just a variant on lossy compression. The original code was just fine. - Daniel P.S. My code was wrong since lst is initialized to nil; the nconc creates a new list in this case. Initializing lst to something like (list 1) and later using (cdr lst) results in the same ~90s timings as before. From dherring at tentpost.com Sun Jan 28 01:53:44 2007 From: dherring at tentpost.com (Daniel Herring) Date: Sat, 27 Jan 2007 20:53:44 -0500 (EST) Subject: [fetter-devel] Verrazano Speedup, take 2 In-Reply-To: References: Message-ID: FWIW, running Verrazano against Qt4.2 (a) isn't working for me and (b) is very slow. Thus, I'm trying to fix (b) to help debug (a). Profiling indicates that most of the time is being spent in (read-macros), hence my focus. >From my read of parser.lisp, (create-macro-nodes) sets the "current file" when a line begins with "# ", and (create-macro-node) only processes lines beginning with "#define ". When Verrazano obtains the Qt macros, it calls `gccxml --preprocess -dDI -I/opt/qt4.2/include vzntemp.cpp > vzntemp.mac` where vzntemp.cpp is simply the two lines #include "Qt/QtCore" const int __verrazano_binding = 1; The resulting vzntemp.mac is 1.5MB in size, but only 215KB consists of lines starting with "#"; the rest is blank lines or function definitions. Examples: void qDebug(const char *, ...) __attribute__ ((format (printf, 1, 2))) ; class QString; QString qt_error_string(int errorCode = -1); void qCritical(const char *, ...) __attribute__ ((format (printf, 1, 2))) ; Thus, I modified (read-macros) to only store non-empty lines beginning with #\# (since the rest are never used). verrazano/frontend/parser.lisp:74 ; read macro definitions from the macro file (defun read-macros (path) (let ((macs nil)) (with-open-file (in path :direction :input :if-exists :supersede) (loop for line = (read-line in nil 'eof) until (eq line 'eof) do (if (and (not (equal "" line)) (eq #\# (char line 0))) (push-end (split-sequence #\Space line) macs)))) macs)) This results in fast operation (10s versus 90s), with the added benefit of (hopefully) not breaking the macro processing. Later, Daniel From rayiner at gmail.com Wed Jan 31 21:10:16 2007 From: rayiner at gmail.com (Rayiner Hashem) Date: Wed, 31 Jan 2007 16:10:16 -0500 Subject: [fetter-devel] Verrazano Speedup, take 2 In-Reply-To: References: Message-ID: Ok, nice optimization. It shouldn't break anything, so it shouldn't hurt to incorporate it. Let me know what kind of progress you make against Qt 4.2. I suspect the vtable layout code is not smart enough to handle a library that complex (yet). On a somewhat unrelated note, I've finagled my way into using Lisp and Cairo to do a software simulation/mockup for a design competition here at school. This will hopefully generate some improvements in Vzn by virtue of my having to use it myself for the next few months :) On 1/27/07, Daniel Herring wrote: > FWIW, running Verrazano against Qt4.2 (a) isn't working for me and (b) is > very slow. Thus, I'm trying to fix (b) to help debug (a). Profiling > indicates that most of the time is being spent in (read-macros), hence my > focus. > > >From my read of parser.lisp, (create-macro-nodes) sets the "current file" > when a line begins with "# ", and (create-macro-node) only processes lines > beginning with "#define ". > > When Verrazano obtains the Qt macros, it calls > `gccxml --preprocess -dDI -I/opt/qt4.2/include vzntemp.cpp > vzntemp.mac` > where vzntemp.cpp is simply the two lines > #include "Qt/QtCore" > const int __verrazano_binding = 1; > > The resulting vzntemp.mac is 1.5MB in size, but only 215KB consists of > lines starting with "#"; the rest is blank lines or function definitions. > Examples: > void qDebug(const char *, ...) > __attribute__ ((format (printf, 1, 2))) > ; > > class QString; > QString qt_error_string(int errorCode = -1); > void qCritical(const char *, ...) > __attribute__ ((format (printf, 1, 2))) > ; > > > Thus, I modified (read-macros) to only store non-empty lines beginning > with #\# (since the rest are never used). > > verrazano/frontend/parser.lisp:74 > ; read macro definitions from the macro file > (defun read-macros (path) > (let ((macs nil)) > (with-open-file (in path :direction :input :if-exists :supersede) > (loop for line = (read-line in nil 'eof) > until (eq line 'eof) > do (if (and (not (equal "" line)) (eq #\# (char line 0))) > (push-end (split-sequence #\Space line) macs)))) > macs)) > > > This results in fast operation (10s versus 90s), with the added benefit of > (hopefully) not breaking the macro processing. > > Later, > Daniel > _______________________________________________ > fetter-devel mailing list > fetter-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/fetter-devel >