From john at fremlin.org Thu Oct 1 10:51:30 2009 From: john at fremlin.org (John Fremlin) Date: Thu, 01 Oct 2009 10:51:30 +0000 Subject: [cl-irregsexp-devel] some smaller patches In-Reply-To: (Attila Lendvai's message of "Wed, 30 Sep 2009 21:25:03 +0200") References: <87ocosieeb.fsf-genuine-vii@john.fremlin.org> Message-ID: <87r5tn9ykt.fsf-genuine-vii@john.fremlin.org> Hi Attila, Attila Lendvai writes: [...] > arguably in this case it should be :maximum-length but max/min is so > well spread that it's one of the very few abbreviations that we accept > internally in public API. but it's your lib... my patches are only > suggestions! :) I should revisit the code anyway to add single and double floats and as you are so nice about it, I accept your suggestion :-) [...] >> You change it to an nreverse! > > > do some profiling... ;) > > it's not faster for sure, but i doubt you'll be able to point out the > difference. Thinking about it the whole idea of building up a list of things to concatenate (now with nreverse) and then concatenating them in a weird way is wrong. (The nreverse should be high up on the profile and if it's not then that's a bug showing that the rest is too slow.) There shouldn't be the big difference between byte-vectors and strings either. I will get together a benchmark and make a better implementation. [...] >> Why are you using BODY here for a non-code form? > > > it plays much better with slime indentation, but that's about all. > > i usually freely exchange &body and &rest as they have no other > effects than that (and documentation-wise it's arguable that the body > there really is a body, that body is just a DSL not lisp...) That makes sense. Good points. >> This makes no difference I believe ^^^^ > > > profile it. on sbcl with-output-to-string creates a > string-output-stream which is more expensive when pretty-printing > stuff is initialized. Thanks for the info! >> This rewrite seems in aid of faster string concatenation. I have a good >> strcat somewhere, I can put that in. Do you have a benchmark? > > > not for this specific case, but i went through quite a few profiling > sessions and it seemed to be the fastest string concatenation (on > sbcl). Before tpd2 dumped strings altogether I had a fast strcat. However, copying strings multiple times for no reason surely isn't a good idea, and that's what's happening here. I will try to come up with a better way. Should be done before Monday, if not shout! From attila.lendvai at gmail.com Thu Oct 1 11:04:44 2009 From: attila.lendvai at gmail.com (Attila Lendvai) Date: Thu, 1 Oct 2009 13:04:44 +0200 Subject: [cl-irregsexp-devel] some smaller patches In-Reply-To: <87r5tn9ykt.fsf-genuine-vii@john.fremlin.org> References: <87ocosieeb.fsf-genuine-vii@john.fremlin.org> <87r5tn9ykt.fsf-genuine-vii@john.fremlin.org> Message-ID: > However, copying strings multiple times for no reason surely isn't a > good idea, and that's what's happening here. I will try to come up with > a better way. > > Should be done before Monday, if not shout! great! although i must note that it has no direct effect on me currently, so don't fall in the trap of too early, too much and unprofiled optimization... the only reason i've touched that part of the code is the iter dependency, which is arguable anyway. we like and use iter more than loop, but it's at the bottom of our TODO to write an iter lib using a proper walker and properly updating the lexenv... we have several transitive cl-ppcre dependencies, and all our own usages are in non-performance-critical code, so for now we don't bring in cl-irregsexp as a +1 dep. -- attila From john at fremlin.org Tue Oct 6 11:24:27 2009 From: john at fremlin.org (John Fremlin) Date: Tue, 06 Oct 2009 11:24:27 +0000 Subject: [cl-irregsexp-devel] Much better replace that does not copy the main string every replacement In-Reply-To: (Attila Lendvai's message of "Thu, 1 Oct 2009 16:22:05 +0200") References: <87ocosieeb.fsf-genuine-vii@john.fremlin.org> <87r5tn9ykt.fsf-genuine-vii@john.fremlin.org> <87iqez9xqp.fsf-genuine-vii@john.fremlin.org> Message-ID: <87ws38rcic.fsf-genuine-vii@john.fremlin.org> Pushed major rewrite of all replace stuff to common-lisp.net. Changed my simple benchmarks from being several orders of magnitude slower than cl-ppcre to being much faster than cl-ppcre. (defun bench (n &key (convertor 'identity)) (let ((s (make-string n))) (loop for i below n by 2 do (setf (char s i) #\r)) (let ((s (funcall convertor s))) (time (cl-irregsexp:match-replace-all s ("r" "replaced")))))) (defun bench-no-replace (n &key (convertor 'identity)) (let ((s (make-string n))) (loop for i below n by 2 do (setf (char s i) #\r)) (let ((s (funcall convertor s))) (eq s (time (cl-irregsexp:match-replace-one s ("q" "replaced"))))))) (defun bench-cl-ppcre (n) (let ((s (make-string n))) (loop for i below n by 2 do (setf (char s i) #\r)) (time (cl-ppcre:regex-replace-all "r" s "replaced")))) (defun bench-cl-ppcre-no-replace (n) (let ((s (make-string n))) (loop for i below n by 2 do (setf (char s i) #\r)) (time (cl-ppcre:regex-replace-all "q" s "replaced")))) There is a worrying bug (in SBCL?) that blows up like this in the find-random test. Guess that should be isolated, but everything seems fine on ClozureCL so . . . failed AVER: (ZEROP (HASH-TABLE-COUNT #)) This is probably a bug in SBCL itself. (Alternatively, SBCL might have been corrupted by bad user code, e.g. by an undefined Lisp operation like (FMAKUNBOUND 'COMPILE), or by stray pointers from alien code or from unsafe Lisp code; or there might be a bug in the OS or hardware that SBCL is running on.) If it seems to be a bug in SBCL itself, the maintainers would like to know about it. Bug reports are welcome on the SBCL mailing lists, which you can find at . [Condition of type SB-INT:BUG] Restarts: 0: [RETEST] Rerun the test # 1: [IGNORE] Signal an exceptional test failure and abort the test #. 2: [EXPLAIN] Ignore the rest of the tests and explain current results 3: [RETRY] Retry SLIME REPL evaluation request. 4: [ABORT] Return to SLIME's top level. 5: [TERMINATE-THREAD] Terminate this thread (#) Backtrace: 0: (SB-INT:BUG "~@")[:EXTERNAL] 1: (SB-IMPL::%FAILED-AVER (ZEROP (HASH-TABLE-COUNT (SB-C::CORE-OBJECT-PATCH-TABLE SB-C::OBJECT)))) 2: (SB-C::FIX-CORE-SOURCE-INFO # # #) 3: (SB-C::%COMPILE ..)[:EXTERNAL] [...]