[cl-ppcre-devel] Excessive Memory Usage by CREATE-SCANNER

[Edi Weitz]

On Fri, 06 Jul 2007 12:31:03 +0300, Volkan YAZICI <yazicivo at ttnet.net.tr> wrote:

> While trying to build a parse tree list using CREATE-SCANNER, in the
> attempts after first 2-3 tries, below code exhausts the whole system
> memory and halts the system.
>
> (defparameter *markup-transformations*
>   (loop for (element syntax) on
> 	(list
> 	 ;; Link formatting.
> 	 :link-internal "\\[\\["
> 	 :link-external "\\[http://"
> 	 :link-external "\\[https://"
> 	 :link-external "\\[ftp://"
> 	 ;; Text formatting.
> 	 :text-italic-bold "'''''"
> 	 :text-italic "'''"
> 	 :text-bold "''"
> 	 :text-underline "\\_\\_"
> 	 :text-monospace "`"
> 	 :text-superscript "\\^"
> 	 :text-subscript ",,"
> 	 ;; Formattings requiring a fresh line.
> 	 :header-3 "\\n=== "
> 	 :header-2 "\\n== "
> 	 :header-1 "\\n= "
> 	 :code-start "\\n{{{"
> 	 :code-end "\\n}}}"
> 	 :blockquote "\\n  ")
> 	by #'cddr collect
> 	(list element (cl-ppcre:create-scanner
> 		       (string-append "^" syntax)))))
>
> After 15 minutes, still no OOM reactions. (SBCL 1.0.6) So I needed
> restart the machine. (And therefore, I cannot narrow down the
> problematic part of the code by bisectioning.) Do you have any ideas
> about the erronous line?

Works fine for me with SBCL 1.0.5 on Linux.  I put the above code in a
file, defined STRING-APPEND in the obvious way using FORMAT, and
compiled and loaded the file a couple of times without problems.  If
you can reproduce the problem, then maybe you should try to print
something (and force output) for each iteration, so you can see when
printing stops.

(Which version of CL-PPCRE did you use, BTW?)

HTH,
Edi.