[tbnl-devel] Problem with file uploads

[Aurelio Bignoli]

Tiarnán Ó Corráin writes:
 > However, I seem to be running into a problem with either mod lisp or
 > tbnl, for images >=~1MB. When I try to upload an image of this site,
 > an internal server error is returned after a few seconds, and the
 > converted image (using imagemagick) appears in the thumbnail gallery
 > with a portion of the bottom shown in grey, as if the image did not
 > upload completely to the server.

I had the same problem trying to upload a 23MB TIFF file to a
PIII/733MHz server running Linux, Apache 1.3 and CMUCL. The cause was
the "excessive" speed of Apache: the Lisp server is not so fast in
handling POST body, its socket buffer is filled up by mod_lisp, which
gets a write error and closes the connection.

To solve the problem, I added some declarations in rfc2388.lisp and
modified the inner loop to read the POST request body in chunks of
4096 byte, using the "Content-length" HTTP header. The result was a
25% increase in file transfer speed, I was able to handle about
3.8MB/sec.

Not bad, but not enough for data coming from a 100Mbit/sec LAN. So I
had also to "enlarge" the socket buffer with a call to the setsockopt
Unix system call.

If someone is interested, I could post my changes. With a caveat:
request body is read using READ-SEQUENCE; if the connection is closed
during transfer, the corresponding server thread hangs indefinitely.

Regards,
Aurelio