From vsedach at gmail.com  Mon Oct 26 15:46:23 2009
From: vsedach at gmail.com (Vladimir Sedach)
Date: Mon, 26 Oct 2009 09:46:23 -0600
Subject: [teepeedee2-devel] More TPD2 patches.
In-Reply-To: <873a56ihdb.fsf-genuine-vii@john.fremlin.org>
References: <c07e645b0910191808h5647af78v8d6401709516c722@mail.gmail.com>
	<873a56ihdb.fsf-genuine-vii@john.fremlin.org>
Message-ID: <c07e645b0910260846r67d71c84o5dad8c796e2ccb39@mail.gmail.com>

> To emphasise again, I am not going to apply patches which just change
> the names of variables, expand out the my stuff to make it longer and
> change the indentation, etc.

Ok, I'll stop sending those.

>> Subject: [PATCH] Refactored sendbuf.lisp to make the code easier to
>> read.
>
> For whom? ;-)

Anybody who doesn't know what my-defun etc. does.

>> +
>> +(defvar request-remote-ip*)
>> +(defvar request-remote-port*) ;; TODO
>> +(defvar request-headers*) ;; TODO
>> +(defvar request-method*)
>> +(defvar request-uri*)
>> +(defvar request-path*)
>> +(defvar request-raw-post-data*) ;; (file upload) TODO
>> +(defvar request-parameters*)
>
> Why do these things have only one earmuff?

That's a convention I borrowed from Hunchentoot 1.0.

> I kind of like this idea, and I guess I will test it out on my benchmark
> game to see if it is feasible. Have you done any benchmarks yourself?

Nope, still haven't gotten my code to run. However, I'm not sure this
(dynamically scoped variables) actually works. Is there any point
during the request dispatching where it can happen that the current
continuation is saved and the thread begins servicing another request?
If that's so then obviously this approach doesn't work.

I actually redid the patch to pass a struct containing all these
fields as an (anaphoric) parameter to the lambda that defpage
generates, which would work in the above scenario, however there's
still a bug there I haven't tracked down.

> This definitely will not fly. Definitely do not depend on .webapp in
> .http! Better to move the constant.

Yes, but I wanted to leave that for a separate patch.

>> -(defun default-http-error-page (dispatcher path params)
>> - ?(declare (ignore params path dispatcher))
>> +(defun default-http-error-page ()
>> ? ?(with-sendbuf ()
>> ? ? ?"<h1>I made a mistake. Sorry for the inconvenience.</h1>"))
>
> I want the path and parameters to be passed to this function.

*If* the dynamic vars work, then you can get at them via request-path*
and request-parameters*. Otherwise, yeah, this is going to need the
request details as an argument.

>> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?("x-forwarded-for" ;; FIXME
>> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? (setf request-remote-ip*
>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? (match-x-forwarded-for value)))))))
>
> Why the FIXME?

I'm not sure having x-forwarded-for clobber the request-remote-ip
value is the right thing to do.

>> diff --git a/src/lib/macros.lisp b/src/lib/macros.lisp
>> index 35b0c76..e554106 100644
>> --- a/src/lib/macros.lisp
>> +++ b/src/lib/macros.lisp
>> @@ -178,16 +178,3 @@
>> ?(defmacro let-current-values (vars &body body)
>> ? ?`(let ,(loop for v in vars collect `(,v ,v))
>> ? ? ? , at body))
>> -
>> -
>> -(defmacro with-preserve-specials (specials &body body)
>> - ?(let ((tmps (mapcar (lambda(x)(gensym (symbol-name x))) specials)))
>> - ? ?`(let ,(loop for s in specials
>> - ? ? ? ? ? ? ?for m in tmps
>> - ? ? ? ? ? ? ?collect `(,m (when (boundp ',s),s)))
>> - ? ? ? (macrolet ((with-specials-restored (&body body)
>> - ? ? ? ? ? ? ? `(let ,',(loop for s in specials
>> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?for m in tmps
>> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?collect `(,s ,m))
>> - ? ? ? ? ? ? ? ? ?, at body)))
>> - ? ? ?, at body))))
>
> Why the hell are you deleting these macros?

It's much easier to understand (and just as easy to write) that code
just using LET.

>> -(defconstant-string +channel-page-name+ "/*channel*")
>> -
>> -(defconstant-string +action-page-name+ "/*action*")
>> -
> Why are you deleting these?

What's the difference between typing +action-page-name+ and
"/*action*"? The URI is what needs to be the same. Having an alias for
it just makes the code harder to grep.

Vladimir



From not at just.any.name  Tue Oct 27 08:53:00 2009
From: not at just.any.name (John Fremlin)
Date: Tue, 27 Oct 2009 08:53:00 +0000
Subject: [teepeedee2-devel] More TPD2 patches.
In-Reply-To: <c07e645b0910260846r67d71c84o5dad8c796e2ccb39@mail.gmail.com>
	(Vladimir Sedach's message of "Mon, 26 Oct 2009 09:46:23 -0600")
References: <c07e645b0910191808h5647af78v8d6401709516c722@mail.gmail.com>
	<873a56ihdb.fsf-genuine-vii@john.fremlin.org>
	<c07e645b0910260846r67d71c84o5dad8c796e2ccb39@mail.gmail.com>
Message-ID: <87skd5fcb7.fsf-genuine-vii@john.fremlin.org>

Hi Vladimir,

I'm about to knock up a TODO list for tpd2. I am encouraged that you now
understand the sendbuf stuff -- I would like to rewrite it using
Stelian's proper buffer pinning (if it is finished), and maybe a better
representation using an array instead of list head and tails. . .

Vladimir Sedach <vsedach at gmail.com> writes:
[...]
>>> +(defvar request-remote-ip*)
>>> +(defvar request-remote-port*) ;; TODO
>>> +(defvar request-headers*) ;; TODO
>>> +(defvar request-method*)
>>> +(defvar request-uri*)
>>> +(defvar request-path*)
>>> +(defvar request-raw-post-data*) ;; (file upload) TODO
>>> +(defvar request-parameters*)
>>
>> Why do these things have only one earmuff?
>
> That's a convention I borrowed from Hunchentoot 1.0.

Looking at the Hunchentoot mechanism, I think that it does something like
this

(defun remote-addr* (&optional (request *request*))
  "Returns the address the current request originated from."
  (remote-addr request))

So that it has just one special variable (i.e. *request*) and the star
is on the function is to indicate that it by default accesses that *request*.

This seems like quite a sensible approach, as it avoids an explosion of
special variables.

>> I kind of like this idea, and I guess I will test it out on my benchmark
>> game to see if it is feasible. Have you done any benchmarks yourself?
>
> Nope, still haven't gotten my code to run. However, I'm not sure this
> (dynamically scoped variables) actually works. Is there any point
> during the request dispatching where it can happen that the current
> continuation is saved and the thread begins servicing another request?
> If that's so then obviously this approach doesn't work.

I think that now the request dispatching waits until all the HTTP
request headers have been sent before doing anything at all (before, it
could partially handle one header line at a time but cl-cont made that
very slow).

Then if there is no request body (i.e. not a POST) then it dispatches
straightaway. Otherwise it might do a context switches (I think).

Note the channel page handler for an example which does not return
immediately and so would use the hated let-preserving-specials :-)

[...]
>> I want the path and parameters to be passed to this function.
>
> *If* the dynamic vars work, then you can get at them via request-path*
> and request-parameters*. Otherwise, yeah, this is going to need the
> request details as an argument.

O sorry, that was stupid of me.

>
>>> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?("x-forwarded-for" ;; FIXME
>>> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? (setf request-remote-ip*
>>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? (match-x-forwarded-for value)))))))
>>
>> Why the FIXME?
>
> I'm not sure having x-forwarded-for clobber the request-remote-ip
> value is the right thing to do.

Yes, I wanted to make that a configuration option somehow. But as I
always use tpd2 behind another server, I haven't done it.

I guess that should be a configuration option on the dispatcher (not
sure really).

[...]
> What's the difference between typing +action-page-name+ and
> "/*action*"? The URI is what needs to be the same. Having an alias for
> it just makes the code harder to grep.

I was a bit worried about having the earmuffs and it is easier to change
it in one place.

Otherwise I agree that maybe it is the wrong side of the balance.