[hunchentoot-devel] Hunchentoot effective DOS-attack
Peter Stiernström
peter.stiernstrom at blixtvik.se
Mon Jul 6 11:53:54 UTC 2009
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hans Hübner wrote:
> Peter,
>
> try this:
>
> http://bknr.net/trac/changeset/4434?format=diff&new=4434
Hello Hans,
I still get an unbound-variable condition for hunchentoot:*acceptor* but
the trace look a bit different:
The variable HUNCHENTOOT:*ACCEPTOR* is unbound.
BREAK was entered because of *BREAK-ON-SIGNALS* (now rebound to NIL).
[Condition of type SIMPLE-CONDITION]
Restarts:
0: [CONTINUE] Return from BREAK.
1: [REASSIGN] Return from BREAK and assign a new value to
*BREAK-ON-SIGNALS*.
2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "Hunchentoot
worker (client: 127.0.0.1:46427)" RUNNING {AF2EA11}>)
Backtrace:
0: (BREAK "~A~%BREAK was entered because of *BREAK-ON-SIGNALS* ~\n
(now rebound to NIL)." #<UNBOUND-VARIABLE *ACCEPTOR*
{AF30361}>)
1: (SIGNAL #<UNBOUND-VARIABLE *ACCEPTOR* {AF30361}>)[:EXTERNAL]
2: (ERROR UNBOUND-VARIABLE)[:EXTERNAL]
3: (SB-KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER #<unavailable argument>
#.(SB-SYS:INT-SAP #XB5724028) #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
#XB5723CDC :TYPE (* (STRUCT SB-VM::OS-CONTEXT-T-STRUCT))> (14))
4: (SB-KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER #<unavailable argument>
#.(SB-SYS:INT-SAP #XB5724028) #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
#XB5723CDC :TYPE (* (STRUCT SB-VM::OS-CONTEXT-T-STRUCT))> (14))[:EX..
5: (SB-KERNEL:INTERNAL-ERROR #.(SB-SYS:INT-SAP #XB5723CDC)
#<unavailable argument>)
6: ("foreign function: #x806480C")
7: ("foreign function: #x8052BCC")
8: ("foreign function: #x8056BD3")
9: ((LAMBDA ()))
10: ((FLET SB-THREAD::WITH-MUTEX-THUNK))
11: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]477))
12: (SB-THREAD::CALL-WITH-MUTEX ..)
13: ((LAMBDA ()))
14: ("foreign function: #x806480C")
15: ("foreign function: #x8052C21")
16: ("foreign function: #x805BD9D")
17: ("foreign function: #xB7FB84C0")
I did set *break-on-signals* to 'unbound-variable and my modified
function now looks like this:
(defmethod handle-incoming-connection ((taskmaster
one-thread-per-connection-taskmaster) socket)
(let ((*acceptor* (taskmaster-acceptor taskmaster)))
(handler-case
(bt:make-thread (lambda ()
(process-connection *acceptor* socket))
:name (format nil "Hunchentoot worker \(client:
~A)" (client-as-string socket)))
(error (e)
(log-message *lisp-errors-log-level*
"Error while creating worker thread for new
incoming connection: ~A" e)))))
/Peter
>
> -Hans
>
> On Mon, Jul 6, 2009 at 13:21, Peter
> Stiernström<peter.stiernstrom at blixtvik.se> wrote:
> Hello Hans,
>
> Hans Hübner wrote:
>>>> Peter, as I cannot reproduce the problem myself, can you please post a
>>>> stack backtrace? Thanks!
> The variable HUNCHENTOOT:*ACCEPTOR* is unbound.
> BREAK was entered because of *BREAK-ON-SIGNALS* (now rebound to NIL).
> [Condition of type SIMPLE-CONDITION]
>
> Restarts:
> 0: [CONTINUE] Return from BREAK.
> 1: [REASSIGN] Return from BREAK and assign a new value to
> *BREAK-ON-SIGNALS*.
> 2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "Hunchentoot
> listener (*:8000)" RUNNING {C42AAE9}>)
>
> Backtrace:
> 0: (BREAK "~A~%BREAK was entered because of *BREAK-ON-SIGNALS* ~\n
> (now rebound to NIL)." #<UNBOUND-VARIABLE *ACCEPTOR*
> {AD37FC1}>)
> 1: (SIGNAL #<UNBOUND-VARIABLE *ACCEPTOR* {AD37FC1}>)[:EXTERNAL]
> 2: (ERROR UNBOUND-VARIABLE)[:EXTERNAL]
> 3: (SB-KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER #<unavailable argument>
> #.(SB-SYS:INT-SAP #XB5DFFEF0) #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
> #XB5DFFBDC :TYPE (* (STRUCT SB-VM::OS-CONTEXT-T-STRUCT))> (14))
> 4: (SB-KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER #<unavailable argument>
> #.(SB-SYS:INT-SAP #XB5DFFEF0) #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
> #XB5DFFBDC :TYPE (* (STRUCT SB-VM::OS-CONTEXT-T-STRUCT))> (14))[:EX..
> 5: (SB-KERNEL:INTERNAL-ERROR #.(SB-SYS:INT-SAP #XB5DFFBDC)
> #<unavailable argument>)
> 6: ("foreign function: #x806480C")
> 7: ("foreign function: #x8052BCC")
> 8: ("foreign function: #x8056BD3")
> 9: (HUNCHENTOOT:LOG-MESSAGE :ERROR "Error while creating worker thread
> for new incoming connection: ~A")[:EXTERNAL]
> 10: ((SB-PCL::FAST-METHOD HUNCHENTOOT:HANDLE-INCOMING-CONNECTION
> (HUNCHENTOOT:ONE-THREAD-PER-CONNECTION-TASKMASTER T)) ..)
> 11: ((SB-PCL::FAST-METHOD HUNCHENTOOT:ACCEPT-CONNECTIONS
> (HUNCHENTOOT:ACCEPTOR)) #<unavailable argument> #<unavailable argument>
> #<HUNCHENTOOT:ACCEPTOR (host *, port 8000)>)
> 12: ((FLET SB-THREAD::WITH-MUTEX-THUNK))
> 13: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]477))
> 14: (SB-THREAD::CALL-WITH-MUTEX ..)
> 15: ((LAMBDA ()))
> 16: ("foreign function: #x806480C")
> 17: ("foreign function: #x8052C21")
> 18: ("foreign function: #x805BD9D")
> 19: ("foreign function: #xB7FB84C0")
>
> I found that not attempting to log a message solved this remaining problem.
>
> Thank you very much for your help!
>
> /Peter
>
>>>> 2009/7/6 Peter Stiernström <peter.stiernstrom at blixtvik.se>:
>>>> Hans,
>>>>
>>>> I should have realised I was editing the lispworks implementation myself
>>>> had I only had a cup of coffee this morning :-P
>>>>
>>>> Now when I tried your patch out I still get hangups but this is another
>>>> kind of hangup:
>>>>
>>>> debugger invoked on a UNBOUND-VARIABLE in thread #<THREAD "Hunchentoot
>>>> listener (*:8000)" RUNNING {AD467D1}>:
>>>> The variable HUNCHENTOOT:*ACCEPTOR* is unbound.
>>>>
>>>> I am trying this out with the hunchentoot default page by the way.
>>>>
>>>> /Peter
>>>>
>>>> Hans Hübner wrote:
>>>>>>> Peter,
>>>>>>>
>>>>>>> sorry, I am not able to reproduce the problem myself, so I made the
>>>>>>> patch blindly and made a mistake, working on the Lispworks code rather
>>>>>>> than the portable code that affects you. Please look at the first
>>>>>>> hunk of this diff:
>>>>>>>
>>>>>>> http://bknr.net/trac/changeset/4430?format=diff&new=4430
>>>>>>>
>>>>>>> and let me know if that does it for you.
>>>>>>>
>>>>>>> Sorry,
>>>>>>> Hans
>>>>>>>
>>>>>>> On Mon, Jul 6, 2009 at 11:09, Peter
>>>>>>> Stiernström<peter.stiernstrom at blixtvik.se> wrote:
>>>>>>> Hi Hans,
>>>>>>>
>>>>>>> Hans Hübner wrote:
>>>>>>>>>> Hi Peter,
>>>>>>>>>>
>>>>>>>>>> 2009/7/6 Peter Stiernström <peter.stiernstrom at blixtvik.se>
>>>>>>>>>>> I had a quick peek at client-as-string and it does indeed not handle the
>>>>>>>>>>> sb-bsd-sockets:not-connected-error and if I put a handler-case around
>>>>>>>>>>> body of client-as-string returning nil when the exception appears I can
>>>>>>>>>>> not reproduce my problem anymore.
>>>>>>>>>>>
>>>>>>>>>>> This wouldn't provide a portable solution though :-P
>>>>>>>>>> This really is a bug that requires fixing in usocket. There are
>>>>>>>>>> multiple issues:
>>>>>>>>>>
>>>>>>>>>> - As shown in the backtrace, the problem is that the getpeername()
>>>>>>>>>> call fails for a socket that is not connected anymore. SBCL does the
>>>>>>>>>> right thing by signalling a condition. Other implementations (I
>>>>>>>>>> checked CCL) seem to behave differently, i.e. return NIL in such a
>>>>>>>>>> case.
>>>>>>>>>>
>>>>>>>>>> - Usocket does not unify the behavior, so the implementation specific
>>>>>>>>>> condition percolates to the caller, Hunchentoot in this case.
>>>>>>>>>>
>>>>>>>>>> - Hunchentoot is not prepared to handle conditions when creating a new
>>>>>>>>>> worker thread. The call to CLIENT-AS-STRING is made when creating the
>>>>>>>>>> name for the handler process of a new incoming connection, and that is
>>>>>>>>>> done in the context of the server thread. Therefore, the signalled
>>>>>>>>>> condition will stop the acceptor process.
>>>>>>>>>>
>>>>>>>>>> I spent some thoughts on how a "portable solution" would look like,
>>>>>>>>>> but given that the Lisp implementations vary greatly in behavior,
>>>>>>>>>> fixing usocket would be a pretty large task that I don't have the time
>>>>>>>>>> to do right now. Thus, I would propose this patch:
>>>>>>>>>>
>>>>>>>>>> http://bknr.net/trac/changeset/4428?format=diff&new=4428
>>>>>>>>>>
>>>>>>>>>> It handles all conditions that are signaled during worker process
>>>>>>>>>> creation, under the theory that we want to prevent the acceptor
>>>>>>>>>> process from crashing under all circumstances. This is not a clean
>>>>>>>>>> solution in that it may paper over bugs, but given the limited number
>>>>>>>>>> of function invocations that are surrounded by a HANDLER-CASE, I'd say
>>>>>>>>>> that this is a proper intermediate fix.
>>>>>>>>>>
>>>>>>>>>> Please give it a try and let me know if it solves the problem for you.
>>>>>>> I tried your suggested patch but I can't seem to be able to get it to
>>>>>>> catch the exception for me. Were you able to duplicate the error
>>>>>>> yourself to verify that the suggested patch does indeed work? Since I am
>>>>>>> using the 1.0.0 hunchentoot release I couldn't just apply your patch but
>>>>>>> added the handler-case by hand and thus ended up with this in
>>>>>>> taskmaster.lisp:
>>>>>>>
>>>>>>> (defmethod handle-incoming-connection ((taskmaster
>>>>>>> one-thread-per-connection-taskmaster) handle)
>>>>>>> (incf *worker-counter*)
>>>>>>> ;; check if we need to perform a global GC
>>>>>>> (when (and *cleanup-interval*
>>>>>>> (zerop (mod *worker-counter* *cleanup-interval*)))
>>>>>>> (when *cleanup-function*
>>>>>>> (funcall *cleanup-function*)))
>>>>>>> (handler-case
>>>>>>> (mp:process-run-function (format nil "Hunchentoot worker \(client:
>>>>>>> ~{~A:~A~})"
>>>>>>> (multiple-value-list
>>>>>>> (get-peer-address-and-port handle)))
>>>>>>> nil #'process-connection
>>>>>>> (taskmaster-acceptor taskmaster) handle)
>>>>>>> (error (cond)
>>>>>>> (log-message *lisp-errors-log-level*
>>>>>>> "Error while creating worker thread for new incoming
>>>>>>> connection: ~A" cond))))
>>>>>>>
>>>>>>> /Peter
>>>>>>>
>>>>>>>
>>>>>>>>>> -Hans
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> tbnl-devel site list
>>>>>>>>>> tbnl-devel at common-lisp.net
>>>>>>>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>> _______________________________________________
>>>> tbnl-devel site list
>>>> tbnl-devel at common-lisp.net
>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>>>>> _______________________________________________
>>>>>>> tbnl-devel site list
>>>>>>> tbnl-devel at common-lisp.net
>>>>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>> _______________________________________________
>>>> tbnl-devel site list
>>>> tbnl-devel at common-lisp.net
>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>
_______________________________________________
tbnl-devel site list
tbnl-devel at common-lisp.net
http://common-lisp.net/mailman/listinfo/tbnl-devel
>>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
- --
Med vänlig hälsning,
Peter Stiernström
0708-810932
Blixtvik AB
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkpR5dIACgkQ0brSZD05ZzDvbwCgmnIIeWo0dNZZs/tDzUO6cCqD
UQAAoKedq3OpsV9ru6BK0DINB5SCBpya
=PEA/
-----END PGP SIGNATURE-----
More information about the Tbnl-devel
mailing list