[hunchentoot-devel] Hunchentoot effective DOS-attack
Hans Hübner
hans.huebner at gmail.com
Mon Jul 6 11:59:29 UTC 2009
Oh well, *ACCEPTOR* was not inherited by the new process, so use this:
http://bknr.net/trac/changeset/4435?format=diff&new=4435
-Hans
On Mon, Jul 6, 2009 at 13:53, Peter
Stiernström<peter.stiernstrom at blixtvik.se> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hans Hübner wrote:
>
>> Peter,
>>
>> try this:
>>
>> http://bknr.net/trac/changeset/4434?format=diff&new=4434
>
> Hello Hans,
>
> I still get an unbound-variable condition for hunchentoot:*acceptor* but
> the trace look a bit different:
>
> The variable HUNCHENTOOT:*ACCEPTOR* is unbound.
> BREAK was entered because of *BREAK-ON-SIGNALS* (now rebound to NIL).
> [Condition of type SIMPLE-CONDITION]
>
> Restarts:
> 0: [CONTINUE] Return from BREAK.
> 1: [REASSIGN] Return from BREAK and assign a new value to
> *BREAK-ON-SIGNALS*.
> 2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "Hunchentoot
> worker (client: 127.0.0.1:46427)" RUNNING {AF2EA11}>)
>
> Backtrace:
> 0: (BREAK "~A~%BREAK was entered because of *BREAK-ON-SIGNALS* ~\n
> (now rebound to NIL)." #<UNBOUND-VARIABLE *ACCEPTOR*
> {AF30361}>)
> 1: (SIGNAL #<UNBOUND-VARIABLE *ACCEPTOR* {AF30361}>)[:EXTERNAL]
> 2: (ERROR UNBOUND-VARIABLE)[:EXTERNAL]
> 3: (SB-KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER #<unavailable argument>
> #.(SB-SYS:INT-SAP #XB5724028) #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
> #XB5723CDC :TYPE (* (STRUCT SB-VM::OS-CONTEXT-T-STRUCT))> (14))
> 4: (SB-KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER #<unavailable argument>
> #.(SB-SYS:INT-SAP #XB5724028) #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
> #XB5723CDC :TYPE (* (STRUCT SB-VM::OS-CONTEXT-T-STRUCT))> (14))[:EX..
> 5: (SB-KERNEL:INTERNAL-ERROR #.(SB-SYS:INT-SAP #XB5723CDC)
> #<unavailable argument>)
> 6: ("foreign function: #x806480C")
> 7: ("foreign function: #x8052BCC")
> 8: ("foreign function: #x8056BD3")
> 9: ((LAMBDA ()))
> 10: ((FLET SB-THREAD::WITH-MUTEX-THUNK))
> 11: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]477))
> 12: (SB-THREAD::CALL-WITH-MUTEX ..)
> 13: ((LAMBDA ()))
> 14: ("foreign function: #x806480C")
> 15: ("foreign function: #x8052C21")
> 16: ("foreign function: #x805BD9D")
> 17: ("foreign function: #xB7FB84C0")
>
>
> I did set *break-on-signals* to 'unbound-variable and my modified
> function now looks like this:
>
> (defmethod handle-incoming-connection ((taskmaster
> one-thread-per-connection-taskmaster) socket)
> (let ((*acceptor* (taskmaster-acceptor taskmaster)))
> (handler-case
> (bt:make-thread (lambda ()
> (process-connection *acceptor* socket))
> :name (format nil "Hunchentoot worker \(client:
> ~A)" (client-as-string socket)))
> (error (e)
> (log-message *lisp-errors-log-level*
> "Error while creating worker thread for new
> incoming connection: ~A" e)))))
>
> /Peter
>
>>
>> -Hans
>>
>> On Mon, Jul 6, 2009 at 13:21, Peter
>> Stiernström<peter.stiernstrom at blixtvik.se> wrote:
>> Hello Hans,
>>
>> Hans Hübner wrote:
>>>>> Peter, as I cannot reproduce the problem myself, can you please post a
>>>>> stack backtrace? Thanks!
>> The variable HUNCHENTOOT:*ACCEPTOR* is unbound.
>> BREAK was entered because of *BREAK-ON-SIGNALS* (now rebound to NIL).
>> [Condition of type SIMPLE-CONDITION]
>>
>> Restarts:
>> 0: [CONTINUE] Return from BREAK.
>> 1: [REASSIGN] Return from BREAK and assign a new value to
>> *BREAK-ON-SIGNALS*.
>> 2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "Hunchentoot
>> listener (*:8000)" RUNNING {C42AAE9}>)
>>
>> Backtrace:
>> 0: (BREAK "~A~%BREAK was entered because of *BREAK-ON-SIGNALS* ~\n
>> (now rebound to NIL)." #<UNBOUND-VARIABLE *ACCEPTOR*
>> {AD37FC1}>)
>> 1: (SIGNAL #<UNBOUND-VARIABLE *ACCEPTOR* {AD37FC1}>)[:EXTERNAL]
>> 2: (ERROR UNBOUND-VARIABLE)[:EXTERNAL]
>> 3: (SB-KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER #<unavailable argument>
>> #.(SB-SYS:INT-SAP #XB5DFFEF0) #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
>> #XB5DFFBDC :TYPE (* (STRUCT SB-VM::OS-CONTEXT-T-STRUCT))> (14))
>> 4: (SB-KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER #<unavailable argument>
>> #.(SB-SYS:INT-SAP #XB5DFFEF0) #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
>> #XB5DFFBDC :TYPE (* (STRUCT SB-VM::OS-CONTEXT-T-STRUCT))> (14))[:EX..
>> 5: (SB-KERNEL:INTERNAL-ERROR #.(SB-SYS:INT-SAP #XB5DFFBDC)
>> #<unavailable argument>)
>> 6: ("foreign function: #x806480C")
>> 7: ("foreign function: #x8052BCC")
>> 8: ("foreign function: #x8056BD3")
>> 9: (HUNCHENTOOT:LOG-MESSAGE :ERROR "Error while creating worker thread
>> for new incoming connection: ~A")[:EXTERNAL]
>> 10: ((SB-PCL::FAST-METHOD HUNCHENTOOT:HANDLE-INCOMING-CONNECTION
>> (HUNCHENTOOT:ONE-THREAD-PER-CONNECTION-TASKMASTER T)) ..)
>> 11: ((SB-PCL::FAST-METHOD HUNCHENTOOT:ACCEPT-CONNECTIONS
>> (HUNCHENTOOT:ACCEPTOR)) #<unavailable argument> #<unavailable argument>
>> #<HUNCHENTOOT:ACCEPTOR (host *, port 8000)>)
>> 12: ((FLET SB-THREAD::WITH-MUTEX-THUNK))
>> 13: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]477))
>> 14: (SB-THREAD::CALL-WITH-MUTEX ..)
>> 15: ((LAMBDA ()))
>> 16: ("foreign function: #x806480C")
>> 17: ("foreign function: #x8052C21")
>> 18: ("foreign function: #x805BD9D")
>> 19: ("foreign function: #xB7FB84C0")
>>
>> I found that not attempting to log a message solved this remaining problem.
>>
>> Thank you very much for your help!
>>
>> /Peter
>>
>>>>> 2009/7/6 Peter Stiernström <peter.stiernstrom at blixtvik.se>:
>>>>> Hans,
>>>>>
>>>>> I should have realised I was editing the lispworks implementation myself
>>>>> had I only had a cup of coffee this morning :-P
>>>>>
>>>>> Now when I tried your patch out I still get hangups but this is another
>>>>> kind of hangup:
>>>>>
>>>>> debugger invoked on a UNBOUND-VARIABLE in thread #<THREAD "Hunchentoot
>>>>> listener (*:8000)" RUNNING {AD467D1}>:
>>>>> The variable HUNCHENTOOT:*ACCEPTOR* is unbound.
>>>>>
>>>>> I am trying this out with the hunchentoot default page by the way.
>>>>>
>>>>> /Peter
>>>>>
>>>>> Hans Hübner wrote:
>>>>>>>> Peter,
>>>>>>>>
>>>>>>>> sorry, I am not able to reproduce the problem myself, so I made the
>>>>>>>> patch blindly and made a mistake, working on the Lispworks code rather
>>>>>>>> than the portable code that affects you. Please look at the first
>>>>>>>> hunk of this diff:
>>>>>>>>
>>>>>>>> http://bknr.net/trac/changeset/4430?format=diff&new=4430
>>>>>>>>
>>>>>>>> and let me know if that does it for you.
>>>>>>>>
>>>>>>>> Sorry,
>>>>>>>> Hans
>>>>>>>>
>>>>>>>> On Mon, Jul 6, 2009 at 11:09, Peter
>>>>>>>> Stiernström<peter.stiernstrom at blixtvik.se> wrote:
>>>>>>>> Hi Hans,
>>>>>>>>
>>>>>>>> Hans Hübner wrote:
>>>>>>>>>>> Hi Peter,
>>>>>>>>>>>
>>>>>>>>>>> 2009/7/6 Peter Stiernström <peter.stiernstrom at blixtvik.se>
>>>>>>>>>>>> I had a quick peek at client-as-string and it does indeed not handle the
>>>>>>>>>>>> sb-bsd-sockets:not-connected-error and if I put a handler-case around
>>>>>>>>>>>> body of client-as-string returning nil when the exception appears I can
>>>>>>>>>>>> not reproduce my problem anymore.
>>>>>>>>>>>>
>>>>>>>>>>>> This wouldn't provide a portable solution though :-P
>>>>>>>>>>> This really is a bug that requires fixing in usocket. There are
>>>>>>>>>>> multiple issues:
>>>>>>>>>>>
>>>>>>>>>>> - As shown in the backtrace, the problem is that the getpeername()
>>>>>>>>>>> call fails for a socket that is not connected anymore. SBCL does the
>>>>>>>>>>> right thing by signalling a condition. Other implementations (I
>>>>>>>>>>> checked CCL) seem to behave differently, i.e. return NIL in such a
>>>>>>>>>>> case.
>>>>>>>>>>>
>>>>>>>>>>> - Usocket does not unify the behavior, so the implementation specific
>>>>>>>>>>> condition percolates to the caller, Hunchentoot in this case.
>>>>>>>>>>>
>>>>>>>>>>> - Hunchentoot is not prepared to handle conditions when creating a new
>>>>>>>>>>> worker thread. The call to CLIENT-AS-STRING is made when creating the
>>>>>>>>>>> name for the handler process of a new incoming connection, and that is
>>>>>>>>>>> done in the context of the server thread. Therefore, the signalled
>>>>>>>>>>> condition will stop the acceptor process.
>>>>>>>>>>>
>>>>>>>>>>> I spent some thoughts on how a "portable solution" would look like,
>>>>>>>>>>> but given that the Lisp implementations vary greatly in behavior,
>>>>>>>>>>> fixing usocket would be a pretty large task that I don't have the time
>>>>>>>>>>> to do right now. Thus, I would propose this patch:
>>>>>>>>>>>
>>>>>>>>>>> http://bknr.net/trac/changeset/4428?format=diff&new=4428
>>>>>>>>>>>
>>>>>>>>>>> It handles all conditions that are signaled during worker process
>>>>>>>>>>> creation, under the theory that we want to prevent the acceptor
>>>>>>>>>>> process from crashing under all circumstances. This is not a clean
>>>>>>>>>>> solution in that it may paper over bugs, but given the limited number
>>>>>>>>>>> of function invocations that are surrounded by a HANDLER-CASE, I'd say
>>>>>>>>>>> that this is a proper intermediate fix.
>>>>>>>>>>>
>>>>>>>>>>> Please give it a try and let me know if it solves the problem for you.
>>>>>>>> I tried your suggested patch but I can't seem to be able to get it to
>>>>>>>> catch the exception for me. Were you able to duplicate the error
>>>>>>>> yourself to verify that the suggested patch does indeed work? Since I am
>>>>>>>> using the 1.0.0 hunchentoot release I couldn't just apply your patch but
>>>>>>>> added the handler-case by hand and thus ended up with this in
>>>>>>>> taskmaster.lisp:
>>>>>>>>
>>>>>>>> (defmethod handle-incoming-connection ((taskmaster
>>>>>>>> one-thread-per-connection-taskmaster) handle)
>>>>>>>> (incf *worker-counter*)
>>>>>>>> ;; check if we need to perform a global GC
>>>>>>>> (when (and *cleanup-interval*
>>>>>>>> (zerop (mod *worker-counter* *cleanup-interval*)))
>>>>>>>> (when *cleanup-function*
>>>>>>>> (funcall *cleanup-function*)))
>>>>>>>> (handler-case
>>>>>>>> (mp:process-run-function (format nil "Hunchentoot worker \(client:
>>>>>>>> ~{~A:~A~})"
>>>>>>>> (multiple-value-list
>>>>>>>> (get-peer-address-and-port handle)))
>>>>>>>> nil #'process-connection
>>>>>>>> (taskmaster-acceptor taskmaster) handle)
>>>>>>>> (error (cond)
>>>>>>>> (log-message *lisp-errors-log-level*
>>>>>>>> "Error while creating worker thread for new incoming
>>>>>>>> connection: ~A" cond))))
>>>>>>>>
>>>>>>>> /Peter
>>>>>>>>
>>>>>>>>
>>>>>>>>>>> -Hans
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> tbnl-devel site list
>>>>>>>>>>> tbnl-devel at common-lisp.net
>>>>>>>>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>>> _______________________________________________
>>>>> tbnl-devel site list
>>>>> tbnl-devel at common-lisp.net
>>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>>>>>> _______________________________________________
>>>>>>>> tbnl-devel site list
>>>>>>>> tbnl-devel at common-lisp.net
>>>>>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>> _______________________________________________
>> tbnl-devel site list
>> tbnl-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>>> _______________________________________________
>>>>> tbnl-devel site list
>>>>> tbnl-devel at common-lisp.net
>>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>
>
>> _______________________________________________
>> tbnl-devel site list
>> tbnl-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>
> - --
> Med vänlig hälsning,
>
> Peter Stiernström
> 0708-810932
> Blixtvik AB
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAkpR5dIACgkQ0brSZD05ZzDvbwCgmnIIeWo0dNZZs/tDzUO6cCqD
> UQAAoKedq3OpsV9ru6BK0DINB5SCBpya
> =PEA/
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>
More information about the Tbnl-devel
mailing list