[drakma-devel] Bug report: Overeager encoding of parameters

[Eric Benson]

RFC 2396 has been superseded by RFC 3986, although that does not affect this case.

The freedb server is using plus signs in HTTP requests to substitute for spaces, which are used in CDDBP requests. That appears to be supported by this recommendation from W3:

http://www.w3.org/Addressing/URL/4_URI_Recommentations.html

"Within the query string,
the plus sign is reserved as shorthand notation for a space.
Therefore, real plus signs must be encoded.
This method was used to make query URIs easier to pass in systems which did
not allow spaces."

For the record, neither IE nor Firefox encodes plus signs when they are entered as part of the query. However, they probably should! Spaces are encoded as plus signs. If I left the spaces in the CDDB query alone when passing them to http-request, they would be encoded as plus signs.

It seems to me that http-request should make encoding of the parameters
optional, since a client program may need to deal with encoded as well
as unencoded strings. Otherwise, a caller of http-request that already
has encoded parameters must explicitly decode them before calling. A
keyword argument that specifies whether parameters are already encoded,
defaulting to nil, would be a useful addition. Most HTTP client APIs separate the encoding of URLs and query strings from the network request, leaving it up to the caller to handle the encoding explicitly.


* (drakma:http-request "http://freedb.org/~cddb/cddb.cgi" :parameters '(("cmd" . "cddb lscat") ("hello" . "user host freedb 1.0") ("proto" . "6")))
GET /~cddb/cddb.cgi?cmd=cddb+lscat&hello=user+host+freedb+1.0&proto=6 HTTP/1.1
Host: freedb.org
User-Agent: Drakma/0.11.5 (SBCL 1.0.11.debian; Linux; 2.6.24-19-generic; http://weitz.de/drakma/)
Accept: */*
Connection: close

HTTP/1.1 200 OK
Date: Mon, 07 Jul 2008 16:02:17 GMT
Server: Apache/2.0.54 (Debian GNU/Linux)
Expires: Mon Jul  7 18:02:17 2008
Connection: close
Transfer-Encoding: chunked
Content-Type: text/plain; charset=UTF-8

"210 OK, category list follows (until terminating `.') 
data 
folk 
jazz 
misc 
rock 
country 
blues 
newage 
reggae 
classical 
soundtrack 
. 
"
200
((:DATE . "Mon, 07 Jul 2008 16:02:17 GMT")
 (:SERVER . "Apache/2.0.54 (Debian GNU/Linux)")
 (:EXPIRES . "Mon Jul  7 18:02:17 2008") (:CONNECTION . "close")
 (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/plain; charset=UTF-8"))
#<PURI:URI http://freedb.org/~cddb/cddb.cgi?cmd=cddb+lscat&hello=user+host+freedb+1.0&proto=6>
#<FLEXI-STREAMS:FLEXI-IO-STREAM {BAE7859}>
T
"OK"
* 





----- Original Message ----
From: Edi Weitz <edi at agharta.de>
To: General interest list for Drakma and Chunga <drakma-devel at common-lisp.net>
Sent: Monday, July 7, 2008 7:41:47 AM
Subject: Re: [drakma-devel] Bug report: Overeager encoding of parameters

On Mon, 7 Jul 2008 07:28:13 -0700 (PDT), Eric Benson <eric_a_benson at yahoo.com> wrote:

> It appears that drakma is encoding plus signs (+) in query
> parameters, turning them into %2B when they should be left alone.

According to 3.4 of RFC 2396 the character "+" is reserved in query
components.  Encoding plus signs in GET requests is also something
Firefox and Internet Explorer do.

> This confuses the server in this case.

Maybe the server is confused anyway?

Edi.
_______________________________________________
drakma-devel mailing list
drakma-devel at common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel