From achambers at mcna.net Sat Nov 13 04:07:17 2010 From: achambers at mcna.net (achambers at mcna.net) Date: Fri, 12 Nov 2010 23:07:17 -0500 (EST) Subject: [postmodern-devel] Parameterized sql strings Message-ID: <1289621237.443323012@192.168.2.228> Hi, Am I correct in thinking that there is currently no way to parameterize raw SQL strings to be sent through postmodern? I realize most SQL can be expressed as S-SQL but it doesn't seem to support COPY syntax yet. I added the function `sql-fmt' for this purpose and am submitting it to the list in case it's useful for others.... (defun sql-fmt (fmt &rest args) (apply 'format nil fmt (mapcar 'sql-ize args))) In my repo [1], I added this to the :export list of both s-sql and postmodern packages. Please pull the change or let me know if you need this changed in any way. [1]: https://github.com/cddr/Postmodern/commits/master Cheers, Andy From marijnh at gmail.com Sat Nov 13 07:28:29 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 13 Nov 2010 08:28:29 +0100 Subject: [postmodern-devel] Parameterized sql strings In-Reply-To: <1289621237.443323012@192.168.2.228> References: <1289621237.443323012@192.168.2.228> Message-ID: Hi Andy, I also just replied on github -- basically, you can just *use* raw strings wherever Postmodern expects an s-sql expression (the two can be distinguished at compile time) and it will work. Best, Marijn From slobodan.milnovic at gmail.com Tue Nov 16 00:54:30 2010 From: slobodan.milnovic at gmail.com (=?UTF-8?Q?Slobodan_Milnovi=C4=87?=) Date: Tue, 16 Nov 2010 01:54:30 +0100 Subject: [postmodern-devel] Problems with doquery and simple-date Message-ID: Hi, There is probably some simple reason why these won't work, but I'm not as good in lisp to understand it. This works: CL-USER> (simple-date:decode-timestamp (caar (postmodern:query (:limit (:select (:dot 'sem 'avl 'timestamp) :from (:dot 'sem 'avl)) 1)))) 2010 1 12 8 35 50 0 This doesn't: CL-USER> (postmodern:doquery (:limit (:select (:dot 'sem 'avl 'timestamp) :from (:dot 'sem 'avl)) 1) (ivo) (simple-date:decode-timestamp ivo)) NIL I don't understand the difference, especially since for other stuff doquery works. Please advise. From marijnh at gmail.com Tue Nov 16 07:29:57 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 16 Nov 2010 08:29:57 +0100 Subject: [postmodern-devel] Problems with doquery and simple-date In-Reply-To: References: Message-ID: Let's take Postmodern out of the picture for a second... CL-USER> (decode-universal-time (get-universal-time)) 19 28 8 16 11 2010 1 NIL -1 CL-USER> (dotimes (i 1) (decode-universal-time (get-universal-time))) NIL See the pattern? A loop doesn't return the value of its body, typically, since that body will be executed multiple times. From slobodan.milnovic at gmail.com Tue Nov 16 07:39:26 2010 From: slobodan.milnovic at gmail.com (=?UTF-8?Q?Slobodan_Milnovi=C4=87?=) Date: Tue, 16 Nov 2010 08:39:26 +0100 Subject: [postmodern-devel] Problems with doquery and simple-date In-Reply-To: References: Message-ID: On Tue, Nov 16, 2010 at 08:29, Marijn Haverbeke wrote: > See the pattern? > > A loop doesn't return the value of its body, typically, since that > body will be executed multiple times. Yes, thank you, I do understand now. It did the work, but because of loop it didn't return what I have wrongly assumed it should. From sabra.crolleton at gmail.com Wed Nov 17 04:05:43 2010 From: sabra.crolleton at gmail.com (Sabra Crolleton) Date: Tue, 16 Nov 2010 20:05:43 -0800 Subject: [postmodern-devel] Parameterized sql strings In-Reply-To: <1289621237.443323012@192.168.2.228> References: <1289621237.443323012@192.168.2.228> Message-ID: Andy, Am I correct in thinking that you are pointing out that while (query "select name from countries where id=$1" 27) (("Singapore")) works (query "copy $1 to$2" "countries" "/home/sabra/test/test.dat") does not work? Sabra On Fri, Nov 12, 2010 at 8:07 PM, wrote: > Hi, > > Am I correct in thinking that there is currently no way to parameterize raw > SQL strings to be sent through postmodern? I realize most SQL can be > expressed as S-SQL but it doesn't seem to support COPY syntax yet. I added > the function `sql-fmt' for this purpose and am submitting it to the list in > case it's useful for others.... > > (defun sql-fmt (fmt &rest args) > (apply 'format nil fmt (mapcar 'sql-ize args))) > > In my repo [1], I added this to the :export list of both s-sql and > postmodern packages. Please pull the change or let me know if you need this > changed in any way. > > [1]: https://github.com/cddr/Postmodern/commits/master > > Cheers, > Andy > > > > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From achambers at mcna.net Wed Nov 17 04:20:16 2010 From: achambers at mcna.net (achambers at mcna.net) Date: Tue, 16 Nov 2010 23:20:16 -0500 (EST) Subject: [postmodern-devel] Parameterized sql strings In-Reply-To: References: <1289621237.443323012@192.168.2.228> Message-ID: <1289967616.49181113@192.168.2.229> Yes. That's what I'm saying. Marijnh provided a work-around with the following commit... https://github.com/marijnh/Postmodern/commit/d1f46a1c2b80a80861bafad7498fd63539cc7bb5 -- Andy From slobodan.milnovic at gmail.com Mon Nov 22 18:05:51 2010 From: slobodan.milnovic at gmail.com (=?UTF-8?Q?Slobodan_Milnovi=C4=87?=) Date: Mon, 22 Nov 2010 19:05:51 +0100 Subject: [postmodern-devel] Multiple connections and connection pools Message-ID: Hi, I have a bit of a problem understanding how multiple connections and connection pools work. What I would like to do is to open an connection to the database, read several rows from one table with doquery, and for each read row analyze data, and if conditions are met, write a row to another table. I'd rather not use memory to store the results from the first query because I am a bit memory constrained. I have tried the following (just the simplification of the actual code), for which I get the message "Database error: This connection is still processing another query." (defvar *db-parameters* (list "databasename" "databaseuser" "databasepassword" "dbhostaddress" :pooled-p t)) (defmacro with-db (&body body) `(postmodern:with-connection *db-parameters* , at body)) (defun process-some-stuff () (with-db (postmodern:doquery (read some stuff from a table) (several variables) (analyze row) (if some conditions (with-db (postmodern:query (insert stuff into another table))))))) Also, I have tried using two postmodern:connection and then after each with-db, but the error is the same. I understand that when I use ":pooled-p t" postmodern will use some kind of pool, but I have'nt been able to understand how to use it properly, nor did I manage to find any kind of solution on the internet. The test.lisp in postmodern source directory did test the pool, but each time the connection was imediatelly closed after openning it, and there were no examples like in my situation. Perhaps the nesting of the querries is not allowed in postmodern? From marijnh at gmail.com Tue Nov 23 12:13:22 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 23 Nov 2010 13:13:22 +0100 Subject: [postmodern-devel] Multiple connections and connection pools In-Reply-To: References: Message-ID: Hi Slobodan, Firstly, your snippet works for me (if I substitute meaningful stuff for your placeholders), and should work -- the inner with-db should open (or fetch from the pool) a fresh connection, that can be used even while the outer connection is processing a query (doquery will run its body while the database is returning the result rows). I tested against PostgreSQL 9.0. So, since you don't really describe what kind of error you are getting, you'll have to be more specific (maybe give me a full example, including tables and some test data) for me to be able to debug this. Best, Marijn From slobodan.milnovic at gmail.com Wed Nov 24 22:26:17 2010 From: slobodan.milnovic at gmail.com (=?UTF-8?Q?Slobodan_Milnovi=C4=87?=) Date: Wed, 24 Nov 2010 23:26:17 +0100 Subject: [postmodern-devel] Multiple connections and connection pools In-Reply-To: References: Message-ID: Dear Mr. Haverbeke, It was an stupid mistake on my side, I had another query in the code I have been playing with, and that one wasn't with the "with-connection" and ":pooled-p". I appologize for wasting your time, I will try my best not to do so again. On Tue, Nov 23, 2010 at 13:13, Marijn Haverbeke wrote: > Hi Slobodan, > > Firstly, your snippet works for me (if I substitute meaningful stuff > for your placeholders), and should work -- the inner with-db should > open (or fetch from the pool) a fresh connection, that can be used > even while the outer connection is processing a query (doquery will > run its body while the database is returning the result rows). I > tested against PostgreSQL 9.0. > > So, since you don't really describe what kind of error you are > getting, you'll have to be more specific (maybe give me a full > example, including tables and some test data) for me to be able to > debug this. > > Best, > Marijn > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > From marijnh at gmail.com Thu Nov 25 07:05:52 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Thu, 25 Nov 2010 08:05:52 +0100 Subject: [postmodern-devel] Multiple connections and connection pools In-Reply-To: References: Message-ID: Good to hear you've solved the issue. Don't worry about the occasional false alarm -- it makes the mailing list look alive ;) From sabra.crolleton at gmail.com Sat Nov 27 23:09:51 2010 From: sabra.crolleton at gmail.com (Sabra Crolleton) Date: Sat, 27 Nov 2010 15:09:51 -0800 Subject: [postmodern-devel] Question on returning multi-dimensional arrays Message-ID: Hello all, I'm having trouble trying to get a array result out of a database. Assume (query "create table test2 ( id integer, name text, stuff1 integer[] , stuff2 integer[][])") (query "insert into test2 values (5,'t5',array[21,78,93,24],array[[83,92],[16,27]])") If I'm at a psql command line, then select stuff2[1:2][1] from test2 where id=5; returns {{83},{16}} If I try to use postmodern, eg: (query "select stuff2[1:2][1] from test2 where id=5") I get an error message about junk in the result: junk in string "{83" [Condition of type SB-INT:SIMPLE-PARSE-ERROR] I'm likely missing something obvious. Everything is fine if I'm only trying to get a single item out of the array, but ideally I'd like to just pull the entire array out directly into a variable. Any suggestions or pointers would be appreciated. Sabra -------------- next part -------------- An HTML attachment was scrubbed... URL: From marijnh at gmail.com Sun Nov 28 13:19:16 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sun, 28 Nov 2010 14:19:16 +0100 Subject: [postmodern-devel] Question on returning multi-dimensional arrays In-Reply-To: References: Message-ID: Hi Sabra, I hadn't taken multi-dimensional arrays into account when writing the array parsing/serializing code. I've pushed an update to the git repo that should resolve your issue. Best, Marijn From redline6561 at gmail.com Mon Nov 29 22:50:57 2010 From: redline6561 at gmail.com (Brit Butler) Date: Mon, 29 Nov 2010 17:50:57 -0500 Subject: [postmodern-devel] Quoted lists with QUERY or SQL Message-ID: Hello, I'm working on a Postmodern backend for weblocks. To conform to their Store API I need to implement a count-persistent-objects method. I would like to be able to write something like: (defmethod count-persistent-objects ((store list) class-name &key where &allow-other-keys) (let ((sql-expr `(:select (:count '*) :from ,class-name ,@(when where (list :where where))))) (with-connection store (query sql-expr :single)))) However a simple test of this results in a type error: The value of CL-POSTGRES::QUERY is (:SELECT (:COUNT '*) :FROM QUOTES), which is not of type STRING. Is there a reason that query doesn't support evaluated lists? If so, how would you recommend this be done? Regards, Brit Butler -------------- next part -------------- An HTML attachment was scrubbed... URL: From marijnh at gmail.com Tue Nov 30 08:36:56 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 30 Nov 2010 09:36:56 +0100 Subject: [postmodern-devel] Quoted lists with QUERY or SQL In-Reply-To: References: Message-ID: > Is there a reason that query doesn't support evaluated lists? Yes. These macros also accept string input, and assume that everything that's not a list starting with a keyword will evaluate to a string. > If so, how would you recommend this be done? You can either add an sql-compile around your sql-expr, which does a run-time compilation of an s-sql expression (normally they happen at compile time), or do something like this: (defmethod count-persistent-objects ((store list) class-name &key (where t) &allow-other-keys) (with-connection store (sql (:select (:count '*) :from class-name :where (:raw (if (stringp where) where (sql-compile where))))))) Best, Marijn From redline6561 at gmail.com Tue Nov 30 14:51:20 2010 From: redline6561 at gmail.com (Brit Butler) Date: Tue, 30 Nov 2010 09:51:20 -0500 Subject: [postmodern-devel] Quoted lists with QUERY or SQL In-Reply-To: References: Message-ID: sql-compile is exactly what I was looking for. Thanks Marijn. And thank you for Postmodern! Regards, Brit On Tue, Nov 30, 2010 at 3:36 AM, Marijn Haverbeke wrote: > > Is there a reason that query doesn't support evaluated lists? > > Yes. These macros also accept string input, and assume that everything > that's not a list starting with a keyword will evaluate to a string. > > > If so, how would you recommend this be done? > > You can either add an sql-compile around your sql-expr, which does a > run-time compilation of an s-sql expression (normally they happen at > compile time), or do something like this: > > (defmethod count-persistent-objects ((store list) class-name > &key (where t) &allow-other-keys) > (with-connection store > (sql (:select (:count '*) :from class-name :where > (:raw (if (stringp where) where (sql-compile where))))))) > > Best, > Marijn > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: