From marijnh at gmail.com Mon Mar 1 13:07:48 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 1 Mar 2010 14:07:48 +0100 Subject: [postmodern-devel] Release 1.15 Message-ID: I just bundled the changes (all relatively minor) made in the past year as release 1.15, so that those not pulling directly from darcs can also see them. Cheers, Marijn From dj at danieljanus.pl Wed Mar 3 01:53:11 2010 From: dj at danieljanus.pl (Daniel Janus) Date: Wed, 03 Mar 2010 02:53:11 +0100 Subject: [postmodern-devel] one-way array/record/coercion support Message-ID: <1267581191.8117.9.camel@news.danieljanus.pl> Dear list, first of all, many thanks to Marijn and all the contributors for the excellent piece of code that is Postmodern. In my project I'm using it to connect to a database that heavily uses some of the advanced PostgreSQL functionality, including PL/pgSQL functions taking arrays of records as arguments. I found S-SQL to be a little lacking in expressiveness for that, so I've quickly hacked some methods that let me pass Lisp vectors to SQL and QUERY (they will be rendered into PostgreSQL array literals): ;;; code starts here (in-package :cl-postgres) (defmethod to-sql-string ((arg vector)) (cond ((typep arg '(vector (unsigned-byte 8))) (values (escape-bytes arg) t)) (t (format nil "ARRAY[~{~A~^, ~}]" (mapcar #'s-sql::sql-ize (coerce arg 'list)))))) (defmethod to-sql-string ((arg list)) (format nil "(~{~A~^, ~})" (mapcar #'s-sql::sql-ize arg))) (in-package :s-sql) (def-sql-op :coerce (form type) `(,@(sql-expand form) "::" ,type)) ;;; code ends here I'm sending it along in case it is useful to anyone else. Or perhaps it might make its way to the main repo? For the latter, presumably a full support would be needed, but I didn't find time to dive into Postmodern's way of passing back PostgreSQL-returned data into Lisp values of proper types, as I currently don't need that. Best regards, Daniel Janus From attila.lendvai at gmail.com Wed Mar 3 08:10:12 2010 From: attila.lendvai at gmail.com (Attila Lendvai) Date: Wed, 3 Mar 2010 09:10:12 +0100 Subject: [postmodern-devel] one-way array/record/coercion support In-Reply-To: <1267581191.8117.9.camel@news.danieljanus.pl> References: <1267581191.8117.9.camel@news.danieljanus.pl> Message-ID: > In my project I'm using it to connect to a database that heavily uses > some of the advanced PostgreSQL functionality, including PL/pgSQL functions > taking arrays of records as arguments. ?I found S-SQL to be a little > lacking in expressiveness for that, so I've quickly hacked some methods > that let me pass Lisp vectors to SQL and QUERY (they will be rendered > into PostgreSQL array literals): on a somewhat related note, i'm working on extending/reworking the binding mechanism so that users can add methods for lisp types that serialize lisp values directly into the socket stream. for that i've added a serialize-for-postgres generic (that by default calls to-sql-string), but i need to spend more time to think through how this could be integrated better (e.g. i'd like to support serializers without an intermediate byte vector buffer). but i'm not happy with the current code yet... although inserting local-time:timestamp's without turning them into strings works already. -- attila From marijnh at gmail.com Sat Mar 6 14:40:54 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 6 Mar 2010 15:40:54 +0100 Subject: [postmodern-devel] one-way array/record/coercion support In-Reply-To: <1267581191.8117.9.camel@news.danieljanus.pl> References: <1267581191.8117.9.camel@news.danieljanus.pl> Message-ID: Hi Daniel, Thanks for the code. However, :coerce is already there, named :type, and the way to-sql-string is currently used, it should A) not use S-SQL functions, since cl-postgres does not depend on S-SQL, and B) serialise to a format that can also be used to pass parameters to prepared queries. I'm not that knowledgeable about Postgres' textual formats, but it appears that a value like ARRAY[1, 2, 3] can not be written as a parameter: Database error 22P02: array value must start with "{" or dimension information If you feel like getting to the bottom of this, feel free to submit an alternative patch. If not, array serialisation will have to wait a bit more. Best, Marijn From kentilton at gmail.com Sat Mar 6 20:26:34 2010 From: kentilton at gmail.com (Ken Tilton) Date: Sat, 6 Mar 2010 15:26:34 -0500 Subject: [postmodern-devel] one-way array/record/coercion support In-Reply-To: References: <1267581191.8117.9.camel@news.danieljanus.pl> Message-ID: On Sat, Mar 6, 2010 at 9:40 AM, Marijn Haverbeke wrote: > Hi Daniel, > > Thanks for the code. However, :coerce is already there, named :type, > and the way to-sql-string is currently used, it should A) not use > S-SQL functions, since cl-postgres does not depend on S-SQL, and B) > serialise to a format that can also be used to pass parameters to > prepared queries. I'm not that knowledgeable about Postgres' textual > formats, but it appears that a value like ARRAY[1, 2, 3] can not be > written as a parameter: > > Database error 22P02: array value must start with "{" or dimension > information > Pardon a vague response, but in my crack t postgres recently I do remember a minor aggravation with the two ways of saying "array". Probably just need to sort out which syntax to use when. kt -------------- next part -------------- An HTML attachment was scrubbed... URL: From dj at danieljanus.pl Sat Mar 6 22:37:17 2010 From: dj at danieljanus.pl (Daniel Janus) Date: Sat, 06 Mar 2010 23:37:17 +0100 Subject: [postmodern-devel] one-way array/record/coercion support In-Reply-To: References: <1267581191.8117.9.camel@news.danieljanus.pl> Message-ID: <1267915037.25926.16.camel@thymus> Hi Marijn, > Thanks for the code. However, :coerce is already there, named :type, Didn't notice it -- thanks! > and the way to-sql-string is currently used, it should A) not use > S-SQL functions, since cl-postgres does not depend on S-SQL, Then S-SQL could contain a method TO-SQL-STRING specializing on vectors and lists, much the same way as it now contains a specialization on symbols (see s-sql.lisp, lines 263-264). > , and B) > serialise to a format that can also be used to pass parameters to > prepared queries. I'm not that knowledgeable about Postgres' textual > formats, but it appears that a value like ARRAY[1, 2, 3] can not be > written as a parameter: > > Database error 22P02: array value must start with "{" or dimension > information Could you elaborate on how you are getting this error? I don't seem to have problems when passing arrays to prepared statements in this way: weekword=# prepare testme(int[]) as select array_lower($1, 1) as lo, array_upper($1, 1) as hi; PREPARE weekword=# execute testme(array[1,2,3]); lo | hi ----+---- 1 | 3 > If you feel like getting to the bottom of this, feel free to submit an > alternative patch. If not, array serialisation will have to wait a bit > more. Thanks for the comments! I'll try to do my best to work this out in a correct way. Best, Daniel From marijnh at gmail.com Sun Mar 7 08:13:18 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sun, 7 Mar 2010 09:13:18 +0100 Subject: [postmodern-devel] one-way array/record/coercion support In-Reply-To: <1267915037.25926.16.camel@thymus> References: <1267581191.8117.9.camel@news.danieljanus.pl> <1267915037.25926.16.camel@thymus> Message-ID: > Then S-SQL could contain a method TO-SQL-STRING specializing on vectors > and lists, much the same way as it now contains a specialization on > symbols (see s-sql.lisp, lines 263-264). Actually, there should probably be a separate method on the s-sql side, which calls to-sql-string on types it's not specialised on, and has its own specialisations for symbol. I'll look into this. > Could you elaborate on how you are getting this error? ?I don't seem > to have problems when passing arrays to prepared statements in this > way: > > weekword=# prepare testme(int[]) as select array_lower($1, 1) as lo, > array_upper($1, 1) as hi; > PREPARE > weekword=# execute testme(array[1,2,3]); That's the client-side parser helping you out. What I did was prepare the query in Postmodern, and then trying to execute it with a vector as one of the arguments. This will cause cl-postgres to call to-sql-string on the parameter values before stuffing them into the socket. Apparently, at that level, only the {} syntax is supported. From peter.stiernstrom at blixtvik.se Tue Mar 9 09:20:15 2010 From: peter.stiernstrom at blixtvik.se (=?ISO-8859-1?Q?Peter_Stiernstr=F6m?=) Date: Tue, 09 Mar 2010 10:20:15 +0100 Subject: [postmodern-devel] Postgres transaction warnings and save-dao Message-ID: <4B9612CF.5060100@blixtvik.se> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, I'm using postmodern with great success but I keep seeing these two warnings in my log: - "Postgres warning: there is no transaction in progress" - "Warning while processing connection: Postgres warning: there is already a transaction in progress" As far as I can tell these are merely warnings but they're filling up my logs so I had a look at what was causing it and I found that save-dao has a (with-transaction () ...) wrapping an insert-dao and I was calling save-dao from within my own transaction block. Now if just remove the with-transaction from save-dao an duplicate key violation will ruin the outer transaction by just aborting it (regardless of the error handling code. However if I add a savepoint just before trying insert-dao and in the error handling code use rollback-savepoint to restore to this savepoint I can make the warnings go away. The problem now being that save-dao expects to be called within a with-transaction form. So I was wondering if there is some way to tell if we are within a transaction (e.g. within-transaction-p)? Then I would be able to make a modification that handles both cases. It might then look something like this: (defun save-dao (dao) (if (within-transaction-p) (with-savepoint save-dao (handler-case (progn (insert-dao dao) t) (cl-postgres-error:unique-violation () (rollback-savepoint save-dao) (update-dao dao) nil)))) (handler-case (with-transaction () (insert-dao dao) t) (cl-postgres-error:unique-violation () (update-dao dao) nil))) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkuWEs8ACgkQ0brSZD05ZzAj7wCfbhPynKyGMStzsRP/3MIGpOSH 6IcAni232P2lQ8/hAh5+03NmFr0FSbMs =EIRi -----END PGP SIGNATURE----- From marijnh at gmail.com Tue Mar 9 10:14:41 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 9 Mar 2010 11:14:41 +0100 Subject: [postmodern-devel] Postgres transaction warnings and save-dao In-Reply-To: <4B9612CF.5060100@blixtvik.se> References: <4B9612CF.5060100@blixtvik.se> Message-ID: Hello Peter, It seems that transaction was a misguided attempt (apparently I believed postgres supported nested transactions at the time) to prevent the failure to insert from abandoning the transaction. I think the cleanest solution at this point is to add a variant, save-dao/transaction, that uses a savepoint, and declare the regular save-dao to only be safe outside of transactions. I've pushed this change to the repository. Best, Marijn From daniel at whitehouse.id.au Tue Mar 9 11:04:20 2010 From: daniel at whitehouse.id.au (Daniel White) Date: Tue, 9 Mar 2010 22:04:20 +1100 Subject: [postmodern-devel] Postgres transaction warnings and save-dao In-Reply-To: References: <4B9612CF.5060100@blixtvik.se> Message-ID: I've also had the need for determining the state of a transaction. Previously, I'd defined a local WITH-TRANSACTION macro that bound a special variable *TRANSACTION* with the transaction in progress. I have attached a patch that does just that. WITHIN-TRANSACTION-P will return T if a transaction is open. I've not exported *TRANSACTION* because I'm unsure whether this should be part of the public interface. I'm yet to come up with a less hands-on approach to nested transactions, but this allows options for experiments outside of the postmodern proper. For example, my ENSURE-TRANSACTION macro. Really, I should be using savepoints, but this allows me some more flexibility in how I layer my database access methods. I'm not sure if this is appropriate, but if you think it is useful enough, I can prepare a patch to include this as well. (defmacro ensure-transaction ((&optional name) &body body) `(flet ((exec-body () (let ,(if name `((,name postmodern::*transaction*))) , at body))) (if (within-transaction-p) (exec-body) (with-transaction (,name) (exec-body))))) -- Daniel White -------------- next part -------------- A non-text attachment was scrubbed... Name: add-function_-within_transaction_p_-to-determine-if-a-transaction-is-open.dpatch Type: application/octet-stream Size: 32941 bytes Desc: not available URL: From marijnh at gmail.com Tue Mar 9 11:16:35 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 9 Mar 2010 12:16:35 +0100 Subject: [postmodern-devel] Postgres transaction warnings and save-dao In-Reply-To: References: <4B9612CF.5060100@blixtvik.se> Message-ID: Hey Daniel, I'd say something like this works best as a project-local hack, rather than part of the library interface. The user can perfectly well do (execute "begin") outside of the macro, causing the *transaction* variable to no longer reflect the actual situation. Best, Marijn From peter.stiernstrom at blixtvik.se Tue Mar 9 12:14:45 2010 From: peter.stiernstrom at blixtvik.se (=?ISO-8859-1?Q?Peter_Stiernstr=F6m?=) Date: Tue, 09 Mar 2010 13:14:45 +0100 Subject: [postmodern-devel] Postgres transaction warnings and save-dao In-Reply-To: References: <4B9612CF.5060100@blixtvik.se> Message-ID: <4B963BB5.8080209@blixtvik.se> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Marijn Haverbeke wrote: > Hey Daniel, > > I'd say something like this works best as a project-local hack, rather > than part of the library interface. The user can perfectly well do > (execute "begin") outside of the macro, causing the *transaction* > variable to no longer reflect the actual situation. I agree that the pragmatic solution suggested is not a perfect one but I'm leaning this way myself since I'm using with-transaction very consistently. The best way would of course be if one could interrogate postgresql about its transaction state so that it could catch explicit statements like (execute "begin"). I implemented within-transaction-p using a special variable and a small modification to within-transaction to bind it and with these tools save-dao can be fixed for both when within and outside of a transaction without generating any warnings. For me the problem with the explicit solution of save-dao and save-dao/transaction is that it can't be used to build any transaction agnostic abstractions. Anyhow, this solution is good enough for now (and me). Thanks for all the input! /Peter > Best, > Marijn > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkuWO7UACgkQ0brSZD05ZzDgBACgytukmnKbo8B9h5MfgEAzv4MA np4AnjHYjpTdFHXMg4kKKC6W5mMbeQZ5 =M80q -----END PGP SIGNATURE----- From daniel at whitehouse.id.au Tue Mar 9 12:07:47 2010 From: daniel at whitehouse.id.au (Daniel White) Date: Tue, 9 Mar 2010 23:07:47 +1100 Subject: [postmodern-devel] Postgres transaction warnings and save-dao In-Reply-To: References: <4B9612CF.5060100@blixtvik.se> Message-ID: On Tue, Mar 9, 2010 at 10:16 PM, Marijn Haverbeke wrote: > > I'd say something like this works best as a project-local hack, rather > than part of the library interface. The user can perfectly well do > (execute "begin") outside of the macro, causing the *transaction* > variable to no longer reflect the actual situation. Maybe with a better docstring explicitly stating that it is limited to only knowing of transactions initiated with the WITH-TRANSACTION macro. -- Daniel White From marianomontone at gmail.com Fri Mar 19 17:37:44 2010 From: marianomontone at gmail.com (Mariano Montone) Date: Fri, 19 Mar 2010 14:37:44 -0300 Subject: [postmodern-devel] Developing with postmodern Message-ID: <4BA3B668.8040301@gmail.com> Hi! When developing with postmodern I'm having two problems: If a query is wrong, my slime connection seems to get disconnected or waiting for something (I have to hit C-c c to get the REPL back). Besides, when an error ocurrs, I don't get notified immediatly, but after I make another query, in which case I get a "the database is processing another query" (the screwed up query). So I have to restart the connection or the application (I'm developing a Web application)... :( Do someone else have the same problems? How do you fix them? Thanks! Mariano From marijnh at gmail.com Fri Mar 19 22:17:35 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Fri, 19 Mar 2010 23:17:35 +0100 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: <4BA3B668.8040301@gmail.com> References: <4BA3B668.8040301@gmail.com> Message-ID: Hello Mariano, I haven't seen this issue before. Are those queries made from the REPL, or from another thread? Also, which CL implementation, and which Postgres version are you using? Best, Marijn From lucas.r.hope at gmail.com Sat Mar 20 03:43:57 2010 From: lucas.r.hope at gmail.com (Lucas Hope) Date: Sat, 20 Mar 2010 14:43:57 +1100 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: References: <4BA3B668.8040301@gmail.com> Message-ID: <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> I get this sometimes when I have done a "wrong" query that is wrong in that it would take a very long time to complete, not a malformed SQL statement. Then I interrupt the process from the lisp/slime side, but the PostgreSQL side is still happily executing. The toplevel connection is still waiting for the result, and hence you have to reconnect toplevel (or better, kill the rogue postgres process). This behaviour seems correct to me, or at least suitable enough. A minor improvement might be to kill the query if you interrupt and cancel the lisp-side process, but that sounds like too much trouble. So it depends how "wrong" wrong is. -Luke On Sat, Mar 20, 2010 at 9:17 AM, Marijn Haverbeke wrote: > Hello Mariano, > > I haven't seen this issue before. Are those queries made from the > REPL, or from another thread? Also, which CL implementation, and which > Postgres version are you using? > > Best, > Marijn > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > -- --------------------------------------------------- Dr Lucas Hope - (646) 2332123 after 3pm US EST Machine Learning and Software Engineering Consultant Melbourne, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From marijnh at gmail.com Sat Mar 20 08:08:25 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 20 Mar 2010 09:08:25 +0100 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> Message-ID: > This behaviour seems correct to me, or at least suitable enough. A minor > improvement might be to kill the query if you interrupt and cancel the > lisp-side process, but that sounds like too much trouble. I think there actually is unwinding code that tries to do this, but apparently it doesn't work. I'll make a note for myself to test that again sometime. From marianomontone at gmail.com Sat Mar 20 14:23:42 2010 From: marianomontone at gmail.com (Mariano Montone) Date: Sat, 20 Mar 2010 11:23:42 -0300 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> Message-ID: <4BA4DA6E.2080605@gmail.com> Lucas Hope escribi?: > I get this sometimes when I have done a "wrong" query that is wrong in > that it would take a very long time to complete, not a malformed SQL > statement. > > Then I interrupt the process from the lisp/slime side, but the > PostgreSQL side is still happily executing. The toplevel connection is > still waiting for the result, and hence you have to reconnect toplevel > (or better, kill the rogue postgres process). > > This behaviour seems correct to me, or at least suitable enough. A > minor improvement might be to kill the query if you interrupt and > cancel the lisp-side process, but that sounds like too much trouble. > > So it depends how "wrong" wrong is. Mmm...maybe that's what is happening. I'll check. Thanks Mariano From marianomontone at gmail.com Sat Mar 20 14:29:07 2010 From: marianomontone at gmail.com (Mariano Montone) Date: Sat, 20 Mar 2010 11:29:07 -0300 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: References: <4BA3B668.8040301@gmail.com> Message-ID: <4BA4DBB3.7050805@gmail.com> Marijn Haverbeke escribi?: > Hello Mariano, > > I haven't seen this issue before. Are those queries made from the > REPL, or from another thread? Also, which CL implementation, and which > Postgres version are you using? > I'm using SBCL and the repo version of postmodern with postgresql 8.3.8. The queries are made from the web app. I guess it may have to do with what Lucas says. I'll be back after doing some tests and having a clearer idea of what may be happening. Thanks Mariano From marijnh at gmail.com Sat Mar 20 19:02:43 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 20 Mar 2010 20:02:43 +0100 Subject: [postmodern-devel] one-way array/record/coercion support In-Reply-To: References: <1267581191.8117.9.camel@news.danieljanus.pl> <1267915037.25926.16.camel@thymus> Message-ID: Hey list, I finally got around to implementing a proper serialisation for arrays, both as query text and as prepared-query parameters (which requires two wildly different formats --- ARRAY[E'a', E'b'] versus {"a", "b"}). They should now Just Work. Also added a :[] sql-op, so you can do (:[] 'myarr 1) and it'll expand to (myarr)[1] -- including superfluous parentheses, since in a lot of situations they are required. Best, Marijn From marijnh at gmail.com Sat Mar 20 19:21:46 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 20 Mar 2010 20:21:46 +0100 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> Message-ID: > I think there actually is unwinding code that tries to do this, but > apparently it doesn't work. I'll make a note for myself to test that > again sometime. This is currently not implemented, I found out. I also remembered why -- the protocol for canceling requests is rather cumbersome, and at the time I didn't feel it was important enough to implement. See http://www.postgresql.org/docs/current/static/protocol-flow.html#AEN80105 for the details. From marianomontone at gmail.com Mon Mar 22 17:40:53 2010 From: marianomontone at gmail.com (Mariano Montone) Date: Mon, 22 Mar 2010 14:40:53 -0300 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> Message-ID: <4BA7ABA5.8090006@gmail.com> Marijn Haverbeke escribi?: >> I think there actually is unwinding code that tries to do this, but >> apparently it doesn't work. I'll make a note for myself to test that >> again sometime. >> > > This is currently not implemented, I found out. I also remembered why > -- the protocol for canceling requests is rather cumbersome, and at > the time I didn't feel it was important enough to implement. See > http://www.postgresql.org/docs/current/static/protocol-flow.html#AEN80105 > for the details. > So, I have no way of telling postgres to stop processing the query? Actually, what I have are unsynchronized DAO objects (no any sort of long queries), so postmodern complains, but when that happens, the database hangs and I cannot query it again after I have fixed the object-table synchronization. Mariano From marijnh at gmail.com Mon Mar 22 17:48:14 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 22 Mar 2010 18:48:14 +0100 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: <4BA7ABA5.8090006@gmail.com> References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> <4BA7ABA5.8090006@gmail.com> Message-ID: > So, I have no way of telling postgres to stop processing the query? > Actually, what I have are unsynchronized DAO objects (no any sort of > long queries), Can you show some code that demonstrates what you mean by an unsynchronized DAO object? From marianomontone at gmail.com Mon Mar 22 17:57:53 2010 From: marianomontone at gmail.com (Mariano Montone) Date: Mon, 22 Mar 2010 14:57:53 -0300 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> <4BA7ABA5.8090006@gmail.com> Message-ID: <4BA7AFA1.6070709@gmail.com> Marijn Haverbeke escribi?: >> So, I have no way of telling postgres to stop processing the query? >> Actually, what I have are unsynchronized DAO objects (no any sort of >> long queries), >> > > Can you show some code that demonstrates what you mean by an > unsynchronized DAO object? > When you have an CLOS (DAO) object's attribute and it doesn't exist in the database table. Or otherwise, when you have an attribute in the table that is not defined in the DAO class. I think you get a DAO synchronization error when any of those happen. That is fine with me, but I would like to get rid of the hung up and be able to go on querying the db. For example: (defclass project-activity () ((userid :accessor userid :initarg :userid :initform 0 :col-type integer :documentation "The id of the user we are registering the activity for") (projectid :accessor projectid :initarg :projectid :initform 0 :col-type integer :documentation "The id of the project in which activity was detected") (value :accessor value :initarg :value :initform 0 :col-type integer :documentation "Each time user's activity in the project is detected, this value is incremented. This is used to determine the most active projects")) (:metaclass dao-class) (:documentation "Instances of this class are created when an user modifies aproject (i.e. creates, modifies, or comments one of its issues)")) If any of the attributes is not in the database, for instance 'value', then I get the error and have to restart my application. From marijnh at gmail.com Mon Mar 22 19:31:59 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 22 Mar 2010 20:31:59 +0100 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: <4BA7AFA1.6070709@gmail.com> References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> <4BA7ABA5.8090006@gmail.com> <4BA7AFA1.6070709@gmail.com> Message-ID: > If any of the attributes is not in the database, for instance 'value', > then I get the error and have to restart my application. Still not very clear -- how can the value not be in the database? You mean it is null for the record you are retrieving? From marianomontone at gmail.com Mon Mar 22 20:44:05 2010 From: marianomontone at gmail.com (Mariano Montone) Date: Mon, 22 Mar 2010 17:44:05 -0300 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> <4BA7ABA5.8090006@gmail.com> <4BA7AFA1.6070709@gmail.com> Message-ID: <4BA7D695.6090408@gmail.com> Marijn Haverbeke escribi?: >> If any of the attributes is not in the database, for instance 'value', >> then I get the error and have to restart my application. >> > > Still not very clear -- how can the value not be in the database? You > mean it is null for the record you are retrieving? > No. The column 'value' is not in the table. The database schema does not match the class definition. I have some of this kind of errors because I'm porting an application from clsql to postmodern and clsql didn't check that the database and the code matched (at least not as strongly as postmodern), and I have some cases in which the data doesn't match the code. Anyway, I'm not sure the problem arises only when a DAO synchronization error ocurrs, but I have to do more tests to tell you that. From marijnh at gmail.com Mon Mar 22 21:04:13 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 22 Mar 2010 22:04:13 +0100 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: <4BA7D695.6090408@gmail.com> References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> <4BA7ABA5.8090006@gmail.com> <4BA7AFA1.6070709@gmail.com> <4BA7D695.6090408@gmail.com> Message-ID: > No. The column 'value' is not in the table. The database schema does not > match the class definition. Ah, now I get it. Still, creating a situation like that, I immediately get this error: Database error 42703: column "y" does not exist Query: INSERT INTO foo (x, y) VALUES (1, 2) [Condition of type cl-postgres-error:syntax-error-or-access-violation] ... and the connection is fine. So, if you want me to look into this, a script that isolates the issue and can be ran against an empty database to produce the error would probably help. Cheers, Marijn From dj at danieljanus.pl Mon Mar 22 21:07:06 2010 From: dj at danieljanus.pl (Daniel Janus) Date: Mon, 22 Mar 2010 22:07:06 +0100 Subject: [postmodern-devel] one-way array/record/coercion support In-Reply-To: References: <1267581191.8117.9.camel@news.danieljanus.pl> <1267915037.25926.16.camel@thymus> Message-ID: <1269292026.2000.0.camel@news.danieljanus.pl> Hi Marijn, > I finally got around to implementing a proper serialisation for > arrays, both as query text and as prepared-query parameters (which > requires two wildly different formats --- ARRAY[E'a', E'b'] versus > {"a", "b"}). They should now Just Work. Also added a :[] sql-op, so > you can do (:[] 'myarr 1) and it'll expand to (myarr)[1] -- including > superfluous parentheses, since in a lot of situations they are > required. This is extremely great, thanks a million! -Daniel From marianomontone at gmail.com Mon Mar 22 21:10:53 2010 From: marianomontone at gmail.com (Mariano Montone) Date: Mon, 22 Mar 2010 18:10:53 -0300 Subject: [postmodern-devel] Developing with postmodern In-Reply-To: References: <4BA3B668.8040301@gmail.com> <69fb852b1003192043p5ff24b50sa1b49615a9f2ffb7@mail.gmail.com> <4BA7ABA5.8090006@gmail.com> <4BA7AFA1.6070709@gmail.com> <4BA7D695.6090408@gmail.com> Message-ID: <4BA7DCDD.50703@gmail.com> Marijn Haverbeke escribi?: >> No. The column 'value' is not in the table. The database schema does not >> match the class definition. >> > > Ah, now I get it. Still, creating a situation like that, I immediately > get this error: > > Database error 42703: column "y" does not exist > Query: INSERT INTO foo (x, y) VALUES (1, 2) > [Condition of type cl-postgres-error:syntax-error-or-access-violation] > > ... and the connection is fine. > > So, if you want me to look into this, a script that isolates the issue > and can be ran against an empty database to produce the error would > probably help. > Ok, so maybe it is a problem in my setup. I'll be back with a script like you say if things get to difficult to put up with. I can go on developing the application for the moment. Thanks for your help. Mariano From archimag at gmail.com Fri Mar 26 21:56:22 2010 From: archimag at gmail.com (Andrey Moskvitin) Date: Sat, 27 Mar 2010 00:56:22 +0300 Subject: [postmodern-devel] PATCH: New query formats - plists and plist Message-ID: Hi, I propose to add new query formats: diff --git a/doc/postmodern.html b/doc/postmodern.html index 3691ae8..9ac466f 100644 --- a/doc/postmodern.html +++ b/doc/postmodern.html @@ -202,6 +202,10 @@ row. :list, :rowReturn a single row as a list. + :plistsReturn a list of plists which map column + names to values,with the names represented as + keywords. + :plistReturn a single row as an plist. :alistsReturn a list of alists which map column names to values,with the names represented as keywords. diff --git a/postmodern/query.lisp b/postmodern/query.lisp index 3e5bfa5..921d03d 100644 --- a/postmodern/query.lisp +++ b/postmodern/query.lisp @@ -9,6 +9,15 @@ :for symbol :in symbols :collect (cons symbol (next-field field)))))) +;; Like symbol-alist-row-reader, but return plist +(def-row-reader symbol-plist-row-reader (fields) + (let ((symbols (map 'list (lambda (desc) (from-sql-name (field-name desc))) fields))) + (loop :while (next-row) + :collect (loop :for field :across fields + :for symbol :in symbols + :collect symbol + :collect (next-field field))))) + ;; A row-reader for reading only a single column, and returning a list ;; of single values. (def-row-reader column-row-reader (fields) @@ -22,6 +31,8 @@ (:list list-row-reader t) (:rows list-row-reader nil) (:row list-row-reader t) + (:plists symbol-plist-row-reader nil) + (:plist symbol-plist-row-reader t) (:alists symbol-alist-row-reader nil) (:alist symbol-alist-row-reader t) (:str-alists alist-row-reader nil) Moskvitin Andrey -------------- next part -------------- An HTML attachment was scrubbed... URL: From marijnh at gmail.com Sat Mar 27 06:51:26 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 27 Mar 2010 07:51:26 +0100 Subject: [postmodern-devel] PATCH: New query formats - plists and plist In-Reply-To: References: Message-ID: Hi Andrey, Thanks for submitting a patch. But -- why plists? Do you have some library for which you need results like this? Best, Marijn On Fri, Mar 26, 2010 at 10:56 PM, Andrey Moskvitin wrote: > Hi, > I propose to add new query formats: > diff --git a/doc/postmodern.html b/doc/postmodern.html > index 3691ae8..9ac466f 100644 > --- a/doc/postmodern.html > +++ b/doc/postmodern.html > @@ -202,6 +202,10 @@ > ?? ? ? row. > ?? ? ? :list, :rowReturn a > ?? ? ? single row as a list. > + ? ? ?:plistsReturn a list of plists which > map column > + ? ? ?names to values,with the names represented as > + ? ? ?keywords. > + ? ? ?:plistReturn a single row as an > plist. > ?? ? ? :alistsReturn a list of alists which > map column > ?? ? ? names to values,with the names represented as > ?? ? ? keywords. > diff --git a/postmodern/query.lisp b/postmodern/query.lisp > index 3e5bfa5..921d03d 100644 > --- a/postmodern/query.lisp > +++ b/postmodern/query.lisp > @@ -9,6 +9,15 @@ > ?? ? ? ? ? ? ? ? ? ? ? ? ?:for symbol :in symbols > ?? ? ? ? ? ? ? ? ? ? ? ? ?:collect (cons symbol (next-field field)))))) > > +;; Like symbol-alist-row-reader, but return plist > +(def-row-reader symbol-plist-row-reader (fields) > + ?(let ((symbols (map 'list (lambda (desc) (from-sql-name (field-name > desc))) fields))) > + ? ?(loop :while (next-row) > + ? ? ? ? ?:collect (loop :for field :across fields > + ? ? ? ? ? ? ? ? ? ? ? ? :for symbol :in symbols > + ? ? ? ? ? ? ? ? ? ? ? ? :collect symbol > + ? ? ? ? ? ? ? ? ? ? ? ? :collect (next-field field))))) > + > ?;; A row-reader for reading only a single column, and returning a list > ?;; of single values. > ?(def-row-reader column-row-reader (fields) > @@ -22,6 +31,8 @@ > ?? ? (:list list-row-reader t) > ?? ? (:rows list-row-reader nil) > ?? ? (:row list-row-reader t) > + ? ?(:plists symbol-plist-row-reader nil) > + ? ?(:plist symbol-plist-row-reader t) > ?? ? (:alists symbol-alist-row-reader nil) > ?? ? (:alist symbol-alist-row-reader t) > ?? ? (:str-alists alist-row-reader nil) > > Moskvitin Andrey > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > > From archimag at gmail.com Sat Mar 27 07:00:09 2010 From: archimag at gmail.com (Andrey Moskvitin) Date: Sat, 27 Mar 2010 10:00:09 +0300 Subject: [postmodern-devel] PATCH: New query formats - plists and plist In-Reply-To: References: Message-ID: > But -- why plists? Do you have some > library for which you need results like this? Yes, my template library cl-closure-template ( http://code.google.com/p/cl-closure-template/) required plists as input data. In my experience code based on the plist is usually easier to write and understand. Andrey -------------- next part -------------- An HTML attachment was scrubbed... URL: From marijnh at gmail.com Sat Mar 27 07:53:20 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 27 Mar 2010 08:53:20 +0100 Subject: [postmodern-devel] PATCH: New query formats - plists and plist In-Reply-To: References: Message-ID: > Yes, my template library cl-closure-template Fair enough. Applied and pushed. From marijnh at gmail.com Sat Mar 27 07:56:57 2010 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 27 Mar 2010 08:56:57 +0100 Subject: [postmodern-devel] Git? Message-ID: Hello list, Would it cause problems, effort, or needless pain for any of you if I switched Postmodern development from darcs to git? I've been using git in pretty much all of my other work, and it's been so ingrained in my system that I'm starting to have trouble working with darcs. The whole project history can be converted using darcs-to-git. Also, I am planning to move over hosting of the project from common-lisp.net to my own server, since I have more control there (not to mention a better uptime record). If I make the switch to git, I'll probably make the server switch at the same time. Best, Marijn From archimag at gmail.com Sat Mar 27 08:51:30 2010 From: archimag at gmail.com (Andrey Moskvitin) Date: Sat, 27 Mar 2010 11:51:30 +0300 Subject: [postmodern-devel] Git? In-Reply-To: References: Message-ID: > Git? That would be great! Andrey 2010/3/27 Marijn Haverbeke > Hello list, > > Would it cause problems, effort, or needless pain for any of you if I > switched Postmodern development from darcs to git? I've been using git > in pretty much all of my other work, and it's been so ingrained in my > system that I'm starting to have trouble working with darcs. > > The whole project history can be converted using darcs-to-git. > > Also, I am planning to move over hosting of the project from > common-lisp.net to my own server, since I have more control there (not > to mention a better uptime record). If I make the switch to git, I'll > probably make the server switch at the same time. > > Best, > Marijn > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lispnik at gmail.com Sat Mar 27 08:38:20 2010 From: lispnik at gmail.com (Ivan Boldyrev) Date: Sat, 27 Mar 2010 14:38:20 +0600 Subject: [postmodern-devel] Git? In-Reply-To: References: Message-ID: On Sat, Mar 27, 2010 at 1:56 PM, Marijn Haverbeke wrote: > Would it cause problems, effort, or needless pain for any of you if I > switched Postmodern development from darcs to git? I believe it will reduce problems, effort and needless pain :) -- Ivan Boldyrev From maciej at pasternacki.net Sat Mar 27 10:15:05 2010 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Sat, 27 Mar 2010 11:15:05 +0100 Subject: [postmodern-devel] Git? In-Reply-To: References: Message-ID: On 2010-03-27, at 08:56, Marijn Haverbeke wrote: > Would it cause problems, effort, or needless pain for any of you if I > switched Postmodern development from darcs to git? +1 - it would rather reduce problems. -- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- http://www.3ofcoins.net/ From dj at danieljanus.pl Tue Mar 30 17:56:23 2010 From: dj at danieljanus.pl (Daniel Janus) Date: Tue, 30 Mar 2010 19:56:23 +0200 Subject: [postmodern-devel] simple-date:day-of-week doc mismatch? Message-ID: <1269971783.3422.1.camel@news.danieljanus.pl> Hi, DB> (make-date nil nil nil) # DB> (day-of-week *) 1 That would indicate Monday, according to the docs, while my system calendar tells me it's Tuesday today... What gives? (I'm on SBCL 1.0.34.) Best regards, Daniel