From mslamin at gmail.com Wed Oct 1 00:27:48 2008 From: mslamin at gmail.com (Mark Slamin) Date: Tue, 30 Sep 2008 19:27:48 -0500 Subject: [postmodern-devel] doquery - iterating rows Message-ID: <939680080809301727o5b2281fev12db5055c8306e30@mail.gmail.com> Sorry, more questions. I'm trying to iterate down rows in the database. Using doquery like this: (defun iterate-rows () (doquery (query (:select 'name :from 'htable) :list) (xname) (print xname))) I receive the following error: The value of CL-POSTGRES::QUERY is ("(A B)"), which is not of type STRING. [Condition of type SIMPLE-TYPE-ERROR] I'm not sure what to do with that. Also, on a broader scale, what I'm trying to do is iterate through the rows of a large database and I don't want to load the entire results of the query into memory. Ideally I just want to load one record with all its fields, perform some operations on that data, write the record back if its been updated, and then move to the next record. Is doquery the best method for that? Or is there something else that would work better? Thank you, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From marijnh at gmail.com Wed Oct 1 06:15:12 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 1 Oct 2008 08:15:12 +0200 Subject: [postmodern-devel] doquery - iterating rows In-Reply-To: <939680080809301727o5b2281fev12db5055c8306e30@mail.gmail.com> References: <939680080809301727o5b2281fev12db5055c8306e30@mail.gmail.com> Message-ID: Hello Mark, DOQUERY is indeed what you need here, but you got the arguments all wrong -- which, looking at the documentation, is not very surprising. I'll add a decent example there. For now, I think you want this: (defun iterate-rows () (doquery (:select 'name :from 'htable) (xname) (print xname))) Best, Marijn 2008/10/1 Mark Slamin : > Sorry, more questions. > I'm trying to iterate down rows in the database. > Using doquery like this: > I receive the following error: > The value of CL-POSTGRES::QUERY is ("(A B)"), which is not of type STRING. > [Condition of type SIMPLE-TYPE-ERROR] > I'm not sure what to do with that. > Also, on a broader scale, what I'm trying to do is iterate through the rows > of a large database and I don't want to load the entire results of the query > into memory. Ideally I just want to load one record with all its fields, > perform some operations on that data, write the record back if its been > updated, and then move to the next record. > Is doquery the best method for that? Or is there something else that would > work better? > Thank you, > Mark > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > > From maciej at pasternacki.net Sat Oct 11 10:18:31 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Sat, 11 Oct 2008 12:18:31 +0200 Subject: [postmodern-devel] patch: DAO constraints Message-ID: <79B67F26-63DC-40A9-B707-EED73A389508@pasternacki.net> Hello, Since it's not my first patch for Postmodern, and most probably not the last one, I can as well subscribe to the list and send patches properly instead of in a private e-mail. I just made a small update to DAOs to use constraints (foreign keys, checks, indices) for DAO slots and classes. Doesn't seem to break anything, and makes it possible (or almost possible) to drop separate .sql definition files and just use DAO-TABLE-DEFINITION. What's missing for me is arbitrary SQL for e.g. defining needed functions, enum types and so on before creating table -- but I'm not sure what approach should I take: move it all to DAOs (as e.g. :BEFORE- CREATE-TABLE and :AFTER-CREATE-TABLE class slots), or just make init function for my package, which would call needed queries and use DAO- TABLE-DEFINITION just to create the tables. Would there be any advantage to :BEFORE-... and :AFTER-... slots, or is the init function the preferred way to go? Regards, Maciej. -------------- next part -------------- A non-text attachment was scrubbed... Name: dao-constraints.cset Type: application/octet-stream Size: 26293 bytes Desc: not available URL: -------------- next part -------------- -- -><- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- From marijnh at gmail.com Sat Oct 11 10:24:51 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 11 Oct 2008 12:24:51 +0200 Subject: [postmodern-devel] patch: DAO constraints In-Reply-To: <79B67F26-63DC-40A9-B707-EED73A389508@pasternacki.net> References: <79B67F26-63DC-40A9-B707-EED73A389508@pasternacki.net> Message-ID: Hey Maciej, I have no time to look into this right away -- but did you see the deftable stuff? The idea is that you combine all the stuff needed to define a table or view in a deftable form. I intentionally didn't include complicated functionality in the DAO syntax, since they are just access objects, not definitions, and trying to support everything there is an endless labour. Constraints can be added after a table has been defined, which is how I usually do it (if I remember correctly theres already a !foreign operator, and similar ones can be defined for other useful constraints). Cheers, Marijn 2008/10/11 Maciej Pasternacki : > Hello, > > Since it's not my first patch for Postmodern, and most probably not the last > one, I can as well subscribe to the list and send patches properly instead > of in a private e-mail. > > I just made a small update to DAOs to use constraints (foreign keys, checks, > indices) for DAO slots and classes. Doesn't seem to break anything, and > makes it possible (or almost possible) to drop separate .sql definition > files and just use DAO-TABLE-DEFINITION. > > What's missing for me is arbitrary SQL for e.g. defining needed functions, > enum types and so on before creating table -- but I'm not sure what approach > should I take: move it all to DAOs (as e.g. :BEFORE-CREATE-TABLE and > :AFTER-CREATE-TABLE class slots), or just make init function for my package, > which would call needed queries and use DAO-TABLE-DEFINITION just to create > the tables. Would there be any advantage to :BEFORE-... and :AFTER-... > slots, or is the init function the preferred way to go? > > Regards, > Maciej. > > > -- > -><- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- > > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > > From maciej at pasternacki.net Sat Oct 11 10:51:16 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Sat, 11 Oct 2008 12:51:16 +0200 Subject: [postmodern-devel] patch: DAO constraints In-Reply-To: References: <79B67F26-63DC-40A9-B707-EED73A389508@pasternacki.net> Message-ID: Hello, DEFTABLE seems fine (or "almost" fine, I'll have to actually try it) for before/after code as I need it, but the constraints inside DAOs seem to be what I actually need. Here's what I'm trying to achieve: separate single-component library with basic functionality (for e.g. Wordpress-like taxonomy applicable to any "item" in any "site", or for user accounts with registration, activation, password recovery and such); data model entities represented by DAOs; DAO-level inheritance to support project-specific functionality (e.g. user-profile columns, specific foreign keys for taxonomy, and so on). With constraints supported by DAOs it's possible to actually inherit the real schema, and DEFTABLE doesn't support inheritance -- it seems it would be hard to add, and it wouldn't fit well there. Putting constraints in DEFTABLE would break on DAO inheritance, and putting them in DAOs makes it possible not to tinker with hand-written SQL and fitting it to DAOs for every specific use case. However, the more I think about before/after SQL forms, the more it seems they don't actually belong to DAOs and using DEFTABLE or custom init functions will be a better way. BTW, thanks for pointing out the DEFTABLE -- it named a completely different feature before (or there was a similar name), and I didn't notice it's a new one. Regards, Maciej On 2008-10-11, at 12:24, Marijn Haverbeke wrote: > Hey Maciej, > > I have no time to look into this right away -- but did you see the > deftable stuff? The idea is that you combine all the stuff needed to > define a table or view in a deftable form. I intentionally didn't > include complicated functionality in the DAO syntax, since they are > just access objects, not definitions, and trying to support everything > there is an endless labour. Constraints can be added after a table has > been defined, which is how I usually do it (if I remember correctly > theres already a !foreign operator, and similar ones can be defined > for other useful constraints). > > Cheers, > Marijn > > > 2008/10/11 Maciej Pasternacki : >> Hello, >> >> Since it's not my first patch for Postmodern, and most probably not >> the last >> one, I can as well subscribe to the list and send patches properly >> instead >> of in a private e-mail. >> >> I just made a small update to DAOs to use constraints (foreign >> keys, checks, >> indices) for DAO slots and classes. Doesn't seem to break >> anything, and >> makes it possible (or almost possible) to drop separate .sql >> definition >> files and just use DAO-TABLE-DEFINITION. >> >> What's missing for me is arbitrary SQL for e.g. defining needed >> functions, >> enum types and so on before creating table -- but I'm not sure what >> approach >> should I take: move it all to DAOs (as e.g. :BEFORE-CREATE-TABLE and >> :AFTER-CREATE-TABLE class slots), or just make init function for my >> package, >> which would call needed queries and use DAO-TABLE-DEFINITION just >> to create >> the tables. Would there be any advantage to :BEFORE-... >> and :AFTER-... >> slots, or is the init function the preferred way to go? >> >> Regards, >> Maciej. >> >> >> -- >> -><- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- >> >> >> _______________________________________________ >> postmodern-devel mailing list >> postmodern-devel at common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel >> >> > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel -- -><- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- From marijnh at gmail.com Sat Oct 11 11:12:46 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 11 Oct 2008 13:12:46 +0200 Subject: [postmodern-devel] patch: DAO constraints In-Reply-To: References: <79B67F26-63DC-40A9-B707-EED73A389508@pasternacki.net>

Message-ID: > and DEFTABLE doesn't support inheritance Just to be clear -- deftable just wraps a list of strings or stuff that can be evaluated to strings, so everything that can be expressed in SQL can be expressed in deftable. Best, Marijn From larsnostdal at gmail.com Mon Oct 13 12:15:21 2008 From: larsnostdal at gmail.com (Lars Rune =?ISO-8859-1?Q?N=F8stdal?=) Date: Mon, 13 Oct 2008 14:15:21 +0200 Subject: [postmodern-devel] (:nextval a-symbol) Message-ID: <1223900121.7962.16.camel@blackbox> took me a while to figure out that only strings are supported for :nextval in a :col-default context, so: lnostdal at blackbox:~/programming/lisp/postmodern/postmodern$ darcs diff -u diff -rN -u old-postmodern/postmodern/table.lisp new-postmodern/postmodern/table.lisp --- old-postmodern/postmodern/table.lisp 2008-10-13 14:11:29.000000000 +0200 +++ new-postmodern/postmodern/table.lisp 2008-10-13 14:11:29.000000000 +0200 @@ -293,6 +293,10 @@ ,(loop :for slot :in (dao-column-slots table) :collect `(,(slot-definition-name slot) :type ,(column-type slot) ,@(when (slot-boundp slot 'col-default) - `(:default ,(column-default slot))))) + `(:default ,(let ((col-def (column-default slot))) + (when (eq :nextval (first col-def)) + (setf (second col-def) + (to-identifier (second col-def)))) + col-def))))) ,@(when (dao-keys table) `((:primary-key ,@(dao-keys table))))))) ..then.. (defclass user () ((id :col-type integer :reader id-of :col-default (:nextval user-id-seq)) ...)) ..i don't know. -- Lars Rune N?stdal || AJAX/Comet GUI type stuff for Common Lisp http://nostdal.org/ || http://groups.google.com/group/symbolicweb From marijnh at gmail.com Mon Oct 13 13:40:23 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 13 Oct 2008 15:40:23 +0200 Subject: [postmodern-devel] (:nextval a-symbol) In-Reply-To: <1223900121.7962.16.camel@blackbox> References: <1223900121.7962.16.camel@blackbox> Message-ID: > took me a while to figure out that only strings are supported > for :nextval in a :col-default context, so: Cool. Any reason you didn't patch the :nextval operator itself? Cheers, Marijn From larsnostdal at gmail.com Mon Oct 13 20:24:55 2008 From: larsnostdal at gmail.com (Lars Rune =?ISO-8859-1?Q?N=F8stdal?=) Date: Mon, 13 Oct 2008 22:24:55 +0200 Subject: [postmodern-devel] (:nextval a-symbol) In-Reply-To: References: <1223900121.7962.16.camel@blackbox> Message-ID: <1223929491.7962.36.camel@blackbox> On Mon, 2008-10-13 at 15:40 +0200, Marijn Haverbeke wrote: > > took me a while to figure out that only strings are supported > > for :nextval in a :col-default context, so: > > Cool. Any reason you didn't patch the :nextval operator itself? > > Cheers, > Marijn ..I think my last message got lost somewhere. This: diff -rN -u old-postmodern/postmodern/util.lisp new-postmodern/postmodern/util.lisp --- old-postmodern/postmodern/util.lisp 2008-10-13 22:24:03.000000000 +0200 +++ new-postmodern/postmodern/util.lisp 2008-10-13 22:24:03.000000000 +0200 @@ -9,7 +9,7 @@ (defun sequence-next (sequence) "Shortcut for getting the next value from a sequence." - (query (:select (:nextval (to-identifier sequence))) :single)) + (query (:select (:nextval sequence)) :single)) (defmacro make-list-query (relkind) "Helper macro for the functions that list tables, sequences, and diff -rN -u old-postmodern/s-sql/s-sql.lisp new-postmodern/s-sql/s-sql.lisp --- old-postmodern/s-sql/s-sql.lisp 2008-10-13 22:24:03.000000000 +0200 +++ new-postmodern/s-sql/s-sql.lisp 2008-10-13 22:24:03.000000000 +0200 @@ -684,6 +684,9 @@ (def-sql-op :drop-sequence (name) `("DROP SEQUENCE " ,@(sql-expand name))) +(def-sql-op :nextval (name) + `("nextval('" ,@(sql-expand name) "')")) + (def-sql-op :create-view (name query) ;; does not allow to specify the columns of the view yet `("CREATE VIEW " ,(to-sql-name name) " AS " ,@(sql-expand query))) ..seems to work. -- Lars Rune N?stdal || AJAX/Comet GUI type stuff for Common Lisp http://nostdal.org/ || http://groups.google.com/group/symbolicweb From maciej at pasternacki.net Tue Oct 14 11:13:20 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Tue, 14 Oct 2008 13:13:20 +0200 Subject: [postmodern-devel] patch: DAO constraints In-Reply-To: References: <79B67F26-63DC-40A9-B707-EED73A389508@pasternacki.net>

Message-ID: <5E9394D9-540E-45F2-BC65-CB0039539755@pasternacki.net> On 2008-10-11, at 13:12, Marijn Haverbeke wrote: >> and DEFTABLE doesn't support inheritance > > Just to be clear -- deftable just wraps a list of strings or stuff > that can be evaluated to strings, so everything that can be expressed > in SQL can be expressed in deftable. I understand how DEFTABLE works (I think so), and it doesn't support what I actually need. I want to use DAO inheritance, which is supported, but currently to use constraints in DAO tables, I need to specify them separately for parent and child class. DEFTABLE doesn't help here, because it allows only to define a specific table, without any extension, application of same constraints to "child" tables, and so on. Maybe initialization function taking class or class name as an argument would help -- however, it seems to me that constraints belong to DAO classes in the same way column types do. Regards, Maciek -- -><- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- From marijnh at gmail.com Tue Oct 14 20:32:13 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 14 Oct 2008 22:32:13 +0200 Subject: [postmodern-devel] patch: DAO constraints In-Reply-To: <5E9394D9-540E-45F2-BC65-CB0039539755@pasternacki.net> References: <79B67F26-63DC-40A9-B707-EED73A389508@pasternacki.net>

<5E9394D9-540E-45F2-BC65-CB0039539755@pasternacki.net> Message-ID: > however, it seems to me that constraints belong > to DAO classes in the same way column types do. The way I see it, DAO classes are purely access object declarations. The column types are already borderline irrelevant, but since it's often extremely convenient to only have to list your columns once, I include them there. Constraints can easily be specified outside of CREATE TABLE statements, so there's no pressing reason to include them in DAO definitions. Best, Marijn From maciej at pasternacki.net Tue Oct 21 17:24:23 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Tue, 21 Oct 2008 19:24:23 +0200 Subject: [postmodern-devel] [patch] ALTER TABLE, named constraints, typos Message-ID: Hello, I attach: support for ALTER TABLE statement (initial, only defining constraints) and named table constraints in s-sql, and fix for a typo in documentation and for a compiler warning I found on my way. Regards, Maciej -- -><- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- -------------- next part -------------- A non-text attachment was scrubbed... Name: alter-table.cset Type: application/octet-stream Size: 31362 bytes Desc: not available URL: From marijnh at gmail.com Wed Oct 22 14:20:03 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 22 Oct 2008 16:20:03 +0200 Subject: [postmodern-devel] [patch] ALTER TABLE, named constraints, typos In-Reply-To: References: Message-ID: Hi Maciej, Thanks for those. I've pushed them. Though not documented them. Best, Marijn From maciej at pasternacki.net Wed Oct 22 14:25:37 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Wed, 22 Oct 2008 16:25:37 +0200 Subject: [postmodern-devel] [patch] ALTER TABLE, named constraints, typos In-Reply-To: References: Message-ID: <582EFE67-BA48-4D6F-971E-9B48DCE557B0@pasternacki.net> On 2008-10-22, at 16:20, Marijn Haverbeke wrote: > Thanks for those. I've pushed them. Though not documented them. I added documentation for those (unless HTML files in doc/ aren't primary docs and I should update somewhere else). While we're at it, I attach one more patch: support for UPDATE ... RETURNING in S-SQL. Regards, Maciej -- -><- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- -------------- next part -------------- A non-text attachment was scrubbed... Name: update-returning.cset Type: application/octet-stream Size: 25175 bytes Desc: not available URL: From marijnh at gmail.com Wed Oct 22 14:46:17 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 22 Oct 2008 16:46:17 +0200 Subject: [postmodern-devel] (:nextval a-symbol) In-Reply-To: <1223929491.7962.36.camel@blackbox> References: <1223900121.7962.16.camel@blackbox> <1223929491.7962.36.camel@blackbox> Message-ID: Hello Lars, Your patch breaks passing strings to :nextval, which might be problematic for existing code. I've pushed a variation that allows both (:nextval "foo") and (:nextval 'foo). Best, Marijn From marijnh at gmail.com Wed Oct 22 14:49:39 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 22 Oct 2008 16:49:39 +0200 Subject: [postmodern-devel] [patch] ALTER TABLE, named constraints, typos In-Reply-To: <582EFE67-BA48-4D6F-971E-9B48DCE557B0@pasternacki.net> References: <582EFE67-BA48-4D6F-971E-9B48DCE557B0@pasternacki.net> Message-ID: Hey Maciej, Yes, the html files are the definite docs. Your new patch has also been pushed. Cheers, Marijn From vishnevsky at gmail.com Tue Oct 28 07:24:08 2008 From: vishnevsky at gmail.com (Yuri Vishnevsky) Date: Tue, 28 Oct 2008 09:24:08 +0200 Subject: [postmodern-devel] Incorrect :cast compilation Message-ID: <23ea12db0810280024j76d7fcas1662226d1dd558f1@mail.gmail.com> Hi. I've upgraded from postmodern-0.24 to 1.13 recently and noticed that :cast is incorectly escaped in sql expressions. (sql-compile '(:cast (:as value :BIGINT))) produces \"cast\"(value AS bigint) which results in postgres error. Removing "cast" from s-sql::*postgres-reserved-words* seems to fix the things. -------------- next part -------------- An HTML attachment was scrubbed... URL: From marijnh at gmail.com Tue Oct 28 07:55:39 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 28 Oct 2008 08:55:39 +0100 Subject: [postmodern-devel] Incorrect :cast compilation In-Reply-To: <23ea12db0810280024j76d7fcas1662226d1dd558f1@mail.gmail.com> References: <23ea12db0810280024j76d7fcas1662226d1dd558f1@mail.gmail.com> Message-ID: Hello Yuri, > (sql-compile '(:cast (:as value :BIGINT))) Aha -- I hadn't used CAST before, the fact that :as works here is kind of incidental. CAST does exactly the same as the '::' operator, though -- see the :type operator in S-SQL. Just removing 'cast' from the list of reserved words is not a good idea, since then you can't have rows named 'cast' in your tables anymore. I could add an operator like this... (def-sql-op :cast (value type) `("CAST(" ,@(sql-expand value) " AS " ,(to-type-name type) ")")) ... but maybe it makes more sense not to support both syntaxes and just use :type instead? Cheers, Marijn From vishnevsky at gmail.com Tue Oct 28 11:10:14 2008 From: vishnevsky at gmail.com (Yuri Vishnevsky) Date: Tue, 28 Oct 2008 13:10:14 +0200 Subject: [postmodern-devel] Incorrect :cast compilation In-Reply-To: References: <23ea12db0810280024j76d7fcas1662226d1dd558f1@mail.gmail.com> Message-ID: <23ea12db0810280410l6d8cf2c3k79c7cd4eaaa846aa@mail.gmail.com> Hello, Marijn Thanks for your reply. ... but maybe it makes more sense not to support both syntaxes and > just use :type instead? > > OK. I'll switch to using :type - it just works :) -------------- next part -------------- An HTML attachment was scrubbed... URL: