From zaries at global.co.za Wed Dec 3 14:46:44 2008 From: zaries at global.co.za (Phil) Date: Wed, 03 Dec 2008 16:46:44 +0200 Subject: [postmodern-devel] Dash in Postgresql Column name - "query-dao" Message-ID: <1228315604.29523.13.camel@scatha> query-dao does not seem to like column names with a dash in them like column-name and I suspect it would have the same problem with spaces. I know that the preffered for postgresql would be column_name but I though that being able to use the dash would simplify going from db to dao object to html and back, well at least for me. Is column names with dashes just not supported, a bug or on the wish list? Thanx From maciej at pasternacki.net Wed Dec 3 15:08:26 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Wed, 3 Dec 2008 16:08:26 +0100 Subject: [postmodern-devel] Dash in Postgresql Column name - "query-dao" In-Reply-To: <1228315604.29523.13.camel@scatha> References: <1228315604.29523.13.camel@scatha> Message-ID: <94597117-AC6A-4E25-9CA7-38E37D01EFC2@pasternacki.net> On 2008-12-03, at 15:46, Phil wrote: > query-dao does not seem to like column names with a dash in them like > column-name and I suspect it would have the same problem with spaces. > > I know that the preffered for postgresql would be column_name but I > though that being able to use the dash would simplify going from db to > dao object to html and back, well at least for me. > > Is column names with dashes just not supported, a bug or on the wish > list? Everything is supported with :RAW sql-op: (select-dao 'foo (:= (:raw "\"foo-bar\"") 23)) calls query (SELECT * FROM foo WHERE ("foo-bar" = 23)) (SELECT-DAO is a shortcut for QUERY-DAO, where you supply only the WHERE clause). There doesn't seem to exist any other way now, but it may be possible to add a bit of macro sugar. Marijn, have you thought of exporting S-SQL::DEF-SQL-OP, so that users can create their own "S-SQL macros"? This may come useful in some cases, like above, or to create reusable parametrized query fragments. With complex queries, macros may come in handy. Regards, Maciej -- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- http://www.3ofcoins.net/ From zaries at global.co.za Wed Dec 3 15:28:17 2008 From: zaries at global.co.za (Phil) Date: Wed, 03 Dec 2008 17:28:17 +0200 Subject: [postmodern-devel] Dash in Postgresql Column name - "query-dao" In-Reply-To: <94597117-AC6A-4E25-9CA7-38E37D01EFC2@pasternacki.net> References: <1228315604.29523.13.camel@scatha> <94597117-AC6A-4E25-9CA7-38E37D01EFC2@pasternacki.net> Message-ID: <1228318097.29523.19.camel@scatha> On Wed, 2008-12-03 at 16:08 +0100, Maciej Pasternacki wrote: > On 2008-12-03, at 15:46, Phil wrote: > > > query-dao does not seem to like column names with a dash in them like > > column-name and I suspect it would have the same problem with spaces. > > > > I know that the preffered for postgresql would be column_name but I > > though that being able to use the dash would simplify going from db to > > dao object to html and back, well at least for me. > > > > Is column names with dashes just not supported, a bug or on the wish > > list? > > Everything is supported with :RAW sql-op: (select-dao 'foo (:= (:raw > "\"foo-bar\"") 23)) calls query (SELECT * FROM foo WHERE ("foo-bar" = > 23)) (SELECT-DAO is a shortcut for QUERY-DAO, where you supply only > the WHERE clause). There doesn't seem to exist any other way now, but > it may be possible to add a bit of macro sugar. > > Marijn, have you thought of exporting S-SQL::DEF-SQL-OP, so that users > can create their own "S-SQL macros"? This may come useful in some > cases, like above, or to create reusable parametrized query > fragments. With complex queries, macros may come in handy. > > Regards, > Maciej > Ok maybe I should have been a bit more precise in my description. I was not using the columns names in the query. Here follows a quick repeatable test: CREATE TABLE test ( id integer NOT NULL, "some" character varying NOT NULL, "test-dash" character varying ) WITH (OIDS=FALSE); ALTER TABLE test OWNER TO postgres; insert into test (id,"some","test-dash") values(1, 'some','dash-1'); (defclass test () ((id :col-type string :initarg :id :accessor biographical-id) (some :col-type (or db-null string) :initarg :some :accessor biographical-some) (test-dash :col-type (or db-null string) :initarg :test-dash :accessor biographical-test-dash)) (:metaclass postmodern:dao-class) (:keys id)) (postmodern:query-dao 'test (:select '* :from 'test)) That last bit produces the following error: No slot named test-dash in class TEST. DAO out of sync with table, or incorrect query used. [Condition of type SIMPLE-ERROR] From maciej at pasternacki.net Wed Dec 3 16:16:57 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Wed, 3 Dec 2008 17:16:57 +0100 Subject: [postmodern-devel] Dash in Postgresql Column name - "query-dao" In-Reply-To: <1228318097.29523.19.camel@scatha> References: <1228315604.29523.13.camel@scatha> <94597117-AC6A-4E25-9CA7-38E37D01EFC2@pasternacki.net> <1228318097.29523.19.camel@scatha> Message-ID: On 2008-12-03, at 16:28, Phil wrote: >>> query-dao does not seem to like column names with a dash in them >>> like >>> column-name and I suspect it would have the same problem with >>> spaces. >>> >>> I know that the preffered for postgresql would be column_name but I >>> though that being able to use the dash would simplify going from >>> db to >>> dao object to html and back, well at least for me. >>> >>> Is column names with dashes just not supported, a bug or on the wish >>> list? >> >> Everything is supported with :RAW sql-op: (select-dao 'foo (:= (:raw >> "\"foo-bar\"") 23)) calls query (SELECT * FROM foo WHERE ("foo-bar" = >> 23)) (SELECT-DAO is a shortcut for QUERY-DAO, where you supply only >> the WHERE clause). There doesn't seem to exist any other way now, >> but >> it may be possible to add a bit of macro sugar. > > Ok maybe I should have been a bit more precise in my description. I > was > not using the columns names in the query. > > Here follows a quick repeatable test: > > CREATE TABLE test > ( > id integer NOT NULL, > "some" character varying NOT NULL, > "test-dash" character varying > ) > WITH (OIDS=FALSE); > ALTER TABLE test OWNER TO postgres; > > insert into test (id,"some","test-dash") values(1, 'some','dash-1'); > > (defclass test () > ((id :col-type string :initarg :id > :accessor biographical-id) > (some :col-type (or db-null string) :initarg :some > :accessor biographical-some) > (test-dash :col-type (or db-null string) :initarg :test-dash > :accessor biographical-test-dash)) > (:metaclass postmodern:dao-class) > (:keys id)) > > (postmodern:query-dao 'test (:select '* :from 'test)) > > That last bit produces the following error: > > No slot named test-dash in class TEST. DAO out of sync with table, or > incorrect query used. > [Condition of type SIMPLE-ERROR] I see. It was SELECT-DAO that made me think there's problem in the query. What you write about is not supported at the moment, and while it's easy to add this, it's not trivial. I just looked into postmodern/table.lisp and I see that in class DIRECT-COLUMN-SLOT there is a slot SQL-NAME. It would seem trivial to add an initarg, e.g. :COL-SQL-NAME, for this slot, and modify SHARED- INITIALIZE of DIRECT-COLUMN-SLOT to set this slot only if it's not already bound by an initarg. This way it would be possible to add :COL-SQL-NAME initarg to the slot to explicitly set column name, with upside being it wouldn't need to be a symbol. However, this slot seems to be rarely used, and there are many cases where simple TO-SQL-NAME is called on slot name, which would need to be fixed. Also, column name would need to be properly double-quoted everywhere. Your problem seems interesting and it looks like it would be useful to support your case. If all that was needed would be adding the initarg to :COL-SQL-NAME, the patch would be probably already attached to this post ;) but the way it is, I can only give these starting points so that you can fix it yourself. It shouldn't be hard, but testing the patch may be a bit time-consuming. Regards, Maciej. -- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- http://www.3ofcoins.net/ From zaries at global.co.za Wed Dec 3 16:59:53 2008 From: zaries at global.co.za (Phil) Date: Wed, 03 Dec 2008 18:59:53 +0200 Subject: [postmodern-devel] Dash in Postgresql Column name - "query-dao" In-Reply-To: References: <1228315604.29523.13.camel@scatha> <94597117-AC6A-4E25-9CA7-38E37D01EFC2@pasternacki.net> <1228318097.29523.19.camel@scatha> Message-ID: <1228323593.29523.28.camel@scatha> On Wed, 2008-12-03 at 17:16 +0100, Maciej Pasternacki wrote: > On 2008-12-03, at 16:28, Phil wrote: > > >>> query-dao does not seem to like column names with a dash in them > >>> like > >>> column-name and I suspect it would have the same problem with > >>> spaces. > >>> > >>> I know that the preffered for postgresql would be column_name but I > >>> though that being able to use the dash would simplify going from > >>> db to > >>> dao object to html and back, well at least for me. > >>> > >>> Is column names with dashes just not supported, a bug or on the wish > >>> list? > >> > >> Everything is supported with :RAW sql-op: (select-dao 'foo (:= (:raw > >> "\"foo-bar\"") 23)) calls query (SELECT * FROM foo WHERE ("foo-bar" = > >> 23)) (SELECT-DAO is a shortcut for QUERY-DAO, where you supply only > >> the WHERE clause). There doesn't seem to exist any other way now, > >> but > >> it may be possible to add a bit of macro sugar. > > > > Ok maybe I should have been a bit more precise in my description. I > > was > > not using the columns names in the query. > > > > Here follows a quick repeatable test: > > > > CREATE TABLE test > > ( > > id integer NOT NULL, > > "some" character varying NOT NULL, > > "test-dash" character varying > > ) > > WITH (OIDS=FALSE); > > ALTER TABLE test OWNER TO postgres; > > > > insert into test (id,"some","test-dash") values(1, 'some','dash-1'); > > > > (defclass test () > > ((id :col-type string :initarg :id > > :accessor biographical-id) > > (some :col-type (or db-null string) :initarg :some > > :accessor biographical-some) > > (test-dash :col-type (or db-null string) :initarg :test-dash > > :accessor biographical-test-dash)) > > (:metaclass postmodern:dao-class) > > (:keys id)) > > > > (postmodern:query-dao 'test (:select '* :from 'test)) > > > > That last bit produces the following error: > > > > No slot named test-dash in class TEST. DAO out of sync with table, or > > incorrect query used. > > [Condition of type SIMPLE-ERROR] > > > I see. It was SELECT-DAO that made me think there's problem in the > query. What you write about is not supported at the moment, and while > it's easy to add this, it's not trivial. > > I just looked into postmodern/table.lisp and I see that in class > DIRECT-COLUMN-SLOT there is a slot SQL-NAME. It would seem trivial to > add an initarg, e.g. :COL-SQL-NAME, for this slot, and modify SHARED- > INITIALIZE of DIRECT-COLUMN-SLOT to set this slot only if it's not > already bound by an initarg. This way it would be possible to > add :COL-SQL-NAME initarg to the slot to explicitly set column name, > with upside being it wouldn't need to be a symbol. > > However, this slot seems to be rarely used, and there are many cases > where simple TO-SQL-NAME is called on slot name, which would need to > be fixed. Also, column name would need to be properly double-quoted > everywhere. > > Your problem seems interesting and it looks like it would be useful to > support your case. If all that was needed would be adding the initarg > to :COL-SQL-NAME, the patch would be probably already attached to this > post ;) but the way it is, I can only give these starting points so > that you can fix it yourself. It shouldn't be hard, but testing the > patch may be a bit time-consuming. > > Regards, > Maciej. > Maciej thanx for the reply. Unfortunatley what I know about LISP is dangerous and what I know about CLOS is non existant (think I read to many of Paul Grahams articles, not a fan of objects...lol). I had to re-read your explanation a couple of times just to get the jist of it...lol I would be more than willing to do the grunt work if it would be something that could be incorperated into postmodern, but I would need a lot more guidence. I dont want to go hacking postmodern for myself (with bad lisp style) and then end up with a version that is broken if and when postmodern is updated. Regards Phil From marijnh at gmail.com Wed Dec 3 18:05:52 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 3 Dec 2008 19:05:52 +0100 Subject: [postmodern-devel] Dash in Postgresql Column name - "query-dao" In-Reply-To: <1228323593.29523.28.camel@scatha> References: <1228315604.29523.13.camel@scatha> <94597117-AC6A-4E25-9CA7-38E37D01EFC2@pasternacki.net> <1228318097.29523.19.camel@scatha> <1228323593.29523.28.camel@scatha> Message-ID: Hello Phil, Postmodern works this way because I wasn't even aware that one could use column names like that -- though actually, seeing the way quotes work, it's rather obvious. Currently the assumption 'dashes have to be converted to underscores' is present all through the system. Is support for such column names very important for you? My first recommendation would be to just rename the thing to use an underscore... but if you have a good reason to use a dash, I guess we'll have to look into another solution. Best, Marijn On Wed, Dec 3, 2008 at 5:59 PM, Phil wrote: > On Wed, 2008-12-03 at 17:16 +0100, Maciej Pasternacki wrote: >> On 2008-12-03, at 16:28, Phil wrote: >> >> >>> query-dao does not seem to like column names with a dash in them >> >>> like >> >>> column-name and I suspect it would have the same problem with >> >>> spaces. >> >>> >> >>> I know that the preffered for postgresql would be column_name but I >> >>> though that being able to use the dash would simplify going from >> >>> db to >> >>> dao object to html and back, well at least for me. >> >>> >> >>> Is column names with dashes just not supported, a bug or on the wish >> >>> list? >> >> >> >> Everything is supported with :RAW sql-op: (select-dao 'foo (:= (:raw >> >> "\"foo-bar\"") 23)) calls query (SELECT * FROM foo WHERE ("foo-bar" = >> >> 23)) (SELECT-DAO is a shortcut for QUERY-DAO, where you supply only >> >> the WHERE clause). There doesn't seem to exist any other way now, >> >> but >> >> it may be possible to add a bit of macro sugar. >> > >> > Ok maybe I should have been a bit more precise in my description. I >> > was >> > not using the columns names in the query. >> > >> > Here follows a quick repeatable test: >> > >> > CREATE TABLE test >> > ( >> > id integer NOT NULL, >> > "some" character varying NOT NULL, >> > "test-dash" character varying >> > ) >> > WITH (OIDS=FALSE); >> > ALTER TABLE test OWNER TO postgres; >> > >> > insert into test (id,"some","test-dash") values(1, 'some','dash-1'); >> > >> > (defclass test () >> > ((id :col-type string :initarg :id >> > :accessor biographical-id) >> > (some :col-type (or db-null string) :initarg :some >> > :accessor biographical-some) >> > (test-dash :col-type (or db-null string) :initarg :test-dash >> > :accessor biographical-test-dash)) >> > (:metaclass postmodern:dao-class) >> > (:keys id)) >> > >> > (postmodern:query-dao 'test (:select '* :from 'test)) >> > >> > That last bit produces the following error: >> > >> > No slot named test-dash in class TEST. DAO out of sync with table, or >> > incorrect query used. >> > [Condition of type SIMPLE-ERROR] >> >> >> I see. It was SELECT-DAO that made me think there's problem in the >> query. What you write about is not supported at the moment, and while >> it's easy to add this, it's not trivial. >> >> I just looked into postmodern/table.lisp and I see that in class >> DIRECT-COLUMN-SLOT there is a slot SQL-NAME. It would seem trivial to >> add an initarg, e.g. :COL-SQL-NAME, for this slot, and modify SHARED- >> INITIALIZE of DIRECT-COLUMN-SLOT to set this slot only if it's not >> already bound by an initarg. This way it would be possible to >> add :COL-SQL-NAME initarg to the slot to explicitly set column name, >> with upside being it wouldn't need to be a symbol. >> >> However, this slot seems to be rarely used, and there are many cases >> where simple TO-SQL-NAME is called on slot name, which would need to >> be fixed. Also, column name would need to be properly double-quoted >> everywhere. >> >> Your problem seems interesting and it looks like it would be useful to >> support your case. If all that was needed would be adding the initarg >> to :COL-SQL-NAME, the patch would be probably already attached to this >> post ;) but the way it is, I can only give these starting points so >> that you can fix it yourself. It shouldn't be hard, but testing the >> patch may be a bit time-consuming. >> >> Regards, >> Maciej. >> > > Maciej thanx for the reply. Unfortunatley what I know about LISP is > dangerous and what I know about CLOS is non existant (think I read to > many of Paul Grahams articles, not a fan of objects...lol). I had to > re-read your explanation a couple of times just to get the jist of > it...lol > > I would be more than willing to do the grunt work if it would be > something that could be incorperated into postmodern, but I would need a > lot more guidence. I dont want to go hacking postmodern for myself (with > bad lisp style) and then end up with a version that is broken if and > when postmodern is updated. > > Regards > Phil > > > > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > From xach at xach.com Wed Dec 3 19:16:03 2008 From: xach at xach.com (Zach Beane) Date: Wed, 3 Dec 2008 14:16:03 -0500 Subject: [postmodern-devel] Dash in Postgresql Column name - "query-dao" In-Reply-To: References: <1228315604.29523.13.camel@scatha> <94597117-AC6A-4E25-9CA7-38E37D01EFC2@pasternacki.net> <1228318097.29523.19.camel@scatha> <1228323593.29523.28.camel@scatha> Message-ID: <20081203191603.GJ28088@xach.com> On Wed, Dec 03, 2008 at 07:05:52PM +0100, Marijn Haverbeke wrote: > Hello Phil, > > Postmodern works this way because I wasn't even aware that one could > use column names like that -- though actually, seeing the way quotes > work, it's rather obvious. Currently the assumption 'dashes have to be > converted to underscores' is present all through the system. Is > support for such column names very important for you? My first > recommendation would be to just rename the thing to use an > underscore... but if you have a good reason to use a dash, I guess > we'll have to look into another solution. I can't comment on dashes specifically, but "foo" in SQL is like |foo| in Lisp. It's a sometimes-useful escape hatch to put all kinds of junk in table names, column names, etc. I can't say it's a *good* idea to make a table or column named "Hello, World %-)" but the double-quotes make it possible. Zach From zaries at global.co.za Wed Dec 3 20:07:01 2008 From: zaries at global.co.za (Phil) Date: Wed, 03 Dec 2008 22:07:01 +0200 Subject: [postmodern-devel] Dash in Postgresql Column name - "query-dao" In-Reply-To: References: <1228315604.29523.13.camel@scatha> <94597117-AC6A-4E25-9CA7-38E37D01EFC2@pasternacki.net> <1228318097.29523.19.camel@scatha> <1228323593.29523.28.camel@scatha> Message-ID: <1228334821.29523.51.camel@scatha> Hi Marijn The only reason I wanted to use the dashes is that it would have made my life slightly easier. I have a semi-generic program that takes a list of "descriptive names", "column names" and table and then produces a grid + detail widget in html. I was just trying to keep all the code that it generated lispy looking with the least amount of work. Usually I would not use the underscore in my DB syntax, just because all the qouting in stored procs/queries becomes a pain. This db will only be accessed via lisp so I decided it was worth the break from convention. For the future it might be worth it thow. Thanx for all the help. Regards Phil On Wed, 2008-12-03 at 19:05 +0100, Marijn Haverbeke wrote: > Hello Phil, > > Postmodern works this way because I wasn't even aware that one could > use column names like that -- though actually, seeing the way quotes > work, it's rather obvious. Currently the assumption 'dashes have to be > converted to underscores' is present all through the system. Is > support for such column names very important for you? My first > recommendation would be to just rename the thing to use an > underscore... but if you have a good reason to use a dash, I guess > we'll have to look into another solution. > > Best, > Marijn > > > On Wed, Dec 3, 2008 at 5:59 PM, Phil wrote: > > On Wed, 2008-12-03 at 17:16 +0100, Maciej Pasternacki wrote: > >> On 2008-12-03, at 16:28, Phil wrote: > >> > >> >>> query-dao does not seem to like column names with a dash in them > >> >>> like > >> >>> column-name and I suspect it would have the same problem with > >> >>> spaces. > >> >>> > >> >>> I know that the preffered for postgresql would be column_name but I > >> >>> though that being able to use the dash would simplify going from > >> >>> db to > >> >>> dao object to html and back, well at least for me. > >> >>> > >> >>> Is column names with dashes just not supported, a bug or on the wish > >> >>> list? > >> >> > >> >> Everything is supported with :RAW sql-op: (select-dao 'foo (:= (:raw > >> >> "\"foo-bar\"") 23)) calls query (SELECT * FROM foo WHERE ("foo-bar" = > >> >> 23)) (SELECT-DAO is a shortcut for QUERY-DAO, where you supply only > >> >> the WHERE clause). There doesn't seem to exist any other way now, > >> >> but > >> >> it may be possible to add a bit of macro sugar. > >> > > >> > Ok maybe I should have been a bit more precise in my description. I > >> > was > >> > not using the columns names in the query. > >> > > >> > Here follows a quick repeatable test: > >> > > >> > CREATE TABLE test > >> > ( > >> > id integer NOT NULL, > >> > "some" character varying NOT NULL, > >> > "test-dash" character varying > >> > ) > >> > WITH (OIDS=FALSE); > >> > ALTER TABLE test OWNER TO postgres; > >> > > >> > insert into test (id,"some","test-dash") values(1, 'some','dash-1'); > >> > > >> > (defclass test () > >> > ((id :col-type string :initarg :id > >> > :accessor biographical-id) > >> > (some :col-type (or db-null string) :initarg :some > >> > :accessor biographical-some) > >> > (test-dash :col-type (or db-null string) :initarg :test-dash > >> > :accessor biographical-test-dash)) > >> > (:metaclass postmodern:dao-class) > >> > (:keys id)) > >> > > >> > (postmodern:query-dao 'test (:select '* :from 'test)) > >> > > >> > That last bit produces the following error: > >> > > >> > No slot named test-dash in class TEST. DAO out of sync with table, or > >> > incorrect query used. > >> > [Condition of type SIMPLE-ERROR] > >> > >> > >> I see. It was SELECT-DAO that made me think there's problem in the > >> query. What you write about is not supported at the moment, and while > >> it's easy to add this, it's not trivial. > >> > >> I just looked into postmodern/table.lisp and I see that in class > >> DIRECT-COLUMN-SLOT there is a slot SQL-NAME. It would seem trivial to > >> add an initarg, e.g. :COL-SQL-NAME, for this slot, and modify SHARED- > >> INITIALIZE of DIRECT-COLUMN-SLOT to set this slot only if it's not > >> already bound by an initarg. This way it would be possible to > >> add :COL-SQL-NAME initarg to the slot to explicitly set column name, > >> with upside being it wouldn't need to be a symbol. > >> > >> However, this slot seems to be rarely used, and there are many cases > >> where simple TO-SQL-NAME is called on slot name, which would need to > >> be fixed. Also, column name would need to be properly double-quoted > >> everywhere. > >> > >> Your problem seems interesting and it looks like it would be useful to > >> support your case. If all that was needed would be adding the initarg > >> to :COL-SQL-NAME, the patch would be probably already attached to this > >> post ;) but the way it is, I can only give these starting points so > >> that you can fix it yourself. It shouldn't be hard, but testing the > >> patch may be a bit time-consuming. > >> > >> Regards, > >> Maciej. > >> > > > > Maciej thanx for the reply. Unfortunatley what I know about LISP is > > dangerous and what I know about CLOS is non existant (think I read to > > many of Paul Grahams articles, not a fan of objects...lol). I had to > > re-read your explanation a couple of times just to get the jist of > > it...lol > > > > I would be more than willing to do the grunt work if it would be > > something that could be incorperated into postmodern, but I would need a > > lot more guidence. I dont want to go hacking postmodern for myself (with > > bad lisp style) and then end up with a version that is broken if and > > when postmodern is updated. > > > > Regards > > Phil > > > > > > > > > > _______________________________________________ > > postmodern-devel mailing list > > postmodern-devel at common-lisp.net > > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > > > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel From zaries at global.co.za Thu Dec 11 14:59:22 2008 From: zaries at global.co.za (Phil) Date: Thu, 11 Dec 2008 16:59:22 +0200 Subject: [postmodern-devel] Building s-sql on the fly Message-ID: <1229007562.9412.7.camel@scatha> Hi I am looking for an example of how to create a s-sql query on the fly. I am sure I saw something like that along the line on some website but I cant find it again. Thanx From info at jensteich.de Thu Dec 11 15:05:05 2008 From: info at jensteich.de (Jens Teich) Date: Thu, 11 Dec 2008 16:05:05 +0100 Subject: [postmodern-devel] Building s-sql on the fly In-Reply-To: <1229007562.9412.7.camel@scatha> References: <1229007562.9412.7.camel@scatha> Message-ID: <49412C21.30309@jensteich.de> Phil schrieb: > Hi > > I am looking for an example of how to create a s-sql query on the fly. I > am sure I saw something like that along the line on some website but I > cant find it again. Are you looking for things like that: (defun element (&key guid guid-parent type-id type-not-equal) (let ((sql-guid (if guid (postmodern:sql (:= 'guid guid)) (postmodern:sql t))) (sql-guid-parent (if guid-parent (postmodern:sql (:= 'struct-guid-parent-node guid-parent)) (postmodern:sql t))) (sql-type-id (if type-id (postmodern:sql (:= 'type-id type-id)) (postmodern:sql t))) (sql-type-not-equal (if type-not-equal (postmodern:sql (:not (:= 'type-id type-not-equal))) (postmodern:sql t)))) (postmodern:query (:order-by (:select 'guid 'name 'type-id 'name-css :from 'projekt-elemente :where (:and (:raw sql-guid) (:raw sql-guid-parent) (:raw sql-type-id) (:raw sql-type-not-equal) (:= 'lang-id *lang*))) 'struct-sort-order)))) Jens From maciej at pasternacki.net Thu Dec 11 15:08:00 2008 From: maciej at pasternacki.net (Maciek Pasternacki) Date: Thu, 11 Dec 2008 16:08:00 +0100 Subject: [postmodern-devel] Building s-sql on the fly In-Reply-To: <1229007562.9412.7.camel@scatha> (Phil's message of "Thu\, 11 Dec 2008 16\:59\:22 +0200") References: <1229007562.9412.7.camel@scatha> Message-ID: Phil writes: > I am looking for an example of how to create a s-sql query on the fly. I > am sure I saw something like that along the line on some website but I > cant find it again. Take a look at http://repo.or.cz/w/cl-trane.git?a=blob;f=src/taxonomy.lisp;h=5776b2b42dde277f9478c0df621ffa9742106c06;hb=HEAD Query building takes place e.g. on lines 274-304 and 365-394. Second example may be interesting: it's a way to generate CASE statement from list of values (it may be generalized to receive alists, I just needed the order), to effectively make an inline-dictionary without writing the CASE statement by hand. Regards, Maciej. -- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- http://www.3ofcoins.net/ From zaries at global.co.za Thu Dec 11 15:32:04 2008 From: zaries at global.co.za (Phil) Date: Thu, 11 Dec 2008 17:32:04 +0200 Subject: [postmodern-devel] Building s-sql on the fly In-Reply-To: References: <1229007562.9412.7.camel@scatha> Message-ID: <1229009524.9412.11.camel@scatha> On Thu, 2008-12-11 at 16:08 +0100, Maciek Pasternacki wrote: > Phil writes: > > > I am looking for an example of how to create a s-sql query on the fly. I > > am sure I saw something like that along the line on some website but I > > cant find it again. > > Take a look at http://repo.or.cz/w/cl-trane.git?a=blob;f=src/taxonomy.lisp;h=5776b2b42dde277f9478c0df621ffa9742106c06;hb=HEAD > > Query building takes place e.g. on lines 274-304 and 365-394. Second > example may be interesting: it's a way to generate CASE statement from > list of values (it may be generalized to receive alists, I just needed > the order), to effectively make an inline-dictionary without writing > the CASE statement by hand. > > Regards, > Maciej. > Firstly thanx for the example. Secondly, this might be a stupid question but being new to lisp I would have expected those functions to be macro's you are using the `. Could you please explain the reasoning behind them being functions. From zaries at global.co.za Thu Dec 11 15:36:47 2008 From: zaries at global.co.za (Phil) Date: Thu, 11 Dec 2008 17:36:47 +0200 Subject: [postmodern-devel] Building s-sql on the fly In-Reply-To: <49412C21.30309@jensteich.de> References: <1229007562.9412.7.camel@scatha> <49412C21.30309@jensteich.de> Message-ID: <1229009807.9412.16.camel@scatha> On Thu, 2008-12-11 at 16:05 +0100, Jens Teich wrote: > Phil schrieb: > > Hi > > > > I am looking for an example of how to create a s-sql query on the fly. I > > am sure I saw something like that along the line on some website but I > > cant find it again. > > Are you looking for things like that: > > (defun element (&key guid guid-parent type-id type-not-equal) > (let ((sql-guid (if guid > (postmodern:sql (:= 'guid guid)) > (postmodern:sql t))) > (sql-guid-parent (if guid-parent > (postmodern:sql > (:= 'struct-guid-parent-node guid-parent)) > (postmodern:sql t))) > (sql-type-id (if type-id > (postmodern:sql (:= 'type-id type-id)) > (postmodern:sql t))) > (sql-type-not-equal (if type-not-equal > (postmodern:sql > (:not (:= 'type-id type-not-equal))) > (postmodern:sql t)))) > (postmodern:query (:order-by > (:select 'guid 'name 'type-id 'name-css > :from 'projekt-elemente > :where (:and > (:raw sql-guid) > (:raw sql-guid-parent) > (:raw sql-type-id) > (:raw sql-type-not-equal) > (:= 'lang-id *lang*))) > 'struct-sort-order)))) > > Jens > Thanx Jens that gives me some insight into a couple of things, but I dont know the amount of statements I will have in the where that is why I have to build the query dynamically. The example Maciej gave completes the puzzle. Phil From maciej at pasternacki.net Thu Dec 11 16:03:11 2008 From: maciej at pasternacki.net (Maciek Pasternacki) Date: Thu, 11 Dec 2008 17:03:11 +0100 Subject: [postmodern-devel] Building s-sql on the fly In-Reply-To: <1229009524.9412.11.camel@scatha> (Phil's message of "Thu\, 11 Dec 2008 17\:32\:04 +0200") References: <1229007562.9412.7.camel@scatha> <1229009524.9412.11.camel@scatha> Message-ID: Phil writes: >> > I am looking for an example of how to create a s-sql query on the fly. I >> > am sure I saw something like that along the line on some website but I >> > cant find it again. >> >> Take a look at http://repo.or.cz/w/cl-trane.git?a=blob;f=src/taxonomy.lisp;h=5776b2b42dde277f9478c0df621ffa9742106c06;hb=HEAD >> >> Query building takes place e.g. on lines 274-304 and 365-394. Second >> example may be interesting: it's a way to generate CASE statement from >> list of values (it may be generalized to receive alists, I just needed >> the order), to effectively make an inline-dictionary without writing >> the CASE statement by hand. > > Firstly thanx for the example. > > Secondly, this might be a stupid question but being new to lisp I would > have expected those functions to be macro's you are using the `. Could > you please explain the reasoning behind them being functions. Let's start from macros. Macros are just functions that are called after reading in the code (in macroexpansion time, to be exact), that take literal parts of code as arguments, and return other piece of code code in form of list as a return value. Let's take trivial macro: (defmacro add-two (form) `(+ 2 ,form)) (add-two (* 5 foo)) gets a list '(* 5 foo) as its argument, and macroexpands to (is equivalent to) (+ 2 (* 5 foo)). And macro definition is equivalent to building the resulting list by hand: (defmacro add-two (form) (list 'plus 2 form)) (actually, backtick would be more efficient, as ` is able to reuse some conses, but these are details now). I do the other way around: often I do need to construct a list from template, e.g. '(1 2 (4 3) ). I could write (list 1 2 (list 4 3) something), but I can use backtick as well: `(1 2 (3 4) ,something). Try to evaluate backtick-and-comma forms in REPL, play around and see yourself how it works. Backtick is "quasi-quote": it quotes everything that is not preceded by comma, or comma-at (,@), and it's most widely used in macros (since these are usually templates - but don't actually need to be), but it can be used to build complicated nested lists from templates inside functions as well. Hope this helps, regards, Maciej. -- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- http://www.3ofcoins.net/ From marijnh at gmail.com Thu Dec 11 15:05:12 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Thu, 11 Dec 2008 16:05:12 +0100 Subject: [postmodern-devel] Building s-sql on the fly In-Reply-To: <1229007562.9412.7.camel@scatha> References: <1229007562.9412.7.camel@scatha> Message-ID: Hello Phil, Maybe you mean sql-compile? http://common-lisp.net/project/postmodern/s-sql.html#sql-compile Best, Marijn From zaries at global.co.za Thu Dec 11 18:58:27 2008 From: zaries at global.co.za (Phil) Date: Thu, 11 Dec 2008 20:58:27 +0200 Subject: [postmodern-devel] Building s-sql on the fly In-Reply-To: References: <1229007562.9412.7.camel@scatha> <1229009524.9412.11.camel@scatha> Message-ID: <1229021907.9412.58.camel@scatha> On Thu, 2008-12-11 at 17:03 +0100, Maciek Pasternacki wrote: > Phil writes: > > >> > I am looking for an example of how to create a s-sql query on the fly. I > >> > am sure I saw something like that along the line on some website but I > >> > cant find it again. > >> > >> Take a look at http://repo.or.cz/w/cl-trane.git?a=blob;f=src/taxonomy.lisp;h=5776b2b42dde277f9478c0df621ffa9742106c06;hb=HEAD > >> > >> Query building takes place e.g. on lines 274-304 and 365-394. Second > >> example may be interesting: it's a way to generate CASE statement from > >> list of values (it may be generalized to receive alists, I just needed > >> the order), to effectively make an inline-dictionary without writing > >> the CASE statement by hand. > > > > Firstly thanx for the example. > > > > Secondly, this might be a stupid question but being new to lisp I would > > have expected those functions to be macro's you are using the `. Could > > you please explain the reasoning behind them being functions. > > Let's start from macros. Macros are just functions that are called > after reading in the code (in macroexpansion time, to be exact), that > take literal parts of code as arguments, and return other piece of > code code in form of list as a return value. > > Let's take trivial macro: > > (defmacro add-two (form) > `(+ 2 ,form)) > > (add-two (* 5 foo)) gets a list '(* 5 foo) as its argument, and > macroexpands to (is equivalent to) (+ 2 (* 5 foo)). > > And macro definition is equivalent to building the resulting list by hand: > > (defmacro add-two (form) > (list 'plus 2 form)) > > (actually, backtick would be more efficient, as ` is able to reuse > some conses, but these are details now). > > I do the other way around: often I do need to construct a list from > template, e.g. '(1 2 (4 3) ). I could write (list 1 2 > (list 4 3) something), but I can use backtick as well: `(1 2 (3 4) > ,something). Try to evaluate backtick-and-comma forms in REPL, play > around and see yourself how it works. Backtick is "quasi-quote": it > quotes everything that is not preceded by comma, or comma-at (,@), and > it's most widely used in macros (since these are usually templates - > but don't actually need to be), but it can be used to build > complicated nested lists from templates inside functions as well. > > Hope this helps, > regards, > Maciej. > And all the behaviorists chorused AH_HAAAAA!! Here is what how I understand this: 1. ` creates a list 2. Where the reader finds a , it evalutes what follows before the list is build, thus the result/value of what follows the , is passed into the list. ;Example not using a , CL-USER> `(1 2 3 4 (format nil "5")) (1 2 3 4 (FORMAT NIL "5")) ;Example using a , CL-USER> `(1 2 3 4 ,(format nil "5")) (1 2 3 4 "5") ;The following is the equavilant of the preceding but it uses list and not the shorthand ` CL-USER> (list 1 3 4 (format nil "5")) (1 3 4 "5") 3. The difference between using ` in a macro and a function is the timing of when the list is build and thus the time that stuff after a , is evaluated. In a macro the list is build at compile time and the evaluation of stuff after a , is also done at compile time. CL-USER> (defmacro test-macro () `(+ 1 2 3 4 ,(parse-integer "5"))) TEST-MACRO ;Simulate at compile time CL-USER> (macroexpand '(test-macro)) (+ 1 2 3 4 5) T ;When run CL-USER> (test-macro) 15 From tsufiev at gmail.com Tue Dec 16 17:42:02 2008 From: tsufiev at gmail.com (Timur Sufiev) Date: Tue, 16 Dec 2008 20:42:02 +0300 Subject: [postmodern-devel] s-sql patch for postmodern 1.13: separate condition Message-ID: <289a24fb0812160942h2b6ae8b2g96be42533097236d@mail.gmail.com> We learned that it is useful to a) have separate condition to distinguish s-sql parsing errors from others b) have :delete-from syntax extended: sometimes it is necessary to return keys of deleted rows. Both patches (generated by git) are in attach. It would be very nice to see them in upstream. -------------- next part -------------- A non-text attachment was scrubbed... Name: s-sql.delete-from.patch Type: application/octet-stream Size: 1021 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: s-sql.sql-error.patch Type: application/octet-stream Size: 10496 bytes Desc: not available URL: From marijnh at gmail.com Wed Dec 17 07:55:11 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 17 Dec 2008 08:55:11 +0100 Subject: [postmodern-devel] s-sql patch for postmodern 1.13: separate condition In-Reply-To: <289a24fb0812160942h2b6ae8b2g96be42533097236d@mail.gmail.com> References: <289a24fb0812160942h2b6ae8b2g96be42533097236d@mail.gmail.com> Message-ID: Hello Timur, Thanks for the patches! The :delete-from one is perfect. The sql-error one is a good idea, but your patch is completely broken (you can't do (error 'sql-error "~a" val)). I've committed something similar to the repository. (Also, submitting patches created by 'darcs send -o patchfilename' makes it easier for me to merge them.) Cheers, Marijn From tsufiev at gmail.com Wed Dec 17 11:54:49 2008 From: tsufiev at gmail.com (Timur Sufiev) Date: Wed, 17 Dec 2008 14:54:49 +0300 Subject: [postmodern-devel] s-sql patch for postmodern 1.13: separate condition In-Reply-To: References: <289a24fb0812160942h2b6ae8b2g96be42533097236d@mail.gmail.com> Message-ID: <289a24fb0812170354q436bfd61hf501022743ddc885@mail.gmail.com> Marjin, thanks for a sql-error nice patch, our one really contained the find&replace-typo. 2008/12/17 Marijn Haverbeke : > Hello Timur, > > Thanks for the patches! The :delete-from one is perfect. The sql-error > one is a good idea, but your patch is completely broken (you can't do > (error 'sql-error "~a" val)). I've committed something similar to the > repository. (Also, submitting patches created by 'darcs send -o > patchfilename' makes it easier for me to merge them.) > > Cheers, > Marijn > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > From maciej at pasternacki.net Sun Dec 21 22:50:30 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Sun, 21 Dec 2008 23:50:30 +0100 Subject: [postmodern-devel] Patch: query callback support Message-ID: <4CDEA1D3-9B2F-4B3F-878F-7ECD6D873C3E@pasternacki.net> Hello, Recently I wanted to start profiling my webapp and display some stats in footer, namely number of queries and total time spent waiting for the database. Only option provided by Postmodern was using CL- POSTGRES:*QUERY-LOG* and reparsing its output, but this would be just awful Perl-like approach, and the time given would be very inaccurate (most of my queries are short, and query log rounds time to full milliseconds, so most queries show as 0ms). I added *QUERY-CALLBACK* support for cl-postgres, which makes CL-Postgres call given function with query string and internal time units spent after every query, moved *QUERY-LOG* support to separate function LOG-QUERY, which I made default value of *QUERY-CALLBACK* for backwards compatibility. Darcs changeset is attached. Regards, Maciej. -- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- http://www.3ofcoins.net/ -------------- next part -------------- A non-text attachment was scrubbed... Name: query-callback.cset.gz Type: application/x-gzip Size: 9173 bytes Desc: not available URL: From marijnh at gmail.com Mon Dec 22 11:34:33 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 22 Dec 2008 12:34:33 +0100 Subject: [postmodern-devel] Patch: query callback support In-Reply-To: <4CDEA1D3-9B2F-4B3F-878F-7ECD6D873C3E@pasternacki.net> References: <4CDEA1D3-9B2F-4B3F-878F-7ECD6D873C3E@pasternacki.net> Message-ID: Hey Maciej, Thanks for the patch. Has been applied. I think I will remove *query-log* in a future version, it was just a debugging hack anyway, and this callback system is more general and pleasant. Best, Marijn 2008/12/21 Maciej Pasternacki : > Hello, > > Recently I wanted to start profiling my webapp and display some stats in > footer, namely number of queries and total time spent waiting for the > database. Only option provided by Postmodern was using > CL-POSTGRES:*QUERY-LOG* and reparsing its output, but this would be just > awful Perl-like approach, and the time given would be very inaccurate (most > of my queries are short, and query log rounds time to full milliseconds, so > most queries show as 0ms). I added *QUERY-CALLBACK* support for > cl-postgres, which makes CL-Postgres call given function with query string > and internal time units spent after every query, moved *QUERY-LOG* support > to separate function LOG-QUERY, which I made default value of > *QUERY-CALLBACK* for backwards compatibility. > > Darcs changeset is attached. > > Regards, > Maciej. > > -- > Maciej Pasternacki -><- http://www.pasternacki.net/ -><- > http://www.3ofcoins.net/ > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel > > From maciej at pasternacki.net Mon Dec 22 22:56:48 2008 From: maciej at pasternacki.net (Maciej Pasternacki) Date: Mon, 22 Dec 2008 23:56:48 +0100 Subject: [postmodern-devel] Patch: query callback support In-Reply-To: References: <4CDEA1D3-9B2F-4B3F-878F-7ECD6D873C3E@pasternacki.net> Message-ID: <0244BDEA-A169-4C5E-B571-392AD58D2061@pasternacki.net> *QUERY-LOG* is actually quite convenient debugging hack, it's really useful to have it on and see actual queries scrolling on Slime REPL, especially when I generate s-sql on the fly. It may be useful to keep it, only maybe change default *QUERY-CALLBACK* to NIL to avoid the call overhead when default *QUERY-LOG* is NIL anyway. Regards, Maciej. On 2008-12-22, at 12:34, Marijn Haverbeke wrote: > Hey Maciej, > > Thanks for the patch. Has been applied. I think I will remove > *query-log* in a future version, it was just a debugging hack anyway, > and this callback system is more general and pleasant. > > Best, > Marijn > > > 2008/12/21 Maciej Pasternacki : >> Hello, >> >> Recently I wanted to start profiling my webapp and display some >> stats in >> footer, namely number of queries and total time spent waiting for the >> database. Only option provided by Postmodern was using >> CL-POSTGRES:*QUERY-LOG* and reparsing its output, but this would be >> just >> awful Perl-like approach, and the time given would be very >> inaccurate (most >> of my queries are short, and query log rounds time to full >> milliseconds, so >> most queries show as 0ms). I added *QUERY-CALLBACK* support for >> cl-postgres, which makes CL-Postgres call given function with query >> string >> and internal time units spent after every query, moved *QUERY-LOG* >> support >> to separate function LOG-QUERY, which I made default value of >> *QUERY-CALLBACK* for backwards compatibility. >> >> Darcs changeset is attached. >> >> Regards, >> Maciej. >> >> -- >> Maciej Pasternacki -><- http://www.pasternacki.net/ -><- >> http://www.3ofcoins.net/ >> >> _______________________________________________ >> postmodern-devel mailing list >> postmodern-devel at common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel >> >> > > _______________________________________________ > postmodern-devel mailing list > postmodern-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel -- Maciej Pasternacki -><- http://www.pasternacki.net/ -><- http://www.3ofcoins.net/ From sabra.crolleton at gmail.com Sun Dec 28 22:20:24 2008 From: sabra.crolleton at gmail.com (Sabra Crolleton) Date: Sun, 28 Dec 2008 14:20:24 -0800 Subject: [postmodern-devel] Looking for hints on converting from clsql to postmodern Message-ID: <200812281420.24911.sabra.crolleton@gmail.com> One of our postgresql databases has a heavy proportion of foreign language text (all encoded in utf-8). ?I was starting to learn lisp by porting a webapplication and got part way into it before I realized that using clsql may require me to manually encode and decode everything from latin to utf-8. It was suggested that postmodern may be a way out of that. The app is still in early stages, so I'm hoping that converting from clsql to postmodern won't be too bad, but any hints on things to watch out would be appreciated. Sabra From marijnh at gmail.com Mon Dec 29 07:38:36 2008 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 29 Dec 2008 08:38:36 +0100 Subject: [postmodern-devel] Looking for hints on converting from clsql to postmodern In-Reply-To: <200812281420.24911.sabra.crolleton@gmail.com> References: <200812281420.24911.sabra.crolleton@gmail.com> Message-ID: Hi Sabra, It has been a while since I've used CLSQL, but converting should be relatively straightforward. If you're using 'join' objects, you'll have to overhaul that, since Postmodern DAOs work completely differencly (they don't join, for one thing). Also look into the database-connection-error (http://common-lisp.net/project/postmodern/cl-postgres.html#database-connection-error) condition and :reconnect restart if you want to make something that doesn't crash when there are connection problems. ... Basically, just skim the documentation, and you should be fine. Best, Marijn