From marijnh at gmail.com  Sat Sep  1 14:17:17 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Sat, 1 Sep 2007 16:17:17 +0200
Subject: [postmodern-devel] Submarine announcement
In-Reply-To: <19e19e410708300213i43bfea1av430e6004865f90b6@mail.gmail.com>
References: <19e19e410708280849q41a1a64dwd893085f77f39f60@mail.gmail.com>
	<befaa7aa0708290317y6c265e23v23dea89a019eff5@mail.gmail.com>
	<19e19e410708300213i43bfea1av430e6004865f90b6@mail.gmail.com>
Message-ID: <befaa7aa0709010717w19f93cbfge8c39241cb6e85d0@mail.gmail.com>

Here is one way in which you could approach it: On the Lisp side, you
can just let people define tables in any order, since non-existant
foreign targets aren't a problem there. When creating a whole
'database schema' at once in the database, just create the tables
first, and then the keys, and let any errors generated by non-existant
tables go through. When creating or resetting a single table, if it
has any foreign keys into tables that do not exist, check whether such
a table is defined on the Lisp side -- if it is, raise an exception
with a 'create this table' restart, otherwise, raise an unrecoverable
exception.

(This might, of course, have some problems that I haven't thought of
yet, but I think it would be workable.)

Will submarine also have possibilities to define sequences, stored
procs, and views? It would be cool to have a single place to define
whole db schemas in Lisp, and then have convenient ways to create them
all at once in an actual database. -- And of course, the hypothetical
'automatically detect differences between the schema defined in Lisp
and the one existing in the database and interactively update the
database' functionality might also be something you can think about --
if Jaap allows you to waste another two months on this, hah.

Cheers,
Marijn


From ryszard.szopa at gmail.com  Sun Sep  2 15:45:40 2007
From: ryszard.szopa at gmail.com (Ryszard Szopa)
Date: Sun, 2 Sep 2007 17:45:40 +0200
Subject: [postmodern-devel] Submarine announcement
In-Reply-To: <befaa7aa0709010717w19f93cbfge8c39241cb6e85d0@mail.gmail.com>
References: <19e19e410708280849q41a1a64dwd893085f77f39f60@mail.gmail.com>
	<befaa7aa0708290317y6c265e23v23dea89a019eff5@mail.gmail.com>
	<19e19e410708300213i43bfea1av430e6004865f90b6@mail.gmail.com>
	<befaa7aa0709010717w19f93cbfge8c39241cb6e85d0@mail.gmail.com>
Message-ID: <19e19e410709020845h5440513el24e782d56ba0bf@mail.gmail.com>

On 9/1/07, Marijn Haverbeke <marijnh at gmail.com> wrote:
> Here is one way in which you could approach it: On the Lisp side, you
> can just let people define tables in any order, since non-existant
> foreign targets aren't a problem there. When creating a whole
> 'database schema' at once in the database, just create the tables
> first, and then the keys, and let any errors generated by non-existant
> tables go through. When creating or resetting a single table, if it
> has any foreign keys into tables that do not exist, check whether such
> a table is defined on the Lisp side -- if it is, raise an exception
> with a 'create this table' restart, otherwise, raise an unrecoverable
> exception.
>
> (This might, of course, have some problems that I haven't thought of
> yet, but I think it would be workable.)

I was thinking about something similar, but I was afraid of one
situation. Imagine the user actually forgets to define a class he
references to. Swallowing all the db errors would mean that he has
something that seems to work, and only in some specific situations
will start behaving strangely. This may lead to errors very hard to
debug.

Also, the creation of an SQL table is a shared-initialize :after
method of db-class. This means that there's no need of an additional
command to creating a table or all the tables -- it's all done at
definition time. The drawback of this approach is that I cannot
compute the dependencies between tables and reorder them so that
Postgres is happy.

On Friday I've implemented nearly exactly what I have written in my
letter from Thursday (when there's a non-existing referenced table
problem, the constraint adding function is stored; after the
initialization of each new db-class, all delayed constraints are
tried). And, apparently, it seems to work. When the user tries to make
an instance of a class that has unresolved dependencies, submarine
raises an error.

> Will submarine also have possibilities to define sequences, stored
> procs, and views? It would be cool to have a single place to define
> whole db schemas in Lisp, and then have convenient ways to create them
> all at once in an actual database. -- And of course, the hypothetical
> 'automatically detect differences between the schema defined in Lisp
> and the one existing in the database and interactively update the
> database' functionality might also be something you can think about --
> if Jaap allows you to waste another two months on this, hah.

I think Jaap won't mind, as I am going back to Poland next Friday.
(Uh-oh... That also means he'll stop paying me... oh, rats... ;))

The "checking if the table in the database complies to the
specification of the db-class" of submarine is nearly finished
(checking constraints is the only thing left to be written).

As for my plans for the future: I will be thinking about inheritance
(fortunately Postgres supports multiple inheritance), schemata (I
think they should be somehow related CL packages, because they seem to
be doing a very similar job). As for stored procs: I am not really
sure what could I do with them (it's a pity Postgres doesn't support
writing procedures in CL, like it does for Scheme).

Also, I would find it very fortunate if S-SQL supported ALTER,
constraints and stuff like that (I am using just quick hacks to
generate the appropriate SQL code right now, and I can't say to be
very happy about it). As you are on vacation, it probably means I
should do it myself. :P

BTW: how did it take for you to get a project at c-l.net? Because I
sent them an e-mail as they wanted and I still haven't received any
answer.

Bests,
    -- Richard
-- 
http://szopa.tasak.gda.pl/


From marijnh at gmail.com  Sun Sep  2 17:44:41 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Mon, 3 Sep 2007 05:44:41 +1200
Subject: [postmodern-devel] Submarine announcement
In-Reply-To: <19e19e410709020845h5440513el24e782d56ba0bf@mail.gmail.com>
References: <19e19e410708280849q41a1a64dwd893085f77f39f60@mail.gmail.com>
	<befaa7aa0708290317y6c265e23v23dea89a019eff5@mail.gmail.com>
	<19e19e410708300213i43bfea1av430e6004865f90b6@mail.gmail.com>
	<befaa7aa0709010717w19f93cbfge8c39241cb6e85d0@mail.gmail.com>
	<19e19e410709020845h5440513el24e782d56ba0bf@mail.gmail.com>
Message-ID: <befaa7aa0709021044i4cf0c906rd01cd8dcbb901ff8@mail.gmail.com>

On 9/3/07, Ryszard Szopa <ryszard.szopa at gmail.com> wrote:
> I was thinking about something similar, but I was afraid of one
> situation. Imagine the user actually forgets to define a class he
> references to. Swallowing all the db errors would mean that he has
> something that seems to work, and only in some specific situations
> will start behaving strangely. This may lead to errors very hard to
> debug.

Note that I didn't talk about swallowing any db errors -- my idea was
to just fail to create the table entirely when some dependencies were
missing.

> Also, the creation of an SQL table is a shared-initialize :after
> method of db-class. This means that there's no need of an additional
> command to creating a table or all the tables -- it's all done at
> definition time. The drawback of this approach is that I cannot
> compute the dependencies between tables and reorder them so that
> Postgres is happy.

I think this is an awful idea -- keeping the definition and the actual
creation of tables separately allows a flexibility which can save you
a ton of headaches.

> Also, I would find it very fortunate if S-SQL supported ALTER,
> constraints and stuff like that (I am using just quick hacks to
> generate the appropriate SQL code right now, and I can't say to be
> very happy about it). As you are on vacation, it probably means I
> should do it myself. :P

It is something I didn't need often enough to bother about (I tend to
write alter table commands directly at an SQL prompt). Also, alter is
quite a complicated statement. But if you write support for it I will,
of course, be happy to add it to the repo.

> BTW: how did it take for you to get a project at c-l.net? Because I
> sent them an e-mail as they wanted and I still haven't received any
> answer.

Took a while before I received a response as well -- if I remember
correctly the owner was on some kind of trip at the time. But once I
got a response, everything was handled smoothly and quickly.

Cheers,
Marijn


From ryszard.szopa at gmail.com  Mon Sep  3 10:47:20 2007
From: ryszard.szopa at gmail.com (Ryszard Szopa)
Date: Mon, 3 Sep 2007 12:47:20 +0200
Subject: [postmodern-devel] Submarine announcement
In-Reply-To: <befaa7aa0709021044i4cf0c906rd01cd8dcbb901ff8@mail.gmail.com>
References: <19e19e410708280849q41a1a64dwd893085f77f39f60@mail.gmail.com>
	<befaa7aa0708290317y6c265e23v23dea89a019eff5@mail.gmail.com>
	<19e19e410708300213i43bfea1av430e6004865f90b6@mail.gmail.com>
	<befaa7aa0709010717w19f93cbfge8c39241cb6e85d0@mail.gmail.com>
	<19e19e410709020845h5440513el24e782d56ba0bf@mail.gmail.com>
	<befaa7aa0709021044i4cf0c906rd01cd8dcbb901ff8@mail.gmail.com>
Message-ID: <19e19e410709030347o157638a9yd75e5b6b025e735b@mail.gmail.com>

On 9/2/07, Marijn Haverbeke <marijnh at gmail.com> wrote:

> > Also, the creation of an SQL table is a shared-initialize :after
> > method of db-class. This means that there's no need of an additional
> > command to creating a table or all the tables -- it's all done at
> > definition time. The drawback of this approach is that I cannot
> > compute the dependencies between tables and reorder them so that
> > Postgres is happy.
>
> I think this is an awful idea -- keeping the definition and the actual
> creation of tables separately allows a flexibility which can save you
> a ton of headaches.

Well, for me it works fine this way. Also, this is only the default
behavior. If you don't provide a connection spec to defdao (macro
creating a subclass of dao with the metaclass set do db-class), then
no SQL queries will be executed (I use this feature for making
fixtures and base classes on the Lisp side that shouldn't have any
effect on the Postgres side).

Do you think this is enough for saving that ton of headaches? :) If it
is not, please don't hesitate to give examples of problems that may
arise, and I will try to do something about them.

Cheers,

    -- Richard

-- 
http://szopa.tasak.gda.pl/


From marijnh at gmail.com  Tue Sep  4 14:24:02 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Tue, 4 Sep 2007 16:24:02 +0200
Subject: [postmodern-devel] Submarine announcement
In-Reply-To: <19e19e410709030347o157638a9yd75e5b6b025e735b@mail.gmail.com>
References: <19e19e410708280849q41a1a64dwd893085f77f39f60@mail.gmail.com>
	<befaa7aa0708290317y6c265e23v23dea89a019eff5@mail.gmail.com>
	<19e19e410708300213i43bfea1av430e6004865f90b6@mail.gmail.com>
	<befaa7aa0709010717w19f93cbfge8c39241cb6e85d0@mail.gmail.com>
	<19e19e410709020845h5440513el24e782d56ba0bf@mail.gmail.com>
	<befaa7aa0709021044i4cf0c906rd01cd8dcbb901ff8@mail.gmail.com>
	<19e19e410709030347o157638a9yd75e5b6b025e735b@mail.gmail.com>
Message-ID: <befaa7aa0709040724x46acbf6fna106e67320f6b50a@mail.gmail.com>

> Do you think this is enough for saving that ton of headaches? :) If it
> is not, please don't hesitate to give examples of problems that may
> arise, and I will try to do something about them.

Will Submarine allow DAO-style classes to be defined for the created
tables? These classes will have to defined every time the program is
compiled/used, while the database tables only have to be created when
a new DB is initialized. Also, the problem with the foreign keys to
non-existant tables seems a lot more manageable to me if definition
and creation are two different commands. But I haven't thought deeply
about any of this, so just do what seems workable to you.

Marijn


From ryszard.szopa at gmail.com  Wed Sep  5 14:15:01 2007
From: ryszard.szopa at gmail.com (Ryszard Szopa)
Date: Wed, 5 Sep 2007 16:15:01 +0200
Subject: [postmodern-devel] postmodern patch for CMUCL
Message-ID: <19e19e410709050714n4d1278f8od2be086391c64673@mail.gmail.com>

Hi,

I wanted to see how submarine works under CMUCL, and it failed all the
tests. It turned out, however, that the blame is on the side of
Postmodern: it couldn't connect to the database, and also failed all
its tests. The problem was that in some place it was trying to compare
with >= a number and nil.

With some effort I managed to find the guilty piece of code. It was
read-ascii-string in communicate.lisp. In that function, if
byte-length is not nil, it is compared with bytes-read. Byte-length,
however, was declared as fixnum. This probably caused CMUCL to
optimized it away as not-NIL in any conditional expressions (imagine
my surpise when evaluating

(when byte-length (break "bytes-read: ~A byte-length: ~A" bytes-read
byte-length))

threw a break and showed byte-length to be nil :))

Also, binding the utf-8 stuff in package.lisp was indeed an awful
hack... It's been a long time since I've used grep on source code to
find who calls a function (yeah, slime is addicting;).

Now Postmodern passes on CMUCL all its tests except two, table and
transaction. I'll take a look at them if I have time.

Cheers,

    -- Richard

-- 
http://szopa.tasak.gda.pl/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cmucl_bugfix.patch
Type: text/x-patch
Size: 5594 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/postmodern-devel/attachments/20070905/cbbe396f/attachment.bin>

From marijnh at gmail.com  Fri Sep  7 07:54:57 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Fri, 7 Sep 2007 09:54:57 +0200
Subject: [postmodern-devel] postmodern patch for CMUCL
In-Reply-To: <19e19e410709050714n4d1278f8od2be086391c64673@mail.gmail.com>
References: <19e19e410709050714n4d1278f8od2be086391c64673@mail.gmail.com>
Message-ID: <befaa7aa0709070054w74cf62d1g63f175012821671e@mail.gmail.com>

Thanks! Will be applied as soon as I can get a secure shell running somewhere.

Cheers,
Marijn


From marijnh at gmail.com  Mon Sep 17 12:48:34 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Mon, 17 Sep 2007 14:48:34 +0200
Subject: [postmodern-devel] query with keyword :on
In-Reply-To: <002a01c7e370$4f0af930$6e02a8c0@JensSony>
References: <010401c7dc2e$20e58eb0$6b02a8c0@JensSony>
	<befaa7aa0708130121n315e21afuf337518bacfe65d5@mail.gmail.com>
	<023901c7ddcf$7d2a6e50$6b02a8c0@JensSony>
	<befaa7aa0708160021h5a96511bw541ed7e3c997b280@mail.gmail.com>
	<015b01c7e059$79b5feb0$6e02a8c0@JensSony>
	<befaa7aa0708190445l1e83105euf9a414b49a49917@mail.gmail.com>
	<002a01c7e370$4f0af930$6e02a8c0@JensSony>
Message-ID: <befaa7aa0709170548o637da40eifceb1e32f7203ed1@mail.gmail.com>

Hello Jens  (and other lispworks users),

I finally got a chance to do some coding, and I pushed a patch that
should fix this problem -- could you try it out and tell me if it
works at all. The only lisp interpreter I had available was Franz'
telnet repl, so testing has been minimal, but the changes weren't that
drastic, so there is a not unreasonable chance that they are correct.

Cheers,
Marijn


From marijnh at gmail.com  Mon Sep 17 13:22:38 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Mon, 17 Sep 2007 15:22:38 +0200
Subject: [postmodern-devel] with-connection problem
In-Reply-To: <19e19e410708170805o4a28f882u61b4f783c0be9fab@mail.gmail.com>
References: <01a301c7e0c0$7906cd20$6e02a8c0@JensSony>
	<19e19e410708170805o4a28f882u61b4f783c0be9fab@mail.gmail.com>
Message-ID: <befaa7aa0709170622m3239a739sb417b2ec4023eeb7@mail.gmail.com>

Hey Ryszard,

I didn't actually read your mail very well last time -- your
suggestion wasn't an incompatible change. The function is a good idea,
I added it. I also realized how completely useless my take on
with-connection was in any kind of serious application, so I also made
a with-connection* macro which does evaluate its
connection-specification.

At least, I think I did. The changes are completely untested -- could
you kindly test them and tell me how it goes?

Cheers,
Marijn

(BTW, does anyone know what's up with the darcsweb thingy for
http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=postmodern-postmodern;a=summary
? It does seem to work for other projects, but postmodern breaks it...
which is a shame.)


From marijnh at gmail.com  Mon Sep 17 13:28:29 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Mon, 17 Sep 2007 15:28:29 +0200
Subject: [postmodern-devel] with-connection problem
In-Reply-To: <befaa7aa0709170622m3239a739sb417b2ec4023eeb7@mail.gmail.com>
References: <01a301c7e0c0$7906cd20$6e02a8c0@JensSony>
	<19e19e410708170805o4a28f882u61b4f783c0be9fab@mail.gmail.com>
	<befaa7aa0709170622m3239a739sb417b2ec4023eeb7@mail.gmail.com>
Message-ID: <befaa7aa0709170628x16d129e7je3540aa88c30fda7@mail.gmail.com>

> (BTW, does anyone know what's up with the darcsweb thingy for
> http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=postmodern-postmodern;a=summary
> ? It does seem to work for other projects, but postmodern breaks it...
> which is a shame.)

Never mind -- it was a weird Croatian letter that I accidentally added
to a patch description while typing on a Croatian keyboard. I
unrecorded it and replaced it by a patch with a regular ascii name,
and everything works again.

Lesson: Darcsweb does not do unicode very well.

Cheers,
Marijn


From info at jensteich.de  Mon Sep 17 19:51:25 2007
From: info at jensteich.de (Jens Teich)
Date: Mon, 17 Sep 2007 21:51:25 +0200
Subject: Fw: [postmodern-devel] query with keyword :on
Message-ID: <008101c7f964$22cec310$290101c0@JensSony>

 Marijn Haverbeke wrote:
> Hello Jens  (and other lispworks users),
> 
> I finally got a chance to do some coding, and I pushed a patch that
> should fix this problem -- could you try it out and tell me if it
> works at all. The only lisp interpreter I had available was Franz'
> telnet repl, so testing has been minimal, but the changes weren't
> that drastic, so there is a not unreasonable chance that they are
> correct. 
 
 First short test looks very good!
 
 Thanks
 Jens
 
 ------------
 
 WEBCAT 15 > (with-db
              (postmodern:sql
               (:select 'pmw.wert
                :from (:as 'projekt-merkmale-werte 'pmw)
                :inner-join (:as 'projekt-elemente       'pe)
                :on         (:= 'pmw.guid-produkt 'pe.guid-objekt)
                :where (:and
                        (:= 'pe.guid "rzt732819vlurc5827955302wwxz8b")
                        (:= 'pmw.merkmal-id "MID1008519")
                        (:= 'pmw.lang-id "de")))))
 "(SELECT pmw.wert FROM projekt_merkmale_werte AS pmw INNER JOIN
 projekt_elemente AS pe ON (pmw.guid_produkt = pe.guid_objekt) WHERE
 ((pe.guid = 'rzt732819vlurc5827955302wwxz8b') and (pmw.merkmal_id =
 'MID1008519') and (pmw.lang_id = 'de')))"   
 
 WEBCAT 16 > (with-db
              (postmodern:query
               (:select 'pmw.wert
                :from (:as 'projekt-merkmale-werte 'pmw)
                :inner-join (:as 'projekt-elemente       'pe)
                :on         (:= 'pmw.guid-produkt 'pe.guid-objekt)
                :where (:and
                        (:= 'pe.guid "rzt732819vlurc5827955302wwxz8b")
                        (:= 'pmw.merkmal-id "MID1008519")
                        (:= 'pmw.lang-id "de")))))
 NIL
 
 WEBCAT 17 > (with-db
              (postmodern:query
               (:select 'pmw.wert
                :from (:as 'projekt-merkmale-werte 'pmw)
                :inner-join (:as 'projekt-elemente       'pe)
                :on         (:= 'pmw.guid-produkt 'pe.guid-objekt)
                :where (:and
                        (:= 'pmw.merkmal-id "MID1008519")
                        (:= 'pmw.lang-id "de")))))
 (("30 N")
  ("30 N")
  ("5 N")
  ("5 N")
  ("30 N")
  ("5 N")
  ("5 N")
  ("30 N")
  (" 5 N") .....


From ryszard.szopa at gmail.com  Tue Sep 18 01:11:35 2007
From: ryszard.szopa at gmail.com (Ryszard Szopa)
Date: Tue, 18 Sep 2007 03:11:35 +0200
Subject: [postmodern-devel] with-connection problem
In-Reply-To: <befaa7aa0709170622m3239a739sb417b2ec4023eeb7@mail.gmail.com>
References: <01a301c7e0c0$7906cd20$6e02a8c0@JensSony>
	<19e19e410708170805o4a28f882u61b4f783c0be9fab@mail.gmail.com>
	<befaa7aa0709170622m3239a739sb417b2ec4023eeb7@mail.gmail.com>
Message-ID: <19e19e410709171811o61d90246p1474801ef473dcff@mail.gmail.com>

On 9/17/07, Marijn Haverbeke <marijnh at gmail.com> wrote:

> At least, I think I did. The changes are completely untested -- could
> you kindly test them and tell me how it goes?

I pulled all the changes from the repo and ran your tests. Fiveam said:
 Did 27 checks.
    Pass: 27 (100%)
    Skip: 0 ( 0%)
    Fail: 0 ( 0%)

That means everything is OK. Unless, of course, they introduced really
ugly bugs that aren't even covered by the tests. :P

    -- Richard
-- 
http://szopa.tasak.gda.pl/


From marijnh at gmail.com  Tue Sep 18 14:25:07 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Tue, 18 Sep 2007 16:25:07 +0200
Subject: [postmodern-devel] with-connection problem
In-Reply-To: <19e19e410709171811o61d90246p1474801ef473dcff@mail.gmail.com>
References: <01a301c7e0c0$7906cd20$6e02a8c0@JensSony>
	<19e19e410708170805o4a28f882u61b4f783c0be9fab@mail.gmail.com>
	<befaa7aa0709170622m3239a739sb417b2ec4023eeb7@mail.gmail.com>
	<19e19e410709171811o61d90246p1474801ef473dcff@mail.gmail.com>
Message-ID: <befaa7aa0709180725r5dc1d835u986a17ec5a074bf9@mail.gmail.com>

> I pulled all the changes from the repo and ran your tests. Fiveam said:
>  Did 27 checks.
>     Pass: 27 (100%)
>     Skip: 0 ( 0%)
>     Fail: 0 ( 0%)

Oh, you seem to be under the impression that I actually introduced
test cases for the changes -- how naive ;) . I didn't, but a quick
usage of call-with-connection (your with-connection-fun) and
with-connection* (a variant of with-connection which evaluates the
specification list) should show any problems I introduced. I stared at
the code for a few minutes, so it shouldn't contain any glaring
mistakes, but you never know.

Cheers,
Marijn


From ryszard.szopa at gmail.com  Tue Sep 18 16:06:33 2007
From: ryszard.szopa at gmail.com (Ryszard Szopa)
Date: Tue, 18 Sep 2007 18:06:33 +0200
Subject: [postmodern-devel] with-connection problem
In-Reply-To: <befaa7aa0709180725r5dc1d835u986a17ec5a074bf9@mail.gmail.com>
References: <01a301c7e0c0$7906cd20$6e02a8c0@JensSony>
	<19e19e410708170805o4a28f882u61b4f783c0be9fab@mail.gmail.com>
	<befaa7aa0709170622m3239a739sb417b2ec4023eeb7@mail.gmail.com>
	<19e19e410709171811o61d90246p1474801ef473dcff@mail.gmail.com>
	<befaa7aa0709180725r5dc1d835u986a17ec5a074bf9@mail.gmail.com>
Message-ID: <19e19e410709180906n13156b82ke3a98667e0e203c1@mail.gmail.com>

On 9/18/07, Marijn Haverbeke <marijnh at gmail.com> wrote:

> Oh, you seem to be under the impression that I actually introduced
> test cases for the changes -- how naive ;) . I didn't, but a quick

Oh. I thought you reimplemented WITH-CONNECTION using
CALL-WITH-CONNECTION (nb. that's a lot better name for that function)
in order to avoid code reduplication blahblahblah ;), so the tests for
WITH-CONNECTION would reveal any errors. Yeah, I definitely should
look more thoroughly at the patches I apply in oder to test them :)

> usage of call-with-connection (your with-connection-fun) and
> with-connection* (a variant of with-connection which evaluates the
> specification list) should show any problems I introduced. I stared at
> the code for a few minutes, so it shouldn't contain any glaring
> mistakes, but you never know.

Apparently, you missed a comma before SPEC in WITH-CONNECTION*, so
there was an undefined variable error. After applying the attached
patch:

CL-USER> (let ((test "test"))
	   (pomo:with-connection* (list test "richard" "dupa" "localhost")
	     (pomo:query (:select 1))))
((1))
CL-USER> (let ((test "test"))
	   (pomo:call-with-connection (list test "richard" "dupa" "localhost")
				      (lambda ()
					(pomo:query (:select 1)))))
((1))

I hope it's OK now.

Bests,

    -- Richard
-- 
http://szopa.tasak.gda.pl/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: with-connection-star.patch
Type: text/x-patch
Size: 5988 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/postmodern-devel/attachments/20070918/570cd328/attachment.bin>

From ryszard.szopa at gmail.com  Tue Sep 18 18:36:33 2007
From: ryszard.szopa at gmail.com (Ryszard Szopa)
Date: Tue, 18 Sep 2007 20:36:33 +0200
Subject: [postmodern-devel] porting Postmodern to SQLite
Message-ID: <19e19e410709181136vdd25646le0bc740e512f6b15@mail.gmail.com>

Marijn,

I was talking with a friend about Submarine and Postmodern and he
complained about the fact that Postmodern (and thus Submarine) are
PostgreSQL specific. I agreed that this is a shame: for some smaller
projects Postgres is a bit overkill and you may want to use something
smaller, like SQLite.

You write in your docs that "...postmodern has no intention of being
portable across different SQL implementations (it embraces
non-standard Postgres features)." What do you mean by that? Is there
anything Postgres specific outside of CL-POSTGRES? How difficult do
you think it would be to port Postmodern to SQLite? (And: how long did
it take to write CL-POSTGRES?)

Cheers,

    -- Richard
-- 
http://szopa.tasak.gda.pl/


From marijnh at gmail.com  Wed Sep 19 10:37:25 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Wed, 19 Sep 2007 12:37:25 +0200
Subject: [postmodern-devel] porting Postmodern to SQLite
In-Reply-To: <19e19e410709181136vdd25646le0bc740e512f6b15@mail.gmail.com>
References: <19e19e410709181136vdd25646le0bc740e512f6b15@mail.gmail.com>
Message-ID: <befaa7aa0709190337lc71701dp9cf5e9839151e226@mail.gmail.com>

Hello Ryszard,

As you are probably aware, the SQL standard is quite limited, and all
database systems I've ever used go beyond it and provide extra
functionality. They tend to all use different syntax for these
extensions, and provide slightly different ones, so I didn't want to
bother with making my library support these different syntaxes or be
some kind of lowest common denominator -- partially because I am lazy,
and partially because I think PostgreSQL is a very solid system which
is pretty easy to set up -- there is rarely a very strong need to use
anything else.

I am afraid that I couldn't quite give a precise list of
Postgres-specific features that Postmodern uses, but just writing a
cl-postgres replacement for SQLite will definitely not be enough --
that only concerns the client/server protocol, and doesn't solve the
problem of SQL syntax and differences in supported features.

I have no idea how well the SQLite interface is documented, and
whether you'd want to implement it in CL or use FFI. Postgres has good
documentation about its protocol, so after studying that, writing the
client was not that hard (apart from having to implement some stuff
like binary floating point decoding).

Cheers,
Marijn


From lucindo at gmail.com  Sat Sep 22 18:13:14 2007
From: lucindo at gmail.com (Renato Lucindo)
Date: Sat, 22 Sep 2007 15:13:14 -0300
Subject: [postmodern-devel] postmodern:connected-p
Message-ID: <49376cdb0709221113j9fded13k95cd52ae1596434f@mail.gmail.com>

Hello,

The function postmodern:connected-p should fail when its argument is a
variable bounded to nil? For instance, here is some code using the package
:postmodern :

(defun connect-to-database ()
  (when (not (connected-p *database*))
    (connect-toplevel *db-name* *db-user* *db-pass* *db-host*)))

This code fails because *database* is nil at first.

If the function cl-postgress:database-open-p was changed into this:

(defun database-open-p (connection)
  "Returns a boolean indicating whether the given connection is currently
connected."
  (and connection
       (connection-socket connection)
       (open-stream-p (connection-socket connection))))

the code above would work as expected.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/postmodern-devel/attachments/20070922/99cd2601/attachment.html>

From marijnh at gmail.com  Mon Sep 24 08:57:17 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Mon, 24 Sep 2007 10:57:17 +0200
Subject: [postmodern-devel] postmodern:connected-p
In-Reply-To: <49376cdb0709221113j9fded13k95cd52ae1596434f@mail.gmail.com>
References: <49376cdb0709221113j9fded13k95cd52ae1596434f@mail.gmail.com>
Message-ID: <befaa7aa0709240157o101ef762p5051dacc4c67847f@mail.gmail.com>

Hello Renato,

> The function postmodern:connected-p should fail when its argument is a
> variable bounded to nil?

Yes, I think it should. It is defined on connection objects, nothing
else. Having it take NIL might be convenient, but that kind of tricks
are the road to (*shrudder*) a Perl-like mess of shortcuts and bizarre
rules. Just use (AND *DATABASE* (CONNECTED-P *DATABASE*)) in your
code.

Regards,
Marijn


From sjatkins at mac.com  Wed Sep 26 04:33:56 2007
From: sjatkins at mac.com (Samantha Atkins)
Date: Tue, 25 Sep 2007 21:33:56 -0700
Subject: [postmodern-devel] is postmodern thread safe
Message-ID: <22221009-6367-4B8C-BC51-695EA7DC74BD@mac.com>

I would like to find a Lisp database interface that is.  I intend to  
build a general ODBMS layer in lisp on top of such.  To do so I need  
serious multi-threaded support.  Think persistence layer to a web app  
platform.   Does postmodern handle multi-threaded use?   If it makes a  
difference my Lisp of choice is SBCL.

- samantha


From attila.lendvai at gmail.com  Wed Sep 26 09:06:05 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Wed, 26 Sep 2007 11:06:05 +0200
Subject: [postmodern-devel] is postmodern thread safe
In-Reply-To: <22221009-6367-4B8C-BC51-695EA7DC74BD@mac.com>
References: <22221009-6367-4B8C-BC51-695EA7DC74BD@mac.com>
Message-ID: <b54b34af0709260206oba0b346r38ec09df847238aa@mail.gmail.com>

> I would like to find a Lisp database interface that is.  I intend to
> build a general ODBMS layer in lisp on top of such.  To do so I need

make sure to check out cl-perec and cl-rdbms. perec aims to be an sql
based transparent persistency layer on CLOS - transparent to the
extent it's feasible.

YMMV and there's not much documentation, but there are tons of tests
and it's used in a production system (as a backend for a webapp).

we are using cl-postgres (part of postmodern) and we had no problem
with multithreaded use.

-- 
 attila


From marijnh at gmail.com  Wed Sep 26 09:44:45 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Wed, 26 Sep 2007 11:44:45 +0200
Subject: [postmodern-devel] is postmodern thread safe
In-Reply-To: <22221009-6367-4B8C-BC51-695EA7DC74BD@mac.com>
References: <22221009-6367-4B8C-BC51-695EA7DC74BD@mac.com>
Message-ID: <befaa7aa0709260244r5e4ebf09y2b4295a2d5513fce@mail.gmail.com>

Hold it! The competition is advertising on *my* list now? Hah, never
mind, CL-RDBMS is a nice system, but to answer your question about
Postmodern: Yes, it is thread-safe, as long as you do not have two
threads use the same connection object at the same time. Using
WITH-CONNECTION, it is easy to give each thread its own connection.
The only part of the library that uses global mutable state is the
connection pool, and I used bordeaux-threads' mutexes to prevent
problems there.

Cheers,
Marijn


From attila.lendvai at gmail.com  Wed Sep 26 11:26:31 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Wed, 26 Sep 2007 13:26:31 +0200
Subject: [postmodern-devel] is postmodern thread safe
In-Reply-To: <befaa7aa0709260244r5e4ebf09y2b4295a2d5513fce@mail.gmail.com>
References: <22221009-6367-4B8C-BC51-695EA7DC74BD@mac.com>
	<befaa7aa0709260244r5e4ebf09y2b4295a2d5513fce@mail.gmail.com>
Message-ID: <b54b34af0709260426g69ef72c3jbf87bd57f05be17a@mail.gmail.com>

> Hold it! The competition is advertising on *my* list now? Hah, never
> mind, CL-RDBMS is a nice system, but to answer your question about

oh, come on, they are not competitiors, or maybe a little bit but they
are all opensource after all... :) i'm just trying to get people's
attention to cl-perec to consider patching it instead of
reimplementing it.

seriously, all the three projects have different goals, although all
of them are in the db/persistency category. cl-rdbms is even using the
wire-protocol part of Postmodern in the postgres backend.

but either way, sorry for the adv. i'm usually advertising the
alternatives to my own projects, too. more information is always a net
win.

-- 
 attila


From henrik at evahjelte.com  Wed Sep 26 11:44:14 2007
From: henrik at evahjelte.com (Henrik Hjelte)
Date: Wed, 26 Sep 2007 13:44:14 +0200
Subject: [postmodern-devel] is postmodern thread safe
In-Reply-To: <befaa7aa0709260244r5e4ebf09y2b4295a2d5513fce@mail.gmail.com>
References: <22221009-6367-4B8C-BC51-695EA7DC74BD@mac.com>
	<befaa7aa0709260244r5e4ebf09y2b4295a2d5513fce@mail.gmail.com>
Message-ID: <50e8e4f60709260444u583ec3f3q7ed42939d168670b@mail.gmail.com>

There is also a new backend for elephant based on
postmodern(cl-postgres) called ele-postmodern. It is thread safe and
process safe. It is in alpha state but is already used by two
production web sites. It is not, like perec, an object relational
mapper, but merely uses postgresql as a low level storage mechanism.

Even though I make this ad about the ele-postmodern backend, I believe
that there is much to do in this area. We have plans to make a new
even better persistence solution, where better means safe, highly
scalable for lots of processes, good performance, nice high level
features. Maybe you are interested in coordinating efforts?

/Henrik Hjelte


From lucindo at gmail.com  Wed Sep 26 19:07:15 2007
From: lucindo at gmail.com (Renato Lucindo)
Date: Wed, 26 Sep 2007 16:07:15 -0300
Subject: [postmodern-devel] Selecting dao's
Message-ID: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>

Hi,

There is some way to select a limited number of daos without using query-dao?

I think in something like that: the function select-dao accepting some
key parameters, like :limit, :offset, :order-by...

(select-dao 'my-entity :limit 100)

to be same as

(query-dao 'my-entity (:limit (:select '* :from 'my-entity) 100))

I think this would be useful in many applications, and allow the
developer to write code without SQL.

[]'s

Lucindo


From ryszard.szopa at gmail.com  Thu Sep 27 00:29:31 2007
From: ryszard.szopa at gmail.com (Ryszard Szopa)
Date: Thu, 27 Sep 2007 02:29:31 +0200
Subject: [postmodern-devel] Selecting dao's
In-Reply-To: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
References: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
Message-ID: <19e19e410709261729l7fd7e6fek54a6e2d6e1b43f5d@mail.gmail.com>

On 9/26/07, Renato Lucindo <lucindo at gmail.com> wrote:

> There is some way to select a limited number of daos without using query-dao?
>
> I think in something like that: the function select-dao accepting some
> key parameters, like :limit, :offset, :order-by...

SELECT-DAO already accepts an optional argument, i.e. TEST. Adding
additional arguments could be done in three ways: (a) by keeping TEST
optional and making the rest keyword arguments, (b) by making all
arguments optional, or (c) by making all arguments keywords. Approach
(a) doesn't seem nice: we mix optional and keyword arguments, and we
cannot naturally make e.g. a query with a limit but without a test.
(b) doesn't seem sensible, because you would have to memorize the
order of the arguments, plus the second part of the argument against
(a). (c) breaks older code, which usually is perceived as bad.

>
> (select-dao 'my-entity :limit 100)
>
> to be same as
>
> (query-dao 'my-entity (:limit (:select '* :from 'my-entity) 100))
>
> I think this would be useful in many applications, and allow the
> developer to write code without SQL.

Well, writing the (:select '* :from... indeed may be annoying. But do
you feel it is annoying enough to risk any of the problems I mentioned
above?

Cheers,

    -- Richard
-- 
http://szopa.tasak.gda.pl/


From lucindo at gmail.com  Thu Sep 27 03:48:21 2007
From: lucindo at gmail.com (Renato Lucindo)
Date: Thu, 27 Sep 2007 00:48:21 -0300
Subject: [postmodern-devel] Selecting dao's
In-Reply-To: <19e19e410709261729l7fd7e6fek54a6e2d6e1b43f5d@mail.gmail.com>
References: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
	<19e19e410709261729l7fd7e6fek54a6e2d6e1b43f5d@mail.gmail.com>
Message-ID: <49376cdb0709262048t3869a9o564a6692ea9e8aa0@mail.gmail.com>

I think (a) is just a problem in sense the of the mixing optional and
keyword arguments because on the select-dao macro the generated sql
aways have a 'where' clause, with the default being "where true".

"select .... where true limit ? offset ?" and/or "select .... where
true order-by ?" should work.

Off course, the (:limit ...) and (:order-by ...) should only be added
to the S-SQL inside select-dao when the keywords paramaters are
specified.

On 9/26/07, Ryszard Szopa <ryszard.szopa at gmail.com> wrote:
> On 9/26/07, Renato Lucindo <lucindo at gmail.com> wrote:
>
> > There is some way to select a limited number of daos without using query-dao?
> >
> > I think in something like that: the function select-dao accepting some
> > key parameters, like :limit, :offset, :order-by...
>
> SELECT-DAO already accepts an optional argument, i.e. TEST. Adding
> additional arguments could be done in three ways: (a) by keeping TEST
> optional and making the rest keyword arguments, (b) by making all
> arguments optional, or (c) by making all arguments keywords. Approach
> (a) doesn't seem nice: we mix optional and keyword arguments, and we
> cannot naturally make e.g. a query with a limit but without a test.
> (b) doesn't seem sensible, because you would have to memorize the
> order of the arguments, plus the second part of the argument against
> (a). (c) breaks older code, which usually is perceived as bad.
>
> >
> > (select-dao 'my-entity :limit 100)
> >
> > to be same as
> >
> > (query-dao 'my-entity (:limit (:select '* :from 'my-entity) 100))
> >
> > I think this would be useful in many applications, and allow the
> > developer to write code without SQL.
>
> Well, writing the (:select '* :from... indeed may be annoying. But do
> you feel it is annoying enough to risk any of the problems I mentioned
> above?
>
> Cheers,
>
>     -- Richard
> --
> http://szopa.tasak.gda.pl/
> _______________________________________________
> postmodern-devel mailing list
> postmodern-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel
>


From marijnh at gmail.com  Thu Sep 27 06:27:10 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Thu, 27 Sep 2007 08:27:10 +0200
Subject: [postmodern-devel] Selecting dao's
In-Reply-To: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
References: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
Message-ID: <befaa7aa0709262327m50b74fb7w97c0d0cae03cb06a@mail.gmail.com>

Hi Renato,

I guess you are right that the interface of SELECT-DAO is a bit
limited, but as Ryszard says it would be hard to extend its
functionality in a clean way, and, after playing with a
keyword-argument implementation for a bit I concluded that providing
all this functionality in such a function would basically be yet
another syntax to do the same thing. So my advice is to just use
QUERY-DAO for more complex cases.

Regards,
Marijn


From lucindo at gmail.com  Fri Sep 28 14:09:54 2007
From: lucindo at gmail.com (Renato Lucindo)
Date: Fri, 28 Sep 2007 11:09:54 -0300
Subject: [postmodern-devel] Selecting dao's
In-Reply-To: <befaa7aa0709262327m50b74fb7w97c0d0cae03cb06a@mail.gmail.com>
References: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
	<befaa7aa0709262327m50b74fb7w97c0d0cae03cb06a@mail.gmail.com>
Message-ID: <49376cdb0709280709j66c18eaft801d138a90ec03ea@mail.gmail.com>

Hi,

I changed my code to use this function instead of (quey-dao ...)

(defun select-limited-dao (type &key (test t) limit offset order-by desc)
  (let ((query `(:select * :from ,type :where ,(if (stringp test)
`(:raw ,test) test))))
    (progn
      (if order-by (setf query `(:order-by ,query ,order-by)))
      (if desc (setf query `(:desc ,query)))
      (if limit (setf query `(:limit ,query ,limit ,(if offset offset ()))))
      (query-dao type (sql-compile query)))))

But I got a strange error testing this function. The problem is with :desc

> (select-limited-dao 'entity :order-by 'column :desc t)
Database error 42601: syntax error at or near "DESC"
Query: ((SELECT * FROM entity WHERE true) ORDER BY column) DESC

Then I tried with quey-dao and got the same error:

> (query-dao 'entity (:desc (:order-by (:select '* :from 'entity :where t) 'column)))
Database error 42601: syntax error at or near "DESC"
Query: ((SELECT * FROM entity WHERE true) ORDER BY column) DESC

I think I'm using :desc in a wrong way.

The query sent to Postgress is: "((SELECT * FROM entity WHERE true)
ORDER BY column) DESC"

This quey works: "(SELECT * FROM entity WHERE true) ORDER BY column DESC"

I'm using Postgres 8.2 under OS X 10.4.10 and SBCL 1.0.6

On 9/27/07, Marijn Haverbeke <marijnh at gmail.com> wrote:
> Hi Renato,
>
> I guess you are right that the interface of SELECT-DAO is a bit
> limited, but as Ryszard says it would be hard to extend its
> functionality in a clean way, and, after playing with a
> keyword-argument implementation for a bit I concluded that providing
> all this functionality in such a function would basically be yet
> another syntax to do the same thing. So my advice is to just use
> QUERY-DAO for more complex cases.
>
> Regards,
> Marijn
> _______________________________________________
> postmodern-devel mailing list
> postmodern-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel
>


From marijnh at gmail.com  Fri Sep 28 15:58:24 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Fri, 28 Sep 2007 17:58:24 +0200
Subject: [postmodern-devel] Selecting dao's
In-Reply-To: <49376cdb0709280709j66c18eaft801d138a90ec03ea@mail.gmail.com>
References: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
	<befaa7aa0709262327m50b74fb7w97c0d0cae03cb06a@mail.gmail.com>
	<49376cdb0709280709j66c18eaft801d138a90ec03ea@mail.gmail.com>
Message-ID: <befaa7aa0709280858q4ab9c536m96776088e478bcb6@mail.gmail.com>

Hi,

The documentation is a bit sparse on this, but :DESC should be used on
the elements of an :ORDER-BY clause, not on the whole expression, as
in:

(:select '* :from 'bozo :order-by (:desc 'age))

Cheers,
Marijn


From marijnh at gmail.com  Fri Sep 28 16:01:19 2007
From: marijnh at gmail.com (Marijn Haverbeke)
Date: Fri, 28 Sep 2007 18:01:19 +0200
Subject: [postmodern-devel] Selecting dao's
In-Reply-To: <befaa7aa0709280858q4ab9c536m96776088e478bcb6@mail.gmail.com>
References: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
	<befaa7aa0709262327m50b74fb7w97c0d0cae03cb06a@mail.gmail.com>
	<49376cdb0709280709j66c18eaft801d138a90ec03ea@mail.gmail.com>
	<befaa7aa0709280858q4ab9c536m96776088e478bcb6@mail.gmail.com>
Message-ID: <befaa7aa0709280901h5d532920i9b998294f3515993@mail.gmail.com>

On 9/28/07, some nitwit <marijnh at gmail.com> wrote:
> (:select '* :from 'bozo :order-by (:desc 'age))

Except, of course, that the correct syntax is (:order-by (:select '*
:from 'bozo) (:desc 'age))


From lucindo at gmail.com  Fri Sep 28 17:15:16 2007
From: lucindo at gmail.com (Renato Lucindo)
Date: Fri, 28 Sep 2007 14:15:16 -0300
Subject: [postmodern-devel] Selecting dao's
In-Reply-To: <befaa7aa0709280901h5d532920i9b998294f3515993@mail.gmail.com>
References: <49376cdb0709261207p3052cad9n83a710896b8b1c6a@mail.gmail.com>
	<befaa7aa0709262327m50b74fb7w97c0d0cae03cb06a@mail.gmail.com>
	<49376cdb0709280709j66c18eaft801d138a90ec03ea@mail.gmail.com>
	<befaa7aa0709280858q4ab9c536m96776088e478bcb6@mail.gmail.com>
	<befaa7aa0709280901h5d532920i9b998294f3515993@mail.gmail.com>
Message-ID: <49376cdb0709281015j453471fejd538a4a0693ddaed@mail.gmail.com>

I changed my code and all seems to be working fine now.

Thanks

On 9/28/07, Marijn Haverbeke <marijnh at gmail.com> wrote:
> On 9/28/07, some nitwit <marijnh at gmail.com> wrote:
> > (:select '* :from 'bozo :order-by (:desc 'age))
>
> Except, of course, that the correct syntax is (:order-by (:select '*
> :from 'bozo) (:desc 'age))
> _______________________________________________
> postmodern-devel mailing list
> postmodern-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel
>