From attila.lendvai at gmail.com  Tue Aug 19 08:40:57 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 19 Aug 2008 10:40:57 +0200
Subject: [Cl-perec-devel] Fwd: chasing after a strange bug in my app
In-Reply-To: <20080819070947.GA10884@tehran.lain.pl>
References: <20080819053437.GA9527@tehran.lain.pl>
	<20080819070947.GA10884@tehran.lain.pl>
Message-ID: <b54b34af0808190140v30760abdw1b5d4697cb2103ee@mail.gmail.com>

Tomi, take a look at this if you have some free cycles:

---------- Forwarded message ----------
From: Stanislaw Halik <sthalik at tehran.lain.pl>
Date: Tue, Aug 19, 2008 at 9:09 AM
Subject: Re: chasing after a strange bug in my app
To: attila.lendvai at gmail.com


On Tue, Aug 19, 2008, Stanislaw Halik wrote:
[...]

Found it! In case you're interested, it's caused by partial eval.
Redefining the method solves the problem without fully disabling partial
eval.

(defmethod prc::%partial-eval-syntax ((variable prc::dynamic-variable) query)
 variable)

--
The great peril of our existence lies in the fact that our diet consists
entirely of souls. -- Inuit saying


From leslie.polzer at gmx.net  Thu Aug 21 11:39:04 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Thu, 21 Aug 2008 13:39:04 +0200 (CEST)
Subject: [Cl-perec-devel] Special requirements
Message-ID: <61461.84.157.16.47.1219318744.squirrel@mail.stardawn.org>


Hi,

I'm contemplating moving from Elephant to cl-perec.

My project has some special requirements:

  * Persistent objects and values are accessed by multiple threads.

  * Persistent objects and values are accessed by multiple processes.

  * Processes share a central storage 'ITEMS'. Additionally, each
    process has a private storage 'WORLD'.

  * Objects from the central storage need to be copied from ITEMS
    to WORLD on a regular basis, including their associations
    (I currently do this with a recursive clone-by-value).

With this in mind, two questions:

  Will cl-perec in its current state cope with this?
  In negative case, are there workarounds?

  What happens when I have a schema change and I update
  the live processes one by one?

Thanks!

  Leslie

-- 
LinkedIn Profile: http://www.linkedin.com/in/polzer
Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
Blog: http://blog.viridian-project.de/



From leslie.polzer at gmx.net  Thu Aug 21 11:40:53 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Thu, 21 Aug 2008 13:40:53 +0200 (CEST)
Subject: [Cl-perec-devel] Deferring persistence
Message-ID: <62675.84.157.16.47.1219318853.squirrel@mail.stardawn.org>


Forgot one question in the other message:

Can I defer (or entirely cancel) storage of an object until
some point after MAKE-INSTANCE?

  Leslie



From attila.lendvai at gmail.com  Thu Aug 21 13:38:14 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Thu, 21 Aug 2008 15:38:14 +0200
Subject: [Cl-perec-devel] Special requirements
In-Reply-To: <61461.84.157.16.47.1219318744.squirrel@mail.stardawn.org>
References: <61461.84.157.16.47.1219318744.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808210638s73e3d81eya8c28e31a748454e@mail.gmail.com>

hi,

> I'm contemplating moving from Elephant to cl-perec.
>
> My project has some special requirements:
>
>  * Persistent objects and values are accessed by multiple threads.
>
>  * Persistent objects and values are accessed by multiple processes.


i'm not sure what you mean by processes. i assume OS processes and OS
threads here, so multiple threaded lisp vm's.

in perec persistent-objects are EQ inside a transaction, but separate
transactions can not share anything, not even inside a lisp vm. there
are two primitives to deal with this:

 - REVIVE-INSTANCE which works on a place and can "import" a "dead"
   instance from a previous already closed transaction

 - LOAD-INSTANCE which can "import" an instance into the current
   transaction (by loading it from the db, or if requested copying the
   cached slots from the dead instance without checking the db)

so, object identity in perec means that persistent instances are EQ
inside a transaction but otherwise if you want to check object
identity then you need to EQL check the id-of slot of the instances.


>  * Processes share a central storage 'ITEMS'. Additionally, each
>    process has a private storage 'WORLD'.
>
>  * Objects from the central storage need to be copied from ITEMS
>    to WORLD on a regular basis, including their associations
>    (I currently do this with a recursive clone-by-value).


perec is an rdbms mapper that helps you transfer data from/to the
database. this also defines your possibilities. so, you can either
separate your worlds using rdbms associations or useing different db
instances.

we use the former in our setup: we have 9 nodes in a cluster, each
node has a cluster-node instance looked up by the machine name when
the node starts. some of the persistent state is associated to these
instances.


> With this in mind, two questions:
>
>  Will cl-perec in its current state cope with this?
>  In negative case, are there workarounds?


i'm not sure about your requirements, but maybe the descriptions above
give you a picture.


>  What happens when I have a schema change and I update
>  the live processes one by one?


bad idea... might work in progressive changes (e.g. making a column
bigger), but what happens if the first restarted instance drops a
column the others are expecting to be there...?


> Can I defer (or entirely cancel) storage of an object until
> some point after MAKE-INSTANCE?


(make-persistent (make-instance 'foo :persistent nil))

but the multiple sides of associations are not working yet in
transient instances (iow, transient collections are not yet
implemented to store changes made to an association of a transient
instance)

hth,

-- 
 attila


From leslie.polzer at gmx.net  Thu Aug 21 17:27:19 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Thu, 21 Aug 2008 19:27:19 +0200 (CEST)
Subject: [Cl-perec-devel] Special requirements
In-Reply-To: <b54b34af0808210638s73e3d81eya8c28e31a748454e@mail.gmail.com>
References: <61461.84.157.16.47.1219318744.squirrel@mail.stardawn.org>
	<b54b34af0808210638s73e3d81eya8c28e31a748454e@mail.gmail.com>
Message-ID: <60743.84.157.16.47.1219339639.squirrel@mail.stardawn.org>


Dear Attila,

thanks for the answers so far.


> i'm not sure what you mean by processes. i assume OS processes and OS
> threads here, so multiple threaded lisp vm's.

Yes, I mean OS processes and threads.


> in perec persistent-objects are EQ inside a transaction, but separate
> transactions can not share anything, not even inside a lisp vm.

Will slot changes in one thread/process be seen by the other one?
I suppose a slot write will propagate immediately to the db, right?

Also, I'm not sure I understand the concept of dead/alive objects.
When is an object dead, and when is it alive?


> persistent instances are EQ inside a transaction but otherwise
> if you want to check object identity then you need to EQL check
> the id-of slot of the instances.

Good to know, thanks.


> perec is an rdbms mapper that helps you transfer data from/to the
> database. this also defines your possibilities. so, you can either
> separate your worlds using rdbms associations or useing different db
> instances.

How would I use different databases with perec?


> i'm not sure about your requirements, but maybe the descriptions above
> give you a picture.

We're slowly getting there, I guess. :)


> bad idea... might work in progressive changes (e.g. making a column
> bigger), but what happens if the first restarted instance drops a
> column the others are expecting to be there...?

Yes, that's the case I'm worried about. But I guess I have to
invent something on my own here.


> but the multiple sides of associations are not working yet in
> transient instances (iow, transient collections are not yet
> implemented to store changes made to an association of a transient
> instance)

Speaking of associations, have you pondered offering an alternative
interface to DEFASSOCIATION? This seems more natural to me:

(defpclass brother ()
  ((siblings :type (set siblings)))) ; hope I got the syntax right

  Leslie



From attila.lendvai at gmail.com  Thu Aug 21 18:37:52 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Thu, 21 Aug 2008 20:37:52 +0200
Subject: [Cl-perec-devel] Special requirements
In-Reply-To: <60743.84.157.16.47.1219339639.squirrel@mail.stardawn.org>
References: <61461.84.157.16.47.1219318744.squirrel@mail.stardawn.org>
	<b54b34af0808210638s73e3d81eya8c28e31a748454e@mail.gmail.com>
	<60743.84.157.16.47.1219339639.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808211137k71b532c4vcab6d5095cc2b7dd@mail.gmail.com>

>> in perec persistent-objects are EQ inside a transaction, but separate
>> transactions can not share anything, not even inside a lisp vm.
>
> Will slot changes in one thread/process be seen by the other one?
> I suppose a slot write will propagate immediately to the db, right?


currently each slot change ends up in an sql update. it could be
optimized to gather slot changes until the transaction is actually
comitted, but it changes the semantics slightly and there are also
other lower fruits before that optimization.

what is visible in other transactions depends on the transaction
isolation level of your db and the timing of the lisp transactions.
your db is most probably in read-comitted unless you changed it, which
means that changes made in a transaction is not visible in other
transactions until it is comitted.


> Also, I'm not sure I understand the concept of dead/alive objects.
> When is an object dead, and when is it alive?


dead means that the transaction in which it was loaded in is already
closed. alive means that there's an open transaction in which this
instance is alive in. some operations don't work on dead instances
(i.e. accessors to not cached slots)


>> perec is an rdbms mapper that helps you transfer data from/to the
>> database. this also defines your possibilities. so, you can either
>> separate your worlds using rdbms associations or useing different db
>> instances.
>
> How would I use different databases with perec?


rebind *database* with different connection specification before using
a with-transaction block. (note that mixing different db types is not
supported/tested/etc, only different db instance of the same kind)


>> but the multiple sides of associations are not working yet in
>> transient instances (iow, transient collections are not yet
>> implemented to store changes made to an association of a transient
>> instance)
>
> Speaking of associations, have you pondered offering an alternative
> interface to DEFASSOCIATION? This seems more natural to me:
>
> (defpclass brother ()
>  ((siblings :type (set siblings)))) ; hope I got the syntax right


this should work already but it's not throughly tested because
associations don't have more cost than these slots and most of the
time it comes handy if you have a link in both directions. so we
mostly use associations.

-- 
 attila


From leslie.polzer at gmx.net  Thu Aug 21 19:11:01 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Thu, 21 Aug 2008 21:11:01 +0200 (CEST)
Subject: [Cl-perec-devel] Special requirements
In-Reply-To: <b54b34af0808211137k71b532c4vcab6d5095cc2b7dd@mail.gmail.com>
References: <61461.84.157.16.47.1219318744.squirrel@mail.stardawn.org>
	<b54b34af0808210638s73e3d81eya8c28e31a748454e@mail.gmail.com>
	<60743.84.157.16.47.1219339639.squirrel@mail.stardawn.org>
	<b54b34af0808211137k71b532c4vcab6d5095cc2b7dd@mail.gmail.com>
Message-ID: <59723.84.157.16.47.1219345861.squirrel@mail.stardawn.org>


> currently each slot change ends up in an sql update. it could be
> optimized to gather slot changes until the transaction is actually
> comitted, but it changes the semantics slightly and there are also
> other lower fruits before that optimization.

Ah, so I need to explicitly wrap all slot writes/uncached reads
in WITH-TRANSACTION?

Also, is there a perec-idiomatic way to set the pg isolation level?



From leslie.polzer at gmx.net  Sat Aug 23 08:13:57 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Sat, 23 Aug 2008 10:13:57 +0200 (CEST)
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re:
 [cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
Message-ID: <61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>


> this is due to your defclass-star loading problem (which i can't reproduce).

Okay, got that fixed.

Next is:

; /home/sky/projects/mystic/hg.beta1/packages/cl-perec/persistence/export.fasl written
; compilation finished in 0:00:00

debugger invoked on a SIMPLE-ERROR in thread #<THREAD "initial thread" RUNNING {A698869}>:
  There is no class named TIMESTAMP.

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY ] Retry performing #<ASDF:LOAD-OP NIL {DDDB241}> on
              #<CL-PEREC-SYSTEM::LOCAL-CL-SOURCE-FILE "copy" {DDDB221}>.
...


I pruned all FASLs from my system but still get this.

cl-perec and dependencies are all up to date.

  Leslie



From attila.lendvai at gmail.com  Sat Aug 23 09:27:27 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Sat, 23 Aug 2008 11:27:27 +0200
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re:
	[cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
	<61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>

> debugger invoked on a SIMPLE-ERROR in thread #<THREAD "initial thread" RUNNING {A698869}>:
>  There is no class named TIMESTAMP.
>
> Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
>
> restarts (invokable by number or by possibly-abbreviated name):
>  0: [RETRY ] Retry performing #<ASDF:LOAD-OP NIL {DDDB241}> on
>              #<CL-PEREC-SYSTEM::LOCAL-CL-SOURCE-FILE "copy" {DDDB221}>.
> ...

you need the new 1.0 branch of local-time:
http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi

-- 
 attila


From leslie.polzer at gmx.net  Sat Aug 23 09:52:45 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Sat, 23 Aug 2008 11:52:45 +0200 (CEST)
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re: 
	[cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
	<61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
	<b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>
Message-ID: <64886.88.73.251.191.1219485165.squirrel@mail.stardawn.org>


>>  There is no class named TIMESTAMP.

> you need the new 1.0 branch of local-time:
> http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi

Got that now, but I can't find the type TIMESTAMP there.

Postmodern exports this class, but it's neither loaded nor used
by perec.

Am I missing something else?



From attila.lendvai at gmail.com  Sat Aug 23 09:56:45 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Sat, 23 Aug 2008 11:56:45 +0200
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re:
	[cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <64886.88.73.251.191.1219485165.squirrel@mail.stardawn.org>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
	<61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
	<b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>
	<64886.88.73.251.191.1219485165.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808230256i6c29c790mbe89a59acdb22406@mail.gmail.com>

>> you need the new 1.0 branch of local-time:
>> http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi
>
> Got that now, but I can't find the type TIMESTAMP there.
>
> Postmodern exports this class, but it's neither loaded nor used
> by perec.
>
> Am I missing something else?

probably an rmfasl? :)

we have this:

function rmfasl ()
{
  for i in $*; do
    for dir in `find ~/.fasls/ -type d -wholename "*$i*"`; do
      if [ -e "$dir" ]; then
        echo Deleting "$dir"
        rm -rf "$dir"
      fi
    done
  done
}

but it also needs asdf-binary-locations to redirect fasl's to ~/.fasls/

details available at
http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=cl-dwim-environment;a=summary

also make sure asdf is loading from the correct local-time directory.

-- 
 attila


From leslie.polzer at gmx.net  Sat Aug 23 10:45:22 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Sat, 23 Aug 2008 12:45:22 +0200 (CEST)
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re: 
	[cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <b54b34af0808230256i6c29c790mbe89a59acdb22406@mail.gmail.com>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
	<61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
	<b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>
	<64886.88.73.251.191.1219485165.squirrel@mail.stardawn.org>
	<b54b34af0808230256i6c29c790mbe89a59acdb22406@mail.gmail.com>
Message-ID: <61320.88.73.251.191.1219488322.squirrel@mail.stardawn.org>


> also make sure asdf is loading from the correct local-time directory.

Ensured, and deleted all FASLs. But I still get the error.

Where is type TIMESTAMP supposed to be defined? I can't
find the definition in cl-dwim-local-time.



From attila.lendvai at gmail.com  Sat Aug 23 18:39:42 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Sat, 23 Aug 2008 20:39:42 +0200
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re:
	[cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <61320.88.73.251.191.1219488322.squirrel@mail.stardawn.org>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
	<61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
	<b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>
	<64886.88.73.251.191.1219485165.squirrel@mail.stardawn.org>
	<b54b34af0808230256i6c29c790mbe89a59acdb22406@mail.gmail.com>
	<61320.88.73.251.191.1219488322.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808231139q2104a84bn4d57b81a2dd7de9e@mail.gmail.com>

> Where is type TIMESTAMP supposed to be defined? I can't
> find the definition in cl-dwim-local-time.

that local-time repo under cl-dwim was a bit stale, so i've removed it.

you need the official local-time repo from the local-time project
(i've just merged the new l-t codebase back in the official tree).

sorry for the confusion, it's my bad... :|

-- 
 attila


From leslie.polzer at gmx.net  Sat Aug 23 18:48:47 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Sat, 23 Aug 2008 20:48:47 +0200 (CEST)
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re: 
	[cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <b54b34af0808231139q2104a84bn4d57b81a2dd7de9e@mail.gmail.com>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
	<61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
	<b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>
	<64886.88.73.251.191.1219485165.squirrel@mail.stardawn.org>
	<b54b34af0808230256i6c29c790mbe89a59acdb22406@mail.gmail.com>
	<61320.88.73.251.191.1219488322.squirrel@mail.stardawn.org>
	<b54b34af0808231139q2104a84bn4d57b81a2dd7de9e@mail.gmail.com>
Message-ID: <64245.88.73.251.191.1219517327.squirrel@mail.stardawn.org>


> that local-time repo under cl-dwim was a bit stale, so i've removed it.
>
> you need the official local-time repo from the local-time project
> (i've just merged the new l-t codebase back in the official tree).
>
> sorry for the confusion, it's my bad... :|

Compiles fine now, thank you very much. :)



From leslie.polzer at gmx.net  Sat Aug 23 19:38:20 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Sat, 23 Aug 2008 21:38:20 +0200 (CEST)
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re:  
	[cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <64245.88.73.251.191.1219517327.squirrel@mail.stardawn.org>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
	<61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
	<b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>
	<64886.88.73.251.191.1219485165.squirrel@mail.stardawn.org>
	<b54b34af0808230256i6c29c790mbe89a59acdb22406@mail.gmail.com>
	<61320.88.73.251.191.1219488322.squirrel@mail.stardawn.org>
	<b54b34af0808231139q2104a84bn4d57b81a2dd7de9e@mail.gmail.com>
	<64245.88.73.251.191.1219517327.squirrel@mail.stardawn.org>
Message-ID: <64788.88.73.251.191.1219520300.squirrel@mail.stardawn.org>


> Compiles fine now, thank you very much. :)

So we're now at the stage of run-time errors. ;)

(asdf:oos 'asdf:load-op :cl-perec)

(defparameter *database*
  (make-instance 'postgresql-postmodern
                 :connection-specification '(:host "localhost" :port 5433
                                             :database "test"
                                             :user-name "sky" :password "sky")))
(in-package prc)

(defpclass* foo () ((name :type (text 20)) (age :type integer-16)))

(with-transaction
  (make-instance 'foo :name "bar" :age 5))

fails with:

  There is no applicable method for the generic function
    #<STANDARD-GENERIC-FUNCTION CREATED-INSTANCES-OF (1)>
  when called with arguments
    (#<#<STANDARD-CLASS NIL {C9330A1}> :begin-executed-p #t {C917001}>).

I understand that I might have forgot to setup some thing
or another, but I don't know what that might be...

  Leslie



From attila.lendvai at gmail.com  Sat Aug 23 20:14:22 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Sat, 23 Aug 2008 22:14:22 +0200
Subject: [Cl-perec-devel] There is no class named TIMESTAMP. (was: Re:
	[cl-rdbms-devel] Problem with dml.lisp)
In-Reply-To: <64788.88.73.251.191.1219520300.squirrel@mail.stardawn.org>
References: <61962.88.73.217.30.1219423089.squirrel@mail.stardawn.org>
	<b54b34af0808221116j45359e57l212bc63c54b6ab0e@mail.gmail.com>
	<61550.88.73.251.191.1219479237.squirrel@mail.stardawn.org>
	<b54b34af0808230227m4ef26cd6y5a42d856d10956e9@mail.gmail.com>
	<64886.88.73.251.191.1219485165.squirrel@mail.stardawn.org>
	<b54b34af0808230256i6c29c790mbe89a59acdb22406@mail.gmail.com>
	<61320.88.73.251.191.1219488322.squirrel@mail.stardawn.org>
	<b54b34af0808231139q2104a84bn4d57b81a2dd7de9e@mail.gmail.com>
	<64245.88.73.251.191.1219517327.squirrel@mail.stardawn.org>
	<64788.88.73.251.191.1219520300.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808231314w54e83dabrc6ca38aa24875898@mail.gmail.com>

> (defparameter *database*
>  (make-instance 'postgresql-postmodern
>                 :connection-specification '(:host "localhost" :port 5433
>                                             :database "test"
>                                             :user-name "sky" :password "sky")))

you need to specify the perec transaction class for cl-rdbms. add this
next to the connection specification:

:transaction-mixin 'cl-perec::transaction-mixin

see cl-perec-test.postgresql.asd for more details. (it also adds the
transaction-t-mixin, but you don't need that, it's the time machine
stuff).

also you can try (asdf:operate 'asdf:test-op
:cl-perec-test.postgresql) to start the standard perec testsuite.

-- 
 attila


From leslie.polzer at gmx.net  Sun Aug 24 11:59:27 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Sun, 24 Aug 2008 13:59:27 +0200 (CEST)
Subject: [Cl-perec-devel] Associations and auxiliary information
Message-ID: <64380.88.73.244.169.1219579167.squirrel@mail.stardawn.org>


We often want an association to carry extra information about
the relationship. In the shop example, each product in a
basket has a quantity assigned to it via

(defpclass* products-in-basket ()
  ((quantity :type integer-16))
  (:documentation "Specifies the quantity of a product in a basket"))

;; defassociation is used to define 1-1, 1-n, m-n persistent associations
;; referential integrity is kept between the two slots in the two owner classes
(defassociation*
  ((:class basket :slot products-in-basket :type (set products-in-basket))
   (:class products-in-basket :slot basket :type basket)))

That seems a bit clumsy to me.
Is there a reason for not being able to say something like

(defassociation*
  ((:class basket :slot products-in-basket :type (set products-in-basket))
   (:class products-in-basket :slot basket :type basket)))
  (quantity :type integer-16))

?

  Leslie



From leslie.polzer at gmx.net  Sun Aug 24 13:48:15 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Sun, 24 Aug 2008 15:48:15 +0200 (CEST)
Subject: [Cl-perec-devel] Associations and auxiliary information
Message-ID: <62279.88.73.244.169.1219585695.squirrel@mail.stardawn.org>


> (defassociation*
>   ((:class basket :slot products-in-basket :type (set products-in-basket))
>    (:class products-in-basket :slot basket :type basket)))
>   (quantity :type integer-16))

Err... that should read:

(defassociation*
  ((:class basket :slot products :type (set products))
   (:class product :slot basket :type basket)))
  (quantity :type integer-16))

of course.

  Leslie



From levente.meszaros at gmail.com  Mon Aug 25 08:17:28 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Mon, 25 Aug 2008 10:17:28 +0200
Subject: [Cl-perec-devel] Deferring persistence
In-Reply-To: <62675.84.157.16.47.1219318853.squirrel@mail.stardawn.org>
References: <62675.84.157.16.47.1219318853.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808250117j4ef2acf3j9b7b96105abd8cf@mail.gmail.com>

You can call (make-instance 'foo :persistent #f) and later on
(make-persistent foo-instance).

There is event (with-making-transient-instances) and
(with-making-persistent-instances)

On Thu, Aug 21, 2008 at 1:40 PM, Leslie P. Polzer <leslie.polzer at gmx.net> wrote:
>
> Forgot one question in the other message:
>
> Can I defer (or entirely cancel) storage of an object until
> some point after MAKE-INSTANCE?
>
>  Leslie
>
> _______________________________________________
> Cl-perec-devel mailing list
> Cl-perec-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/cl-perec-devel
>



-- 
There's no perfectoin


From levente.meszaros at gmail.com  Mon Aug 25 11:51:15 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Mon, 25 Aug 2008 13:51:15 +0200
Subject: [Cl-perec-devel] Associations and auxiliary information
In-Reply-To: <62279.88.73.244.169.1219585695.squirrel@mail.stardawn.org>
References: <62279.88.73.244.169.1219585695.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808250451t3aa7ef8en4b746896582531bc@mail.gmail.com>

On Sun, Aug 24, 2008 at 3:48 PM, Leslie P. Polzer <leslie.polzer at gmx.net> wrote:
>
>> (defassociation*
>>   ((:class basket :slot products-in-basket :type (set products-in-basket))
>>    (:class products-in-basket :slot basket :type basket)))
>>   (quantity :type integer-16))
>
> Err... that should read:
>
> (defassociation*
>  ((:class basket :slot products :type (set products))
>   (:class product :slot basket :type basket)))
>  (quantity :type integer-16))
>
> of course.
I know what you mean by that, we have already discussed internally
that something similar would be a nice to have but not critical.

At the moment we add a separate class with the slots and associate it
with both ends. BTW, this scheme works for n-ary associations too,
which is not directly supported.

levy
-- 
There's no perfectoin


From levente.meszaros at gmail.com  Mon Aug 25 11:56:42 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Mon, 25 Aug 2008 13:56:42 +0200
Subject: [Cl-perec-devel] Special requirements
In-Reply-To: <59723.84.157.16.47.1219345861.squirrel@mail.stardawn.org>
References: <61461.84.157.16.47.1219318744.squirrel@mail.stardawn.org>
	<b54b34af0808210638s73e3d81eya8c28e31a748454e@mail.gmail.com>
	<60743.84.157.16.47.1219339639.squirrel@mail.stardawn.org>
	<b54b34af0808211137k71b532c4vcab6d5095cc2b7dd@mail.gmail.com>
	<59723.84.157.16.47.1219345861.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808250456p6466059dm7c9972c7252cd1e3@mail.gmail.com>

>> currently each slot change ends up in an sql update. it could be
>> optimized to gather slot changes until the transaction is actually
>> comitted, but it changes the semantics slightly and there are also
>> other lower fruits before that optimization.
>
> Ah, so I need to explicitly wrap all slot writes/uncached reads
> in WITH-TRANSACTION?
It depends, we usually put with-transaction high enough so that you can
write complex algorithms which modify the database at once or not at all.

On the other hande we have for example an algorithm which does a complex
computation and it uses all 9 nodes with 4 cores per node and runs in
parallel by
serializing continuations into the database.

>
> Also, is there a perec-idiomatic way to set the pg isolation level?
There is always (cl-rdbms:execute-command "SET TRANSACTION ISOLATION
READ COMMITED")
or something similar.

It's not rare but sometimes easier to use direct SQL, for example, a
hand optimized query.

levy
-- 
There's no perfectoin


From levente.meszaros at gmail.com  Mon Aug 25 12:02:43 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Mon, 25 Aug 2008 14:02:43 +0200
Subject: [Cl-perec-devel] Associations and auxiliary information
In-Reply-To: <62279.88.73.244.169.1219585695.squirrel@mail.stardawn.org>
References: <62279.88.73.244.169.1219585695.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808250502t626314d0j14632197f2f8537a@mail.gmail.com>

On Sun, Aug 24, 2008 at 3:48 PM, Leslie P. Polzer <leslie.polzer at gmx.net> wrote:
>
>> (defassociation*
>>   ((:class basket :slot products-in-basket :type (set products-in-basket))
>>    (:class products-in-basket :slot basket :type basket)))
>>   (quantity :type integer-16))
>
> Err... that should read:
>
> (defassociation*
>  ((:class basket :slot products :type (set products))
>   (:class product :slot basket :type basket)))
>  (quantity :type integer-16))
BTW, we have an ongoing web user interface effort called wui under
cl-dwim which is not yet public.
This is a highly customizable (with lots of generics and layered
functions) meta-gui which can automatically
make, view, edit, query objects of any class whether it is persistent
or not, etc.

We are using it internally but before making it public we have to
factor out parts depending on cl-perec and cl-dwim.

Ask Attila if you are interested.

levy

-- 
There's no perfectoin


From leslie.polzer at gmx.net  Tue Aug 26 07:38:10 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 09:38:10 +0200 (CEST)
Subject: [Cl-perec-devel] [PATCH] Adapt to typo fix in cl-rdbms
Message-ID: <62172.88.73.194.97.1219736290.squirrel@mail.stardawn.org>

That 40k limit for message bodies is annoying.

Patch at this URL:

http://www.nic-nac-project.de/~skypher/with-confirmed-destructive-changes-PEREC.patch



From levente.meszaros at gmail.com  Tue Aug 26 07:45:44 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 09:45:44 +0200
Subject: [Cl-perec-devel] [PATCH] Adapt to typo fix in cl-rdbms
In-Reply-To: <62172.88.73.194.97.1219736290.squirrel@mail.stardawn.org>
References: <62172.88.73.194.97.1219736290.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808260045j73e0bff3v5b9153f5a77005f1@mail.gmail.com>

Thanks, I guess ati will apply it, cl-rdbms has to be fixed too.

levy

On Tue, Aug 26, 2008 at 9:38 AM, Leslie P. Polzer <leslie.polzer at gmx.net> wrote:
> That 40k limit for message bodies is annoying.
>
> Patch at this URL:
>
> http://www.nic-nac-project.de/~skypher/with-confirmed-destructive-changes-PEREC.patch
>
> _______________________________________________
> Cl-perec-devel mailing list
> Cl-perec-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/cl-perec-devel
>



-- 
There's no perfectoin


From attila.lendvai at gmail.com  Tue Aug 26 08:14:38 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 26 Aug 2008 10:14:38 +0200
Subject: [Cl-perec-devel] [PATCH] Adapt to typo fix in cl-rdbms
In-Reply-To: <62172.88.73.194.97.1219736290.squirrel@mail.stardawn.org>
References: <62172.88.73.194.97.1219736290.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808260114x737ef6feu9e3a86c27381a009@mail.gmail.com>

> Patch at this URL:
>
> http://www.nic-nac-project.de/~skypher/with-confirmed-destructive-changes-PEREC.patch

pushed, thanks!

-- 
 attila


From levente.meszaros at gmail.com  Tue Aug 26 08:13:10 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 10:13:10 +0200
Subject: [Cl-perec-devel] Info
Message-ID: <a0d87c1c0808260113uff93b53oe06bb7bcec59b13b@mail.gmail.com>

Hi Lesli,

I just wanted to let you know that if you have any questions, don't
hesitate to contact us.

In case you encounter a bug it's best to let us know about it early,
because we can probably fix it easier than you. The best is if you can
provide a failing test case similar to those you can find in the
source tree. This way we can talk about what exactly is not working
and whether it should really work that way. We are strongly interested
in such things, because we are using cl-perec in production and any
potential bug might be a source of big headaches in the future.

Also if you have suggestions, like the one related to defassociation
(for which we don't have time to implement now), let us know. If you
plan to add features we can also give you hints where to start, but I
guess you have other more important things to do.

BTW: the defassociation stuff could be probably implemented by a new
metaclass. This could be a kind of a persistent association and a
persistent class at the same time. The good thing with this is that it
does not introduce complexities (such as identitfy of the relationship
among two instances) into normal persistent classes and associations.
This is similar to how time dependent values are introduced in the
'tesites' directory.

levy

-- 
There's no perfectoin


From leslie.polzer at gmx.net  Tue Aug 26 09:02:28 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 11:02:28 +0200 (CEST)
Subject: [Cl-perec-devel] *oid-instance-id-sequence-exists*
In-Reply-To: <m28wuk425h.fsf@nocandy.dyndns.org>
References: <f7b974e4-b401-4cdd-9aae-fea2e605527c@m36g2000hse.googlegroups.com>
	<m28wuk425h.fsf@nocandy.dyndns.org>
Message-ID: <62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>


This global special doesn't cope well with multiple database
initialization and scenarios where a database needs to be
reinitialized because it was purged of all tables.

There are at least three possibilities of fixing:

  a) get rid of it

  b) use a hash table to record which db has been initialized

  c) catch db error 42P01 (relation doesn't exist)
     when trying to access the sequence and create it

Solution a) incurs a performance hit, but I'm not sure how
severe it is.

Solution b) fixes the case of multiple databases
that need to be initialized but doesn't solve the issue of
reinitializing.

Not sure about solution c). It's efficient and fixes
both cases, but I'm not sure whether you like it
because it produces db errors. Plus it might hide
some unrelated errors...

Opinions?

  Leslie



From leslie.polzer at gmx.net  Tue Aug 26 09:04:54 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 11:04:54 +0200 (CEST)
Subject: [Cl-perec-devel] PURGE-INSTANCES
Message-ID: <62326.88.73.194.97.1219741494.squirrel@mail.stardawn.org>


Intuitively (or from my working with Elephant) I'd expect
to have PURGE-INSTANCES the behaviour of

(defun purge-instances (list)
  (mapcar #'purge-instance list))

How about changing it and renaming the current function
to PURGE-ALL-INSTANCES?

  Leslie



From levente.meszaros at gmail.com  Tue Aug 26 09:31:32 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 11:31:32 +0200
Subject: [Cl-perec-devel] PURGE-INSTANCES
In-Reply-To: <62326.88.73.194.97.1219741494.squirrel@mail.stardawn.org>
References: <62326.88.73.194.97.1219741494.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808260231j19abff81k7a47688ed1a9e8f3@mail.gmail.com>

On Tue, Aug 26, 2008 at 11:04 AM, Leslie P. Polzer
<leslie.polzer at gmx.net> wrote:
>
> Intuitively (or from my working with Elephant) I'd expect
> to have PURGE-INSTANCES the behaviour of
>
> (defun purge-instances (list)
>  (mapcar #'purge-instance list))
>
> How about changing it and renaming the current function
> to PURGE-ALL-INSTANCES?
There is already a purge-all-instances which is, well, purges all
persistent instances regardless their class. This is not exported but
may come handy when playing with a test database. Accepting your
suggestions would require to rename this too. There are other
*-instances methods which also have to be looket at.

Hmm, since purge-instances is generic, what about adding a method
dispatching on list? It may seem strange for the first look, but it's
not that bad if you think of purge-instances in a really generic way.

levy
-- 
There's no perfectoin


From leslie.polzer at gmx.net  Tue Aug 26 09:43:31 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 11:43:31 +0200 (CEST)
Subject: [Cl-perec-devel] PURGE-INSTANCES
In-Reply-To: <a0d87c1c0808260231j19abff81k7a47688ed1a9e8f3@mail.gmail.com>
References: <62326.88.73.194.97.1219741494.squirrel@mail.stardawn.org>
	<a0d87c1c0808260231j19abff81k7a47688ed1a9e8f3@mail.gmail.com>
Message-ID: <61021.88.73.194.97.1219743811.squirrel@mail.stardawn.org>


> Hmm, since purge-instances is generic, what about adding a method
> dispatching on list? It may seem strange for the first look, but it's
> not that bad if you think of purge-instances in a really generic way.

Sounds good to me. Want a patch?

  Leslie



From levente.meszaros at gmail.com  Tue Aug 26 09:49:25 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 11:49:25 +0200
Subject: [Cl-perec-devel] PURGE-INSTANCES
In-Reply-To: <61021.88.73.194.97.1219743811.squirrel@mail.stardawn.org>
References: <62326.88.73.194.97.1219741494.squirrel@mail.stardawn.org>
	<a0d87c1c0808260231j19abff81k7a47688ed1a9e8f3@mail.gmail.com>
	<61021.88.73.194.97.1219743811.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808260249i12c1e9f4o1ad9071d00812880@mail.gmail.com>

I guess Ati could just hit those few keystrokes... plz!

Unfortunately I work this week on a Java project, so I can't directly
commit from here.

levy

On Tue, Aug 26, 2008 at 11:43 AM, Leslie P. Polzer
<leslie.polzer at gmx.net> wrote:
>
>> Hmm, since purge-instances is generic, what about adding a method
>> dispatching on list? It may seem strange for the first look, but it's
>> not that bad if you think of purge-instances in a really generic way.
>
> Sounds good to me. Want a patch?
>
>  Leslie
>
> _______________________________________________
> Cl-perec-devel mailing list
> Cl-perec-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/cl-perec-devel
>



-- 
There's no perfectoin


From levente.meszaros at gmail.com  Tue Aug 26 10:03:09 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 12:03:09 +0200
Subject: [Cl-perec-devel] *oid-instance-id-sequence-exists*
In-Reply-To: <62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>
References: <f7b974e4-b401-4cdd-9aae-fea2e605527c@m36g2000hse.googlegroups.com>
	<m28wuk425h.fsf@nocandy.dyndns.org>
	<62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808260303t26f724afpddd398bf53f0e595@mail.gmail.com>

On Tue, Aug 26, 2008 at 11:02 AM, Leslie P. Polzer
<leslie.polzer at gmx.net> wrote:
>
> This global special doesn't cope well with multiple database
> initialization and scenarios where a database needs to be
> reinitialized because it was purged of all tables.
Well, this is a problem I see, BTW it also fails if there are any
errors during table creation and the transaction rolls back. I saw it
once if I remember correctly.

> Solution a) incurs a performance hit, but I'm not sure how
> severe it is.
Always checkint the existence in the DB is pretty slow, it's an extra
roundtrip which is nearly 1ms.

> Solution b) fixes the case of multiple databases
> that need to be initialized but doesn't solve the issue of
> reinitializing.
Reinitialization is not that important and if we provide a function
for that we can also clear the bits.

> Not sure about solution c). It's efficient and fixes
> both cases, but I'm not sure whether you like it
> because it produces db errors. Plus it might hide
> some unrelated errors...
I would like to avoid this one if possible.

The best would be not to select the oid at all. We could add it to the
insert using a stored procedure for that column which can handle the
missing case on the server pretty cheeply I guess. I don't know if
inserts can return anything, because the lisp vm needs to get the oid
back. So this might not work, but it would decrease the number of
commands for making an instance with 1. (and if we have time we could
even implement a single insert replace rule for multiple tables too,
so one instance could be 1 command)

We can customize the database in *database* and put the flag there.
This still does not solve the issue of reinitialization better than
the hashtable, but at least we don't need to look it up in hash-table
all the time. I don't remember if there is any support for that in
cl-rdbms or do we need to subclass or database types which would be
not that good.

Well, I'm still hesitating, any ideas ati?

levy

-- 
There's no perfectoin


From leslie.polzer at gmx.net  Tue Aug 26 10:05:53 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 12:05:53 +0200 (CEST)
Subject: [Cl-perec-devel] [PATCH] Fix another typo:
	shared-initialize-around-persistent-class
Message-ID: <62071.88.73.194.97.1219745153.squirrel@mail.stardawn.org>

Another trivium encountered and fixed en passant -- attached.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: shared-initialize-around-persistent-class.patch
Type: text/x-patch
Size: 48401 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/cl-perec-devel/attachments/20080826/218d2ec1/attachment.bin>

From levente.meszaros at gmail.com  Tue Aug 26 10:15:35 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 12:15:35 +0200
Subject: [Cl-perec-devel] [PATCH] Fix another typo:
	shared-initialize-around-persistent-class
In-Reply-To: <62071.88.73.194.97.1219745153.squirrel@mail.stardawn.org>
References: <62071.88.73.194.97.1219745153.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808260315s3d436456s490e4375a4f31ee4@mail.gmail.com>

2008/8/26 Leslie P. Polzer <leslie.polzer at gmx.net>:
> Another trivium encountered and fixed en passant -- attached.

Well, this is the only drawback of SLIME's fuzzy-completion...

levy

-- 
There's no perfectoin


From leslie.polzer at gmx.net  Tue Aug 26 10:25:43 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 12:25:43 +0200 (CEST)
Subject: [Cl-perec-devel] Schema evolution problem
Message-ID: <62794.88.73.194.97.1219746343.squirrel@mail.stardawn.org>


Consider the following, questions as comments in-line:

PRC(7): *db1*
#<POSTGRESQL-POSTMODERN {B026989}>

PRC(8): (defpclass bar nil nil)
#<PERSISTENT-CLASS BAR>

PRC(9): (let ((*database* *db1*)) (with-transaction (defparameter bar (make-instance
'bar))))
BAR

PRC(10): (let ((*database* *db1*)) (with-transaction (describe bar)))
#<BAR :persistent #t 30 {B68B411}>
is an instance of class #<PERSISTENT-CLASS BAR>.
The following slots have :INSTANCE allocation:
 OID                  1986950
 PERSISTENT           T
 TRANSACTION          NIL
 TRANSACTION-EVENT    :CREATED

;; I didn't use REVIVE-/LOAD-INSTANCE here, but no error is signaled.
;; Why?

PRC(12): (defpclass* bar nil ((s :type (or unbound text))))
#<PERSISTENT-CLASS BAR>

PRC(13): (let ((*database* *db1*)) (with-transaction (describe bar)))
#<BAR :persistent #t 30 {B68B411}>
is an instance of class #<PERSISTENT-CLASS BAR>.
The following slots have :INSTANCE allocation:
 OID                  1986950
 PERSISTENT           T
 TRANSACTION          NIL
 TRANSACTION-EVENT    :CREATED
debugger invoked on a SIMPLE-ERROR in thread #<THREAD "initial thread" RUNNING {A698799}>:
  Accessing a persistent #<BAR :persistent #t 30 {B68B411}> while it is not attached to
the current transaction.

;; here an error *is* signaled...
;; continuing...

PRC(14): (let ((*database* *db1*)) (with-transaction (describe (revive-instance bar))))
debugger invoked on a CL-POSTGRES-ERROR:SYNTAX-ERROR-OR-ACCESS-VIOLATION in thread
#<THREAD "initial thread" RUNNING {A698799}>:
  Database error 42703: column "_s" does not exist
Query: SELECT _s FROM _bar WHERE (_oid = $1::BIGINT)

;; now that's the real issue: why hasn't the column for slot S been created?

  Leslie



From attila.lendvai at gmail.com  Tue Aug 26 10:36:22 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 26 Aug 2008 12:36:22 +0200
Subject: [Cl-perec-devel] [PATCH] Fix another typo:
	shared-initialize-around-persistent-class
In-Reply-To: <62071.88.73.194.97.1219745153.squirrel@mail.stardawn.org>
References: <62071.88.73.194.97.1219745153.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808260336v6f37b81asc46f0fde7d30329d@mail.gmail.com>

> Another trivium encountered and fixed en passant -- attached.

pushed, thanks!

-- 
 attila


From levente.meszaros at gmail.com  Tue Aug 26 11:19:51 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 13:19:51 +0200
Subject: [Cl-perec-devel] Schema evolution problem
In-Reply-To: <62794.88.73.194.97.1219746343.squirrel@mail.stardawn.org>
References: <62794.88.73.194.97.1219746343.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808260419yc961506v86e8a6c91d12f7c3@mail.gmail.com>

On Tue, Aug 26, 2008 at 12:25 PM, Leslie P. Polzer
<leslie.polzer at gmx.net> wrote:
>
> Consider the following, questions as comments in-line:
>
> PRC(7): *db1*
> #<POSTGRESQL-POSTMODERN {B026989}>
>
> PRC(8): (defpclass bar nil nil)
> #<PERSISTENT-CLASS BAR>
>
> PRC(9): (let ((*database* *db1*)) (with-transaction (defparameter bar (make-instance
BTW, there is a with-database macro...

> 'bar))))
> BAR
>
> PRC(10): (let ((*database* *db1*)) (with-transaction (describe bar)))
> #<BAR :persistent #t 30 {B68B411}>
> is an instance of class #<PERSISTENT-CLASS BAR>.
> The following slots have :INSTANCE allocation:
>  OID                  1986950
>  PERSISTENT           T
>  TRANSACTION          NIL
>  TRANSACTION-EVENT    :CREATED
>
> ;; I didn't use REVIVE-/LOAD-INSTANCE here, but no error is signaled.
> ;; Why?
This is because there were no persistent slot access, all of those
slots are standard slots. I guess the check should go in
slot-value.lisp where we are reading the persistent flag in svuc, but
even in that case oid, transaction and transaction-event could be read
without signalling an error. I don't know whether if we want to forbid
that. You can always explicityly check if an instance is in the
current transaction. This check supposed to catch errors early and
it's questionable whether if those slots accesses are errors.

> PRC(12): (defpclass* bar nil ((s :type (or unbound text))))
> #<PERSISTENT-CLASS BAR>
>
> PRC(13): (let ((*database* *db1*)) (with-transaction (describe bar)))
> #<BAR :persistent #t 30 {B68B411}>
> is an instance of class #<PERSISTENT-CLASS BAR>.
> The following slots have :INSTANCE allocation:
>  OID                  1986950
>  PERSISTENT           T
>  TRANSACTION          NIL
>  TRANSACTION-EVENT    :CREATED
> debugger invoked on a SIMPLE-ERROR in thread #<THREAD "initial thread" RUNNING {A698799}>:
>  Accessing a persistent #<BAR :persistent #t 30 {B68B411}> while it is not attached to
> the current transaction.
>
> ;; here an error *is* signaled...
> ;; continuing...
>
> PRC(14): (let ((*database* *db1*)) (with-transaction (describe (revive-instance bar))))
> debugger invoked on a CL-POSTGRES-ERROR:SYNTAX-ERROR-OR-ACCESS-VIOLATION in thread
> #<THREAD "initial thread" RUNNING {A698799}>:
>  Database error 42703: column "_s" does not exist
> Query: SELECT _s FROM _bar WHERE (_oid = $1::BIGINT)
>
> ;; now that's the real issue: why hasn't the column for slot S been created?
Because revive-instance does not call ensure-exported on the class.
Currently only make-instance and queries do that, we add it to
load-instance which would add some extra cost there, but I guess that
is neglectable compared to the cost of the RDBMS select.

levy

-- 
There's no perfectoin


From leslie.polzer at gmx.net  Tue Aug 26 11:58:17 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 13:58:17 +0200 (CEST)
Subject: [Cl-perec-devel] Schema evolution problem
In-Reply-To: <a0d87c1c0808260419yc961506v86e8a6c91d12f7c3@mail.gmail.com>
References: <62794.88.73.194.97.1219746343.squirrel@mail.stardawn.org>
	<a0d87c1c0808260419yc961506v86e8a6c91d12f7c3@mail.gmail.com>
Message-ID: <63199.88.73.194.97.1219751897.squirrel@mail.stardawn.org>


> Because revive-instance does not call ensure-exported on the class.
> Currently only make-instance and queries do that, we add it to
> load-instance which would add some extra cost there, but I guess that
> is neglectable compared to the cost of the RDBMS select.

Alright -- do you want me to make that change?

  Leslie



From levente.meszaros at gmail.com  Tue Aug 26 12:08:17 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 14:08:17 +0200
Subject: [Cl-perec-devel] Schema evolution problem
In-Reply-To: <63199.88.73.194.97.1219751897.squirrel@mail.stardawn.org>
References: <62794.88.73.194.97.1219746343.squirrel@mail.stardawn.org>
	<a0d87c1c0808260419yc961506v86e8a6c91d12f7c3@mail.gmail.com>
	<63199.88.73.194.97.1219751897.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808260508x1c7e7004g358b843cf1ebb421@mail.gmail.com>

On Tue, Aug 26, 2008 at 1:58 PM, Leslie P. Polzer <leslie.polzer at gmx.net> wrote:
>
>> Because revive-instance does not call ensure-exported on the class.
>> Currently only make-instance and queries do that, we add it to
>> load-instance which would add some extra cost there, but I guess that
>> is neglectable compared to the cost of the RDBMS select.
>
> Alright -- do you want me to make that change?
I think these kind of simple changes (which takes less than 30
seconds) are better to be done by us than by you recording the patch,
sending it along, and us applying it.

If I were sitting in front of my linux I would have probably already done it...

levy
-- 
There's no perfectoin


From attila.lendvai at gmail.com  Tue Aug 26 12:08:45 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 26 Aug 2008 14:08:45 +0200
Subject: [Cl-perec-devel] *oid-instance-id-sequence-exists*
In-Reply-To: <62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>
References: <f7b974e4-b401-4cdd-9aae-fea2e605527c@m36g2000hse.googlegroups.com>
	<m28wuk425h.fsf@nocandy.dyndns.org>
	<62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808260508w5632ddd4of0c08ce7504478ec@mail.gmail.com>

> This global special doesn't cope well with multiple database
> initialization and scenarios where a database needs to be
> reinitialized because it was purged of all tables.

pull rdbms and perec, i've moved that global into a slot on *database*.

you'll need to slightly change the way you set up your *database*,
consult the perec test diff for the details.

-- 
 attila


From attila.lendvai at gmail.com  Tue Aug 26 12:17:01 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 26 Aug 2008 14:17:01 +0200
Subject: [Cl-perec-devel] Schema evolution problem
In-Reply-To: <a0d87c1c0808260508x1c7e7004g358b843cf1ebb421@mail.gmail.com>
References: <62794.88.73.194.97.1219746343.squirrel@mail.stardawn.org>
	<a0d87c1c0808260419yc961506v86e8a6c91d12f7c3@mail.gmail.com>
	<63199.88.73.194.97.1219751897.squirrel@mail.stardawn.org>
	<a0d87c1c0808260508x1c7e7004g358b843cf1ebb421@mail.gmail.com>
Message-ID: <b54b34af0808260517h3051306dv87d4cb6b3b2f89c4@mail.gmail.com>

>> Alright -- do you want me to make that change?
> I think these kind of simple changes (which takes less than 30
> seconds) are better to be done by us than by you recording the patch,
> sending it along, and us applying it.

Leslie is sending proper darcs patches, so it would be ok, but i've
just pushed this change.

-- 
 attila

ps:  at least he would get the credit that i've forgotten to record in
this last patch. sorry for that!


From levente.meszaros at gmail.com  Tue Aug 26 12:26:11 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 14:26:11 +0200
Subject: [Cl-perec-devel] Schema evolution problem
In-Reply-To: <b54b34af0808260517h3051306dv87d4cb6b3b2f89c4@mail.gmail.com>
References: <62794.88.73.194.97.1219746343.squirrel@mail.stardawn.org>
	<a0d87c1c0808260419yc961506v86e8a6c91d12f7c3@mail.gmail.com>
	<63199.88.73.194.97.1219751897.squirrel@mail.stardawn.org>
	<a0d87c1c0808260508x1c7e7004g358b843cf1ebb421@mail.gmail.com>
	<b54b34af0808260517h3051306dv87d4cb6b3b2f89c4@mail.gmail.com>
Message-ID: <a0d87c1c0808260526s48baf53fvfaa2b94801c319e8@mail.gmail.com>

I don't mind it, just wanted to simplify things, but as for the
credits, you are right.

levy

On Tue, Aug 26, 2008 at 2:17 PM, Attila Lendvai
<attila.lendvai at gmail.com> wrote:
>>> Alright -- do you want me to make that change?
>> I think these kind of simple changes (which takes less than 30
>> seconds) are better to be done by us than by you recording the patch,
>> sending it along, and us applying it.
>
> Leslie is sending proper darcs patches, so it would be ok, but i've
> just pushed this change.
>
> --
>  attila
>
> ps:  at least he would get the credit that i've forgotten to record in
> this last patch. sorry for that!
>



-- 
There's no perfectoin


From attila.lendvai at gmail.com  Tue Aug 26 12:30:24 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 26 Aug 2008 14:30:24 +0200
Subject: [Cl-perec-devel] PURGE-INSTANCES
In-Reply-To: <a0d87c1c0808260249i12c1e9f4o1ad9071d00812880@mail.gmail.com>
References: <62326.88.73.194.97.1219741494.squirrel@mail.stardawn.org>
	<a0d87c1c0808260231j19abff81k7a47688ed1a9e8f3@mail.gmail.com>
	<61021.88.73.194.97.1219743811.squirrel@mail.stardawn.org>
	<a0d87c1c0808260249i12c1e9f4o1ad9071d00812880@mail.gmail.com>
Message-ID: <b54b34af0808260530i6d9d7d82hd5b8cde3787fa641@mail.gmail.com>

> I guess Ati could just hit those few keystrokes... plz!

pushed.

> Unfortunately I work this week on a Java project, so I can't directly
> commit from here.

anybody knows about a fat order for a webapp, so that we can convince
Levy and Tomi to leave that java job? :)

-- 
 attila


From levente.meszaros at gmail.com  Tue Aug 26 12:49:47 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Tue, 26 Aug 2008 14:49:47 +0200
Subject: [Cl-perec-devel] PURGE-INSTANCES
In-Reply-To: <b54b34af0808260530i6d9d7d82hd5b8cde3787fa641@mail.gmail.com>
References: <62326.88.73.194.97.1219741494.squirrel@mail.stardawn.org>
	<a0d87c1c0808260231j19abff81k7a47688ed1a9e8f3@mail.gmail.com>
	<61021.88.73.194.97.1219743811.squirrel@mail.stardawn.org>
	<a0d87c1c0808260249i12c1e9f4o1ad9071d00812880@mail.gmail.com>
	<b54b34af0808260530i6d9d7d82hd5b8cde3787fa641@mail.gmail.com>
Message-ID: <a0d87c1c0808260549h7a904246n71c01108b076f98b@mail.gmail.com>

> anybody knows about a fat order for a webapp, so that we can convince
> Levy and Tomi to leave that java job? :)
Khm... Java, C++, Swig, Perl, several home grown proprietary
languages, bugridden 1% of CL interpreters and other kinds of
animals....

levy
-- 
There's no perfectoin


From leslie.polzer at gmx.net  Tue Aug 26 14:02:58 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 16:02:58 +0200 (CEST)
Subject: [Cl-perec-devel] *oid-instance-id-sequence-exists*
In-Reply-To: <b54b34af0808260508w5632ddd4of0c08ce7504478ec@mail.gmail.com>
References: <f7b974e4-b401-4cdd-9aae-fea2e605527c@m36g2000hse.googlegroups.com>
	<m28wuk425h.fsf@nocandy.dyndns.org>
	<62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>
	<b54b34af0808260508w5632ddd4of0c08ce7504478ec@mail.gmail.com>
Message-ID: <64498.88.73.194.97.1219759378.squirrel@mail.stardawn.org>


> pull rdbms and perec, i've moved that global into a slot on *database*.

Great, thanks for making all these changes so quickly!



From leslie.polzer at gmx.net  Tue Aug 26 14:20:07 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 16:20:07 +0200 (CEST)
Subject: [Cl-perec-devel] *oid-instance-id-sequence-exists*
In-Reply-To: <b54b34af0808260508w5632ddd4of0c08ce7504478ec@mail.gmail.com>
References: <f7b974e4-b401-4cdd-9aae-fea2e605527c@m36g2000hse.googlegroups.com>
	<m28wuk425h.fsf@nocandy.dyndns.org>
	<62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>
	<b54b34af0808260508w5632ddd4of0c08ce7504478ec@mail.gmail.com>
Message-ID: <61265.88.73.194.97.1219760407.squirrel@mail.stardawn.org>


> you'll need to slightly change the way you set up your *database*,
> consult the perec test diff for the details.

For convenience, how about exporting a ready-made class with
database-mixin?

  Leslie



From attila.lendvai at gmail.com  Tue Aug 26 14:24:19 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 26 Aug 2008 16:24:19 +0200
Subject: [Cl-perec-devel] *oid-instance-id-sequence-exists*
In-Reply-To: <61265.88.73.194.97.1219760407.squirrel@mail.stardawn.org>
References: <f7b974e4-b401-4cdd-9aae-fea2e605527c@m36g2000hse.googlegroups.com>
	<m28wuk425h.fsf@nocandy.dyndns.org>
	<62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>
	<b54b34af0808260508w5632ddd4of0c08ce7504478ec@mail.gmail.com>
	<61265.88.73.194.97.1219760407.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808260724k4aacc598td36edd167639bb5@mail.gmail.com>

>> you'll need to slightly change the way you set up your *database*,
>> consult the perec test diff for the details.
>
> For convenience, how about exporting a ready-made class with
> database-mixin?

you'd need one for all rdbms types (oracle, sqlite, etc)... did you
consider that, too?

-- 
 attila


From leslie.polzer at gmx.net  Tue Aug 26 14:36:00 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 16:36:00 +0200 (CEST)
Subject: [Cl-perec-devel] *oid-instance-id-sequence-exists*
In-Reply-To: <b54b34af0808260724k4aacc598td36edd167639bb5@mail.gmail.com>
References: <f7b974e4-b401-4cdd-9aae-fea2e605527c@m36g2000hse.googlegroups.com>
	<m28wuk425h.fsf@nocandy.dyndns.org>
	<62587.88.73.194.97.1219741348.squirrel@mail.stardawn.org>
	<b54b34af0808260508w5632ddd4of0c08ce7504478ec@mail.gmail.com>
	<61265.88.73.194.97.1219760407.squirrel@mail.stardawn.org>
	<b54b34af0808260724k4aacc598td36edd167639bb5@mail.gmail.com>
Message-ID: <62425.88.73.194.97.1219761360.squirrel@mail.stardawn.org>


>> For convenience, how about exporting a ready-made class with
>> database-mixin?
>
> you'd need one for all rdbms types (oracle, sqlite, etc)... did you
> consider that, too?

Each backend could bring its own.

OTOH since the backends are supplied by cl-rdbms it's probably
not worth the hassle of figuring out the responsibilities...

  Leslie



From leslie.polzer at gmx.net  Tue Aug 26 14:37:56 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 16:37:56 +0200 (CEST)
Subject: [Cl-perec-devel] Copying instances across databases
Message-ID: <62504.88.73.194.97.1219761476.squirrel@mail.stardawn.org>


I have been somewhat unsuccessful at coercing perec's metacopy
protocol to copy my objects across databases.

How do I copy a persistent object from db A into db B?

  Leslie



From leslie.polzer at gmx.net  Tue Aug 26 19:10:48 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 21:10:48 +0200 (CEST)
Subject: [Cl-perec-devel] Schema evolution: handling of initforms
Message-ID: <63593.88.73.194.97.1219777848.squirrel@mail.stardawn.org>


I just noticed that existing instances don't get updated
with a specified initform when a slot addition takes place.

Deliberate or a bug?

  Leslie



From leslie.polzer at gmx.net  Tue Aug 26 20:32:54 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Tue, 26 Aug 2008 22:32:54 +0200 (CEST)
Subject: [Cl-perec-devel] Another multi-db bug?
Message-ID: <63926.88.73.194.97.1219782774.squirrel@mail.stardawn.org>


(defcclass* exportable ()
  ((ensure-exported
    (compute-as (export-to-rdbms -self-) -self-)
    :reader ensure-exported
    ...)))

I'm not familiar with COMPUTED-CLASS. How does this work?

When I evolve a class, ENSURE-EXPORTED seems to call
EXPORT-TO-RDBMS only once, meaning that only the first
of multiple databases gets their table updated.

  Leslie



From attila.lendvai at gmail.com  Tue Aug 26 21:04:10 2008
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 26 Aug 2008 23:04:10 +0200
Subject: [Cl-perec-devel] Another multi-db bug?
In-Reply-To: <63926.88.73.194.97.1219782774.squirrel@mail.stardawn.org>
References: <63926.88.73.194.97.1219782774.squirrel@mail.stardawn.org>
Message-ID: <b54b34af0808261404q379016ebnc35cafc3530604ab@mail.gmail.com>

> (defcclass* exportable ()
>  ((ensure-exported
>    (compute-as (export-to-rdbms -self-) -self-)
>    :reader ensure-exported
>    ...)))
>
> I'm not familiar with COMPUTED-CLASS. How does this work?
>
> When I evolve a class, ENSURE-EXPORTED seems to call
> EXPORT-TO-RDBMS only once, meaning that only the first
> of multiple databases gets their table updated.

positive, this is a bug - or maybe an unsupported feature. keep in
mind that the schema export is treated mostly as a development-time
tool that helps making progress while working. at deployment you'll
need to handle things carefully anyway... for example our server
refuses to start when it detects that the schema is out of sync. then
it needs to be started as a repl session with user feedbacks...

the problem is that both databases are in the same
"computed-universe", so when a class is exported in one database then
it gets "computed" (iow, valid) so the export is considered done when
accessing the next database.

you can C-c C-c something/anything again and then the export will run again.

i have no quick fix for this.

what's your use-case for using two databases? is it schema evolution
and the involved data migration?

-- 
 attila


From levente.meszaros at gmail.com  Wed Aug 27 07:53:19 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Wed, 27 Aug 2008 09:53:19 +0200
Subject: [Cl-perec-devel] Schema evolution: handling of initforms
In-Reply-To: <63593.88.73.194.97.1219777848.squirrel@mail.stardawn.org>
References: <63593.88.73.194.97.1219777848.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808270053l1a748a9bq6cf57fdb76e3d6b0@mail.gmail.com>

On Tue, Aug 26, 2008 at 9:10 PM, Leslie P. Polzer <leslie.polzer at gmx.net> wrote:
>
> I just noticed that existing instances don't get updated
> with a specified initform when a slot addition takes place.
>
> Deliberate or a bug?
It is a missing feature and unfortunately it can even make impossible
to load an instance from the database after the slot addition. For
example, if you define a new slot with integer-16 and do not allow
nils, then the database will be incorrect, because the alter table
will add the column with NULL default value which is certainly not an
integer.

If the initform is a constant it could go down to the ALTER TABLE sql
command, probably with some cl-rdbms support its not that difficult to
implement. We don't have tests for this and when occasionally we need
to do such things we workaround by issuing an SQL update.

Sorry for your inconvenience,
levy
-- 
There's no perfectoin


From leslie.polzer at gmx.net  Wed Aug 27 08:15:23 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Wed, 27 Aug 2008 10:15:23 +0200 (CEST)
Subject: [Cl-perec-devel] Another multi-db bug?
In-Reply-To: <b54b34af0808261404q379016ebnc35cafc3530604ab@mail.gmail.com>
References: <63926.88.73.194.97.1219782774.squirrel@mail.stardawn.org>
	<b54b34af0808261404q379016ebnc35cafc3530604ab@mail.gmail.com>
Message-ID: <63075.88.73.223.167.1219824923.squirrel@mail.stardawn.org>


> the problem is that both databases are in the same
> "computed-universe", so when a class is exported in one database then
> it gets "computed" (iow, valid) so the export is considered done when
> accessing the next database.

I suppose that slot should be a list that stores which databases
have been updated and which have not.


> what's your use-case for using two databases? is it schema evolution
> and the involved data migration?

Performance, separation of concerns and convenience e.g. when backing-up
changes.

I need to have several semantically and referentially self-contained
collections of data (e.g. users and their messages).
Each Lisp process has access to exactly one of this and doesn't need
access to the others.

In addition, there is one that is shared by all others, and I need
to clone objects off it. It's some kind of instantiation.

Talking Plato, exactly one of my databases is a shared collection of
"ideals" and the others are separate real-world instances of these
ideal objects.

Thus each process needs access to the public shared storage and to its
own private storage.

I'd appreciate alternate proposals, but so far this has shown to be
the most efficient way.

  Leslie



From leslie.polzer at gmx.net  Wed Aug 27 08:55:20 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Wed, 27 Aug 2008 10:55:20 +0200 (CEST)
Subject: [Cl-perec-devel] Schema evolution: handling of initforms
In-Reply-To: <a0d87c1c0808270053l1a748a9bq6cf57fdb76e3d6b0@mail.gmail.com>
References: <63593.88.73.194.97.1219777848.squirrel@mail.stardawn.org>
	<a0d87c1c0808270053l1a748a9bq6cf57fdb76e3d6b0@mail.gmail.com>
Message-ID: <64172.88.73.223.167.1219827320.squirrel@mail.stardawn.org>


> On Tue, Aug 26, 2008 at 9:10 PM, Leslie P. Polzer <leslie.polzer at gmx.net> wrote:
>>
>> I just noticed that existing instances don't get updated
>> with a specified initform when a slot addition takes place.
>>
>> Deliberate or a bug?
> It is a missing feature and unfortunately it can even make impossible
> to load an instance from the database after the slot addition. For
> example, if you define a new slot with integer-16 and do not allow
> nils, then the database will be incorrect, because the alter table
> will add the column with NULL default value which is certainly not an
> integer.

Sounds bad to me because it breaks a basic assumption about the
MOP and, as you said, the integrity of the database.

I'll look into this.

  Leslie



From leslie.polzer at gmx.net  Wed Aug 27 14:41:51 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Wed, 27 Aug 2008 16:41:51 +0200 (CEST)
Subject: [Cl-perec-devel] Another multi-db bug?
In-Reply-To: <b54b34af0808261404q379016ebnc35cafc3530604ab@mail.gmail.com>
References: <63926.88.73.194.97.1219782774.squirrel@mail.stardawn.org>
	<b54b34af0808261404q379016ebnc35cafc3530604ab@mail.gmail.com>
Message-ID: <62031.88.73.223.167.1219848111.squirrel@mail.stardawn.org>


> what's your use-case for using two databases? is it schema evolution
> and the involved data migration?

I'm contemplating going down to CLSQL's object interface
for this one project. The requirements seem to require a
more conservative approach to data storage...

  Leslie





From levente.meszaros at gmail.com  Wed Aug 27 15:15:03 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Wed, 27 Aug 2008 17:15:03 +0200
Subject: [Cl-perec-devel] Copying instances across databases
In-Reply-To: <62504.88.73.194.97.1219761476.squirrel@mail.stardawn.org>
References: <62504.88.73.194.97.1219761476.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808270815r492d9e6fy901fd0b4ca1af77f@mail.gmail.com>

On Tue, Aug 26, 2008 at 4:37 PM, Leslie P. Polzer <leslie.polzer at gmx.net> wrote:
>
> I have been somewhat unsuccessful at coercing perec's metacopy
> protocol to copy my objects across databases.
>
> How do I copy a persistent object from db A into db B?
This is something we did not do in the past.

a. You can use the export/import interface which can serialize (using
cl-serializer) an object graph into a binary file and read it back in
the other database.
b. If the data to be copied fits into memory you can read it in, swith
to the other database and write it out by using the metacopy protocol.
b2. After read you can set the persistent flag to #f in all instances
and make them persistent in the other database using make-persistent.
c. You can also cusomize the metacopy protocol to switch back and
forth (I don't exactly know how at the moment, no repl at hand)
d. Using direct SQL

I can't decide which one is best for you. It strongly depends on the details.

levy
-- 
There's no perfectoin


From leslie.polzer at gmx.net  Wed Aug 27 17:58:44 2008
From: leslie.polzer at gmx.net (Leslie P. Polzer)
Date: Wed, 27 Aug 2008 19:58:44 +0200 (CEST)
Subject: [Cl-perec-devel] Schema evolution: handling of initforms
In-Reply-To: <a0d87c1c0808270053l1a748a9bq6cf57fdb76e3d6b0@mail.gmail.com>
References: <63593.88.73.194.97.1219777848.squirrel@mail.stardawn.org>
	<a0d87c1c0808270053l1a748a9bq6cf57fdb76e3d6b0@mail.gmail.com>
Message-ID: <64968.88.73.223.167.1219859924.squirrel@mail.stardawn.org>


> If the initform is a constant it could go down to the ALTER TABLE sql
> command, probably with some cl-rdbms support its not that difficult to
> implement. We don't have tests for this and when occasionally we need
> to do such things we workaround by issuing an SQL update.

It's pretty simple if the initform shall be evaluated at
column recomputation (persistence/class.lisp::compute-columns).

If we want to evaluate the initform at ALTER TABLE time, though
(which I suggest), we need to do something about the DEFAULT-VALUE
slot of class SQL-COLUMN (cl-rdbms/syntax/create-table.lisp),
making it hold a form that is evaluated before use.

What do you think?

  Leslie



From levente.meszaros at gmail.com  Thu Aug 28 11:18:18 2008
From: levente.meszaros at gmail.com (=?ISO-8859-1?Q?Levente_M=E9sz=E1ros?=)
Date: Thu, 28 Aug 2008 13:18:18 +0200
Subject: [Cl-perec-devel] Schema evolution: handling of initforms
In-Reply-To: <64968.88.73.223.167.1219859924.squirrel@mail.stardawn.org>
References: <63593.88.73.194.97.1219777848.squirrel@mail.stardawn.org>
	<a0d87c1c0808270053l1a748a9bq6cf57fdb76e3d6b0@mail.gmail.com>
	<64968.88.73.223.167.1219859924.squirrel@mail.stardawn.org>
Message-ID: <a0d87c1c0808280418g5f852a9cl66b798fdf2221d79@mail.gmail.com>

> It's pretty simple if the initform shall be evaluated at
> column recomputation (persistence/class.lisp::compute-columns).
>
> If we want to evaluate the initform at ALTER TABLE time, though
> (which I suggest), we need to do something about the DEFAULT-VALUE
> slot of class SQL-COLUMN (cl-rdbms/syntax/create-table.lisp),
> making it hold a form that is evaluated before use.
>
> What do you think?
I have no objections putting a lambda or function into the
DEFAULT-VALUE of SQL-COLUMN in perec and have it called in rdbms when
altering the table or adding the column. On the other hand putting a
lisp form and calling eval on it is not a good idea.

Also, don't forget that you need to pass the result of the initform to
the writer function of the slot to get the RDBMS values (a lisp value
may be mapped to multiple rdbms columns). I guess putting these
lambdas into compute-columns is ok.

levy
-- 
There's no perfectoin