From alex.mizrahi at gmail.com  Mon Sep  5 14:10:23 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Mon, 05 Sep 2011 17:10:23 +0300
Subject: [elephant-devel] status update on db-postmodern
Message-ID: <j42l8f$vo0$1@dough.gmane.org>

I've fixed problem with db-postmodern, now it passes almost all tests on 
SBCL 1.0.16 (except NIL-SLOT-CLASS).

The problem was that with-btree-cursor using WITHOUT-INTERRUPTS while 
making cursor to prevent potential resource leak, but POSTMODERN depends 
on interrupts for socket communications (select).

I've fixed this problem through use of ALLOW-WITH-INTERRUPTS in 
ele:WITHOUT-INTERRUPTS -- it allows re-enabling interrupts -- and 
WITH-INTERRUPTS around code which works with DB.

This is SBCL-specific solutions, but we don't support WITHOUT-INTERRUPTS 
on other implementations anyway. (Well, we have it for AllegroCL but I 
don't support it.)

Other possible solution is to remove WITHOUT-INTERRUPTS, I don't see 
much value in preventing resource leaks when people cancel threads 
through signals. Canceling threads sucks.

...

There is still a potential problem with select being called when 
interrupts are disabled: schema-related functions which access DB might 
be called while compiling/loading code, and SBCL has internal 
WITHOUT-INTERRUPTS around code which calls our code.  But, well, it 
works anyway.Problem might happen only when you're loading/compiling 
code, so if it already works it should be fine.




From alex.mizrahi at gmail.com  Tue Sep  6 10:34:37 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Tue, 06 Sep 2011 13:34:37 +0300
Subject: [elephant-devel] Are there any postmodern backend users here?
Message-ID: <j44svu$fca$1@dough.gmane.org>

I would like to discuss handling of types, i.e. integer/string keys 
which support range queries (and work faster).

There is a couple of options for handling type mismatches, I want to 
consider possible applications.




From alex.mizrahi at gmail.com  Fri Sep  9 10:23:48 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Fri, 09 Sep 2011 13:23:48 +0300
Subject: [elephant-devel] CL implementation support
Message-ID: <j4cpfl$9us$1@dough.gmane.org>

It looks like something weird happens with CCL's CLOS when we load 
Elephant. Given that we have openmcl-mop-patches.lisp, it might be that 
we break something with out patches. Or something...

Is anybody interested in Elephant on CCL? Or other CL implementations?

If not, it will be SBCL-only until somebody enthusiastic enough comes 
and fixes problems.




From alex.mizrahi at gmail.com  Fri Sep  9 11:17:22 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Fri, 09 Sep 2011 14:17:22 +0300
Subject: [elephant-devel] SBCL errors
In-Reply-To: <4E533290.9050008@chatsubo.net>
References: <4E323FA6.2020000@chatsubo.net>	<CADow0CqMmvgTtk=_0Lk7PtFsry-PF0-khaXCSWevF-vVQnGokQ@mail.gmail.com>
	<4E533290.9050008@chatsubo.net>
Message-ID: <j4csk2$ju$1@dough.gmane.org>

>>> Unexpected Error: #<SB-INT:BUG "~@<failed AVER: ~2I~_~A~:>"
>>> {1002E81E41}>
>>> failed AVER: NSLOTDS

I've found source of this error -- SBCL's internal %UPDATE-CLASS assumes 
that slot-allocation is either :instance or :class (and we have 
:database for persistent slots), so it breaks when you drop a virtual 
slot (i.e. redefine class without this slot).

It is trivial to fix this particular problem, but I'm not sure that rest 
of %UPDATE-CLASS code is correct.

So I didn't submit a patch: it would be better if SBCL developers review 
%UPDATE-CLASS and related code assuming that some slots might be 
"virtual". I just don't understand SBCL internals very well.

You can probably ignore this bug and use latest version of SBCL anyway 
if you abstain from redefining classes with dropped slots :)




From eslick at media.mit.edu  Fri Sep  9 15:53:41 2011
From: eslick at media.mit.edu (Ian Eslick)
Date: Fri, 9 Sep 2011 08:53:41 -0700
Subject: [elephant-devel] CL implementation support
In-Reply-To: <j4cpfl$9us$1@dough.gmane.org>
References: <j4cpfl$9us$1@dough.gmane.org>
Message-ID: <1FF8F88F-AE3E-4A3B-A432-1EA3025D565E@media.mit.edu>

I use CCL for some production CL applications that use Elephant.  Which version of CCL are you having problems with?  I haven't upgraded in awhile - 1.2-1.4 were all good for me.


On Sep 9, 2011, at 3:23 AM, Alex Mizrahi wrote:

> It looks like something weird happens with CCL's CLOS when we load Elephant. Given that we have openmcl-mop-patches.lisp, it might be that we break something with out patches. Or something...
> 
> Is anybody interested in Elephant on CCL? Or other CL implementations?
> 
> If not, it will be SBCL-only until somebody enthusiastic enough comes and fixes problems.
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel




From alex.mizrahi at gmail.com  Fri Sep  9 19:06:50 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Fri, 09 Sep 2011 22:06:50 +0300
Subject: [elephant-devel] CL implementation support
In-Reply-To: <1FF8F88F-AE3E-4A3B-A432-1EA3025D565E@media.mit.edu>
References: <j4cpfl$9us$1@dough.gmane.org>
	<1FF8F88F-AE3E-4A3B-A432-1EA3025D565E@media.mit.edu>
Message-ID: <j4do4a$e25$1@dough.gmane.org>

> I use CCL for some production CL applications that use Elephant.  Which version of CCL are you having problems with?  I haven't upgraded in awhile - 1.2-1.4 were all good for me.

The latest available, 1.7, 64-bit. 1.6 had same problems IIRC.

I've localized a problem a bit: it is triggered by recompiling/reloading 
some files in src/elephant. (Maybe it is just one of problems.) So this 
looks like a problem with CCL itself. Or maybe we're breaking something...

I hope that finding which files trigger error will reveal something 
about its nature. For example,

(load (compile-file "src/elephant/classes.lisp"))

is ok. But when classes.fasl is loaded after <something> some slots just 
vanish on class redefinition.

BDB backend seems to be more vulnerable to this stuff, not just slots 
disappear but superclasses... WTF




From eslick at media.mit.edu  Fri Sep  9 19:14:34 2011
From: eslick at media.mit.edu (Ian Eslick)
Date: Fri, 9 Sep 2011 12:14:34 -0700
Subject: [elephant-devel] CL implementation support
In-Reply-To: <j4do4a$e25$1@dough.gmane.org>
References: <j4cpfl$9us$1@dough.gmane.org>
	<1FF8F88F-AE3E-4A3B-A432-1EA3025D565E@media.mit.edu>
	<j4do4a$e25$1@dough.gmane.org>
Message-ID: <2622CF25-2722-4276-8119-FD31DF536A0F@media.mit.edu>

I worked on this project with Clojure, so they may be willing to help diagnose.  I'll be looking into this later this month when I upgrade the running server and add some features to that old project I mentioned.  -Ian

On Sep 9, 2011, at 12:06 PM, Alex Mizrahi wrote:

>> I use CCL for some production CL applications that use Elephant.  Which version of CCL are you having problems with?  I haven't upgraded in awhile - 1.2-1.4 were all good for me.
> 
> The latest available, 1.7, 64-bit. 1.6 had same problems IIRC.
> 
> I've localized a problem a bit: it is triggered by recompiling/reloading some files in src/elephant. (Maybe it is just one of problems.) So this looks like a problem with CCL itself. Or maybe we're breaking something...
> 
> I hope that finding which files trigger error will reveal something about its nature. For example,
> 
> (load (compile-file "src/elephant/classes.lisp"))
> 
> is ok. But when classes.fasl is loaded after <something> some slots just vanish on class redefinition.
> 
> BDB backend seems to be more vulnerable to this stuff, not just slots disappear but superclasses... WTF
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel




From alex.mizrahi at gmail.com  Mon Sep 12 08:58:41 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Mon, 12 Sep 2011 11:58:41 +0300
Subject: [elephant-devel] CL implementation support
In-Reply-To: <2622CF25-2722-4276-8119-FD31DF536A0F@media.mit.edu>
References: <j4cpfl$9us$1@dough.gmane.org>	<1FF8F88F-AE3E-4A3B-A432-1EA3025D565E@media.mit.edu>	<j4do4a$e25$1@dough.gmane.org>
	<2622CF25-2722-4276-8119-FD31DF536A0F@media.mit.edu>
Message-ID: <j4khk1$n06$1@dough.gmane.org>

> I worked on this project with Clojure, so they may be willing to help diagnose.
> I'll be looking into this later this month when I upgrade the running server
> and add some features to that old project I mentioned.  -Ian

That would be cool, as I'm not comfortable with diving too deep into 
CCL's CLOS implementation.

I've found what happens: if you re-define PERSISTENT-METACLASS (e.g. 
(load "metaclasses.lisp") ) then classes of that metaclass lose their 
slots (CLASS-SLOTS returns NIL) and, perhaps, superclasses too, until 
you redefine them or their parents.

finalize-inheritance doesn't help: it looks like CCL thinks that classes 
are finalized while in fact their content is wiped out.

But what exactly happens with classes when their metaclass is 
redefined... I guess this question is better addressed by guys who wrote 
that CLOS implementation.

...

We trigger this problem because we force re-loading dynamic libraries 
via ASDF:

(defmethod operation-done-p ((o load-op) (c elephant-c-source))
    nil)

So, (asdf:oos 'asdf:load-op :elephant-tests) load Elephant system once.
Then when you work with a certain backend Elephant system it loaded once 
more by that backend's system definition, and ASDF thinks that operation 
is not done, so it loads both dynamic library and everything which 
depends on it, which includes metaclasses.lisp, so PERSISTENT-METACLASS 
is redefined and it fries all PERSISTENT-OBJECT subclasses.

I've tweaked method above changing NIL to T, in this case it passes 
almost all tests:

  Did 508 checks.
     Pass: 507 (99%)
     Skip: 0 ( 0%)
     Fail: 1 ( 0%)

  Failure Details:
  --------------------------------
  LIST-OF-STRINGS-RANDOM []:
       Unexpected Error: #<CCL::ARRAY-ELEMENT-TYPE-ERROR #x302001B44E8D>
NIL doesn't match array element type of <some weird unicode characters here>

...

I think it is funny that current versions of two out of two major open 
source CL implementations have CLOS too broken to run Elephant. Perhaps 
they should include Elephant into their regression tests lol




From alex.mizrahi at gmail.com  Mon Sep 12 15:26:18 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Mon, 12 Sep 2011 18:26:18 +0300
Subject: [elephant-devel] set-valued slots review
Message-ID: <j4l8ar$vd4$1@dough.gmane.org>

I. RATIONALE

There is a number of cases where one might want to store more than one 
value in a slot. An example from our testassociations: person can hold 
multiple jobs, job can have multiple holders.

There is a number of ways how you can hold these multiple values in a 
single slot:

1. Lisp collection like list or array. It is serialized as a whole, so 
you need to be careful to update slot after manipulations. If there are 
many items and list is frequently updated it is less than perfect.
2. Associations: it is, perhaps, ideal when you have relationship 
between classes. (I'll cover it in another message.)
3. pset (or btree): just assign pset instance to slot, then use pset API 
functions on it.
4. New slot-valued slot feature.

So how what is it? It is very similar to storing pset in a slot, and in 
fact it is implemented as a pset under the hood, with a number of 
differences:

1. There are convenience macros like set-insert and set-remove which 
combine slot-value with pset function. But you can use them on 
pset-in-a-slot too, so it is not really a difference.

2. On-demand ("lazy") initialization -- object is created with an 
unbound slot and it is created on first access. So it can save space 
(and time) if set is empty.

3. Pset storage is automatically reclaimed on slot-makunbound.

4. "Fancy" (setf (things-of obj) a-thing) API.

5. Special handling of assigning slot-set and NIL to a slot.

6. In theory we can implement optimized storage (on per-backend basis) 
or add more convenience features.

To be honest, I think that just using pset is good enough and features 
above do not add much. But, on the other hand, it is nice to have an 
official solution for storing multiple values in a slot. So, as it is 
already implemented, I think we should keep it and improve functionality 
and consistency.

II. API

You can use set-valued slots in three possible ways:

1. Macro API:
    (set-insert 1 object 'numbers)
    (set-remove 2 object 'numbers)
    (set-list object 'numbers) => (1)

2. Accessor/pset API:
    (insert-item 1 (numbers-of object))
    (remove-item 2 (numbers-of object))
    (find-item 1 (numbers-of object)) => t (generalized boolean)
    (slot-set-list (numbers-of object)) => (1)
    (map-slot-set (lambda (number) (print number))
          (numbers-of object))

3.
     (setf (numbers-of object) 1)
     (setf (numbers-of object) 2)
     (slot-set-list (numbers-of object)) = (1 2)

I find option `2` the best option -- code is readable and it is natural.
Macro-based is slightly shorter, but you have to quote slot name and it 
is somewhat unusual. I'm not really sure we need it, but perhaps it is 
better to keep it for people who do not like accessors.

I'm in favor of removing option 3 because it is not how SETF should work 
-- it should replace existing value, not add it. And while there are 
possible ways to improve it, I don't think it makes sense to do that 
because insert-item and friends are already good enough.

III. Weird 'features' (aka bugs)

I've mentioned that there is a special handling for the case when you 
SETF another slot-set object or NIL value. Current implementation just 
deallocates old slot-set and sets slot as usual. This actually creates a 
problem:

  * (setf (numbers-of obj) NIL) will make slot broken, none of the APIs 
would work because they assume that there is a slot-set object in a 
slot, not NIL. I guess it was just a way to wipe the set, but it doesn't 
work this way.

  * (setf (numbers-of obj1) (numbers-of obj2)) will make obj1 and obj2 
to share single slot-set. E.g. if you add something to obj1's slot it 
will appear in obj2 and vice versa.

  * (setf (numbers-of obj1) (numbers-of obj1)) will wipe all items in 
pset (and probably will make it broken too.

If we actually want to make slot-set objects transparent and employ 
"just many values in one slot" metaphor, we actually need a special 
treatment for assignment: assignment should replace contents of 
slot-set, not assign the object itself.

Thus when you do

    (setf (numbers-of obj1) (numbers-of obj2))

it should replace whatever set was in obj1 with set of obj2, but 
afterwards these slots should be independent.

As for special handling of NIL, perhaps we should just remove it 
together with a "weird API". Or maybe not, there is less harm from it.

IV. Lacking features

I think it would be great if we also add features for:

1. Initializing slot-set from list. Something like:
    (setf (numbers-of obj1) (make-slot-set :items '(1 2 3 4))

2. Wipe contents of a slot-set. You can do it via slot-makunbound, sure, 
but it is kinda ugly. (setf (numbers-of obj) NIL) is kinda weird because 
otherwise it doesn't recognize lists.
    (setf (numbers-of obj1) (make-slot-set) is, perhaps, appropriate, 
except it would create unnecessary object.




From alex.mizrahi at gmail.com  Mon Sep 12 15:31:35 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Mon, 12 Sep 2011 18:31:35 +0300
Subject: [elephant-devel] set-valued slots review
In-Reply-To: <j4l8ar$vd4$1@dough.gmane.org>
References: <j4l8ar$vd4$1@dough.gmane.org>
Message-ID: <j4l8kn$265$1@dough.gmane.org>

 > IV. Lacking features
>
> I think it would be great if we also add features for:
>
> 1. Initializing slot-set from list. Something like:
> (setf (numbers-of obj1) (make-slot-set :items '(1 2 3 4))
>
> 2. Wipe contents of a slot-set. You can do it via slot-makunbound, sure,
> but it is kinda ugly. (setf (numbers-of obj) NIL) is kinda weird because
> otherwise it doesn't recognize lists.
> (setf (numbers-of obj1) (make-slot-set) is, perhaps, appropriate, except
> it would create unnecessary object.
>
>

Or, alternatively, use lists:

(setf (numbers-of obj) '(1 2 3))
(setf (numbers-of obj) ())

It is kinda asymmetric because accessor will return a slot-set object 
rather than a list, but at least it is more consistent than using setf 
for inserting items.




From alex.mizrahi at gmail.com  Wed Sep 14 12:30:58 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Wed, 14 Sep 2011 15:30:58 +0300
Subject: [elephant-devel] bugs in tests
Message-ID: <j4q6q2$m17$1@dough.gmane.org>

It turns out that for compatibility with RT deftest we redefine 
5am:finishes (not shadowing it) into something which is supposed to work 
with deftest -- it returns NIL if there was error and T otherwise.
There are three problems with it:

1. if `finishes` is in a middle of deftest its output is ignored, i.e. 
it is OK even if there was an ERROR.
2. it just doesn't work in 5am:test, even if you spelled it 5am:finishes
3. And the worst part is that it eats CONDITION as if test was 
successful. And there are conditions which are worse than errors: memory 
corruption, etc.

And this is what happened with inhibit-rollback test -- it crashes CCL 
and makes SBCL signaling memory error.

Besides that, it turns out that cached-test-inherit-ok part of 
caching-style-required did not work, i.e. :cache-style is not inherited.




From alex.mizrahi at gmail.com  Wed Sep 14 14:39:46 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Wed, 14 Sep 2011 17:39:46 +0300
Subject: [elephant-devel] status update
Message-ID: <j4qebj$gg0$1@dough.gmane.org>

Recent patches:

  * set-valued slot tests, optimized postmodern support
  * test-slot-sets enabled in asd

   test-slot-sets thoroughly verifies current behaviour (rather weird)
   optimized postmodern support uses pm-pset for backend storage which 
keeps everything in one btree (database table)

  * disabled testconditions
  * fixes FINISHES macro in tests
  * prevent loading same foreign library (and stuff which "depends" on it)
  * with-interrupts for db-postmodern
  * pm-cursor typo fixed
  * import class-direct-subclasses

Current status: two tests fail both on SBCL 1.0.16 and CCL 1.7:

1. ELEPHANT-DB-PATH-TEST -- as I understand, both tests and function 
ELEPHANT-DB-PATH are buggy as test comments do not quite match 
function's behaviour. I do not quite get how it was supposed to work, so 
I'll leave it as it is for now. I don't quite get why we got this 
failure only now, perhaps reshuffling enabled tests which were not run 
previously (problems from migration to fiveam)

2. CACHING-STYLE-REQUIRED -- I guess some MOP code which handles option 
inheritance is broken (or does not exist).

INHIBIT-ROLLBACK test in testconditions crashes both CCL and SBCL, as 
I've mentioned previously, but I've disabled it for now. (I have no idea 
what it was supposed to test. CCL crashes after db_commit, IIRC, if it 
says you anything.)

I had no response on MOP breakage from SBCL devs so far. I've even made 
a simple repro case for them...




From arno at arnoo.net  Tue Sep 20 07:46:40 2011
From: arno at arnoo.net (Arnaud Betremieux)
Date: Tue, 20 Sep 2011 09:46:40 +0200 (CEST)
Subject: [elephant-devel] Multiple store controllers,
	with each its own set of classes
Message-ID: <fee287742a257ee0739d21d079a046ff.squirrel@alf.arnoo.net>

Hi all,

I'm trying to use two (BDB) store controllers, each with its own set of
persistent classes. The reason is that I want to be able to modify the
secondary store outside the application (it's read-only within it) and
just plug-in a modified version (close-store, change directory,
open-store) at some point without affecting the rest of the data.

If I just use a let binding to set *store-controller* to the secondary
store around make-instance and get-instances-by-class, everything works,
except I can see the class index is stored in the main store-controller,
which is not good, as I want the secondary store to be independant.

If I set the global *store-controller* to the secondary controller after
having opened both, and even close the first one to make sure before I
call defpclass or any other elephant function, the class index is created
in the right store-controller this time, but I get a new one each time I
open the store (empty btree with a new oid).

I have tried understanding and tracing the related parts of code, but that
part is not exactly easy to dive in for an elephant noob, so I was hoping
that someone familiar with the code could explain what's happening, or
that someone would have a trick for using multiple store controllers in
this fashion.

Thanks a lot,
Arnaud






From arno at arnoo.net  Mon Sep 26 12:34:54 2011
From: arno at arnoo.net (Arnaud Betremieux)
Date: Mon, 26 Sep 2011 14:34:54 +0200 (CEST)
Subject: [elephant-devel] Multiple store controllers,
	with each its own set of classes
In-Reply-To: <fee287742a257ee0739d21d079a046ff.squirrel@alf.arnoo.net>
References: <fee287742a257ee0739d21d079a046ff.squirrel@alf.arnoo.net>
Message-ID: <5dd6cab301fec7450367af378d315cb9.squirrel@alf.arnoo.net>

After looking into this for way too long...It turns out I'm a complete
idiot : I forgot SBCL had been updated at the same time I changed my code
to split store controllers, and I was running into the MOP issue of new
SBCL versions.

All my apologies.

> Hi all,
>
> I'm trying to use two (BDB) store controllers, each with its own set of
> persistent classes. The reason is that I want to be able to modify the
> secondary store outside the application (it's read-only within it) and
> just plug-in a modified version (close-store, change directory,
> open-store) at some point without affecting the rest of the data.
>
> If I just use a let binding to set *store-controller* to the secondary
> store around make-instance and get-instances-by-class, everything works,
> except I can see the class index is stored in the main store-controller,
> which is not good, as I want the secondary store to be independant.
>
> If I set the global *store-controller* to the secondary controller after
> having opened both, and even close the first one to make sure before I
> call defpclass or any other elephant function, the class index is created
> in the right store-controller this time, but I get a new one each time I
> open the store (empty btree with a new oid).
>
> I have tried understanding and tracing the related parts of code, but that
> part is not exactly easy to dive in for an elephant noob, so I was hoping
> that someone familiar with the code could explain what's happening, or
> that someone would have a trick for using multiple store controllers in
> this fashion.
>
> Thanks a lot,
> Arnaud
>
>
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>





From killerstorm at newmail.ru  Mon Sep 26 17:05:24 2011
From: killerstorm at newmail.ru (Alex Mizrahi)
Date: Mon, 26 Sep 2011 20:05:24 +0300
Subject: [elephant-devel] Multiple store controllers,
	with each its own set of classes
References: <fee287742a257ee0739d21d079a046ff.squirrel@alf.arnoo.net>
	<5dd6cab301fec7450367af378d315cb9.squirrel@alf.arnoo.net>
Message-ID: <j5qbcn$kh3$1@dough.gmane.org>

 AB> After looking into this for way too long...It turns out I'm a complete
 AB> idiot : I forgot SBCL had been updated at the same time I changed my
 AB> code to split store controllers, and I was running into the MOP issue
 AB> of new SBCL versions.

Hmm, what SBCL version is that? Can you please check against current git 
version?

I'm aware of problems with class redifinition, but what you've described 
does not look like that. 





From arnaud.betremieux at gmail.com  Tue Sep 27 12:13:28 2011
From: arnaud.betremieux at gmail.com (Arnaud Betremieux)
Date: Tue, 27 Sep 2011 14:13:28 +0200
Subject: [elephant-devel] Multiple store controllers,
	with each its own set of classes
Message-ID: <CAEL9bk1+ixYn3N9sJEALF9poB83fzsvvq6x+6PmJTPvthZS-Dg@mail.gmail.com>

Sorry, I should have been clearer : I *assumed* this bug was the cause,
since there had been an SBCL update in the meantime. Maybe there were
other bugs ? Unfortunately, I don't have the previous version I was
testing on anymore. AFAIK, it was an unmodified, somewhat recent (a few
months) version of Elephant from the darcs repository.

Anyway, my latest test was done with the current git version from
gonzojive on github (had to remove the link to make the message pass the
common-lisp.net spam filter) and everything is OK.

My test was as follows :
- defpclass
- open new BDB store
- make-instance
- get-instances-by-class
- quit
- start a new Lisp instance
- defpclass
- open same BDB store
- get-instances-by-class

With the previous version I was using, get-instances-by-class would return
the correct list the first time, nil the second. Tracing map-index showed
that the btree used was a new one (new oid) at each open-store. With the
current git version, both get-instances-by-class return the correct value.

>  AB> After looking into this for way too long...It turns out I'm a
> complete
>  AB> idiot : I forgot SBCL had been updated at the same time I changed
my AB> code to split store controllers, and I was running into the MOP
issue AB> of new SBCL versions.
>
> Hmm, what SBCL version is that? Can you please check against current git
version?
>
> I'm aware of problems with class redifinition, but what you've described
does not look like that.
>



From eslick at media.mit.edu  Tue Sep 27 19:25:39 2011
From: eslick at media.mit.edu (Ian Eslick)
Date: Tue, 27 Sep 2011 12:25:39 -0700
Subject: [elephant-devel] set-valued slots review
In-Reply-To: <j4l8kn$265$1@dough.gmane.org>
References: <j4l8ar$vd4$1@dough.gmane.org> <j4l8kn$265$1@dough.gmane.org>
Message-ID: <08B961DC-2603-4575-BB9A-0EFBB695399A@media.mit.edu>

I was creating the set-valued slot API as a placeholder for something more native, but ran out of time to work on the project before really getting it done right.  As you say, the API is awkward.

I think it would be more convenient to use lists as the slot-level API and have a separate API for set operations.

(slot object) -> list | nil
(setf (slot object) list) -> list
(setf (slot object) nil) -> nil
(setf (slot object) atom) -> error or '(atom)

(insert-item 
(remove-item
(find-item 
(map-slot-set 

In general, feel free to use your judgement in upgrading the API - just check with the list that no one is deeply dependent on a particular feature.  Cached slots, pests, set-valued-slots are all beta features that I don't believe are being used seriously.  I'd be open to removing cached slots - they're just too restricting in how they can be used safely in a threaded and/or multi-server environment.

Thanks,
Ian

On Sep 12, 2011, at 8:31 AM, Alex Mizrahi wrote:

> > IV. Lacking features
>> 
>> I think it would be great if we also add features for:
>> 
>> 1. Initializing slot-set from list. Something like:
>> (setf (numbers-of obj1) (make-slot-set :items '(1 2 3 4))
>> 
>> 2. Wipe contents of a slot-set. You can do it via slot-makunbound, sure,
>> but it is kinda ugly. (setf (numbers-of obj) NIL) is kinda weird because
>> otherwise it doesn't recognize lists.
>> (setf (numbers-of obj1) (make-slot-set) is, perhaps, appropriate, except
>> it would create unnecessary object.
>> 
>> 
> 
> Or, alternatively, use lists:
> 
> (setf (numbers-of obj) '(1 2 3))
> (setf (numbers-of obj) ())
> 
> It is kinda asymmetric because accessor will return a slot-set object rather than a list, but at least it is more consistent than using setf for inserting items.
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel




From eslick at media.mit.edu  Wed Sep 28 00:19:39 2011
From: eslick at media.mit.edu (Ian Eslick)
Date: Tue, 27 Sep 2011 17:19:39 -0700
Subject: [elephant-devel] Reason for
Message-ID: <C7CAEE7D-F6FD-48AB-B533-65314010CA86@media.mit.edu>

Anyone remember why we put this into the elephant.asd file?  I think the goal was to force some action to take place so that all the dynamic libraries were loaded each time.  It was causing the CCL MOP problem discussed earlier.  I may have fixes for the CCL MOP - a few tests still fail which I will diagnose later.  

Part of the win is removing the following line from elephant.asd, but I'm sure I'm casing a regression against another use case this was intended to handle.

(defmethod operation-done-p ((o load-op) (c elephant-c-source))
  nil)

Thank you,
Ian


From killerstorm at newmail.ru  Wed Sep 28 06:57:40 2011
From: killerstorm at newmail.ru (Alex Mizrahi)
Date: Wed, 28 Sep 2011 09:57:40 +0300
Subject: [elephant-devel] Reason for
References: <C7CAEE7D-F6FD-48AB-B533-65314010CA86@media.mit.edu>
Message-ID: <j5ugh9$vl6$1@dough.gmane.org>

 IE> Anyone remember why we put this into the elephant.asd file?

 IE> (defmethod operation-done-p ((o load-op) (c elephant-c-source)) nil)

To make sure that library is loaded at least once? There is no way to check 
whether it is loaded via UFFI.
And if you return T ASDF might skip loading altogether.

I've already fixed this, BTW, through tracking what is loaded. Forgot to 
push the patch...

 IE> It was causing the CCL MOP problem discussed earlier.
 IE>   I may have fixes for the CCL MOP - a few tests still fail which I
 IE> will diagnose later.

Have you read openmcl mailing list? It turns out shared-initialize which 
injects persistent-object direct superclass is kinda broken.

BTW, is there a reason to do it via shared-initialize instead of 
compute-precedence-list?

 IE> Part of the win is removing the following line from elephant.asd,
 IE>  but I'm sure I'm casing a regression against another use case this was
 IE> intended to handle.

Hmm, I haven't checked how ASDF handles it by default, I've tried both NIL 
and T and none of them worked right, IIRC.
But tracking what is loaded is the right thing, right? 





From eslick at media.mit.edu  Wed Sep 28 14:40:29 2011
From: eslick at media.mit.edu (Ian Eslick)
Date: Wed, 28 Sep 2011 07:40:29 -0700
Subject: [elephant-devel] Reason for
In-Reply-To: <j5ugh9$vl6$1@dough.gmane.org>
References: <C7CAEE7D-F6FD-48AB-B533-65314010CA86@media.mit.edu>
	<j5ugh9$vl6$1@dough.gmane.org>
Message-ID: <2076A344-0510-46CE-B037-77828AF25C38@media.mit.edu>

The problem was exposed by loading elephant again without also loading the backend libraries.  I made a few changes so that doesn't happen so easily.  Sounds like you had a better fix.  I haven't dug into this code base, nor done much CL hacking, in almost two years so feeling rusty.  :)

I also made a change for ccl so we don't inject the class if the dependencies argument was not provided (vs passing it along as a nil argument).  I'll look at the alternate placement.

These get ccl to pass all but one test.

Where are you with getting sbcl to build/pass?  I'm testing with 1.0.56

Ian



Sent from my iPhone

On Sep 27, 2011, at 11:57 PM, "Alex Mizrahi" <killerstorm at newmail.ru> wrote:

> IE> Anyone remember why we put this into the elephant.asd file?
> 
> IE> (defmethod operation-done-p ((o load-op) (c elephant-c-source)) nil)
> 
> To make sure that library is loaded at least once? There is no way to check whether it is loaded via UFFI.
> And if you return T ASDF might skip loading altogether.
> 
> I've already fixed this, BTW, through tracking what is loaded. Forgot to push the patch...
> 
> IE> It was causing the CCL MOP problem discussed earlier.
> IE>   I may have fixes for the CCL MOP - a few tests still fail which I
> IE> will diagnose later.
> 
> Have you read openmcl mailing list? It turns out shared-initialize which injects persistent-object direct superclass is kinda broken.
> 
> BTW, is there a reason to do it via shared-initialize instead of compute-precedence-list?
> 
> IE> Part of the win is removing the following line from elephant.asd,
> IE>  but I'm sure I'm casing a regression against another use case this was
> IE> intended to handle.
> 
> Hmm, I haven't checked how ASDF handles it by default, I've tried both NIL and T and none of them worked right, IIRC.
> But tracking what is loaded is the right thing, right? 
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel



From arnaud.betremieux at gmail.com  Wed Sep 28 16:52:21 2011
From: arnaud.betremieux at gmail.com (Arnaud Betremieux)
Date: Wed, 28 Sep 2011 18:52:21 +0200
Subject: [elephant-devel] Migration and derived slots
Message-ID: <CAEL9bk20TjwmzKECb5+J-HADKXZsvP2J9JTEJsHiO-o7trLR2w@mail.gmail.com>

With the latest Git revision on SBCL 1.0.51, when trying to migrate
objects that have a derived slot, elephant tries to call setf
slot-value-using-class on the derived slot, resulting in an error :
"Cannot write computed (derived) slot... for read/index retrieval
only".

It seems to me that copy-persistent-slots shouldn't try to write
derived slots, as the work will be done when (setf
slot-value-using-class) is called on the slots the derived function
depends on.

So I tried to patch the code for that (see attached patch). This seems
to do the trick in that the migration works.

I stumble upon another problem though : the derived slots are unbound
after migration. Apparently, update-derived-slots is called when
copying slot values, but derived-slots-trigger seems to always return
nil... I don't get why.

Here's my test case if anyone is interested (notice that I need to
require bordeaux-threads manually, which is probably a small
dependency bug ?). The first describe shows c=3, as expected, while
the second describe shows c unbound.

(require 'elephant)
(require 'bordeaux-threads)
(use-package 'elephant)

(defvar *store1* (open-store '(:BDB "store1")))
(defvar *store2* (open-store '(:BDB "store2")))

(defpclass test ()
  ((a :type integer :accessor a :initarg :a)
   (b :type integer :accessor b :initarg :b)
   (c :derived-fn (? (obj) (values (+ (a obj) (b obj)) t)))))

(let ((*store-controller* *store1*))
  (describe (make-instance 'test :a 1 :b 2)))
(migrate *store2* *store1*)
(close-store *store1*)
(describe (first (get-instances-by-class 'test)))
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch_migrate_derived
Type: application/octet-stream
Size: 1461 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20110928/10e3272e/attachment.obj>

From alex.mizrahi at gmail.com  Thu Sep 29 07:57:37 2011
From: alex.mizrahi at gmail.com (Alex Mizrahi)
Date: Thu, 29 Sep 2011 10:57:37 +0300
Subject: [elephant-devel] Reason for
In-Reply-To: <2076A344-0510-46CE-B037-77828AF25C38@media.mit.edu>
References: <C7CAEE7D-F6FD-48AB-B533-65314010CA86@media.mit.edu>	<j5ugh9$vl6$1@dough.gmane.org>
	<2076A344-0510-46CE-B037-77828AF25C38@media.mit.edu>
Message-ID: <j618dh$d9$1@dough.gmane.org>

> I also made a change for ccl so we don't inject the class if the dependencies argument was not provided (vs passing it along as a nil argument).  I'll look at the alternate placement.

AMOP spec says that users shouldn't add methods to SHARED-INITIALIZE of 
a metaclass. I think it can be moved to INITIALIZE-INSTANCE  and 
REINITIALIZE-INSTANCE, as they are allowed, and that would also prevent 
problems with internal updates as these functions are not called on 
update-instance-for-*. (I.e. you can get rid of the check of whether 
direct-superclasses were passed or not in that case.)

So that could be a conservative fix (more like bringing it up to the 
spec) which doesn't change the implementation much.

COMPUTE-PRECEDENCE-LIST sounds like a cleaner way to do it (if we don't 
really care for DIRECT superclasses), but I haven't read AMOP on this 
yet, so it is more like a guess.

> These get ccl to pass all but one test.
> Where are you with getting sbcl to build/pass?  I'm testing with 1.0.56

IIRC both CCL and SBCL were passing all tests but 
CACHING-STYLE-REQUIRED. (Again some MOP part which doesn't work.)

But then I've fixed the test suite itself and got more failures. :)

Can you please check inhibit-rollback (in testconditions) after applying 
my patches? It crashes lisp, as I understand, through corrupting BDB 
state. (Problem was previously hidden because of "ignore-conditions".)




From eslick at media.mit.edu  Thu Sep 29 16:51:06 2011
From: eslick at media.mit.edu (Ian Eslick)
Date: Thu, 29 Sep 2011 09:51:06 -0700
Subject: [elephant-devel] Reason for
In-Reply-To: <j618dh$d9$1@dough.gmane.org>
References: <C7CAEE7D-F6FD-48AB-B533-65314010CA86@media.mit.edu>	<j5ugh9$vl6$1@dough.gmane.org>
	<2076A344-0510-46CE-B037-77828AF25C38@media.mit.edu>
	<j618dh$d9$1@dough.gmane.org>
Message-ID: <53CACEFB-78A2-40A4-B38E-65E4FB58FE3C@media.mit.edu>

I recall there being problems taking base class injection out of shared-initialize - can initialize-instance manipulate the class precedence list and direct superclasses at that point in the instance initialization?

It occurs to me that, the easiest way to fix this is by policy; remove the functionality in 'shared-initialize (persistent-metaclass)' from the MOP entirely and mandate that persistent classes use the defpclass macro, or that users inherit from persistent-object manually.  The macro can do the error checking and base-class injection and be done with it.  The tests would have to be updated to stop using the explicit metaclass.

I can make the change and update the tests and then look into the other issues if you concur.

Thanks,
Ian

On Sep 29, 2011, at 12:57 AM, Alex Mizrahi wrote:

>> I also made a change for ccl so we don't inject the class if the dependencies argument was not provided (vs passing it along as a nil argument).  I'll look at the alternate placement.
> 
> AMOP spec says that users shouldn't add methods to SHARED-INITIALIZE of a metaclass. I think it can be moved to INITIALIZE-INSTANCE  and REINITIALIZE-INSTANCE, as they are allowed, and that would also prevent problems with internal updates as these functions are not called on update-instance-for-*. (I.e. you can get rid of the check of whether direct-superclasses were passed or not in that case.)
> 
> So that could be a conservative fix (more like bringing it up to the spec) which doesn't change the implementation much.
> 
> COMPUTE-PRECEDENCE-LIST sounds like a cleaner way to do it (if we don't really care for DIRECT superclasses), but I haven't read AMOP on this yet, so it is more like a guess.
> 
>> These get ccl to pass all but one test.
>> Where are you with getting sbcl to build/pass?  I'm testing with 1.0.56
> 
> IIRC both CCL and SBCL were passing all tests but CACHING-STYLE-REQUIRED. (Again some MOP part which doesn't work.)
> 
> But then I've fixed the test suite itself and got more failures. :)
> 
> Can you please check inhibit-rollback (in testconditions) after applying my patches? It crashes lisp, as I understand, through corrupting BDB state. (Problem was previously hidden because of "ignore-conditions".)
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel