From waldo at trianet.net  Thu Jan  5 22:01:29 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Thu, 5 Jan 2006 17:01:29 -0500
Subject: [elephant-devel] Problem with Elephant and BTree access
Message-ID: <90EC8189-B359-4932-AAFB-7B8909CBE7A7@trianet.net>

Hi,

I'm starting to use BTrees in elephant and am encountering a problem.  
Basically, I added a BTree object to the root under a key named  
"EDI-834-TAPES". I can retrieve the BTree object, but when I try to  
access a BTree element (whether to read or write), I get an error.  
See below:

CL-USER> (in-package ele)
#<Package "ELEPHANT">
ELE> (open-store "testdb")
#<BDB-STORE-CONTROLLER #x8A5C816>
ELE> (setf tapes (get-from-root "EDI-834-TAPES"))
#<BTREE #x8A5805E>
ELE> (get-value 1234 tapes)
CL-USER> (in-package ele)
#<Package "ELEPHANT">
ELE> (open-store "testdb")
#<BDB-STORE-CONTROLLER #x8A5C816>
ELE> (setf tapes (get-from-root "EDI-834-TAPES"))
#<BTREE #x8A5805E>
ELE> (get-value 1234 tapes)

No applicable method for args:
(1234 #<BTREE #x8A5805E>)
to #<STANDARD-GENERIC-FUNCTION GET-VALUE #x84146AE>
    [Condition of type SIMPLE-ERROR]

Restarts:
   0: [ABORT-REQUEST] Abort handling SLIME request.
   1: [ABORT-BREAK] Reset this process
   2: [ABORT] Kill this process

Backtrace:
   0: (#<CCL::STANDARD-KERNEL-METHOD NO-APPLICABLE-METHOD (T)>  
#<STANDARD-GENERIC-FUNCTION GET-VALUE #x84146AE>)
   1: (CCL::CALL-CHECK-REGS 'GET-VALUE)

Basically, the last statement (get-value) should return NIL since  
there is not BTree element within the BTree object.

What am I doing wrong? I tried to "mimic" as close as possible the  
steps in the Blog tutorial but keep getting to the same problem.

Thanks,
Waldo


From read at robertlread.net  Thu Jan  5 23:03:55 2006
From: read at robertlread.net (Robert L. Read)
Date: Thu, 05 Jan 2006 17:03:55 -0600
Subject: [elephant-devel] Problem with Elephant and BTree access
In-Reply-To: <90EC8189-B359-4932-AAFB-7B8909CBE7A7@trianet.net>
References: <90EC8189-B359-4932-AAFB-7B8909CBE7A7@trianet.net>
Message-ID: <1136502236.28637.241.camel@localhost.localdomain>

Let me try to duplicate this.  I'll get back to you tonight.

On Thu, 2006-01-05 at 17:01 -0500, Waldo Rubinstein wrote:

> Hi,
> 
> I'm starting to use BTrees in elephant and am encountering a problem.  
> Basically, I added a BTree object to the root under a key named  
> "EDI-834-TAPES". I can retrieve the BTree object, but when I try to  
> access a BTree element (whether to read or write), I get an error.  
> See below:
> 
> CL-USER> (in-package ele)
> #<Package "ELEPHANT">
> ELE> (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x8A5C816>
> ELE> (setf tapes (get-from-root "EDI-834-TAPES"))
> #<BTREE #x8A5805E>
> ELE> (get-value 1234 tapes)
> CL-USER> (in-package ele)
> #<Package "ELEPHANT">
> ELE> (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x8A5C816>
> ELE> (setf tapes (get-from-root "EDI-834-TAPES"))
> #<BTREE #x8A5805E>
> ELE> (get-value 1234 tapes)
> 
> No applicable method for args:
> (1234 #<BTREE #x8A5805E>)
> to #<STANDARD-GENERIC-FUNCTION GET-VALUE #x84146AE>
>     [Condition of type SIMPLE-ERROR]
> 
> Restarts:
>    0: [ABORT-REQUEST] Abort handling SLIME request.
>    1: [ABORT-BREAK] Reset this process
>    2: [ABORT] Kill this process
> 
> Backtrace:
>    0: (#<CCL::STANDARD-KERNEL-METHOD NO-APPLICABLE-METHOD (T)>  
> #<STANDARD-GENERIC-FUNCTION GET-VALUE #x84146AE>)
>    1: (CCL::CALL-CHECK-REGS 'GET-VALUE)
> 
> Basically, the last statement (get-value) should return NIL since  
> there is not BTree element within the BTree object.
> 
> What am I doing wrong? I tried to "mimic" as close as possible the  
> steps in the Blog tutorial but keep getting to the same problem.
> 
> Thanks,
> Waldo
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060105/50c43524/attachment.html>

From waldo at trianet.net  Fri Jan  6 00:00:55 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Thu, 5 Jan 2006 19:00:55 -0500
Subject: [elephant-devel] Elephant Collections
Message-ID: <28C8229A-9324-4317-93D7-6CC255322BC2@trianet.net>

Hi again,

I'm interested in learning about the collections supported by Elephant.

I have a need to store [very large] "flat" collections. For example,  
I want to store 2 million objects of approximately 1K bytes in size,  
ordered by date (an object attribute).

The only reference to collections in Elephant I have seen is the  
tutorial using btrees and indexed-btrees. What will happen when I  
load this many objects in a btree. When I get a hold of the  
persistent btree, will it keep everything in memory or will it read  
in memory the objects that I fetch from it?

Also, I have another case. I have to store about 100 objects where  
each of these objects has a reference to a collection of other  
objects. The sub-collection of objects don't need to be in any  
particular sort sequence. I was thinking of using a resizable vector.  
However, I was wondering the same questions as above. If I fetch one  
of the 100 objects, will it automatically load in memory all the  
objects in its sub-collection? The sub-collection objects may be  
30,000 elements of approximately 1K bytes in size as well. How will  
it behave if I need to traverse the sub-collection of objects of a  
particular object? Will it, again, load all the elements in memory or  
will it read from DB as needed?

Thanks,
Waldo


From read at robertlread.net  Fri Jan  6 02:40:04 2006
From: read at robertlread.net (Robert L. Read)
Date: Thu, 05 Jan 2006 20:40:04 -0600
Subject: [elephant-devel] Problem with Elephant and BTree access
In-Reply-To: <90EC8189-B359-4932-AAFB-7B8909CBE7A7@trianet.net>
References: <90EC8189-B359-4932-AAFB-7B8909CBE7A7@trianet.net>
Message-ID: <1136515204.6452.2.camel@localhost.localdomain>

This works for me under SBCL; please see the attached typescript.

I can't conjecture what the problem might be; is the script you show
below everything?
Are the packages changes somehow done in a visual environment and there
for not showing?
Try running through my script and let me know what the results are, if
this is still a problem for you.


On Thu, 2006-01-05 at 17:01 -0500, Waldo Rubinstein wrote:

> Hi,
> 
> I'm starting to use BTrees in elephant and am encountering a problem.  
> Basically, I added a BTree object to the root under a key named  
> "EDI-834-TAPES". I can retrieve the BTree object, but when I try to  
> access a BTree element (whether to read or write), I get an error.  
> See below:
> 
> CL-USER> (in-package ele)
> #<Package "ELEPHANT">
> ELE> (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x8A5C816>
> ELE> (setf tapes (get-from-root "EDI-834-TAPES"))
> #<BTREE #x8A5805E>
> ELE> (get-value 1234 tapes)
> CL-USER> (in-package ele)
> #<Package "ELEPHANT">
> ELE> (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x8A5C816>
> ELE> (setf tapes (get-from-root "EDI-834-TAPES"))
> #<BTREE #x8A5805E>
> ELE> (get-value 1234 tapes)
> 
> No applicable method for args:
> (1234 #<BTREE #x8A5805E>)
> to #<STANDARD-GENERIC-FUNCTION GET-VALUE #x84146AE>
>     [Condition of type SIMPLE-ERROR]
> 
> Restarts:
>    0: [ABORT-REQUEST] Abort handling SLIME request.
>    1: [ABORT-BREAK] Reset this process
>    2: [ABORT] Kill this process
> 
> Backtrace:
>    0: (#<CCL::STANDARD-KERNEL-METHOD NO-APPLICABLE-METHOD (T)>  
> #<STANDARD-GENERIC-FUNCTION GET-VALUE #x84146AE>)
>    1: (CCL::CALL-CHECK-REGS 'GET-VALUE)
> 
> Basically, the last statement (get-value) should return NIL since  
> there is not BTree element within the BTree object.
> 
> What am I doing wrong? I tried to "mimic" as close as possible the  
> steps in the Blog tutorial but keep getting to the same problem.
> 
> Thanks,
> Waldo
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060105/cbfd6e08/attachment.html>
-------------- next part --------------
;   printed 11 notes
#<PACKAGE "ELEPHANT-TESTS">
ELE-TESTS> (open-store "../tests/testdb")
#<BDB-STORE-CONTROLLER {A18C361}>
ELE-TESTS> (setq bt (make-btree))
#<BDB-BTREE {A3770B9}>
ELE-TESTS> (add-to-root "EDI-834-TAPES" bt)
#<BDB-BTREE {A3770B9}>
ELE-TESTS> (get-value 1234 tapes)
; Evaluation aborted
ELE-TESTS> (setf tapes (get-from-root "EDI-834-TAPES"))
; in: LAMBDA NIL
;     (SETF ELEPHANT-TESTS::TAPES (ELEPHANT:GET-FROM-ROOT "EDI-834-TAPES"))
; ==>
;   (SETQ ELEPHANT-TESTS::TAPES (ELEPHANT:GET-FROM-ROOT "EDI-834-TAPES"))
; 
; caught WARNING:
;   undefined variable: TAPES

; 
; caught WARNING:
;   This variable is undefined:
;     TAPES
; 
; compilation unit finished
;   caught 2 WARNING conditions
#<BDB-BTREE {A3770B9}>
ELE-TESTS> (get-value 1234 tapes)
NIL
NIL
ELE-TESTS> (open-store "../tests/testdb")
#<BDB-STORE-CONTROLLER {A5BA8F1}>
ELE-TESTS> (setf tapes (get-from-root "EDI-834-TAPES"))

; in: LAMBDA NIL
;     (SETF ELEPHANT-TESTS::TAPES (ELEPHANT:GET-FROM-ROOT "EDI-834-TAPES"))
; ==>
;   (SETQ ELEPHANT-TESTS::TAPES (ELEPHANT:GET-FROM-ROOT "EDI-834-TAPES"))
; 
; caught WARNING:
;   undefined variable: TAPES

; 
; caught WARNING:
;   This variable is undefined:
;     TAPES
; 
; compilation unit finished
;   caught 2 WARNING conditions
#<BDB-BTREE {A5D6AD9}>
ELE-TESTS> (get-value 1234 tapes)
NIL
NIL
ELE-TESTS> 

From waldo at trianet.net  Fri Jan  6 06:59:01 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Fri, 6 Jan 2006 01:59:01 -0500
Subject: [elephant-devel] Problem with Elephant and BTree access
In-Reply-To: <1136515204.6452.2.camel@localhost.localdomain>
References: <90EC8189-B359-4932-AAFB-7B8909CBE7A7@trianet.net>
	<1136515204.6452.2.camel@localhost.localdomain>
Message-ID: <954E97A4-8FA3-4D3E-AACB-1D94B9875F45@trianet.net>

After reading your script, I tried it again in OpenMCL and it worked.  
See below (but please see further below):

Welcome to OpenMCL Version 1.0 (DarwinPPC32)!
? (in-package :ele)
#<Package "ELEPHANT">
? (open-store "testdb")
#<BDB-STORE-CONTROLLER #x89CF54E>
? (setq bt (make-btree))
#<BDB-BTREE #x89CD506>
? (add-to-root "EDI-834-TAPES" bt)
#<BDB-BTREE #x89CD506>
? (close-store)
NIL
? (setq bt nil)
NIL
? (open-store "testdb")
#<BDB-STORE-CONTROLLER #x89CC55E>
? (setq bt (get-from-root "EDI-834-TAPES"))
#<BDB-BTREE #x89CBADE>
? (get-value 1234 bt)
NIL
NIL
? (setf (get-value 1234 bt) "My test")
;Compiler warnings :
;   Undeclared free variable BT, in an anonymous lambda form.
"My test"
? (close-store)
NIL
? (setq bt nil)
NIL
? (open-store "testdb")
#<BDB-STORE-CONTROLLER #x89C9E36>
? (get-value 1234 (get-from-root "EDI-834-TAPES"))
"My test"
T
? (close-store)
NIL

However, when I try it using the format used in the Tutorial, it  
fails. See below:

? (open-store "testdb")
#<BDB-STORE-CONTROLLER #x89C8DEE>
? (setq bt (make-instance 'btree))
#<BTREE #x89C851E>
? (add-to-root "EDI-834-TAPES" bt)
#<BTREE #x89C851E>
? (close-store)
NIL
? (setq bt nil)
NIL
? (open-store "testdb")
#<BDB-STORE-CONTROLLER #x89C7BAE>
? (setq bt (get-from-root "EDI-834-TAPES"))
#<BTREE #x89C7256>
? (get-value 1234 bt)
 > Error in process listener(1): No applicable method for args:
 >                                (1234 #<BTREE #x89C7256>)
 >                                to #<STANDARD-GENERIC-FUNCTION GET- 
VALUE #x84146AE>
 > While executing: #<CCL::STANDARD-KERNEL-METHOD NO-APPLICABLE- 
METHOD (T)>
 > Type :POP to abort.
Type :? for other options.
1 >

The main difference is the way the BTree class is created. In the  
tutorial, it says to use (make-instance 'btree). In your script,  
which worked, you suggest to use (make-btree). Apparently, these are  
two different CLOS classes. I believe that the tutorial is then  
somehow inaccurate.

Thanks,
Waldo

On Jan 5, 2006, at 9:40 PM, Robert L. Read wrote:

> This works for me under SBCL; please see the attached typescript.
>
> I can't conjecture what the problem might be; is the script you  
> show below everything?
> Are the packages changes somehow done in a visual environment and  
> there for not showing?
> Try running through my script and let me know what the results are,  
> if this is still a problem for you.
>
<snip>




From read at robertlread.net  Fri Jan  6 14:07:58 2006
From: read at robertlread.net (Robert L. Read)
Date: Fri, 06 Jan 2006 08:07:58 -0600
Subject: [elephant-devel] Problem with Elephant and BTree access
In-Reply-To: <954E97A4-8FA3-4D3E-AACB-1D94B9875F45@trianet.net>
References: <90EC8189-B359-4932-AAFB-7B8909CBE7A7@trianet.net>
	<1136515204.6452.2.camel@localhost.localdomain>
	<954E97A4-8FA3-4D3E-AACB-1D94B9875F45@trianet.net>
Message-ID: <1136556478.6608.21.camel@localhost.localdomain>

Wonderful!  You have found a bug in my documentation.  I will fix it
today.




On Fri, 2006-01-06 at 01:59 -0500, Waldo Rubinstein wrote:

> After reading your script, I tried it again in OpenMCL and it worked.  
> See below (but please see further below):
> 
> Welcome to OpenMCL Version 1.0 (DarwinPPC32)!
> ? (in-package :ele)
> #<Package "ELEPHANT">
> ? (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x89CF54E>
> ? (setq bt (make-btree))
> #<BDB-BTREE #x89CD506>
> ? (add-to-root "EDI-834-TAPES" bt)
> #<BDB-BTREE #x89CD506>
> ? (close-store)
> NIL
> ? (setq bt nil)
> NIL
> ? (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x89CC55E>
> ? (setq bt (get-from-root "EDI-834-TAPES"))
> #<BDB-BTREE #x89CBADE>
> ? (get-value 1234 bt)
> NIL
> NIL
> ? (setf (get-value 1234 bt) "My test")
> ;Compiler warnings :
> ;   Undeclared free variable BT, in an anonymous lambda form.
> "My test"
> ? (close-store)
> NIL
> ? (setq bt nil)
> NIL
> ? (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x89C9E36>
> ? (get-value 1234 (get-from-root "EDI-834-TAPES"))
> "My test"
> T
> ? (close-store)
> NIL
> 
> However, when I try it using the format used in the Tutorial, it  
> fails. See below:
> 
> ? (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x89C8DEE>
> ? (setq bt (make-instance 'btree))
> #<BTREE #x89C851E>
> ? (add-to-root "EDI-834-TAPES" bt)
> #<BTREE #x89C851E>
> ? (close-store)
> NIL
> ? (setq bt nil)
> NIL
> ? (open-store "testdb")
> #<BDB-STORE-CONTROLLER #x89C7BAE>
> ? (setq bt (get-from-root "EDI-834-TAPES"))
> #<BTREE #x89C7256>
> ? (get-value 1234 bt)
>  > Error in process listener(1): No applicable method for args:
>  >                                (1234 #<BTREE #x89C7256>)
>  >                                to #<STANDARD-GENERIC-FUNCTION GET- 
> VALUE #x84146AE>
>  > While executing: #<CCL::STANDARD-KERNEL-METHOD NO-APPLICABLE- 
> METHOD (T)>
>  > Type :POP to abort.
> Type :? for other options.
> 1 >
> 
> The main difference is the way the BTree class is created. In the  
> tutorial, it says to use (make-instance 'btree). In your script,  
> which worked, you suggest to use (make-btree). Apparently, these are  
> two different CLOS classes. I believe that the tutorial is then  
> somehow inaccurate.
> 
> Thanks,
> Waldo
> 
> On Jan 5, 2006, at 9:40 PM, Robert L. Read wrote:
> 
> > This works for me under SBCL; please see the attached typescript.
> >
> > I can't conjecture what the problem might be; is the script you  
> > show below everything?
> > Are the packages changes somehow done in a visual environment and  
> > there for not showing?
> > Try running through my script and let me know what the results are,  
> > if this is still a problem for you.
> >
> <snip>
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060106/70206f37/attachment.html>

From read at robertlread.net  Fri Jan  6 14:11:28 2006
From: read at robertlread.net (Robert L. Read)
Date: Fri, 06 Jan 2006 08:11:28 -0600
Subject: [elephant-devel] 
	Re: Made any progress on that ACL issue, Andrew B.?
In-Reply-To: <Pine.LNX.4.61.0601060104050.27875@math.uchicago.edu>
References: <200601051937.k05Jb8jd004137@blombos.isi.edu>
	<Pine.LNX.4.61.0601051429520.13827@math.uchicago.edu>
	<1136516085.6452.10.camel@localhost.localdomain>
	<Pine.LNX.4.61.0601060104050.27875@math.uchicago.edu>
Message-ID: <1136556689.6608.25.camel@localhost.localdomain>

Thanks to Andrew Philpot and Andrew Blumberg, I believe the minor fix
noted noted
below makes 0.3.0 completely green on ACL as well as SBCL.

I will commit this change today, and publish 0.3.1 or 0.4.0 today.

Also, Waldo found a serious bug in the documentation; thank your for
that.  I will 
fix that today as well.


On Fri, 2006-01-06 at 01:04 -0600, Andrew Blumberg wrote:

> yes, that sounds like a good idea to me.
> 
>  					- andrew
> 
> On Thu, 5 Jan 2006, Robert L. Read wrote:
> 
> > After commenting out the initform, I am green on all tests including
> > my migration tests.
> >
> > Andrew Blumberg, should I publish this fix into the code base?
> >
> > Mr. Philpot, I will continue to try to help with any additional problems
> > you might have.
> >
> > Thanks to both of you.
> >
> >
> >
> > On Thu, 2006-01-05 at 14:33 -0600, Andrew Blumberg wrote:
> >
> >> hi andrew,
> >>
> >>  	i'm glad to hear the fix works.  thanks for testing it.  just to
> >> be clear, the migration tests are all aimed at moving between different
> >> repositories?  (i spent a fair bit of time getting class update stuff to
> >> be in a reasonable state during the last revision i wrote, and if that's
> >> what is meant by migration i'll be sad).
> >>
> >>  								- andrew
> >>
> >> On Thu, 5 Jan 2006, Andrew Philpot wrote:
> >>
> >>>
> >>> Andrew B. was right all along here.  I rebuilt all from scratch, with
> >>> the initform commented out, and followed the test procedure more
> >>> closely, and I succeeded except for the migration stuff.
> >>>
> >>> 5 out of 107 total tests failed: ELEPHANT-TESTS::MIGRATE1,
> >>>   ELEPHANT-TESTS::MIGRATE2, ELEPHANT-TESTS::MIGRATE3,
> >>>   ELEPHANT-TESTS::MIGRATE4, ELEPHANT-TESTS::MIGRATE5.
> >>>
> >>> Robert, what do you get if you replace the definition of defclass
> >>> PERSISTENT with
> >>>
> >>> (defclass persistent ()
> >>>  ((%oid :accessor oid :initarg :from-oid)
> >>>   ;; This is just an idea for storing connections in the persistent
> >>>   ;; objects; these should be transient as well, if that flag exists!
> >>>   ;; In the case of sleepy cat, this is the controller-db from
> >>>   ;; the store-controller.  In the case of SQL this is
> >>>   ;; the connection spec (since the connection might be broken?)
> >>>   ;; It probably would be better to put a string in here in the case
> >>>   ;; of sleepycat...
> >>>   (dbconnection-spec-pst :type (or list string)
> >>> 			  :accessor :dbcn-spc-pst
> >>> 			  :initarg :dbconnection-spec-pst
> >>> 			  ;; :initform '()
> >>> 			  )
> >>>   )
> >>>  (:documentation
> >>>   "Abstract superclass for all persistent classes (common
> >>> to user-defined classes and collections.)"))
> >>>
> >>> Andrew P.
> >>>
> >>>
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060106/b91ddac5/attachment.html>

From read at robertlread.net  Fri Jan  6 15:52:29 2006
From: read at robertlread.net (Robert L. Read)
Date: Fri, 06 Jan 2006 09:52:29 -0600
Subject: [elephant-devel] Elephant 0.4.0 is released
Message-ID: <1136562750.6608.37.camel@localhost.localdomain>

Elephant 0.4.0 is released.  It can be obtained from the elephant
project main page:
 http://common-lisp.net/project/elephant/

This release only fixes bugs over the 0.3.0 release; however the numbers
seem low
for what I deem to be a relatively stable system so I called it 0.4.0.
The improvements
over 0.3.0 are:

1)  0.4.0 now works with unicode-enables SBCL versions
2)  0.4.0 works with ACL 
3)  a bug in documentation has been fixed.

Thanks to Waldo Rubinstein, Andrew Philpot, Andrew Blumberg and me.

If anybody installs 0.4.0 and gets it to work, I would love a
confirmation email
telling me which LISP implementation you use so that I can build a
matrix
of what works.  I myself only test under SBCL, but I believe this
release will
work with ACL and Open MCL.


----
Robert L. Read, PhD                                     read &T
robertlread.net
Consider visiting Progressive Engineering:
http://robertlread.net/pe
In Austin: 912-8593                                        "Think
globally, Act locally." -- RBF


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060106/052dcaa9/attachment.html>

From waldo at trianet.net  Fri Jan  6 18:20:23 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Fri, 6 Jan 2006 13:20:23 -0500
Subject: [elephant-devel] Just a clarification on using elephant metaclasses
	and problem accessing OID
Message-ID: <591E7C15-3D25-4711-B086-CD575AEC480E@trianet.net>

Hi all,

In playing with CLOS and using the :metadata class option to use  
elephant, I realized that you cannot instantiate a CLOS object that  
uses the :metadata option without first opening an elephant store.

See below:

Welcome to OpenMCL Version 1.0 (DarwinPPC32)!
? (load "lisp/dev/edi.lisp")
;Compiler warnings :
;   Undefined function :STORE-CONTROLLER-SQL (2 references), in MAKE- 
OR-FIND-834-TAPE.
;   Undefined function ELEPHANT::WITH-TRANSACTION-SQL (2 references),  
in MAKE-OR-FIND-834-TAPE.
#P"/Users/waldo/dev/lisp/dev/edi.lisp"
? (setf entry (make-instance 'oonet:edi-entry))
 > Error in process listener(1): No applicable method for args:
 >                                (NIL)
 >                                to #<STANDARD-GENERIC-FUNCTION  
ELEPHANT::NEXT-OID #x8408F0E>
 > While executing: #<CCL::STANDARD-KERNEL-METHOD NO-APPLICABLE- 
METHOD (T)>
 > Type :POP to abort.
Type :? for other options.
1 > :pop

? (ele:open-store "testdb")
#<ELEPHANT:BDB-STORE-CONTROLLER #x8A689D6>
? (setf entry (make-instance 'oonet:edi-entry))
#<COM.OONET.EDI:EDI-ENTRY #x8A66986>
? (ele:close-store)
NIL

The reason I mention this is because it's not clear in the  
documentation that you need to have the store opened.

Now, the problem I have is that I can't access the object's OID value  
as per the documentation. See below:

? (slot-value entry 'oonet::oid)
 > Error in process listener(1): #<COM.OONET.EDI:EDI-ENTRY #x8A66986>  
has no slot named COM.OONET.EDI::OID.
 > While executing: #<CCL::STANDARD-KERNEL-METHOD SLOT-MISSING (T T T  
T)>
 > Type :POP to abort.
Type :? for other options.
1 > :pop

? (slot-value entry 'ele::oid)
 > Error in process listener(1): #<COM.OONET.EDI:EDI-ENTRY #x8A66986>  
has no slot named ELEPHANT::OID.
 > While executing: #<CCL::STANDARD-KERNEL-METHOD SLOT-MISSING (T T T  
T)>
 > Type :POP to abort.
Type :? for other options.
1 > :pop

How can I fetch the OID?

My class is defined roughly as follows:

(defclass edi-entry ()
   (last-name first-name ssn)
   (:metaclass persistent-metaclass))

in

(defpackage :com.oonet.edi
   (:use :common-lisp
	:it.bese.ucw
	:it.bese.yaclml
	:elephant)
   (:nicknames :oonet)
   (:export :edi-entry))

(in-package :com.oonet.edi)

Thanks,
Waldo



From midfield at gmail.com  Fri Jan  6 17:52:57 2006
From: midfield at gmail.com (Ben)
Date: Fri, 6 Jan 2006 11:52:57 -0600
Subject: [elephant-devel] Elephant Collections
In-Reply-To: <28C8229A-9324-4317-93D7-6CC255322BC2@trianet.net>
References: <28C8229A-9324-4317-93D7-6CC255322BC2@trianet.net>
Message-ID: <9157df230601060952s7c7b6d53kd9412e5f13a74324@mail.gmail.com>

the elephant collections are not pulled entirely into memory.  IIRC it
tries to be as lazy as possible as far as loading entries.  you may
have to load a bunch of stuff if you're doing complicated joins.

the same goes for references to other persistant objects.  in general
they are stored in slots by ID and are loaded on-demand.  this is not
true if the pointed-at objects are not persistent -- e.g. don't use
the elephant metaclass.

B

On 1/5/06, Waldo Rubinstein <waldo at trianet.net> wrote:
> Hi again,
>
> I'm interested in learning about the collections supported by Elephant.
>
> I have a need to store [very large] "flat" collections. For example,
> I want to store 2 million objects of approximately 1K bytes in size,
> ordered by date (an object attribute).
>
> The only reference to collections in Elephant I have seen is the
> tutorial using btrees and indexed-btrees. What will happen when I
> load this many objects in a btree. When I get a hold of the
> persistent btree, will it keep everything in memory or will it read
> in memory the objects that I fetch from it?
>
> Also, I have another case. I have to store about 100 objects where
> each of these objects has a reference to a collection of other
> objects. The sub-collection of objects don't need to be in any
> particular sort sequence. I was thinking of using a resizable vector.
> However, I was wondering the same questions as above. If I fetch one
> of the 100 objects, will it automatically load in memory all the
> objects in its sub-collection? The sub-collection objects may be
> 30,000 elements of approximately 1K bytes in size as well. How will
> it behave if I need to traverse the sub-collection of objects of a
> particular object? Will it, again, load all the elements in memory or
> will it read from DB as needed?
>
> Thanks,
> Waldo
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>


From read at robertlread.net  Fri Jan  6 19:13:42 2006
From: read at robertlread.net (Robert L. Read)
Date: Fri, 06 Jan 2006 13:13:42 -0600
Subject: [elephant-devel] Re: more allegro changes you might want to
	incorporate
In-Reply-To: <200601061808.k06I8xBj004250@ubirr.isi.edu>
References: <200601061808.k06I8xBj004250@ubirr.isi.edu>
Message-ID: <1136574823.6608.62.camel@localhost.localdomain>

Andrew has discovered some problems (and even more than this message.)

I thought 4.0 was functional under ACL, but I would now have to say it
is 
questionable.

Unfortunately, I am about to begin an weekend-long LAN party to rebuild
the website of the Esperanto League of North America, so will not 
be able to incorprate Andrew's suggestions until next week.

Many thanks, Andrew for all your work.  If everyone can continue to be 
patient with me, we will continue to have a better-and-better project.


On Fri, 2006-01-06 at 10:08 -0800, Andrew Philpot wrote:

> In src/elephant.lisp, you need three more imports from CLOS (primary
> name is ACLMOP)
> 
>   #+allegro
>   (:import-from :clos
> 		compute-class-precedence-list
> 		validate-superclass
> 		standard-slot-definition
> 		standard-direct-slot-definition
> 		standard-effective-slot-definition
> 		direct-slot-definition-class
> 		effective-slot-definition-class
> 		slot-definition-name
> 		slot-definition-initform
> 		slot-definition-initfunction
> 		compute-effective-slot-definition
> 		class-slots
> 		slot-value-using-class
> 		slot-boundp-using-class
> 		slot-makunbound-using-class
> 		slot-definition-allocation
> 		slot-definition-initargs
> 		class-finalized-p
> 		finalize-inheritance
> 		compute-slots
> 		slot-definition-readers
> 		slot-definition-writers
> 		class-direct-slots)
> 
> In INSTALL and src/controller, there are stray references to '0.3.0'
> Maybe you could use logical pathnames?
> 
> I didn't notice this before, but DESERIALIZE-FROM-BASE64-STRING has
> arguments (x &keys sc) and is called incorrectly in SQL-DUMP-CLCN
> which should probably be something like
> 
> (defmethod sql-dump-clcn ((clcn integer) sc con)
>   (let* (
> 	 (tuples
> 	  (clsql::select [key] [value]
> 		  :from [keyvalue]
> 		  :where [and [= [clctn_id] clcn]]
> 		  :database con
> 		  )))
>     (mapcar #'(lambda (x) 
>       (mapcar #'(lambda (q)
>         (deserialize-from-base64-string q :sc sc)) x))
> 	    tuples)))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060106/ac440001/attachment.html>

From waldo at trianet.net  Fri Jan  6 21:32:29 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Fri, 6 Jan 2006 16:32:29 -0500
Subject: [elephant-devel] Getting the number of nodes in BTree
Message-ID: <6470CCEA-6812-4AA6-B178-1EC3FFC7BFAA@trianet.net>

Is there a way to query an elephant BTree to find out how many  
elements it has similar to (length collection)?

Thanks,
Waldo


From ml13 at onlinehome.de  Sat Jan  7 18:55:07 2006
From: ml13 at onlinehome.de (ml13 at onlinehome.de)
Date: Sat, 7 Jan 2006 19:55:07 +0100
Subject: [elephant-devel] secondary indices
Message-ID: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>

Hi,

while looking at the tutorial, I always get an error while trying to  
work with secondary indices.

I am doing this on openmcl "Version 1.0 (DarwinPPC32)" with BDB4.3.

It looks like the bdb-indexed-tree does not have enough slots??

Anyway, I am very happy to see that elephant has developed so much!!

		Kilian Sprotte


ELE> (defclass appointment ()
          ((date :accessor ap-date :initarg :date :type integer)
           (type :accessor ap-type :initarg :type :type string))
          (:metaclass persistent-metaclass))
#<PERSISTENT-METACLASS APPOINTMENT>

ELE> (defparameter *appointments* (with-transaction () (build-indexed- 
btree s)))
*APPOINTMENTS*

(add-appointment (encode-universal-time 0 0 0 22 12 2004) "Birthday")

Array index 4 out of bounds for #<SLOT-VECTOR  #x8D6863E> .
    [Condition of type SIMPLE-ERROR]

Restarts:
   0: [ABORT-REQUEST] Abort handling SLIME request.
   1: [ABORT-BREAK] Reset this process
   2: [ABORT] Kill this process

Backtrace:
   0: (#<CCL::STANDARD-KERNEL-METHOD SLOT-VALUE-USING-CLASS (STANDARD- 
CLASS T STANDARD-EFFECTIVE-SLOT-DEFINITION)> #<PERSISTENT-METACLASS  
BDB-INDEXED-BTREE> '(ADD-APPOINTMENT (ENCODE-UNIVERSAL-TIME 0 0 0 22  
12 2004) "Birthday") #<STRING-INPUT-STREAM  #x8D66486>)
       Locals:
         CLASS = #<PERSISTENT-METACLASS BDB-INDEXED-BTREE>
         CCL::SLOT-VECTOR = #<SLOT-VECTOR  #x8D6863E>
   1: (#<STANDARD-METHOD (SETF GET-VALUE) (T T BDB-INDEXED-BTREE)>  
#<APPOINTMENT #x8D662CE> 3312658800 #<BDB-INDEXED-BTREE #x8D6862E>)
       Locals:
         VALUE = #<APPOINTMENT #x8CABF7E>
         KEY = 3312658800
         BT = #<BDB-INDEXED-BTREE #x8CAC106>
         SC = #<BDB-STORE-CONTROLLER #x8C7E0BE>
   2: (ADD-APPOINTMENT 3312658800 "Birthday")
       Locals:
         DATE = 3312658800
         TYPE = "Birthday"
         ENV = #<A Mac Pointer #x180E000>
         #:G541 = 1
         #:G543 = NIL
         #:G540 = #<A Mac Pointer #x309450>
         *CURRENT-TRANSACTION* = #<A Null Mac Pointer>
         *AUTO-COMMIT* = T
         #:G546 = #<BDB-INDEXED-BTREE #x8CAC106>
       Catch-tags:
         NIL
         SLEEPYCAT::TRANSACTION
  


From read at robertlread.net  Sat Jan  7 22:01:12 2006
From: read at robertlread.net (Robert L. Read)
Date: Sat, 07 Jan 2006 16:01:12 -0600
Subject: [elephant-devel] secondary indices
In-Reply-To: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
Message-ID: <1136671272.9761.15.camel@localhost.localdomain>

I won't be able to answer this until Tuesday; maybe somebody else can
look at it.

On Sat, 2006-01-07 at 19:55 +0100, ml13 at onlinehome.de wrote:

> Hi,
> 
> while looking at the tutorial, I always get an error while trying to  
> work with secondary indices.
> 
> I am doing this on openmcl "Version 1.0 (DarwinPPC32)" with BDB4.3.
> 
> It looks like the bdb-indexed-tree does not have enough slots??
> 
> Anyway, I am very happy to see that elephant has developed so much!!
> 
> 		Kilian Sprotte
> 
> 
> ELE> (defclass appointment ()
>           ((date :accessor ap-date :initarg :date :type integer)
>            (type :accessor ap-type :initarg :type :type string))
>           (:metaclass persistent-metaclass))
> #<PERSISTENT-METACLASS APPOINTMENT>
> 
> ELE> (defparameter *appointments* (with-transaction () (build-indexed- 
> btree s)))
> *APPOINTMENTS*
> 
> (add-appointment (encode-universal-time 0 0 0 22 12 2004) "Birthday")
> 
> Array index 4 out of bounds for #<SLOT-VECTOR  #x8D6863E> .
>     [Condition of type SIMPLE-ERROR]
> 
> Restarts:
>    0: [ABORT-REQUEST] Abort handling SLIME request.
>    1: [ABORT-BREAK] Reset this process
>    2: [ABORT] Kill this process
> 
> Backtrace:
>    0: (#<CCL::STANDARD-KERNEL-METHOD SLOT-VALUE-USING-CLASS (STANDARD- 
> CLASS T STANDARD-EFFECTIVE-SLOT-DEFINITION)> #<PERSISTENT-METACLASS  
> BDB-INDEXED-BTREE> '(ADD-APPOINTMENT (ENCODE-UNIVERSAL-TIME 0 0 0 22  
> 12 2004) "Birthday") #<STRING-INPUT-STREAM  #x8D66486>)
>        Locals:
>          CLASS = #<PERSISTENT-METACLASS BDB-INDEXED-BTREE>
>          CCL::SLOT-VECTOR = #<SLOT-VECTOR  #x8D6863E>
>    1: (#<STANDARD-METHOD (SETF GET-VALUE) (T T BDB-INDEXED-BTREE)>  
> #<APPOINTMENT #x8D662CE> 3312658800 #<BDB-INDEXED-BTREE #x8D6862E>)
>        Locals:
>          VALUE = #<APPOINTMENT #x8CABF7E>
>          KEY = 3312658800
>          BT = #<BDB-INDEXED-BTREE #x8CAC106>
>          SC = #<BDB-STORE-CONTROLLER #x8C7E0BE>
>    2: (ADD-APPOINTMENT 3312658800 "Birthday")
>        Locals:
>          DATE = 3312658800
>          TYPE = "Birthday"
>          ENV = #<A Mac Pointer #x180E000>
>          #:G541 = 1
>          #:G543 = NIL
>          #:G540 = #<A Mac Pointer #x309450>
>          *CURRENT-TRANSACTION* = #<A Null Mac Pointer>
>          *AUTO-COMMIT* = T
>          #:G546 = #<BDB-INDEXED-BTREE #x8CAC106>
>        Catch-tags:
>          NIL
>          SLEEPYCAT::TRANSACTION
>   
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060107/9857ad1b/attachment.html>

From waldo at trianet.net  Sat Jan  7 23:08:37 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Sat, 7 Jan 2006 18:08:37 -0500
Subject: [elephant-devel] secondary indices
In-Reply-To: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
Message-ID: <D26055F9-9526-4981-A668-0A0D58C605EC@trianet.net>

I was able to replicate it under the same environment. Also, I  
received the same error message when executing add-index while  
*appointments* was empty.

I'm fairly new to elephant so I'll try my best to see what's going on.

- Waldo

On Jan 7, 2006, at 1:55 PM, ml13 at onlinehome.de wrote:

> Hi,
>
> while looking at the tutorial, I always get an error while trying  
> to work with secondary indices.
>
> I am doing this on openmcl "Version 1.0 (DarwinPPC32)" with BDB4.3.
>
> It looks like the bdb-indexed-tree does not have enough slots??
>
> Anyway, I am very happy to see that elephant has developed so much!!
>
> 		Kilian Sprotte
>
>
> ELE> (defclass appointment ()
>          ((date :accessor ap-date :initarg :date :type integer)
>           (type :accessor ap-type :initarg :type :type string))
>          (:metaclass persistent-metaclass))
> #<PERSISTENT-METACLASS APPOINTMENT>
>
> ELE> (defparameter *appointments* (with-transaction () (build- 
> indexed-btree s)))
> *APPOINTMENTS*
>
> (add-appointment (encode-universal-time 0 0 0 22 12 2004) "Birthday")
>
> Array index 4 out of bounds for #<SLOT-VECTOR  #x8D6863E> .
>    [Condition of type SIMPLE-ERROR]
>
> Restarts:
>   0: [ABORT-REQUEST] Abort handling SLIME request.
>   1: [ABORT-BREAK] Reset this process
>   2: [ABORT] Kill this process
>
> Backtrace:
>   0: (#<CCL::STANDARD-KERNEL-METHOD SLOT-VALUE-USING-CLASS  
> (STANDARD-CLASS T STANDARD-EFFECTIVE-SLOT-DEFINITION)> #<PERSISTENT- 
> METACLASS BDB-INDEXED-BTREE> '(ADD-APPOINTMENT (ENCODE-UNIVERSAL- 
> TIME 0 0 0 22 12 2004) "Birthday") #<STRING-INPUT-STREAM  #x8D66486>)
>       Locals:
>         CLASS = #<PERSISTENT-METACLASS BDB-INDEXED-BTREE>
>         CCL::SLOT-VECTOR = #<SLOT-VECTOR  #x8D6863E>
>   1: (#<STANDARD-METHOD (SETF GET-VALUE) (T T BDB-INDEXED-BTREE)>  
> #<APPOINTMENT #x8D662CE> 3312658800 #<BDB-INDEXED-BTREE #x8D6862E>)
>       Locals:
>         VALUE = #<APPOINTMENT #x8CABF7E>
>         KEY = 3312658800
>         BT = #<BDB-INDEXED-BTREE #x8CAC106>
>         SC = #<BDB-STORE-CONTROLLER #x8C7E0BE>
>   2: (ADD-APPOINTMENT 3312658800 "Birthday")
>       Locals:
>         DATE = 3312658800
>         TYPE = "Birthday"
>         ENV = #<A Mac Pointer #x180E000>
>         #:G541 = 1
>         #:G543 = NIL
>         #:G540 = #<A Mac Pointer #x309450>
>         *CURRENT-TRANSACTION* = #<A Null Mac Pointer>
>         *AUTO-COMMIT* = T
>         #:G546 = #<BDB-INDEXED-BTREE #x8CAC106>
>       Catch-tags:
>         NIL
>         SLEEPYCAT::TRANSACTION
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel



From read at robertlread.net  Tue Jan 10 21:23:04 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 10 Jan 2006 15:23:04 -0600
Subject: [elephant-devel] Status of 4.0 ---- apparently OK,
	but some minor problems....
In-Reply-To: <1136574823.6608.62.camel@localhost.localdomain>
References: <200601061808.k06I8xBj004250@ubirr.isi.edu>
	<1136574823.6608.62.camel@localhost.localdomain>
Message-ID: <1136928184.7578.73.camel@localhost.localdomain>

Dear Team,

    I was out of touch this weekend on another project.  Before the
weekend I sent 
a note questioning whether 4.0 really worked with ACL.  Andrew Philpot
has 
been doing a great job sending me suggestions, which I have not been
able to 
keep up with.  Based on what he is sent me, I think it is clear:

1)  4.0 does work with ACL-6.2
2)  4.0 does probably has a type-specific bug with ACL-7.0 (see below)
3) Andrew has pointed out a lot of mis-organization of the code in terms
of circular references and so forth that produce compiler warnings but 
are not fatal.

I have not had time to digest his suggestions; I also am not really an
expert
in this sort of thing although I am learning.  I will continue to work
with him
and will study his suggestions later in the week.  

Basically, it seems like to clarify and improve the code quality, I need
to 
do some significant reorganization.  As far as I know, it remeains true
that 0.4.0
is the best release so far.


Andrew Philpot wrote:




The type declarations in controller.lisp in the following

(defun add-index-from-index (iname v dstibt dstsc)
  (declare (type btree-index v)
           (type indexed-btree dstibt))
  (let ((kf (key-form v)))
    (format t " kf ~A ~%" kf)
    (let ((index
           (build-btree-index dstsc :primary dstibt
                              :key-form kf)))
      ;; Why do I have to do this here?
      (setf (indices dstibt) (make-hash-table))
      (setf (indices-cache dstibt) (make-hash-table))
      (setf (gethash iname (indices-cache dstibt)) index)
      (setf (gethash iname (indices dstibt)) index)
      )
    )
  )

apparently are seen before the type is visible in ACL:

; While compiling ADD-INDEX-FROM-INDEX:
Warning: Ignoring declaration of unrecognized type: BTREE-INDEX
Warning: Ignoring declaration of unrecognized type: INDEXED-BTREE


I still have migration/*test-path-secondary* turned off

In acl6.2

5 out of 106 total tests failed: ELEPHANT-TESTS::MIGRATE1, 
   ELEPHANT-TESTS::MIGRATE2, ELEPHANT-TESTS::MIGRATE3, 
   ELEPHANT-TESTS::MIGRATE4, ELEPHANT-TESTS::MIGRATE5.

In acl7.0

6 out of 106 total tests failed: ELEPHANT-TESTS::ARRAYS-1, ELEPHANT-
TESTS::MIGRATE1, 
   ELEPHANT-TESTS::MIGRATE2, ELEPHANT-TESTS::MIGRATE3, ELEPHANT-
TESTS::MIGRATE4, 
   ELEPHANT-TESTS::MIGRATE5.

The difference is in serializing that funny array with element-type
(UNSIGNED-BYTE 20)

    ELE-TESTS(30):           (in-out-value (make-array '(3 4 5) 
                                           :element-type 
                                           '(unsigned-byte 20)))
     0[1]: (SERIALIZE #<Array of rank 3 @ #x71e874c2>
                      #S(SLEEPYCAT:BUFFER-STREAM :BUFFER
2684465168 :SIZE 0 :POSITION 0 :LENGTH 5184))
    Error: #<unknown object of type number 56 @ #x71b0dc22> fell through
a ETYPECASE form.  The valid
           cases were FIXNUM, NULL, SYMBOL, STRING, PERSISTENT, SINGLE-
FLOAT, DOUBLE-FLOAT, CHARACTER,
           PATHNAME, INTEGER, RATIONAL, CONS, HASH-TABLE, STANDARD-
OBJECT, and ARRAY.
      [condition type: CASE-FAILURE]

    Restart actions (select using :continue):
     0: Return to Top Level (an "abort" restart).
     1: Abort entirely from this (lisp) process.
    [1] ELE-TESTS(31): :reset
     0[1]: returned-by-throwing NIL
    ELE-TESTS(32): 

#Andrew


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060110/3c32bd92/attachment.html>

From read at robertlread.net  Tue Jan 10 21:41:59 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 10 Jan 2006 15:41:59 -0600
Subject: [elephant-devel] Long email from Andrew...
Message-ID: <1136929319.9533.6.camel@localhost.localdomain>

Dear Team and Andrew Philpot,
    This long message from Andrew seemed worth including here.
Summary:
1)  There are compiler warnings that we should fix
2)  ACL is green on the tests
3)  Some other problems are missing.
4)  Andrew calls for a large example application.
5)  Andrew identifies a place where our error handling needs to be
improved.
All of this sound unpleasant, but is actually good:  Andrew is
identifying of
ways to improve the system, and despite these problems is (apparently
from his words below) actually becoming MORE confident that he
can use Elephant.

Thank you, Andrew.  I will try to get back to you in more detail later
this
week.



On Tue, 2006-01-10 at 12:28 -0800, Andrew Philpot wrote:
  Before I step in a bucket again, can you make this very clear to me:
> 
>   With all the changes that you have mailed me, are the tests green
>   under ACL? (barring the migration tests.)
> 
> Most of the edits I sent you in the last couple of days correspond
> from my perspective to quality improvements, not to getting things to
> work.  When I get compiler warnings, I usually address them just so I
> can see the "real" errors when they are signaled.  I know some other
> developers can "see around" them better than I, so I don't 
> 
> A note on getting things to work: my build procedure is to always
> compile elephant itself (:force t), and then to loading clsql as well
> on top.  This nearly eliminates undefined references.  (The only one
> still missing is function :STORE-CONTROLLER-SQL).  But this build
> workaround seems far from ideal, and restructuring the ASDF might be
> appropriate.
> 
> (defun build-elephant ()
>        ;; I have UFFI 1.4.6 installed in service of another
>        ;; application which I can't tinker with
>   (asdf:operate 'asdf:load-op :uffi157)
>   (provide :uffi)
>   (asdf:operate 'asdf:load-op :elephant :force t)
>   (asdf:operate 'asdf:load-op :ele-bdb)
>   (asdf:operate 'asdf:load-op :ele-clsql)
>   (asdf:operate 'asdf:load-op :elephant-tests)
>   (eval (read-from-string "(setq elephant-tests::*test-path-primary* 
> \"/opt/elephant/tests/testdb\")"))
>   (eval (read-from-string "(setq elephant-tests::*test-path-secondary*
nil)"))
>   (eval (read-from-string "(elephant-tests::do-all-tests-spec 
> elephant-tests::*test-path-primary*)"))
>   )
> 
> Having said that, following the above I get:
> 
> 5 out of 106 total tests failed: ELEPHANT-TESTS::MIGRATE1,
>    ELEPHANT-TESTS::MIGRATE2, ELEPHANT-TESTS::MIGRATE3,
>    ELEPHANT-TESTS::MIGRATE4, ELEPHANT-TESTS::MIGRATE5.
> 
> Can I ask whether sleepycat::*elephant-lib-path* (used in a
> load-foreign-library call in sleepycat.lisp) is supposed to be the
> same object (presumably it is the same value) as
> elephant::*elephant-lib-path* defined in controller.lisp?  I get a
> message that the former is not bound when I compile the sleepycat
> stuff.  To my eye, it looks probable that this is another possibly
> inappropriate circular reference; shouldn't sleepycat per se be
> independent of the higher level elephant layer?  I'm not setting the
> former at all right now, but perhaps I should be.  What do you think
> of elephant (and possibly sleepycat) configuration file(s) which would
> group all this stuff in one place.

I can't yet answer about how it should be set, but yes, I agree that
sleepycat.lisp ought in theory to be a pure interface to the sleepycat
API and therefore independent of the other code.  Yes, you have
convinced me the configuration stuff should be rewritten and 
streamlined.

> 
> I'm by no means a UFFI expert.  In compiling, I always get warnings
> about calls to functions named by uninterned symbols (see below).  I
> would think this indicates some kind of mismatch between UFFI and ACL
> or UFFI and elephant, but I don't have enough understand to inspect,
> much less propose a fix.
> 
> ; Fast
loading /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/serial
> izer.xfasl
> Warning: While compiling these undefined functions were referenced:
>          ELEPHANT::WITH-TRANSACTION-SQL from position 7258 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/collection
> s.lisp, 9085 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/collection
> s.lisp, 10024
>
in /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/collect
> ions.lisp
>          :STORE-CONTROLLER-SQL from position 7258 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/collection
> s.lisp, 9085 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/collection
> s.lisp, 10024
>
in /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/collect
> ions.lisp
>          ELEPHANT::SQL-REMOVE-FROM-ROOT from position 10715 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/classes.li
> sp
>          ELEPHANT::PERSISTENT-SLOT-BOUNDP-AUX from position 8117 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/classes.li
> sp, 9924 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/classes.li
> sp, 10239 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/classes.li
> sp
>          :DBCN-SPC from position 1523 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/classes.li
> sp
>          #:G3689 from position 51885 in
>            /nfs/isd3/philpot/lisp/system/elephant/elephant-0.4.0/src/berkeley-d
> b.lisp
>          #:G3616...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
>          ...
> 
> Finally, I think things are working enough to prototype making my
> knowledge base persistent.  Yet, I still get lots of segmentation
> faults and seeming configuration level errors such as "The slot SLOT1
> is missing from the object #<PFOO @ #x720bb22a> of class
> #<PERSISTENT-METACLASS PFOO> during operation SETF."  I'm mostly
> unclear as to why code which works inside a RT:DEFTEST doesn't always
> work at top level, or in my client code.  I seem to do a lot of
> deleting the database, stopping lisp, rebuilding elephant, and trying
> my example again.  I fear I'm corrupting either the BDB store, the
> local Lisp connection to it, and/or CLOS with my missteps.  Symptoms
> are things like 

I have sometimes had to do restart and executed the BDB command
"db_restore".  During development, this generally happens when I 
have a bug that thows me into the debugger.  I suspect it 
happens when a transaction is interrupted.  That is also a 
possible difference between the RT tests and whatever you are
doing.

> 
> Do you have any semi-large applications of ELE-BDB with CLOS?  I am
> kind of looking for a best practices template to follow.

No, I do not -- perhaps someone can mention or contribute one?

I have a large application of my own but I use it with PostGres, 
and I furthermore the way in which it uses Elephant is restricted
compared to very generally operating on slots in a CLOS operation.


> 
> Is it a wart or a feature that SLOT-VALUE and CLOS accessors return
> three values?
> 
> TCE(13): abs
> #<CONCEPT @ #x71edc4ea>
> TCE(14): (class-of abs)
> #<PERSISTENT-METACLASS CONCEPT>
> TCE(15): (name abs)
> "O4||ABSOLUTE"
> 12
> 24
> TCE(16): (slot-value abs 'name)
> "O4||ABSOLUTE"
> 12
> 24
> 
> It seems to me that the following sequence of actions results in a DB
> that you cannot add anything to or read anything from (i.e., until you
> recreate package X).
> 
> 1. Start elephant, open store
> 2. create lisp package X
> 3. put a symbol X::Y into the database
> 4. close store
> 5. restart lisp, elephant, open store
> 6. don't create lisp package X
> 7. Try to access the database -> error
> 
> Is this an intended behavior? (I'll try not create packages X with
> forgettable names in future).

I think this is an example of not having good enough error handling.
Yes, it is a problem that if you try to read an object in and
the package does not exist, it can't be constructed.  The same
is true of the class definition as well.
> 
> In any case, when reading X::Y from the store back into Lisp, if X is
> not available, I think a better error message than "NIL is not a
> package" would be a good idea.

Yes, I think we can improve the robustness along the lines
that you suggest.

> 
> Plodding ahead,
> Andrew
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060110/39a9911c/attachment.html>

From read at robertlread.net  Tue Jan 10 22:06:42 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 10 Jan 2006 16:06:42 -0600
Subject: [elephant-devel] secondary indices
In-Reply-To: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
Message-ID: <1136930803.9533.19.camel@localhost.localdomain>

Dear Kilian,

    Thanks for trying this on openmcl.  This code works for me under
SBCL 9.5.  As we 
all know, CLOS-related stuff like this tends to be more dissimilar
between list implementations
than other parts of LISP.  Since this works under SBCL and has not been
reported under
any other implementation, it might be an implementation-specific
problem.  Unfortunately,
I don't have a PPC machine on which to attempt to duplicate this
problem.

    I will be happy to help you debug this, if you are interested (Waldo
Rubinstein maybe 
even more interested!)
    As a first step, we need to establish what the test regime does
under your OpenMCL.
As you may know, Elephant comes with 106 tests (some of which involve
data migration).
I wouldn't claim they are exhaustive but they are far better than
nothing.  If you are 
willing to take a few extra minutes, please execute this and send me the
transcript:

(asdf:operate 'asdf:load-op :elephant)
(asdf:operate 'asdf:load-op :ele-bdb)
(asdf:operate 'asdf:load-op :elephant-tests)
(in-package "ELEPHANT-TESTS")
(setq *test-path-primary* *testdb-path*)
(setq *test-path-secondary* nil)
(do-all-tests-spec *test-path-primary*)

(on my computer, the actual test execution takes 31 seconds.)

On Sat, 2006-01-07 at 19:55 +0100, ml13 at onlinehome.de wrote:

> Hi,
> 
> while looking at the tutorial, I always get an error while trying to  
> work with secondary indices.
> 
> I am doing this on openmcl "Version 1.0 (DarwinPPC32)" with BDB4.3.
> 
> It looks like the bdb-indexed-tree does not have enough slots??
> 
> Anyway, I am very happy to see that elephant has developed so much!!
> 
> 		Kilian Sprotte
> 
> 
> ELE> (defclass appointment ()
>           ((date :accessor ap-date :initarg :date :type integer)
>            (type :accessor ap-type :initarg :type :type string))
>           (:metaclass persistent-metaclass))
> #<PERSISTENT-METACLASS APPOINTMENT>
> 
> ELE> (defparameter *appointments* (with-transaction () (build-indexed- 
> btree s)))
> *APPOINTMENTS*
> 
> (add-appointment (encode-universal-time 0 0 0 22 12 2004) "Birthday")
> 
> Array index 4 out of bounds for #<SLOT-VECTOR  #x8D6863E> .
>     [Condition of type SIMPLE-ERROR]
> 
> Restarts:
>    0: [ABORT-REQUEST] Abort handling SLIME request.
>    1: [ABORT-BREAK] Reset this process
>    2: [ABORT] Kill this process
> 
> Backtrace:
>    0: (#<CCL::STANDARD-KERNEL-METHOD SLOT-VALUE-USING-CLASS (STANDARD- 
> CLASS T STANDARD-EFFECTIVE-SLOT-DEFINITION)> #<PERSISTENT-METACLASS  
> BDB-INDEXED-BTREE> '(ADD-APPOINTMENT (ENCODE-UNIVERSAL-TIME 0 0 0 22  
> 12 2004) "Birthday") #<STRING-INPUT-STREAM  #x8D66486>)
>        Locals:
>          CLASS = #<PERSISTENT-METACLASS BDB-INDEXED-BTREE>
>          CCL::SLOT-VECTOR = #<SLOT-VECTOR  #x8D6863E>
>    1: (#<STANDARD-METHOD (SETF GET-VALUE) (T T BDB-INDEXED-BTREE)>  
> #<APPOINTMENT #x8D662CE> 3312658800 #<BDB-INDEXED-BTREE #x8D6862E>)
>        Locals:
>          VALUE = #<APPOINTMENT #x8CABF7E>
>          KEY = 3312658800
>          BT = #<BDB-INDEXED-BTREE #x8CAC106>
>          SC = #<BDB-STORE-CONTROLLER #x8C7E0BE>
>    2: (ADD-APPOINTMENT 3312658800 "Birthday")
>        Locals:
>          DATE = 3312658800
>          TYPE = "Birthday"
>          ENV = #<A Mac Pointer #x180E000>
>          #:G541 = 1
>          #:G543 = NIL
>          #:G540 = #<A Mac Pointer #x309450>
>          *CURRENT-TRANSACTION* = #<A Null Mac Pointer>
>          *AUTO-COMMIT* = T
>          #:G546 = #<BDB-INDEXED-BTREE #x8CAC106>
>        Catch-tags:
>          NIL
>          SLEEPYCAT::TRANSACTION
>   
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060110/9273649c/attachment.html>

From waldo at trianet.net  Wed Jan 11 04:26:38 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Tue, 10 Jan 2006 23:26:38 -0500
Subject: [elephant-devel] secondary indices
In-Reply-To: <1136930803.9533.19.camel@localhost.localdomain>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
Message-ID: <1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>

I ran the tests and here is what I got (Note: I never ran this  
before, so not sure what I'm looking for):

Welcome to OpenMCL Version 1.0 (DarwinPPC32)!
? (time (do-all-tests-spec *test-path-primary*))
Doing 106 pending tests of 106 tests total.
FIXNUMS FIXNUM-TYPE-1 BIGNUMS FLOATS RATIONALS BASE-STRINGS STRINGS  
SYMBOLS
CHARS PATHNAMES CONSES HASH-TABLES-1 HASH-TABLES-2 ARRAYS-1 ARRAYS-2
TEST-DEEP-EQUALP OBJECTS CIRCULAR PERSISTENT NON-TRANSIENT-CLASS-SLOT-1
NON-TRANSIENT-CLASS-SLOT-2 TRANSIENT-CLASS-SLOT CLASS-DEFINERS BAD- 
INHERITENCE
MIXES MIXES-RIGHT-SLOTS INHERIT INHERIT-RIGHT-SLOTS INITFORM-CLASSES
INITFORM-TEST INITARG-TEST NO-EVAL-INITFORM REDEFCLASS MAKUNBOUND
Test UPDATE-CLASS failed
Form: (PROGN (DEFCLASS UPDATE-CLASS NIL ((SLOT1 :INITFORM 1 :ACCESSOR  
SLOT1)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
(FOO (MAKE-INSTANCE 'UPDATE-CLASS :SC *STORE-CONTROLLER*))) (DEFCLASS  
UPDATE-CLASS NIL ((SLOT2 :INITFORM 2 :ACCESSOR SLOT2)) (:METACLASS  
PERSISTENT-METACLASS)) (VALUES (SLOT2 FOO) (SIGNALS-CONDITION (SLOT1  
FOO)))))
Expected values: 2
                  T
Actual value: #<UNBOUND-SLOT #x8570FF6>.
Test CHANGE-CLASS failed
Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :INITFORM 1 :ACCESSOR  
SLOT1)) (:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL  
((SLOT1 :INITFORM 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR  
SLOT2)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
(FOO (MAKE-INSTANCE 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE- 
CLASS FOO (FIND-CLASS 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
Expected values: 1
                  2
Actual value: #<UNBOUND-SLOT #x858BBAE>.
Test CHANGE-CLASS2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET  
((FOO (BUILD-BTREE *STORE-CONTROLLER*))) (CHANGE-CLASS FOO (FIND- 
CLASS (IF (TYPEP *STORE-CONTROLLER* 'BDB-STORE-CONTROLLER) 'BDB- 
INDEXED-BTREE 'SQL-INDEXED-BTREE))) (IS-NOT-NULL (INDICES FOO))))
Expected value: T
Actual value: #<UNBOUND-SLOT #x859B756>.
Test CHANGE-CLASS3 failed
Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :ACCESSOR SLOT1))  
(:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL  
((SLOT1 :INITFORM 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR  
SLOT2)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
(FOO (MAKE-INSTANCE 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE- 
CLASS FOO (FIND-CLASS 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
Expected values: 0
                  2
Actual value: #<UNBOUND-SLOT #x8596516>.
BASICPERSISTENCE TESTOID BTREE-MAKE BTREE-PUT BTREE-GET REMOVE-KV  
REMOVED
MAP-BTREE INDEXED-BTREE-MAKE
Test ADD-INDICES failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (SETF INDEX1 (ADD-INDEX INDEXED :INDEX-NAME 'SLOT1 :KEY- 
FORM 'KEY-MAKER)) (SETF INDEX2 (ADD-INDEX INDEXED :INDEX-NAME  
'SLOT2 :KEY-FORM '(LAMBDA (S KEY VALUE) (DECLARE (IGNORE S KEY))  
(VALUES T (SLOT2 VALUE)))))))
Expected value: T
Actual value: NIL.
Test TEST-INDICES failed
Form: (VALUES (= (HASH-TABLE-COUNT (INDICES INDEXED)) 2) (EQ INDEX1  
(GETHASH 'SLOT1 (INDICES INDEXED))) (EQ INDEX2 (GETHASH 'SLOT2  
(INDICES INDEXED))))
Expected values: T
                  T
                  T
Actual value: #<UNBOUND-VARIABLE #x8628626>.
Test INDEXED-PUT failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (LOOP FOR OBJ IN OBJS FOR KEY IN KEYS DO (SETF (GET- 
VALUE KEY INDEXED) OBJ))))
Expected value: T
Actual value: NIL.
Test INDEXED-GET failed
Form: (LOOP FOR KEY IN KEYS FOR I FROM 1 TO 1000 FOR OBJ = (GET-VALUE  
KEY INDEXED) ALWAYS (AND (= (SLOT1 OBJ) I) (= (SLOT2 OBJ) (* I 100))))
Expected value: T
Actual value: #<SIMPLE-ERROR #x864A6A6>.
Test SIMPLE-SLOT-GET failed
Form: (PROGN (SETF (GET-VALUE (NTH 0 KEYS) INDEXED) (NTH 0 OBJS))  
(LET ((OBJ (GET-VALUE 1 INDEX1))) (AND (= (SLOT1 OBJ) 1) (= (SLOT2  
OBJ) (* 1 100)))))
Expected value: T
Actual value: #<SIMPLE-ERROR #x8648FCE>.
Test INDEXED-GET-FROM-SLOT1 failed
Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT1) FOR I FROM 1 TO  
1000 FOR OBJ = (GET-VALUE I INDEX) ALWAYS (= (SLOT1 OBJ) I))
Expected value: T
Actual value: #<SIMPLE-ERROR #x8645B76>.
Test INDEXED-GET-FROM-SLOT2 failed
Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT2) FOR I FROM 1 TO  
1000 FOR OBJ = (GET-VALUE (* I 100) INDEX) ALWAYS (= (SLOT2 OBJ) (* I  
100)))
Expected value: T
Actual value: #<SIMPLE-ERROR #x8641FBE>.
REMOVE-KV-INDEXED
Test NO-KEY-NOR-INDICES failed
Form: (VALUES (GET-VALUE FIRST-KEY INDEXED) (GET-PRIMARY-KEY 1  
INDEX1) (GET-PRIMARY-KEY 100 INDEX2))
Expected values: NIL
                  NIL
                  NIL
Actual value: #<UNBOUND-VARIABLE #x855DEAE>.
Test REMOVE-KV-FROM-SLOT1 failed
Form: (FINISHES (REMOVE-KV 2 INDEX1))
Expected value: T
Actual value: NIL.
Test NO-KEY-NOR-INDICES-SLOT1 failed
Form: (VALUES (GET-VALUE (SECOND KEYS) INDEXED) (GET-PRIMARY-KEY 2  
INDEX1) (GET-PRIMARY-KEY 200 INDEX2))
Expected values: NIL
                  NIL
                  NIL
Actual value: #<UNBOUND-VARIABLE #x8559DAE>.
Test REMOVE-KV-FROM-SLOT2 failed
Form: (FINISHES (REMOVE-KV 300 INDEX2))
Expected value: T
Actual value: NIL.
Test NO-KEY-NOR-INDICES-SLOT2 failed
Form: (VALUES (GET-VALUE (THIRD KEYS) INDEXED) (GET-PRIMARY-KEY 3  
INDEX1) (GET-PRIMARY-KEY 300 INDEX2))
Expected values: NIL
                  NIL
                  NIL
Actual value: #<UNBOUND-VARIABLE #x8555CA6>.
Test MAP-INDEXED failed
Form: (LET ((KS NIL) (VS NIL)) (FLET ((MAPPER (K V) (PUSH K KS) (PUSH  
V VS))) (MAP-BTREE #'MAPPER INDEXED)) (VALUES (AND (SUBSETP KS (CDDDR  
KEYS) :TEST #'EQUALP) (SUBSETP (CDDDR KEYS) KS :TEST #'EQUALP))))
Expected value: T
Actual value: NIL.
Test GET-FIRST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST  
C) (DECLARE (IGNORE HAS V)) (= K 4))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85728C6>.
Test GET-FIRST2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST  
C) (DECLARE (IGNORE HAS V)) (= K 400))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x8582076>.
Test GET-LAST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST  
C) (DECLARE (IGNORE HAS V)) (= K 1000))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x8591826>.
Test GET-LAST2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST  
C) (DECLARE (IGNORE HAS V)) (= K 100000))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85A0D76>.
Test SET failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C  
200) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85CFE4E>.
Test SET2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C  
500) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 500))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85DF0BE>.
Test SET-RANGE failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET- 
RANGE C 199.5) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85EE326>.
Test SET-RANGE2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET- 
RANGE C 501) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 600))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85FD54E>.
Test REM-KV failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET  
((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*))) (LOOP FOR I FROM 0  
TO 10 DO (SETF (GET-VALUE I IBT) (* I I))) (REMOVE-KV 0 IBT) (REMOVE- 
KV 1 IBT) (REMOVE-KV 10 IBT) (EQUAL (LIST (GET-VALUE 0 IBT) (GET- 
VALUE 1 IBT) (GET-VALUE 10 IBT) (GET-VALUE 5 IBT)) '(NIL NIL NIL 25))))
Expected value: T
Actual value: #<SIMPLE-ERROR #x8609BAE>.
Test REM-IDEXKV failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
(SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
FIRST C) (DOTIMES (I 10) (MULTIPLE-VALUE-BIND (HAS KEY VALUE) (CURSOR- 
NEXT C)))) (REMOVE-KV 4 IBT) (REMOVE-KV 5 IBT) (EQUAL (LIST (GET- 
VALUE 4 IBT) (GET-VALUE 5 IBT) (GET-VALUE 6 IBT) (WITH-BTREE-CURSOR  
(C IBT) (CURSOR-FIRST C) (DOTIMES (I 4) (MULTIPLE-VALUE-BIND (HAS KEY  
VALUE) (CURSOR-NEXT C) VALUE)) (MULTIPLE-VALUE-BIND (HAS KEY VALUE)  
(CURSOR-NEXT C) VALUE))) '(NIL NIL 36 49))))
Expected value: T
Actual value: #<SIMPLE-ERROR #x862FAB6>.
MAKE-INDEXED2
Test ADD-INDICES2 failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (SETQ INDEX3 (ADD-INDEX INDEXED2 :INDEX-NAME  
'CRUNCH :KEY-FORM 'CRUNCH))))
Expected value: T
Actual value: NIL.
Test PUT-INDEXED2 failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (LOOP FOR I FROM 0 TO 10000 DO (SETF (GET-VALUE I  
INDEXED2) (- I)))))
Expected value: T
Actual value: NIL.
Test GET-INDEXED2 failed
Form: (LOOP FOR I FROM 0 TO 10000 ALWAYS (= (- I) (GET-VALUE I  
INDEXED2)))
Expected value: T
Actual value: #<TYPE-ERROR #x856D096>.
Test GET-FROM-INDEX3 failed
Form: (LOOP FOR I FROM 0 TO 1000 ALWAYS (= (* I -10) (GET-VALUE I  
INDEX3)))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x856A34E>.
Test DUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (LOOP FOR (MORE K V) = (MULTIPLE-VALUE- 
LIST (CURSOR-FIRST CURS)) THEN (MULTIPLE-VALUE-LIST (CURSOR-NEXT-DUP  
CURS)) WHILE MORE COLLECT V)))
Expected value: (0 -1 -2 -3 -4 -5 -6 -7 -8 -9)
Actual value: #<UNBOUND-VARIABLE #x859571E>.
Test NODUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V) = (MULTIPLE-VALUE-LIST  
(CURSOR-NEXT-NODUP CURS)) FOR I FROM 0 DOWNTO -9990 BY 10 WHILE M  
ALWAYS (= V I))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85A16AE>.
Test PREV-NODUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V) =  
(MULTIPLE-VALUE-LIST (CURSOR-PREV-NODUP CURS)) FOR I FROM -9999 TO -9  
BY 10 WHILE M ALWAYS (= V I))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85CD1AE>.
Test PNODUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V P) = (MULTIPLE-VALUE-LIST  
(CURSOR-PNEXT-NODUP CURS)) FOR I FROM 0 TO 9990 BY 10 WHILE M ALWAYS  
(= P I))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85D86DE>.
Test PPREV-NODUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V P) =  
(MULTIPLE-VALUE-LIST (CURSOR-PPREV-NODUP CURS)) FOR I FROM 9999  
DOWNTO 9 BY 10 WHILE M ALWAYS (= P I))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x85E3BE6>.
Test CUR-DEL1 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
(SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
LAST C) (CURSOR-DELETE C)) (EQUAL (LIST (GET-VALUE 4 IBT) (GET-VALUE  
5 IBT) (GET-VALUE 9 IBT) (GET-VALUE 10 IBT)) '(16 25 NIL 100))))
Expected value: T
Actual value: #<SIMPLE-ERROR #x860F77E>.
Test INDEXED-DELETE failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS)  
(CURSOR-DELETE CURS))))
Expected value: T
Actual value: NIL.
Test TEST-DELETED failed
Form: (VALUES (GET-VALUE 10000 INDEXED2) (GET-VALUE 1000 INDEX3))
Expected values: NIL
                  NIL
Actual value: #<UNBOUND-VARIABLE #x8619B2E>.
Test INDEXED-DELETE2 failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-FIRST CURS)  
(CURSOR-NEXT-DUP CURS) (CURSOR-DELETE CURS))))
Expected value: T
Actual value: NIL.
Test TEST-DELETED2 failed
Form: (VALUES (GET-VALUE 0 INDEXED2) (GET-VALUE 0 INDEX3) (GET-VALUE  
1 INDEXED2) (WITH-BTREE-CURSOR (C INDEX3) (CURSOR-FIRST C) (MULTIPLE- 
VALUE-BIND (M K V) (CURSOR-NEXT C) V)))
Expected values: 0
                  0
                  NIL
                  -2
Actual value: #<UNBOUND-VARIABLE #x86245D6>.
Test CUR-DEL2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
(SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
FIRST C) (CURSOR-NEXT-DUP C) (CURSOR-DELETE C)) (EQUAL (LIST (GET- 
VALUE 1 ID1) (GET-VALUE 0 ID1)) '(1 0))))
Expected value: T
Actual value: #<SIMPLE-ERROR #x863093E>.
Test GET-BOTH failed
Form: (WITH-BTREE-CURSOR (C INDEXED2) (CURSOR-GET-BOTH C 200 -200))
Expected values: T
                  200
                  -200
Actual value: NIL.
Test PGET-BOTH failed
Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P)  
(CURSOR-PGET-BOTH C 10 107) (VALUES K V P)))
Expected values: 10
                  -107
                  107
Actual value: #<UNBOUND-VARIABLE #x864D7B6>.
Test PGET-BOTH-RANGE failed
Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P)  
(CURSOR-PGET-BOTH-RANGE C 10 106.5) (VALUES K V P)))
Expected values: 10
                  -107
                  107
Actual value: #<UNBOUND-VARIABLE #x864B9FE>.
Test PCURSOR failed
Form: (WITH-BTREE-CURSOR (C INDEX3) (VALUES (PCURSOR-PKEY (CURSOR- 
PFIRST C)) (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR- 
PNEXT-NODUP C)) (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY  
(CURSOR-PPREV C)) (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY  
(CURSOR-PLAST C)) (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY  
(CURSOR-PSET-RANGE C 199.5)) (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10  
101)) (PCURSOR-PKEY (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
Expected values: 0
                  2
                  10
                  11
                  10
                  9
                  9999
                  3000
                  2000
                  101
                  112
Actual value: #<UNBOUND-VARIABLE #x8643776>.
Test NEWINDEX failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (SETQ INDEX4 (ADD-INDEX INDEXED2 :INDEX-NAME  
'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T))))
Expected value: T
Actual value: NIL.
Test PCURSOR2 failed
Form: (WITH-BTREE-CURSOR (C INDEX4) (VALUES (PCURSOR-PKEY (CURSOR- 
PFIRST C)) (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR- 
PNEXT-NODUP C)) (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY  
(CURSOR-PPREV C)) (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY  
(CURSOR-PLAST C)) (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY  
(CURSOR-PSET-RANGE C 199.5)) (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10  
101)) (PCURSOR-PKEY (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
Expected values: 0
                  2
                  10
                  11
                  10
                  9
                  9999
                  3000
                  2000
                  101
                  112
Actual value: #<UNBOUND-VARIABLE #x8666A06>.
ADD-GET-REMOVE ADD-GET-REMOVE-SYMBOL EXISTSP PREPARES-SLEEPYCAT TEST- 
SEQ1
TEST-SEQ2 CLEANSUP-SLEEPYCAT REMOVE-ELEMENT
Test MIGRATE1 failed
Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
(ADD-TO-ROOT "x" "y" :STORE-CONTROLLER SC1) (COPY-FROM-KEY "x" SC1  
SC2) (SETF RV (EQUAL (GET-FROM-ROOT "x" :STORE-CONTROLLER SC1) (GET- 
FROM-ROOT "x" :STORE-CONTROLLER SC2)))) (PROGN (SETQ *STORE- 
CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
Expected value: T
Actual value: #<SIMPLE-ERROR #x85779EE>.
Test MIGRATE2 failed
Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
(LET ((IBT (BUILD-BTREE SC1))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET- 
VALUE I IBT) (* I I))) (LET ((MIG (MIGRATE SC2 IBT))) (BTREE-DIFFER  
IBT MIG)))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO- 
COMMIT* *PREV-COMMIT*))))
Expected value: NIL
Actual value: #<SIMPLE-ERROR #x85ACC46>.
Test MIGRATE3 failed
Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
(LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX (ADD-INDEX  
IBT :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T))) (LOOP FOR I  
FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I I))) (LET* ((MIG  
(MIGRATE SC2 IBT)) (NINDEX (GETHASH 'CRUNCH (INDICES IBT)))) (LOOP  
FOR I FROM 0 TO 10 DO (IF (NOT (EQUAL (GET-VALUE I INDEX) (GET-VALUE  
I NINDEX))) (PROGN (FORMAT T "YIKES ~A ~%" I)))) (SETF RV (NOT (BTREE- 
DIFFER IBT MIG))))))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE)  
(SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
Expected value: T
Actual value: #<SIMPLE-ERROR #x85A511E>.
Test MIGRATE4 failed
Form: (FINISHES (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT*  
*AUTO-COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET*  
((SC1 (OPEN-STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH- 
SECONDARY*))) (LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX  
(ADD-INDEX IBT :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T)) (X  
0)) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I I))))))  
(PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV- 
COMMIT*)))))
Expected value: T
Actual value: NIL.
MIGRATE5
52 out of 106 total tests failed: UPDATE-CLASS, CHANGE-CLASS, CHANGE- 
CLASS2, CHANGE-CLASS3, ADD-INDICES,
    TEST-INDICES, INDEXED-PUT, INDEXED-GET, SIMPLE-SLOT-GET,
    INDEXED-GET-FROM-SLOT1, INDEXED-GET-FROM-SLOT2, NO-KEY-NOR-INDICES,
    REMOVE-KV-FROM-SLOT1, NO-KEY-NOR-INDICES-SLOT1, REMOVE-KV-FROM- 
SLOT2,
    NO-KEY-NOR-INDICES-SLOT2, MAP-INDEXED, GET-FIRST, GET-FIRST2, GET- 
LAST,
    GET-LAST2, SET, SET2, SET-RANGE, SET-RANGE2, REM-KV, REM-IDEXKV,
    ADD-INDICES2, PUT-INDEXED2, GET-INDEXED2, GET-FROM-INDEX3, DUP-TEST,
    NODUP-TEST, PREV-NODUP-TEST, PNODUP-TEST, PPREV-NODUP-TEST, CUR- 
DEL1,
    INDEXED-DELETE, TEST-DELETED, INDEXED-DELETE2, TEST-DELETED2, CUR- 
DEL2,
    GET-BOTH, PGET-BOTH, PGET-BOTH-RANGE, PCURSOR, NEWINDEX,  
PCURSOR2, MIGRATE1,
    MIGRATE2, MIGRATE3, MIGRATE4.
(DO-ALL-TESTS-SPEC *TEST-PATH-PRIMARY*) took 8,681 milliseconds  
(8.681 seconds) to run.
Of that, 6,746 milliseconds (6.746 seconds) were spent in user mode
          706 milliseconds (0.706 seconds) were spent in system mode
          1,229 milliseconds (1.229 seconds) were spent executing  
other OS processes.
738 milliseconds (0.738 seconds) was spent in GC.
372,045,168 bytes of memory allocated.
NIL

- Waldo

On Jan 10, 2006, at 5:06 PM, Robert L. Read wrote:

> Dear Kilian,
>
>     Thanks for trying this on openmcl.  This code works for me  
> under SBCL 9.5.  As we
> all know, CLOS-related stuff like this tends to be more dissimilar  
> between list implementations
> than other parts of LISP.  Since this works under SBCL and has not  
> been reported under
> any other implementation, it might be an implementation-specific  
> problem.  Unfortunately,
> I don't have a PPC machine on which to attempt to duplicate this  
> problem.
>
>     I will be happy to help you debug this, if you are interested  
> (Waldo Rubinstein maybe
> even more interested!)
>     As a first step, we need to establish what the test regime does  
> under your OpenMCL.
> As you may know, Elephant comes with 106 tests (some of which  
> involve data migration).
> I wouldn't claim they are exhaustive but they are far better than  
> nothing.  If you are
> willing to take a few extra minutes, please execute this and send  
> me the transcript:
>
> (asdf:operate 'asdf:load-op :elephant)
> (asdf:operate 'asdf:load-op :ele-bdb)
> (asdf:operate 'asdf:load-op :elephant-tests)
> (in-package "ELEPHANT-TESTS")
> (setq *test-path-primary* *testdb-path*)
> (setq *test-path-secondary* nil)
> (do-all-tests-spec *test-path-primary*)
>
> (on my computer, the actual test execution takes 31 seconds.)


From blumberg at math.uchicago.edu  Wed Jan 11 07:38:55 2006
From: blumberg at math.uchicago.edu (Andrew Blumberg)
Date: Wed, 11 Jan 2006 01:38:55 -0600 (CST)
Subject: [elephant-devel] secondary indices
In-Reply-To: <1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
Message-ID: <Pine.LNX.4.61.0601110137160.27574@math.uchicago.edu>

the failure of update-class and change-class(N) are my responsibility, so 
i'll try and debug the problems there.

 								- andrew

On Tue, 10 Jan 2006, Waldo Rubinstein wrote:

> I ran the tests and here is what I got (Note: I never ran this before, so not 
> sure what I'm looking for):
>
> Welcome to OpenMCL Version 1.0 (DarwinPPC32)!
> ? (time (do-all-tests-spec *test-path-primary*))
> Doing 106 pending tests of 106 tests total.
> FIXNUMS FIXNUM-TYPE-1 BIGNUMS FLOATS RATIONALS BASE-STRINGS STRINGS SYMBOLS
> CHARS PATHNAMES CONSES HASH-TABLES-1 HASH-TABLES-2 ARRAYS-1 ARRAYS-2
> TEST-DEEP-EQUALP OBJECTS CIRCULAR PERSISTENT NON-TRANSIENT-CLASS-SLOT-1
> NON-TRANSIENT-CLASS-SLOT-2 TRANSIENT-CLASS-SLOT CLASS-DEFINERS BAD- 
> INHERITENCE
> MIXES MIXES-RIGHT-SLOTS INHERIT INHERIT-RIGHT-SLOTS INITFORM-CLASSES
> INITFORM-TEST INITARG-TEST NO-EVAL-INITFORM REDEFCLASS MAKUNBOUND
> Test UPDATE-CLASS failed
> Form: (PROGN (DEFCLASS UPDATE-CLASS NIL ((SLOT1 :INITFORM 1 :ACCESSOR SLOT1)) 
> (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T) (FOO 
> (MAKE-INSTANCE 'UPDATE-CLASS :SC *STORE-CONTROLLER*))) (DEFCLASS UPDATE-CLASS 
> NIL ((SLOT2 :INITFORM 2 :ACCESSOR SLOT2)) (:METACLASS PERSISTENT-METACLASS)) 
> (VALUES (SLOT2 FOO) (SIGNALS-CONDITION (SLOT1 FOO)))))
> Expected values: 2
>                 T
> Actual value: #<UNBOUND-SLOT #x8570FF6>.
> Test CHANGE-CLASS failed
> Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :INITFORM 1 :ACCESSOR SLOT1)) 
> (:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL ((SLOT1 :INITFORM 
> 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR SLOT2)) (:METACLASS 
> PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T) (FOO (MAKE-INSTANCE 
> 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE-CLASS FOO (FIND-CLASS 
> 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
> Expected values: 1
>                 2
> Actual value: #<UNBOUND-SLOT #x858BBAE>.
> Test CHANGE-CLASS2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET ((FOO 
> (BUILD-BTREE *STORE-CONTROLLER*))) (CHANGE-CLASS FOO (FIND-CLASS (IF (TYPEP 
> *STORE-CONTROLLER* 'BDB-STORE-CONTROLLER) 'BDB-INDEXED-BTREE 
> 'SQL-INDEXED-BTREE))) (IS-NOT-NULL (INDICES FOO))))
> Expected value: T
> Actual value: #<UNBOUND-SLOT #x859B756>.
> Test CHANGE-CLASS3 failed
> Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :ACCESSOR SLOT1)) (:METACLASS 
> PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL ((SLOT1 :INITFORM 0 :ACCESSOR 
> SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR SLOT2)) (:METACLASS 
> PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T) (FOO (MAKE-INSTANCE 
> 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE-CLASS FOO (FIND-CLASS 
> 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
> Expected values: 0
>                 2
> Actual value: #<UNBOUND-SLOT #x8596516>.
> BASICPERSISTENCE TESTOID BTREE-MAKE BTREE-PUT BTREE-GET REMOVE-KV REMOVED
> MAP-BTREE INDEXED-BTREE-MAKE
> Test ADD-INDICES failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) 
> (SETF INDEX1 (ADD-INDEX INDEXED :INDEX-NAME 'SLOT1 :KEY-FORM 'KEY-MAKER)) 
> (SETF INDEX2 (ADD-INDEX INDEXED :INDEX-NAME 'SLOT2 :KEY-FORM '(LAMBDA (S KEY 
> VALUE) (DECLARE (IGNORE S KEY)) (VALUES T (SLOT2 VALUE)))))))
> Expected value: T
> Actual value: NIL.
> Test TEST-INDICES failed
> Form: (VALUES (= (HASH-TABLE-COUNT (INDICES INDEXED)) 2) (EQ INDEX1 (GETHASH 
> 'SLOT1 (INDICES INDEXED))) (EQ INDEX2 (GETHASH 'SLOT2 (INDICES INDEXED))))
> Expected values: T
>                 T
>                 T
> Actual value: #<UNBOUND-VARIABLE #x8628626>.
> Test INDEXED-PUT failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) 
> (LOOP FOR OBJ IN OBJS FOR KEY IN KEYS DO (SETF (GET-VALUE KEY INDEXED) 
> OBJ))))
> Expected value: T
> Actual value: NIL.
> Test INDEXED-GET failed
> Form: (LOOP FOR KEY IN KEYS FOR I FROM 1 TO 1000 FOR OBJ = (GET-VALUE KEY 
> INDEXED) ALWAYS (AND (= (SLOT1 OBJ) I) (= (SLOT2 OBJ) (* I 100))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x864A6A6>.
> Test SIMPLE-SLOT-GET failed
> Form: (PROGN (SETF (GET-VALUE (NTH 0 KEYS) INDEXED) (NTH 0 OBJS)) (LET ((OBJ 
> (GET-VALUE 1 INDEX1))) (AND (= (SLOT1 OBJ) 1) (= (SLOT2 OBJ) (* 1 100)))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x8648FCE>.
> Test INDEXED-GET-FROM-SLOT1 failed
> Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT1) FOR I FROM 1 TO 1000 FOR 
> OBJ = (GET-VALUE I INDEX) ALWAYS (= (SLOT1 OBJ) I))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x8645B76>.
> Test INDEXED-GET-FROM-SLOT2 failed
> Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT2) FOR I FROM 1 TO 1000 FOR 
> OBJ = (GET-VALUE (* I 100) INDEX) ALWAYS (= (SLOT2 OBJ) (* I 100)))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x8641FBE>.
> REMOVE-KV-INDEXED
> Test NO-KEY-NOR-INDICES failed
> Form: (VALUES (GET-VALUE FIRST-KEY INDEXED) (GET-PRIMARY-KEY 1 INDEX1) 
> (GET-PRIMARY-KEY 100 INDEX2))
> Expected values: NIL
>                 NIL
>                 NIL
> Actual value: #<UNBOUND-VARIABLE #x855DEAE>.
> Test REMOVE-KV-FROM-SLOT1 failed
> Form: (FINISHES (REMOVE-KV 2 INDEX1))
> Expected value: T
> Actual value: NIL.
> Test NO-KEY-NOR-INDICES-SLOT1 failed
> Form: (VALUES (GET-VALUE (SECOND KEYS) INDEXED) (GET-PRIMARY-KEY 2 INDEX1) 
> (GET-PRIMARY-KEY 200 INDEX2))
> Expected values: NIL
>                 NIL
>                 NIL
> Actual value: #<UNBOUND-VARIABLE #x8559DAE>.
> Test REMOVE-KV-FROM-SLOT2 failed
> Form: (FINISHES (REMOVE-KV 300 INDEX2))
> Expected value: T
> Actual value: NIL.
> Test NO-KEY-NOR-INDICES-SLOT2 failed
> Form: (VALUES (GET-VALUE (THIRD KEYS) INDEXED) (GET-PRIMARY-KEY 3 INDEX1) 
> (GET-PRIMARY-KEY 300 INDEX2))
> Expected values: NIL
>                 NIL
>                 NIL
> Actual value: #<UNBOUND-VARIABLE #x8555CA6>.
> Test MAP-INDEXED failed
> Form: (LET ((KS NIL) (VS NIL)) (FLET ((MAPPER (K V) (PUSH K KS) (PUSH V VS))) 
> (MAP-BTREE #'MAPPER INDEXED)) (VALUES (AND (SUBSETP KS (CDDDR KEYS) :TEST 
> #'EQUALP) (SUBSETP (CDDDR KEYS) KS :TEST #'EQUALP))))
> Expected value: T
> Actual value: NIL.
> Test GET-FIRST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST C) 
> (DECLARE (IGNORE HAS V)) (= K 4))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85728C6>.
> Test GET-FIRST2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST C) 
> (DECLARE (IGNORE HAS V)) (= K 400))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x8582076>.
> Test GET-LAST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST C) 
> (DECLARE (IGNORE HAS V)) (= K 1000))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x8591826>.
> Test GET-LAST2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST C) 
> (DECLARE (IGNORE HAS V)) (= K 100000))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85A0D76>.
> Test SET failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C 200) 
> (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85CFE4E>.
> Test SET2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C 500) 
> (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 500))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85DF0BE>.
> Test SET-RANGE failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET-RANGE C 
> 199.5) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85EE326>.
> Test SET-RANGE2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET-RANGE C 
> 501) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 600))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85FD54E>.
> Test REM-KV failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET ((IBT 
> (BUILD-INDEXED-BTREE *STORE-CONTROLLER*))) (LOOP FOR I FROM 0 TO 10 DO (SETF 
> (GET-VALUE I IBT) (* I I))) (REMOVE-KV 0 IBT) (REMOVE-KV 1 IBT) (REMOVE-KV 10 
> IBT) (EQUAL (LIST (GET-VALUE 0 IBT) (GET-VALUE 1 IBT) (GET-VALUE 10 IBT) 
> (GET-VALUE 5 IBT)) '(NIL NIL NIL 25))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x8609BAE>.
> Test REM-IDEXKV failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET* ((IBT 
> (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX IBT :INDEX-NAME 
> 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) 
> (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR-FIRST C) (DOTIMES (I 10) 
> (MULTIPLE-VALUE-BIND (HAS KEY VALUE) (CURSOR-NEXT C)))) (REMOVE-KV 4 IBT) 
> (REMOVE-KV 5 IBT) (EQUAL (LIST (GET-VALUE 4 IBT) (GET-VALUE 5 IBT) (GET-VALUE 
> 6 IBT) (WITH-BTREE-CURSOR (C IBT) (CURSOR-FIRST C) (DOTIMES (I 4) 
> (MULTIPLE-VALUE-BIND (HAS KEY VALUE) (CURSOR-NEXT C) VALUE)) 
> (MULTIPLE-VALUE-BIND (HAS KEY VALUE) (CURSOR-NEXT C) VALUE))) '(NIL NIL 36 
> 49))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x862FAB6>.
> MAKE-INDEXED2
> Test ADD-INDICES2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) 
> (SETQ INDEX3 (ADD-INDEX INDEXED2 :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH))))
> Expected value: T
> Actual value: NIL.
> Test PUT-INDEXED2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) 
> (LOOP FOR I FROM 0 TO 10000 DO (SETF (GET-VALUE I INDEXED2) (- I)))))
> Expected value: T
> Actual value: NIL.
> Test GET-INDEXED2 failed
> Form: (LOOP FOR I FROM 0 TO 10000 ALWAYS (= (- I) (GET-VALUE I INDEXED2)))
> Expected value: T
> Actual value: #<TYPE-ERROR #x856D096>.
> Test GET-FROM-INDEX3 failed
> Form: (LOOP FOR I FROM 0 TO 1000 ALWAYS (= (* I -10) (GET-VALUE I INDEX3)))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x856A34E>.
> Test DUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (MORE K V) = (MULTIPLE-VALUE-LIST 
> (CURSOR-FIRST CURS)) THEN (MULTIPLE-VALUE-LIST (CURSOR-NEXT-DUP CURS)) WHILE 
> MORE COLLECT V)))
> Expected value: (0 -1 -2 -3 -4 -5 -6 -7 -8 -9)
> Actual value: #<UNBOUND-VARIABLE #x859571E>.
> Test NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V) = (MULTIPLE-VALUE-LIST 
> (CURSOR-NEXT-NODUP CURS)) FOR I FROM 0 DOWNTO -9990 BY 10 WHILE M ALWAYS (= V 
> I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85A16AE>.
> Test PREV-NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V) = 
> (MULTIPLE-VALUE-LIST (CURSOR-PREV-NODUP CURS)) FOR I FROM -9999 TO -9 BY 10 
> WHILE M ALWAYS (= V I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85CD1AE>.
> Test PNODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V P) = (MULTIPLE-VALUE-LIST 
> (CURSOR-PNEXT-NODUP CURS)) FOR I FROM 0 TO 9990 BY 10 WHILE M ALWAYS (= P 
> I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85D86DE>.
> Test PPREV-NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V P) = 
> (MULTIPLE-VALUE-LIST (CURSOR-PPREV-NODUP CURS)) FOR I FROM 9999 DOWNTO 9 BY 
> 10 WHILE M ALWAYS (= P I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85E3BE6>.
> Test CUR-DEL1 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET* ((IBT 
> (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX IBT :INDEX-NAME 
> 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) 
> (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR-LAST C) (CURSOR-DELETE C)) 
> (EQUAL (LIST (GET-VALUE 4 IBT) (GET-VALUE 5 IBT) (GET-VALUE 9 IBT) (GET-VALUE 
> 10 IBT)) '(16 25 NIL 100))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x860F77E>.
> Test INDEXED-DELETE failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) 
> (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (CURSOR-DELETE CURS))))
> Expected value: T
> Actual value: NIL.
> Test TEST-DELETED failed
> Form: (VALUES (GET-VALUE 10000 INDEXED2) (GET-VALUE 1000 INDEX3))
> Expected values: NIL
>                 NIL
> Actual value: #<UNBOUND-VARIABLE #x8619B2E>.
> Test INDEXED-DELETE2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) 
> (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-FIRST CURS) (CURSOR-NEXT-DUP CURS) 
> (CURSOR-DELETE CURS))))
> Expected value: T
> Actual value: NIL.
> Test TEST-DELETED2 failed
> Form: (VALUES (GET-VALUE 0 INDEXED2) (GET-VALUE 0 INDEX3) (GET-VALUE 1 
> INDEXED2) (WITH-BTREE-CURSOR (C INDEX3) (CURSOR-FIRST C) (MULTIPLE-VALUE-BIND 
> (M K V) (CURSOR-NEXT C) V)))
> Expected values: 0
>                 0
>                 NIL
>                 -2
> Actual value: #<UNBOUND-VARIABLE #x86245D6>.
> Test CUR-DEL2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET* ((IBT 
> (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX IBT :INDEX-NAME 
> 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) 
> (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR-FIRST C) (CURSOR-NEXT-DUP C) 
> (CURSOR-DELETE C)) (EQUAL (LIST (GET-VALUE 1 ID1) (GET-VALUE 0 ID1)) '(1 
> 0))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x863093E>.
> Test GET-BOTH failed
> Form: (WITH-BTREE-CURSOR (C INDEXED2) (CURSOR-GET-BOTH C 200 -200))
> Expected values: T
>                 200
>                 -200
> Actual value: NIL.
> Test PGET-BOTH failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P) 
> (CURSOR-PGET-BOTH C 10 107) (VALUES K V P)))
> Expected values: 10
>                 -107
>                 107
> Actual value: #<UNBOUND-VARIABLE #x864D7B6>.
> Test PGET-BOTH-RANGE failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P) 
> (CURSOR-PGET-BOTH-RANGE C 10 106.5) (VALUES K V P)))
> Expected values: 10
>                 -107
>                 107
> Actual value: #<UNBOUND-VARIABLE #x864B9FE>.
> Test PCURSOR failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (VALUES (PCURSOR-PKEY (CURSOR-PFIRST C)) 
> (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR-PNEXT-NODUP C)) 
> (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY (CURSOR-PPREV C)) 
> (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY (CURSOR-PLAST C)) 
> (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY (CURSOR-PSET-RANGE C 199.5)) 
> (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10 101)) (PCURSOR-PKEY 
> (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
> Expected values: 0
>                 2
>                 10
>                 11
>                 10
>                 9
>                 9999
>                 3000
>                 2000
>                 101
>                 112
> Actual value: #<UNBOUND-VARIABLE #x8643776>.
> Test NEWINDEX failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) 
> (SETQ INDEX4 (ADD-INDEX INDEXED2 :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH 
> :POPULATE T))))
> Expected value: T
> Actual value: NIL.
> Test PCURSOR2 failed
> Form: (WITH-BTREE-CURSOR (C INDEX4) (VALUES (PCURSOR-PKEY (CURSOR-PFIRST C)) 
> (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR-PNEXT-NODUP C)) 
> (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY (CURSOR-PPREV C)) 
> (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY (CURSOR-PLAST C)) 
> (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY (CURSOR-PSET-RANGE C 199.5)) 
> (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10 101)) (PCURSOR-PKEY 
> (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
> Expected values: 0
>                 2
>                 10
>                 11
>                 10
>                 9
>                 9999
>                 3000
>                 2000
>                 101
>                 112
> Actual value: #<UNBOUND-VARIABLE #x8666A06>.
> ADD-GET-REMOVE ADD-GET-REMOVE-SYMBOL EXISTSP PREPARES-SLEEPYCAT TEST-SEQ1
> TEST-SEQ2 CLEANSUP-SLEEPYCAT REMOVE-ELEMENT
> Test MIGRATE1 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO-COMMIT*) 
> (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN-STORE 
> *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*))) (ADD-TO-ROOT 
> "x" "y" :STORE-CONTROLLER SC1) (COPY-FROM-KEY "x" SC1 SC2) (SETF RV (EQUAL 
> (GET-FROM-ROOT "x" :STORE-CONTROLLER SC1) (GET-FROM-ROOT "x" 
> :STORE-CONTROLLER SC2)))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ 
> *AUTO-COMMIT* *PREV-COMMIT*))) RV)
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x85779EE>.
> Test MIGRATE2 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO-COMMIT*) 
> (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN-STORE 
> *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*))) (LET ((IBT 
> (BUILD-BTREE SC1))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I 
> I))) (LET ((MIG (MIGRATE SC2 IBT))) (BTREE-DIFFER IBT MIG)))) (PROGN (SETQ 
> *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV-COMMIT*))))
> Expected value: NIL
> Actual value: #<SIMPLE-ERROR #x85ACC46>.
> Test MIGRATE3 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO-COMMIT*) 
> (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN-STORE 
> *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*))) (LET* ((IBT 
> (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX (ADD-INDEX IBT :INDEX-NAME 'CRUNCH 
> :KEY-FORM 'CRUNCH :POPULATE T))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE 
> I IBT) (* I I))) (LET* ((MIG (MIGRATE SC2 IBT)) (NINDEX (GETHASH 'CRUNCH 
> (INDICES IBT)))) (LOOP FOR I FROM 0 TO 10 DO (IF (NOT (EQUAL (GET-VALUE I 
> INDEX) (GET-VALUE I NINDEX))) (PROGN (FORMAT T "YIKES ~A ~%" I)))) (SETF RV 
> (NOT (BTREE-DIFFER IBT MIG))))))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) 
> (SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x85A511E>.
> Test MIGRATE4 failed
> Form: (FINISHES (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* 
> *AUTO-COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET* ((SC1 
> (OPEN-STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*))) 
> (LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX (ADD-INDEX IBT 
> :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T)) (X 0)) (LOOP FOR I FROM 0 
> TO 10 DO (SETF (GET-VALUE I IBT) (* I I)))))) (PROGN (SETQ *STORE-CONTROLLER* 
> OLD-STORE) (SETQ *AUTO-COMMIT* *PREV-COMMIT*)))))
> Expected value: T
> Actual value: NIL.
> MIGRATE5
> 52 out of 106 total tests failed: UPDATE-CLASS, CHANGE-CLASS, CHANGE-CLASS2, 
> CHANGE-CLASS3, ADD-INDICES,
>   TEST-INDICES, INDEXED-PUT, INDEXED-GET, SIMPLE-SLOT-GET,
>   INDEXED-GET-FROM-SLOT1, INDEXED-GET-FROM-SLOT2, NO-KEY-NOR-INDICES,
>   REMOVE-KV-FROM-SLOT1, NO-KEY-NOR-INDICES-SLOT1, REMOVE-KV-FROM-SLOT2,
>   NO-KEY-NOR-INDICES-SLOT2, MAP-INDEXED, GET-FIRST, GET-FIRST2, GET-LAST,
>   GET-LAST2, SET, SET2, SET-RANGE, SET-RANGE2, REM-KV, REM-IDEXKV,
>   ADD-INDICES2, PUT-INDEXED2, GET-INDEXED2, GET-FROM-INDEX3, DUP-TEST,
>   NODUP-TEST, PREV-NODUP-TEST, PNODUP-TEST, PPREV-NODUP-TEST, CUR-DEL1,
>   INDEXED-DELETE, TEST-DELETED, INDEXED-DELETE2, TEST-DELETED2, CUR-DEL2,
>   GET-BOTH, PGET-BOTH, PGET-BOTH-RANGE, PCURSOR, NEWINDEX, PCURSOR2, 
> MIGRATE1,
>   MIGRATE2, MIGRATE3, MIGRATE4.
> (DO-ALL-TESTS-SPEC *TEST-PATH-PRIMARY*) took 8,681 milliseconds (8.681 
> seconds) to run.
> Of that, 6,746 milliseconds (6.746 seconds) were spent in user mode
>         706 milliseconds (0.706 seconds) were spent in system mode
>         1,229 milliseconds (1.229 seconds) were spent executing other OS 
> processes.
> 738 milliseconds (0.738 seconds) was spent in GC.
> 372,045,168 bytes of memory allocated.
> NIL
>
> - Waldo
>
> On Jan 10, 2006, at 5:06 PM, Robert L. Read wrote:
>
>> Dear Kilian,
>> 
>>     Thanks for trying this on openmcl.  This code works for me under SBCL 
>> 9.5.  As we
>> all know, CLOS-related stuff like this tends to be more dissimilar between 
>> list implementations
>> than other parts of LISP.  Since this works under SBCL and has not been 
>> reported under
>> any other implementation, it might be an implementation-specific problem. 
>> Unfortunately,
>> I don't have a PPC machine on which to attempt to duplicate this problem.
>> 
>>     I will be happy to help you debug this, if you are interested (Waldo 
>> Rubinstein maybe
>> even more interested!)
>>     As a first step, we need to establish what the test regime does under 
>> your OpenMCL.
>> As you may know, Elephant comes with 106 tests (some of which involve data 
>> migration).
>> I wouldn't claim they are exhaustive but they are far better than nothing. 
>> If you are
>> willing to take a few extra minutes, please execute this and send me the 
>> transcript:
>> 
>> (asdf:operate 'asdf:load-op :elephant)
>> (asdf:operate 'asdf:load-op :ele-bdb)
>> (asdf:operate 'asdf:load-op :elephant-tests)
>> (in-package "ELEPHANT-TESTS")
>> (setq *test-path-primary* *testdb-path*)
>> (setq *test-path-secondary* nil)
>> (do-all-tests-spec *test-path-primary*)
>> 
>> (on my computer, the actual test execution takes 31 seconds.)
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>


From ml13 at onlinehome.de  Wed Jan 11 10:41:17 2006
From: ml13 at onlinehome.de (Kilian Sprotte)
Date: Wed, 11 Jan 2006 11:41:17 +0100
Subject: [elephant-devel] secondary indices
In-Reply-To: <1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
Message-ID: <996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>

Hi all,

for the moment, I cannot say anything more that I get the same  
results on openmcl 1.0 as Waldo and also on openmcl 0.14.3.

I append again the transcript with

(defmethod print-object ((obj simple-error) stream)
	     (print-unreadable-object (obj stream :type t)
	       (apply #'format stream (simple-condition-format-control obj)  
(simple-condition-format-arguments obj))))

The #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT-VECTOR   
#x66E4BEE> .> seems to (always?) refer to
the "missing slots" of the bdb-indexed-btree.

I am sorry for not being anymore helpful - I will be quite busy the  
next days - and also I should admit that I am actually a musician,  
who normally would just store his chords into elephant - I mean, I am  
just not sure if I can be of real, valuable help for you. Well, I'll  
try to be helpful where I can, but maybe you shouldn't rely too much  
on what I say :)

Cheers,

	Kilian Sprotte


Doing 106 pending tests of 106 tests total.
FIXNUMS FIXNUM-TYPE-1 BIGNUMS FLOATS RATIONALS BASE-STRINGS STRINGS  
SYMBOLS
CHARS PATHNAMES CONSES HASH-TABLES-1 HASH-TABLES-2 ARRAYS-1 ARRAYS-2
TEST-DEEP-EQUALP OBJECTS CIRCULAR PERSISTENT NON-TRANSIENT-CLASS-SLOT-1
NON-TRANSIENT-CLASS-SLOT-2 TRANSIENT-CLASS-SLOT CLASS-DEFINERS BAD- 
INHERITENCE
MIXES MIXES-RIGHT-SLOTS INHERIT INHERIT-RIGHT-SLOTS INITFORM-CLASSES
INITFORM-TEST INITARG-TEST NO-EVAL-INITFORM REDEFCLASS MAKUNBOUND
Test UPDATE-CLASS failed
Form: (PROGN (DEFCLASS UPDATE-CLASS NIL ((SLOT1 :INITFORM 1 :ACCESSOR  
SLOT1)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
(FOO (MAKE-INSTANCE 'UPDATE-CLASS :SC *STORE-CONTROLLER*))) (DEFCLASS  
UPDATE-CLASS NIL ((SLOT2 :INITFORM 2 :ACCESSOR SLOT2)) (:METACLASS  
PERSISTENT-METACLASS)) (VALUES (SLOT2 FOO) (SIGNALS-CONDITION (SLOT1  
FOO)))))
Expected values: 2
                  T
Actual value: #<UNBOUND-SLOT #x668C2DE>.
Test CHANGE-CLASS failed
Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :INITFORM 1 :ACCESSOR  
SLOT1)) (:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL  
((SLOT1 :INITFORM 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR  
SLOT2)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
(FOO (MAKE-INSTANCE 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE- 
CLASS FOO (FIND-CLASS 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
Expected values: 1
                  2
Actual value: #<UNBOUND-SLOT #x668724E>.
Test CHANGE-CLASS2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET  
((FOO (BUILD-BTREE *STORE-CONTROLLER*))) (CHANGE-CLASS FOO (FIND- 
CLASS (IF (TYPEP *STORE-CONTROLLER* 'BDB-STORE-CONTROLLER) 'BDB- 
INDEXED-BTREE 'SQL-INDEXED-BTREE))) (IS-NOT-NULL (INDICES FOO))))
Expected value: T
Actual value: #<UNBOUND-SLOT #x66984BE>.
Test CHANGE-CLASS3 failed
Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :ACCESSOR SLOT1))  
(:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL  
((SLOT1 :INITFORM 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR  
SLOT2)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
(FOO (MAKE-INSTANCE 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE- 
CLASS FOO (FIND-CLASS 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
Expected values: 0
                  2
Actual value: #<UNBOUND-SLOT #x66934CE>.
BASICPERSISTENCE TESTOID BTREE-MAKE BTREE-PUT BTREE-GET REMOVE-KV  
REMOVED
MAP-BTREE INDEXED-BTREE-MAKE
Test ADD-INDICES failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (SETF INDEX1 (ADD-INDEX INDEXED :INDEX-NAME 'SLOT1 :KEY- 
FORM 'KEY-MAKER)) (SETF INDEX2 (ADD-INDEX INDEXED :INDEX-NAME  
'SLOT2 :KEY-FORM '(LAMBDA (S KEY VALUE) (DECLARE (IGNORE S KEY))  
(VALUES T (SLOT2 VALUE)))))))
Expected value: T
Actual value: NIL.
Test TEST-INDICES failed
Form: (VALUES (= (HASH-TABLE-COUNT (INDICES INDEXED)) 2) (EQ INDEX1  
(GETHASH 'SLOT1 (INDICES INDEXED))) (EQ INDEX2 (GETHASH 'SLOT2  
(INDICES INDEXED))))
Expected values: T
                  T
                  T
Actual value: #<UNBOUND-VARIABLE #x662D8DE>.
Test INDEXED-PUT failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (LOOP FOR OBJ IN OBJS FOR KEY IN KEYS DO (SETF (GET- 
VALUE KEY INDEXED) OBJ))))
Expected value: T
Actual value: NIL.
Test INDEXED-GET failed
Form: (LOOP FOR KEY IN KEYS FOR I FROM 1 TO 1000 FOR OBJ = (GET-VALUE  
KEY INDEXED) ALWAYS (AND (= (SLOT1 OBJ) I) (= (SLOT2 OBJ) (* I 100))))
Expected value: T
Actual value: #<SIMPLE-ERROR No applicable method for args:
(NIL)
to #<STANDARD-GENERIC-FUNCTION SLOT1 #x653FC7E>>.
Test SIMPLE-SLOT-GET failed
Form: (PROGN (SETF (GET-VALUE (NTH 0 KEYS) INDEXED) (NTH 0 OBJS))  
(LET ((OBJ (GET-VALUE 1 INDEX1))) (AND (= (SLOT1 OBJ) 1) (= (SLOT2  
OBJ) (* 1 100)))))
Expected value: T
Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
VECTOR  #x65F449E> .>.
Test INDEXED-GET-FROM-SLOT1 failed
Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT1) FOR I FROM 1 TO  
1000 FOR OBJ = (GET-VALUE I INDEX) ALWAYS (= (SLOT1 OBJ) I))
Expected value: T
Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
VECTOR  #x65F449E> .>.
Test INDEXED-GET-FROM-SLOT2 failed
Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT2) FOR I FROM 1 TO  
1000 FOR OBJ = (GET-VALUE (* I 100) INDEX) ALWAYS (= (SLOT2 OBJ) (* I  
100)))
Expected value: T
Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
VECTOR  #x65F449E> .>.
REMOVE-KV-INDEXED
Test NO-KEY-NOR-INDICES failed
Form: (VALUES (GET-VALUE FIRST-KEY INDEXED) (GET-PRIMARY-KEY 1  
INDEX1) (GET-PRIMARY-KEY 100 INDEX2))
Expected values: NIL
                  NIL
                  NIL
Actual value: #<UNBOUND-VARIABLE #x6645ADE>.
Test REMOVE-KV-FROM-SLOT1 failed
Form: (FINISHES (REMOVE-KV 2 INDEX1))
Expected value: T
Actual value: NIL.
Test NO-KEY-NOR-INDICES-SLOT1 failed
Form: (VALUES (GET-VALUE (SECOND KEYS) INDEXED) (GET-PRIMARY-KEY 2  
INDEX1) (GET-PRIMARY-KEY 200 INDEX2))
Expected values: NIL
                  NIL
                  NIL
Actual value: #<UNBOUND-VARIABLE #x66419D6>.
Test REMOVE-KV-FROM-SLOT2 failed
Form: (FINISHES (REMOVE-KV 300 INDEX2))
Expected value: T
Actual value: NIL.
Test NO-KEY-NOR-INDICES-SLOT2 failed
Form: (VALUES (GET-VALUE (THIRD KEYS) INDEXED) (GET-PRIMARY-KEY 3  
INDEX1) (GET-PRIMARY-KEY 300 INDEX2))
Expected values: NIL
                  NIL
                  NIL
Actual value: #<UNBOUND-VARIABLE #x665D8C6>.
Test MAP-INDEXED failed
Form: (LET ((KS NIL) (VS NIL)) (FLET ((MAPPER (K V) (PUSH K KS) (PUSH  
V VS))) (MAP-BTREE #'MAPPER INDEXED)) (VALUES (AND (SUBSETP KS (CDDDR  
KEYS) :TEST #'EQUALP) (SUBSETP (CDDDR KEYS) KS :TEST #'EQUALP))))
Expected value: T
Actual value: NIL.
Test GET-FIRST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST  
C) (DECLARE (IGNORE HAS V)) (= K 4))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x6669F86>.
Test GET-FIRST2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST  
C) (DECLARE (IGNORE HAS V)) (= K 400))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x667B0BE>.
Test GET-LAST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST  
C) (DECLARE (IGNORE HAS V)) (= K 1000))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x668C1F6>.
Test GET-LAST2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST  
C) (DECLARE (IGNORE HAS V)) (= K 100000))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x669D14E>.
Test SET failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C  
200) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x66ADCBE>.
Test SET2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C  
500) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 500))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x66BE8A6>.
Test SET-RANGE failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET- 
RANGE C 199.5) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x66CF49E>.
Test SET-RANGE2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET- 
RANGE C 501) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 600))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x66C004E>.
Test REM-KV failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET  
((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*))) (LOOP FOR I FROM 0  
TO 10 DO (SETF (GET-VALUE I IBT) (* I I))) (REMOVE-KV 0 IBT) (REMOVE- 
KV 1 IBT) (REMOVE-KV 10 IBT) (EQUAL (LIST (GET-VALUE 0 IBT) (GET- 
VALUE 1 IBT) (GET-VALUE 10 IBT) (GET-VALUE 5 IBT)) '(NIL NIL NIL 25))))
Expected value: T
Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
VECTOR  #x66EEF9E> .>.
Test REM-IDEXKV failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
(SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
FIRST C) (DOTIMES (I 10) (MULTIPLE-VALUE-BIND (HAS KEY VALUE) (CURSOR- 
NEXT C)))) (REMOVE-KV 4 IBT) (REMOVE-KV 5 IBT) (EQUAL (LIST (GET- 
VALUE 4 IBT) (GET-VALUE 5 IBT) (GET-VALUE 6 IBT) (WITH-BTREE-CURSOR  
(C IBT) (CURSOR-FIRST C) (DOTIMES (I 4) (MULTIPLE-VALUE-BIND (HAS KEY  
VALUE) (CURSOR-NEXT C) VALUE)) (MULTIPLE-VALUE-BIND (HAS KEY VALUE)  
(CURSOR-NEXT C) VALUE))) '(NIL NIL 36 49))))
Expected value: T
Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
VECTOR  #x66F773E> .>.
MAKE-INDEXED2
Test ADD-INDICES2 failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (SETQ INDEX3 (ADD-INDEX INDEXED2 :INDEX-NAME  
'CRUNCH :KEY-FORM 'CRUNCH))))
Expected value: T
Actual value: NIL.
Test PUT-INDEXED2 failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (LOOP FOR I FROM 0 TO 10000 DO (SETF (GET-VALUE I  
INDEXED2) (- I)))))
Expected value: T
Actual value: NIL.
Test GET-INDEXED2 failed
Form: (LOOP FOR I FROM 0 TO 10000 ALWAYS (= (- I) (GET-VALUE I  
INDEXED2)))
Expected value: T
Actual value: #<TYPE-ERROR #x664FAEE>.
Test GET-FROM-INDEX3 failed
Form: (LOOP FOR I FROM 0 TO 1000 ALWAYS (= (* I -10) (GET-VALUE I  
INDEX3)))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x664D3FE>.
Test DUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (LOOP FOR (MORE K V) = (MULTIPLE-VALUE- 
LIST (CURSOR-FIRST CURS)) THEN (MULTIPLE-VALUE-LIST (CURSOR-NEXT-DUP  
CURS)) WHILE MORE COLLECT V)))
Expected value: (0 -1 -2 -3 -4 -5 -6 -7 -8 -9)
Actual value: #<UNBOUND-VARIABLE #x665B1D6>.
Test NODUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V) = (MULTIPLE-VALUE-LIST  
(CURSOR-NEXT-NODUP CURS)) FOR I FROM 0 DOWNTO -9990 BY 10 WHILE M  
ALWAYS (= V I))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x6668D3E>.
Test PREV-NODUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V) =  
(MULTIPLE-VALUE-LIST (CURSOR-PREV-NODUP CURS)) FOR I FROM -9999 TO -9  
BY 10 WHILE M ALWAYS (= V I))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x6676486>.
Test PNODUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V P) = (MULTIPLE-VALUE-LIST  
(CURSOR-PNEXT-NODUP CURS)) FOR I FROM 0 TO 9990 BY 10 WHILE M ALWAYS  
(= P I))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x6683656>.
Test PPREV-NODUP-TEST failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V P) =  
(MULTIPLE-VALUE-LIST (CURSOR-PPREV-NODUP CURS)) FOR I FROM 9999  
DOWNTO 9 BY 10 WHILE M ALWAYS (= P I))))
Expected value: T
Actual value: #<UNBOUND-VARIABLE #x66906A6>.
Test CUR-DEL1 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
(SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
LAST C) (CURSOR-DELETE C)) (EQUAL (LIST (GET-VALUE 4 IBT) (GET-VALUE  
5 IBT) (GET-VALUE 9 IBT) (GET-VALUE 10 IBT)) '(16 25 NIL 100))))
Expected value: T
Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
VECTOR  #x66BF17E> .>.
Test INDEXED-DELETE failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS)  
(CURSOR-DELETE CURS))))
Expected value: T
Actual value: NIL.
Test TEST-DELETED failed
Form: (VALUES (GET-VALUE 10000 INDEXED2) (GET-VALUE 1000 INDEX3))
Expected values: NIL
                  NIL
Actual value: #<UNBOUND-VARIABLE #x66C982E>.
Test INDEXED-DELETE2 failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-FIRST CURS)  
(CURSOR-NEXT-DUP CURS) (CURSOR-DELETE CURS))))
Expected value: T
Actual value: NIL.
Test TEST-DELETED2 failed
Form: (VALUES (GET-VALUE 0 INDEXED2) (GET-VALUE 0 INDEX3) (GET-VALUE  
1 INDEXED2) (WITH-BTREE-CURSOR (C INDEX3) (CURSOR-FIRST C) (MULTIPLE- 
VALUE-BIND (M K V) (CURSOR-NEXT C) V)))
Expected values: 0
                  0
                  NIL
                  -2
Actual value: #<UNBOUND-VARIABLE #x66D59E6>.
Test CUR-DEL2 failed
Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
(SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
FIRST C) (CURSOR-NEXT-DUP C) (CURSOR-DELETE C)) (EQUAL (LIST (GET- 
VALUE 1 ID1) (GET-VALUE 0 ID1)) '(1 0))))
Expected value: T
Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
VECTOR  #x66E4BEE> .>.
Test GET-BOTH failed
Form: (WITH-BTREE-CURSOR (C INDEXED2) (CURSOR-GET-BOTH C 200 -200))
Expected values: T
                  200
                  -200
Actual value: NIL.
Test PGET-BOTH failed
Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P)  
(CURSOR-PGET-BOTH C 10 107) (VALUES K V P)))
Expected values: 10
                  -107
                  107
Actual value: #<UNBOUND-VARIABLE #x66E061E>.
Test PGET-BOTH-RANGE failed
Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P)  
(CURSOR-PGET-BOTH-RANGE C 10 106.5) (VALUES K V P)))
Expected values: 10
                  -107
                  107
Actual value: #<UNBOUND-VARIABLE #x66FE846>.
Test PCURSOR failed
Form: (WITH-BTREE-CURSOR (C INDEX3) (VALUES (PCURSOR-PKEY (CURSOR- 
PFIRST C)) (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR- 
PNEXT-NODUP C)) (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY  
(CURSOR-PPREV C)) (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY  
(CURSOR-PLAST C)) (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY  
(CURSOR-PSET-RANGE C 199.5)) (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10  
101)) (PCURSOR-PKEY (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
Expected values: 0
                  2
                  10
                  11
                  10
                  9
                  9999
                  3000
                  2000
                  101
                  112
Actual value: #<UNBOUND-VARIABLE #x66F67E6>.
Test NEWINDEX failed
Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
CONTROLLER*) (SETQ INDEX4 (ADD-INDEX INDEXED2 :INDEX-NAME  
'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T))))
Expected value: T
Actual value: NIL.
Test PCURSOR2 failed
Form: (WITH-BTREE-CURSOR (C INDEX4) (VALUES (PCURSOR-PKEY (CURSOR- 
PFIRST C)) (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR- 
PNEXT-NODUP C)) (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY  
(CURSOR-PPREV C)) (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY  
(CURSOR-PLAST C)) (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY  
(CURSOR-PSET-RANGE C 199.5)) (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10  
101)) (PCURSOR-PKEY (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
Expected values: 0
                  2
                  10
                  11
                  10
                  9
                  9999
                  3000
                  2000
                  101
                  112
Actual value: #<UNBOUND-VARIABLE #x662B3AE>.
ADD-GET-REMOVE ADD-GET-REMOVE-SYMBOL EXISTSP PREPARES-SLEEPYCAT TEST- 
SEQ1
TEST-SEQ2 CLEANSUP-SLEEPYCAT REMOVE-ELEMENT
Test MIGRATE1 failed
Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
(ADD-TO-ROOT "x" "y" :STORE-CONTROLLER SC1) (COPY-FROM-KEY "x" SC1  
SC2) (SETF RV (EQUAL (GET-FROM-ROOT "x" :STORE-CONTROLLER SC1) (GET- 
FROM-ROOT "x" :STORE-CONTROLLER SC2)))) (PROGN (SETQ *STORE- 
CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
Expected value: T
Actual value: #<SIMPLE-ERROR Strategy resolution for this spec  
completely failed!>.
Test MIGRATE2 failed
Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
(LET ((IBT (BUILD-BTREE SC1))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET- 
VALUE I IBT) (* I I))) (LET ((MIG (MIGRATE SC2 IBT))) (BTREE-DIFFER  
IBT MIG)))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO- 
COMMIT* *PREV-COMMIT*))))
Expected value: NIL
Actual value: #<SIMPLE-ERROR Strategy resolution for this spec  
completely failed!>.
Test MIGRATE3 failed
Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
(LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX (ADD-INDEX  
IBT :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T))) (LOOP FOR I  
FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I I))) (LET* ((MIG  
(MIGRATE SC2 IBT)) (NINDEX (GETHASH 'CRUNCH (INDICES IBT)))) (LOOP  
FOR I FROM 0 TO 10 DO (IF (NOT (EQUAL (GET-VALUE I INDEX) (GET-VALUE  
I NINDEX))) (PROGN (FORMAT T "YIKES ~A ~%" I)))) (SETF RV (NOT (BTREE- 
DIFFER IBT MIG))))))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE)  
(SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
Expected value: T
Actual value: #<SIMPLE-ERROR Strategy resolution for this spec  
completely failed!>.
Test MIGRATE4 failed
Form: (FINISHES (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT*  
*AUTO-COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET*  
((SC1 (OPEN-STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH- 
SECONDARY*))) (LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX  
(ADD-INDEX IBT :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T)) (X  
0)) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I I))))))  
(PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV- 
COMMIT*)))))
Expected value: T
Actual value: NIL.
MIGRATE5
52 out of 106 total tests failed: UPDATE-CLASS, CHANGE-CLASS, CHANGE- 
CLASS2, CHANGE-CLASS3, ADD-INDICES,
    TEST-INDICES, INDEXED-PUT, INDEXED-GET, SIMPLE-SLOT-GET,
    INDEXED-GET-FROM-SLOT1, INDEXED-GET-FROM-SLOT2, NO-KEY-NOR-INDICES,
    REMOVE-KV-FROM-SLOT1, NO-KEY-NOR-INDICES-SLOT1, REMOVE-KV-FROM- 
SLOT2,
    NO-KEY-NOR-INDICES-SLOT2, MAP-INDEXED, GET-FIRST, GET-FIRST2, GET- 
LAST,
    GET-LAST2, SET, SET2, SET-RANGE, SET-RANGE2, REM-KV, REM-IDEXKV,
    ADD-INDICES2, PUT-INDEXED2, GET-INDEXED2, GET-FROM-INDEX3, DUP-TEST,
    NODUP-TEST, PREV-NODUP-TEST, PNODUP-TEST, PPREV-NODUP-TEST, CUR- 
DEL1,
    INDEXED-DELETE, TEST-DELETED, INDEXED-DELETE2, TEST-DELETED2, CUR- 
DEL2,
    GET-BOTH, PGET-BOTH, PGET-BOTH-RANGE, PCURSOR, NEWINDEX,  
PCURSOR2, MIGRATE1,
    MIGRATE2, MIGRATE3, MIGRATE4.
NIL
ELE-TESTS>





From waldo at trianet.net  Thu Jan 12 03:48:43 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Wed, 11 Jan 2006 22:48:43 -0500
Subject: [elephant-devel] Problem with transactions
Message-ID: <443366D1-48D7-4D42-9460-B641C4F2AF80@trianet.net>

I'm testing the with-transaction macro bulk loading custom classes  
into elephant and encountered a little snag.

I'm initially loading about 300 objects in the database. The  
performance using with-transaction as opposed to not using it  
(meaning committing after each object is "written") is significant.  
See below:

** not using with-transaction:
CL-USER> (time (oonet:load-834-tape "01053rdSupp.EDI"))
(COM.OONET.EDI:LOAD-834-TAPE "01053rdSupp.EDI") took 2,341  
milliseconds (2.341 seconds) to run.
Of that, 487 milliseconds (0.487 seconds) were spent in user mode
          422 milliseconds (0.422 seconds) were spent in system mode
          1,432 milliseconds (1.432 seconds) were spent executing  
other OS processes.
309,800 bytes of memory allocated.

** using with-transaction:
CL-USER> (time (oonet:load-834-tape "01053rdSupp.EDI"))
(COM.OONET.EDI:LOAD-834-TAPE "01053rdSupp.EDI") took 624 milliseconds  
(0.624 seconds) to run.
Of that, 270 milliseconds (0.270 seconds) were spent in user mode
          19 milliseconds (0.019 seconds) were spent in system mode
          335 milliseconds (0.335 seconds) were spent executing other  
OS processes.
314,160 bytes of memory allocated.

It seems to be approximately 4 times faster using with-transaction,  
which makes sense.

When I try loading a larger set of objects (approximately 13,000),  
the program works as follows:

** not using with-transaction:
CL-USER> (time (oonet:load-834-tape "0106.EDI"))
(COM.OONET.EDI:LOAD-834-TAPE "0106.EDI") took 480,914 milliseconds  
(480.914 seconds) to run.
Of that, 118,046 milliseconds (118.046 seconds) were spent in user mode
          101,881 milliseconds (101.881 seconds) were spent in system  
mode
          260,987 milliseconds (260.987 seconds) were spent executing  
other OS processes.
301 milliseconds (0.301 seconds) was spent in GC.
66,954,792 bytes of memory allocated.

** using with-transaction:
CL-USER> (time (oonet:load-834-tape "RDTLMCNA0106.EDI"))
Berkeley DB error: Cannot allocate memory
    [Condition of type SLEEPYCAT:DB-ERROR]

Restarts:
   0: [ABORT-REQUEST] Abort handling SLIME request.
   1: [ABORT-BREAK] Reset this process
   2: [ABORT] Kill this process

Backtrace:
   0: (#<STANDARD-METHOD SLOT-BOUNDP-USING-CLASS :AROUND  
(ELEPHANT:PERSISTENT-METACLASS ELEPHANT:PERSISTENT-OBJECT  
ELEPHANT::PERSISTENT-SLOT-DEFINITION)> #<ELEPHANT:PERSISTENT- 
METACLASS COM.OONET.EDI:834-ENTRY> #<PERSISTENT-EFFECTIVE-SLOT- 
DEFINITION for database slot COM.OONET.EDI::EFFECTIVE-DATE-QUALIFIER  
#x8BE961E> #<PERSISTENT-EFFECTIVE-SLOT-DEFINITION for database slot  
COM.OONET.EDI::EFFECTIVE-DATE-QUALIFIER #x8BE961E>)
   1: (#<STANDARD-METHOD SHARED-INITIALIZE :AROUND  
(ELEPHANT:PERSISTENT-OBJECT T)> #<COM.OONET.EDI:834-ENTRY #x8CCB1A6> 'T)
   2: (CCL::%%CNM-WITH-ARGS-COMBINED-METHOD-DCODE '(NIL  
#<CCL::STANDARD-KERNEL-METHOD INITIALIZE-INSTANCE (STANDARD-OBJECT) 
 > . 10540825) '((#<STANDARD-METHOD INITIALIZE-INSTANCE :BEFORE #>)  
NIL #<CCL::STANDARD-KERNEL-METHOD INITIALIZE-INSTANCE (STANDARD- 
OBJECT)>) 10540825)
   3: (CCL::%%STANDARD-COMBINED-METHOD-DCODE 'NIL  
#<ELEPHANT:PERSISTENT-METACLASS COM.OONET.EDI:834-ENTRY>)
   4: (CCL::%MAKE-STD-INSTANCE #<COM.OONET.EDI:834-ENTRY #x8CCC40E>  
'("INS" "Y" "18" "001" "25" "A" "" "" "FT"))
   5: (#<CCL::STANDARD-KERNEL-METHOD MAKE-INSTANCE (STANDARD-CLASS)>  
#<ELEPHANT:PERSISTENT-METACLASS COM.OONET.EDI:834-ENTRY>)
   6: (CCL::%MAKE-INSTANCE '(COM.OONET.EDI:834-ENTRY .  
#<ELEPHANT:PERSISTENT-METACLASS COM.OONET.EDI:834-ENTRY>))
   7: (COM.OONET.EDI::PARSE-834-FILE #<FILE-CHARACTER-INPUT-STREAM ("/ 
Users/dsalama/Documents/Projects/MCNA/DHACS/RDTLMCNA0106.EDI"/14)  
#x8BF207E>)
   8: (COM.OONET.EDI:LOAD-834-TAPE "RDTLMCNA0106.EDI")
   9: (CCL::REPORT-TIME '(COM.OONET.EDI:LOAD-834-TAPE  
"RDTLMCNA0106.EDI") #<Anonymous Function #x8BF20EE>)
10: (CCL::CALL-CHECK-REGS 'CCL::REPORT-TIME)
11: (SWANK::EVAL-REGION "(time (oonet:load-834-tape \"RDTLMCNA0106.EDI 
\"))
" 'T)
12: (#<Anonymous Function #x85286E6> "(time (oonet:load-834-tape  
\"RDTLMCNA0106.EDI\"))
")
13: (SWANK::CALL-WITH-BUFFER-SYNTAX #<COMPILED-LEXICAL-CLOSURE  
#x8BF2716>)
14: (CCL::CALL-CHECK-REGS 'SWANK:LISTENER-EVAL)
15: (#<Anonymous Function #x8516CA6> '(SWANK:LISTENER-EVAL "(time  
(oonet:load-834-tape \"RDTLMCNA0106.EDI\"))
") 47 "COMMON-LISP-USER")
16: (#<STANDARD-METHOD SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK (T T)>  
#<Compiled-function SWANK:SWANK-DEBUGGER-HOOK #x8527576> #<COMPILED- 
LEXICAL-CLOSURE #x8BF2756>)
17: (FUNCALL 'SWANK::EVAL-FOR-EMACS)
18: (#<Anonymous Function #x8504C96>)
19: (#<Anonymous Function #x8508D56> #<Anonymous Function #x8504C96>)
20: (SWANK::CALL-WITH-REDIRECTED-IO #<COMPILED-LEXICAL-CLOSURE  
#x8B9D596> #<CONNECTION  #x899E8F6>)
21: (SWANK::CALL-WITH-CONNECTION #<CONNECTION  #x899E8F6> #<Anonymous  
Function #x8504C96>)
22: (SWANK::HANDLE-REQUEST #<CONNECTION  #x899E8F6>)
23: (#<Anonymous Function #x85013B6> #<CONNECTION  #x899E8F6>)
24: (#<Anonymous Function #x8508D56> #<COMPILED-LEXICAL-CLOSURE  
#x89D9606>)
25: (SWANK::CALL-WITH-REDIRECTED-IO #<COMPILED-LEXICAL-CLOSURE  
#x89D95EE> 'NIL)
26: (SWANK::CALL-WITH-CONNECTION #<CONNECTION  #x899E8F6> #<COMPILED- 
LEXICAL-CLOSURE #x89D9606>)
27: (SWANK::CALL-WITH-BINDINGS 'NIL #<COMPILED-LEXICAL-CLOSURE  
#x89D961E>)
28: (CCL::RUN-PROCESS-INITIAL-FORM '(#<COMPILED-LEXICAL-CLOSURE  
#x89D87AE>) #<PROCESS new-repl-thread(38) [Active] #x89D87E6>)
29: (#<Anonymous Function #x80DB61E> '(#<COMPILED-LEXICAL-CLOSURE  
#x89D87AE>) 0)
30: (#<Anonymous Function #x80CE806> 790176 #<LISP-THREAD new-repl- 
thread [tcr @ #x303A80] #x89D88D6>)

It seems it dies after processing about 430 objects.

Is there a fix for this? Is there a work around? How about, is there  
a way that I could commit every 100 objects within a nested  
transaction? Will it help?

Thanks,
Waldo


From read at robertlread.net  Thu Jan 12 04:46:29 2006
From: read at robertlread.net (Robert L. Read)
Date: Wed, 11 Jan 2006 22:46:29 -0600
Subject: [elephant-devel] Problem with transactions
In-Reply-To: <443366D1-48D7-4D42-9460-B641C4F2AF80@trianet.net>
References: <443366D1-48D7-4D42-9460-B641C4F2AF80@trianet.net>
Message-ID: <1137041190.9533.127.camel@localhost.localdomain>

To my knowledge, you won't be able to do nested transactions.

You can of course, write it to do a block of 100 at a time yourself.
That is, you program each transaction to do blocks of 100 by maging
the blocking yourself.

This will probably give you all the speed-up that you require.

It would of course be more convenient if you could write a giant 
transaction, but that is difficult in any system.

I am not absolutely sure, but I suspect based on the error message
that this is being passed up from BDB itself, or is very deep in the 
sleepycat.lisp code.

I would not expect there to be a code work-around that I would
be competent to suggest beyond the one above:  Do one loop of 100
objects inside the transaction, and a loop of 130, or  (+ 1 (mod n
100)),
as the outer loop.


On Wed, 2006-01-11 at 22:48 -0500, Waldo Rubinstein wrote:

> I'm testing the with-transaction macro bulk loading custom classes  
> into elephant and encountered a little snag.
> 
> I'm initially loading about 300 objects in the database. The  
> performance using with-transaction as opposed to not using it  
> (meaning committing after each object is "written") is significant.  
> See below:
> 
> ** not using with-transaction:
> CL-USER> (time (oonet:load-834-tape "01053rdSupp.EDI"))
> (COM.OONET.EDI:LOAD-834-TAPE "01053rdSupp.EDI") took 2,341  
> milliseconds (2.341 seconds) to run.
> Of that, 487 milliseconds (0.487 seconds) were spent in user mode
>           422 milliseconds (0.422 seconds) were spent in system mode
>           1,432 milliseconds (1.432 seconds) were spent executing  
> other OS processes.
> 309,800 bytes of memory allocated.
> 
> ** using with-transaction:
> CL-USER> (time (oonet:load-834-tape "01053rdSupp.EDI"))
> (COM.OONET.EDI:LOAD-834-TAPE "01053rdSupp.EDI") took 624 milliseconds  
> (0.624 seconds) to run.
> Of that, 270 milliseconds (0.270 seconds) were spent in user mode
>           19 milliseconds (0.019 seconds) were spent in system mode
>           335 milliseconds (0.335 seconds) were spent executing other  
> OS processes.
> 314,160 bytes of memory allocated.
> 
> It seems to be approximately 4 times faster using with-transaction,  
> which makes sense.
> 
> When I try loading a larger set of objects (approximately 13,000),  
> the program works as follows:
> 
> ** not using with-transaction:
> CL-USER> (time (oonet:load-834-tape "0106.EDI"))
> (COM.OONET.EDI:LOAD-834-TAPE "0106.EDI") took 480,914 milliseconds  
> (480.914 seconds) to run.
> Of that, 118,046 milliseconds (118.046 seconds) were spent in user mode
>           101,881 milliseconds (101.881 seconds) were spent in system  
> mode
>           260,987 milliseconds (260.987 seconds) were spent executing  
> other OS processes.
> 301 milliseconds (0.301 seconds) was spent in GC.
> 66,954,792 bytes of memory allocated.
> 
> ** using with-transaction:
> CL-USER> (time (oonet:load-834-tape "RDTLMCNA0106.EDI"))
> Berkeley DB error: Cannot allocate memory
>     [Condition of type SLEEPYCAT:DB-ERROR]
> 
> Restarts:
>    0: [ABORT-REQUEST] Abort handling SLIME request.
>    1: [ABORT-BREAK] Reset this process
>    2: [ABORT] Kill this process
> 
> Backtrace:
>    0: (#<STANDARD-METHOD SLOT-BOUNDP-USING-CLASS :AROUND  
> (ELEPHANT:PERSISTENT-METACLASS ELEPHANT:PERSISTENT-OBJECT  
> ELEPHANT::PERSISTENT-SLOT-DEFINITION)> #<ELEPHANT:PERSISTENT- 
> METACLASS COM.OONET.EDI:834-ENTRY> #<PERSISTENT-EFFECTIVE-SLOT- 
> DEFINITION for database slot COM.OONET.EDI::EFFECTIVE-DATE-QUALIFIER  
> #x8BE961E> #<PERSISTENT-EFFECTIVE-SLOT-DEFINITION for database slot  
> COM.OONET.EDI::EFFECTIVE-DATE-QUALIFIER #x8BE961E>)
>    1: (#<STANDARD-METHOD SHARED-INITIALIZE :AROUND  
> (ELEPHANT:PERSISTENT-OBJECT T)> #<COM.OONET.EDI:834-ENTRY #x8CCB1A6> 'T)
>    2: (CCL::%%CNM-WITH-ARGS-COMBINED-METHOD-DCODE '(NIL  
> #<CCL::STANDARD-KERNEL-METHOD INITIALIZE-INSTANCE (STANDARD-OBJECT) 
>  > . 10540825) '((#<STANDARD-METHOD INITIALIZE-INSTANCE :BEFORE #>)  
> NIL #<CCL::STANDARD-KERNEL-METHOD INITIALIZE-INSTANCE (STANDARD- 
> OBJECT)>) 10540825)
>    3: (CCL::%%STANDARD-COMBINED-METHOD-DCODE 'NIL  
> #<ELEPHANT:PERSISTENT-METACLASS COM.OONET.EDI:834-ENTRY>)
>    4: (CCL::%MAKE-STD-INSTANCE #<COM.OONET.EDI:834-ENTRY #x8CCC40E>  
> '("INS" "Y" "18" "001" "25" "A" "" "" "FT"))
>    5: (#<CCL::STANDARD-KERNEL-METHOD MAKE-INSTANCE (STANDARD-CLASS)>  
> #<ELEPHANT:PERSISTENT-METACLASS COM.OONET.EDI:834-ENTRY>)
>    6: (CCL::%MAKE-INSTANCE '(COM.OONET.EDI:834-ENTRY .  
> #<ELEPHANT:PERSISTENT-METACLASS COM.OONET.EDI:834-ENTRY>))
>    7: (COM.OONET.EDI::PARSE-834-FILE #<FILE-CHARACTER-INPUT-STREAM ("/ 
> Users/dsalama/Documents/Projects/MCNA/DHACS/RDTLMCNA0106.EDI"/14)  
> #x8BF207E>)
>    8: (COM.OONET.EDI:LOAD-834-TAPE "RDTLMCNA0106.EDI")
>    9: (CCL::REPORT-TIME '(COM.OONET.EDI:LOAD-834-TAPE  
> "RDTLMCNA0106.EDI") #<Anonymous Function #x8BF20EE>)
> 10: (CCL::CALL-CHECK-REGS 'CCL::REPORT-TIME)
> 11: (SWANK::EVAL-REGION "(time (oonet:load-834-tape \"RDTLMCNA0106.EDI 
> \"))
> " 'T)
> 12: (#<Anonymous Function #x85286E6> "(time (oonet:load-834-tape  
> \"RDTLMCNA0106.EDI\"))
> ")
> 13: (SWANK::CALL-WITH-BUFFER-SYNTAX #<COMPILED-LEXICAL-CLOSURE  
> #x8BF2716>)
> 14: (CCL::CALL-CHECK-REGS 'SWANK:LISTENER-EVAL)
> 15: (#<Anonymous Function #x8516CA6> '(SWANK:LISTENER-EVAL "(time  
> (oonet:load-834-tape \"RDTLMCNA0106.EDI\"))
> ") 47 "COMMON-LISP-USER")
> 16: (#<STANDARD-METHOD SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK (T T)>  
> #<Compiled-function SWANK:SWANK-DEBUGGER-HOOK #x8527576> #<COMPILED- 
> LEXICAL-CLOSURE #x8BF2756>)
> 17: (FUNCALL 'SWANK::EVAL-FOR-EMACS)
> 18: (#<Anonymous Function #x8504C96>)
> 19: (#<Anonymous Function #x8508D56> #<Anonymous Function #x8504C96>)
> 20: (SWANK::CALL-WITH-REDIRECTED-IO #<COMPILED-LEXICAL-CLOSURE  
> #x8B9D596> #<CONNECTION  #x899E8F6>)
> 21: (SWANK::CALL-WITH-CONNECTION #<CONNECTION  #x899E8F6> #<Anonymous  
> Function #x8504C96>)
> 22: (SWANK::HANDLE-REQUEST #<CONNECTION  #x899E8F6>)
> 23: (#<Anonymous Function #x85013B6> #<CONNECTION  #x899E8F6>)
> 24: (#<Anonymous Function #x8508D56> #<COMPILED-LEXICAL-CLOSURE  
> #x89D9606>)
> 25: (SWANK::CALL-WITH-REDIRECTED-IO #<COMPILED-LEXICAL-CLOSURE  
> #x89D95EE> 'NIL)
> 26: (SWANK::CALL-WITH-CONNECTION #<CONNECTION  #x899E8F6> #<COMPILED- 
> LEXICAL-CLOSURE #x89D9606>)
> 27: (SWANK::CALL-WITH-BINDINGS 'NIL #<COMPILED-LEXICAL-CLOSURE  
> #x89D961E>)
> 28: (CCL::RUN-PROCESS-INITIAL-FORM '(#<COMPILED-LEXICAL-CLOSURE  
> #x89D87AE>) #<PROCESS new-repl-thread(38) [Active] #x89D87E6>)
> 29: (#<Anonymous Function #x80DB61E> '(#<COMPILED-LEXICAL-CLOSURE  
> #x89D87AE>) 0)
> 30: (#<Anonymous Function #x80CE806> 790176 #<LISP-THREAD new-repl- 
> thread [tcr @ #x303A80] #x89D88D6>)
> 
> It seems it dies after processing about 430 objects.
> 
> Is there a fix for this? Is there a work around? How about, is there  
> a way that I could commit every 100 objects within a nested  
> transaction? Will it help?
> 
> Thanks,
> Waldo
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060111/6dc7fe5c/attachment.html>

From read at robertlread.net  Wed Jan 11 12:39:15 2006
From: read at robertlread.net (Robert L. Read)
Date: Wed, 11 Jan 2006 06:39:15 -0600
Subject: [elephant-devel] secondary indices
In-Reply-To: <996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
Message-ID: <1136983155.9533.77.camel@localhost.localdomain>

OK, thank you.  

If we can get Elephant into a state that you don't have to be a LISP
hacker to use it,
we will be greatly expanding its usefulness.

I will try to work with Waldo Rubinstein to solve this problem for
openmcl.


On Wed, 2006-01-11 at 11:41 +0100, Kilian Sprotte wrote:

> Hi all,
> 
> for the moment, I cannot say anything more that I get the same  
> results on openmcl 1.0 as Waldo and also on openmcl 0.14.3.
> 
> I append again the transcript with
> 
> (defmethod print-object ((obj simple-error) stream)
> 	     (print-unreadable-object (obj stream :type t)
> 	       (apply #'format stream (simple-condition-format-control obj)  
> (simple-condition-format-arguments obj))))
> 
> The #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT-VECTOR   
> #x66E4BEE> .> seems to (always?) refer to
> the "missing slots" of the bdb-indexed-btree.
> 
> I am sorry for not being anymore helpful - I will be quite busy the  
> next days - and also I should admit that I am actually a musician,  
> who normally would just store his chords into elephant - I mean, I am  
> just not sure if I can be of real, valuable help for you. Well, I'll  
> try to be helpful where I can, but maybe you shouldn't rely too much  
> on what I say :)
> 
> Cheers,
> 
> 	Kilian Sprotte
> 
> 
> Doing 106 pending tests of 106 tests total.
> FIXNUMS FIXNUM-TYPE-1 BIGNUMS FLOATS RATIONALS BASE-STRINGS STRINGS  
> SYMBOLS
> CHARS PATHNAMES CONSES HASH-TABLES-1 HASH-TABLES-2 ARRAYS-1 ARRAYS-2
> TEST-DEEP-EQUALP OBJECTS CIRCULAR PERSISTENT NON-TRANSIENT-CLASS-SLOT-1
> NON-TRANSIENT-CLASS-SLOT-2 TRANSIENT-CLASS-SLOT CLASS-DEFINERS BAD- 
> INHERITENCE
> MIXES MIXES-RIGHT-SLOTS INHERIT INHERIT-RIGHT-SLOTS INITFORM-CLASSES
> INITFORM-TEST INITARG-TEST NO-EVAL-INITFORM REDEFCLASS MAKUNBOUND
> Test UPDATE-CLASS failed
> Form: (PROGN (DEFCLASS UPDATE-CLASS NIL ((SLOT1 :INITFORM 1 :ACCESSOR  
> SLOT1)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
> (FOO (MAKE-INSTANCE 'UPDATE-CLASS :SC *STORE-CONTROLLER*))) (DEFCLASS  
> UPDATE-CLASS NIL ((SLOT2 :INITFORM 2 :ACCESSOR SLOT2)) (:METACLASS  
> PERSISTENT-METACLASS)) (VALUES (SLOT2 FOO) (SIGNALS-CONDITION (SLOT1  
> FOO)))))
> Expected values: 2
>                   T
> Actual value: #<UNBOUND-SLOT #x668C2DE>.
> Test CHANGE-CLASS failed
> Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :INITFORM 1 :ACCESSOR  
> SLOT1)) (:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL  
> ((SLOT1 :INITFORM 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR  
> SLOT2)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
> (FOO (MAKE-INSTANCE 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE- 
> CLASS FOO (FIND-CLASS 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
> Expected values: 1
>                   2
> Actual value: #<UNBOUND-SLOT #x668724E>.
> Test CHANGE-CLASS2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET  
> ((FOO (BUILD-BTREE *STORE-CONTROLLER*))) (CHANGE-CLASS FOO (FIND- 
> CLASS (IF (TYPEP *STORE-CONTROLLER* 'BDB-STORE-CONTROLLER) 'BDB- 
> INDEXED-BTREE 'SQL-INDEXED-BTREE))) (IS-NOT-NULL (INDICES FOO))))
> Expected value: T
> Actual value: #<UNBOUND-SLOT #x66984BE>.
> Test CHANGE-CLASS3 failed
> Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :ACCESSOR SLOT1))  
> (:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL  
> ((SLOT1 :INITFORM 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR  
> SLOT2)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
> (FOO (MAKE-INSTANCE 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE- 
> CLASS FOO (FIND-CLASS 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
> Expected values: 0
>                   2
> Actual value: #<UNBOUND-SLOT #x66934CE>.
> BASICPERSISTENCE TESTOID BTREE-MAKE BTREE-PUT BTREE-GET REMOVE-KV  
> REMOVED
> MAP-BTREE INDEXED-BTREE-MAKE
> Test ADD-INDICES failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (SETF INDEX1 (ADD-INDEX INDEXED :INDEX-NAME 'SLOT1 :KEY- 
> FORM 'KEY-MAKER)) (SETF INDEX2 (ADD-INDEX INDEXED :INDEX-NAME  
> 'SLOT2 :KEY-FORM '(LAMBDA (S KEY VALUE) (DECLARE (IGNORE S KEY))  
> (VALUES T (SLOT2 VALUE)))))))
> Expected value: T
> Actual value: NIL.
> Test TEST-INDICES failed
> Form: (VALUES (= (HASH-TABLE-COUNT (INDICES INDEXED)) 2) (EQ INDEX1  
> (GETHASH 'SLOT1 (INDICES INDEXED))) (EQ INDEX2 (GETHASH 'SLOT2  
> (INDICES INDEXED))))
> Expected values: T
>                   T
>                   T
> Actual value: #<UNBOUND-VARIABLE #x662D8DE>.
> Test INDEXED-PUT failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (LOOP FOR OBJ IN OBJS FOR KEY IN KEYS DO (SETF (GET- 
> VALUE KEY INDEXED) OBJ))))
> Expected value: T
> Actual value: NIL.
> Test INDEXED-GET failed
> Form: (LOOP FOR KEY IN KEYS FOR I FROM 1 TO 1000 FOR OBJ = (GET-VALUE  
> KEY INDEXED) ALWAYS (AND (= (SLOT1 OBJ) I) (= (SLOT2 OBJ) (* I 100))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR No applicable method for args:
> (NIL)
> to #<STANDARD-GENERIC-FUNCTION SLOT1 #x653FC7E>>.
> Test SIMPLE-SLOT-GET failed
> Form: (PROGN (SETF (GET-VALUE (NTH 0 KEYS) INDEXED) (NTH 0 OBJS))  
> (LET ((OBJ (GET-VALUE 1 INDEX1))) (AND (= (SLOT1 OBJ) 1) (= (SLOT2  
> OBJ) (* 1 100)))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
> VECTOR  #x65F449E> .>.
> Test INDEXED-GET-FROM-SLOT1 failed
> Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT1) FOR I FROM 1 TO  
> 1000 FOR OBJ = (GET-VALUE I INDEX) ALWAYS (= (SLOT1 OBJ) I))
> Expected value: T
> Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
> VECTOR  #x65F449E> .>.
> Test INDEXED-GET-FROM-SLOT2 failed
> Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT2) FOR I FROM 1 TO  
> 1000 FOR OBJ = (GET-VALUE (* I 100) INDEX) ALWAYS (= (SLOT2 OBJ) (* I  
> 100)))
> Expected value: T
> Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
> VECTOR  #x65F449E> .>.
> REMOVE-KV-INDEXED
> Test NO-KEY-NOR-INDICES failed
> Form: (VALUES (GET-VALUE FIRST-KEY INDEXED) (GET-PRIMARY-KEY 1  
> INDEX1) (GET-PRIMARY-KEY 100 INDEX2))
> Expected values: NIL
>                   NIL
>                   NIL
> Actual value: #<UNBOUND-VARIABLE #x6645ADE>.
> Test REMOVE-KV-FROM-SLOT1 failed
> Form: (FINISHES (REMOVE-KV 2 INDEX1))
> Expected value: T
> Actual value: NIL.
> Test NO-KEY-NOR-INDICES-SLOT1 failed
> Form: (VALUES (GET-VALUE (SECOND KEYS) INDEXED) (GET-PRIMARY-KEY 2  
> INDEX1) (GET-PRIMARY-KEY 200 INDEX2))
> Expected values: NIL
>                   NIL
>                   NIL
> Actual value: #<UNBOUND-VARIABLE #x66419D6>.
> Test REMOVE-KV-FROM-SLOT2 failed
> Form: (FINISHES (REMOVE-KV 300 INDEX2))
> Expected value: T
> Actual value: NIL.
> Test NO-KEY-NOR-INDICES-SLOT2 failed
> Form: (VALUES (GET-VALUE (THIRD KEYS) INDEXED) (GET-PRIMARY-KEY 3  
> INDEX1) (GET-PRIMARY-KEY 300 INDEX2))
> Expected values: NIL
>                   NIL
>                   NIL
> Actual value: #<UNBOUND-VARIABLE #x665D8C6>.
> Test MAP-INDEXED failed
> Form: (LET ((KS NIL) (VS NIL)) (FLET ((MAPPER (K V) (PUSH K KS) (PUSH  
> V VS))) (MAP-BTREE #'MAPPER INDEXED)) (VALUES (AND (SUBSETP KS (CDDDR  
> KEYS) :TEST #'EQUALP) (SUBSETP (CDDDR KEYS) KS :TEST #'EQUALP))))
> Expected value: T
> Actual value: NIL.
> Test GET-FIRST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST  
> C) (DECLARE (IGNORE HAS V)) (= K 4))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x6669F86>.
> Test GET-FIRST2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST  
> C) (DECLARE (IGNORE HAS V)) (= K 400))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x667B0BE>.
> Test GET-LAST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST  
> C) (DECLARE (IGNORE HAS V)) (= K 1000))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x668C1F6>.
> Test GET-LAST2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST  
> C) (DECLARE (IGNORE HAS V)) (= K 100000))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x669D14E>.
> Test SET failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C  
> 200) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x66ADCBE>.
> Test SET2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C  
> 500) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 500))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x66BE8A6>.
> Test SET-RANGE failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET- 
> RANGE C 199.5) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x66CF49E>.
> Test SET-RANGE2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET- 
> RANGE C 501) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 600))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x66C004E>.
> Test REM-KV failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET  
> ((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*))) (LOOP FOR I FROM 0  
> TO 10 DO (SETF (GET-VALUE I IBT) (* I I))) (REMOVE-KV 0 IBT) (REMOVE- 
> KV 1 IBT) (REMOVE-KV 10 IBT) (EQUAL (LIST (GET-VALUE 0 IBT) (GET- 
> VALUE 1 IBT) (GET-VALUE 10 IBT) (GET-VALUE 5 IBT)) '(NIL NIL NIL 25))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
> VECTOR  #x66EEF9E> .>.
> Test REM-IDEXKV failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
> ((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
> IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
> (SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
> FIRST C) (DOTIMES (I 10) (MULTIPLE-VALUE-BIND (HAS KEY VALUE) (CURSOR- 
> NEXT C)))) (REMOVE-KV 4 IBT) (REMOVE-KV 5 IBT) (EQUAL (LIST (GET- 
> VALUE 4 IBT) (GET-VALUE 5 IBT) (GET-VALUE 6 IBT) (WITH-BTREE-CURSOR  
> (C IBT) (CURSOR-FIRST C) (DOTIMES (I 4) (MULTIPLE-VALUE-BIND (HAS KEY  
> VALUE) (CURSOR-NEXT C) VALUE)) (MULTIPLE-VALUE-BIND (HAS KEY VALUE)  
> (CURSOR-NEXT C) VALUE))) '(NIL NIL 36 49))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
> VECTOR  #x66F773E> .>.
> MAKE-INDEXED2
> Test ADD-INDICES2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (SETQ INDEX3 (ADD-INDEX INDEXED2 :INDEX-NAME  
> 'CRUNCH :KEY-FORM 'CRUNCH))))
> Expected value: T
> Actual value: NIL.
> Test PUT-INDEXED2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (LOOP FOR I FROM 0 TO 10000 DO (SETF (GET-VALUE I  
> INDEXED2) (- I)))))
> Expected value: T
> Actual value: NIL.
> Test GET-INDEXED2 failed
> Form: (LOOP FOR I FROM 0 TO 10000 ALWAYS (= (- I) (GET-VALUE I  
> INDEXED2)))
> Expected value: T
> Actual value: #<TYPE-ERROR #x664FAEE>.
> Test GET-FROM-INDEX3 failed
> Form: (LOOP FOR I FROM 0 TO 1000 ALWAYS (= (* I -10) (GET-VALUE I  
> INDEX3)))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x664D3FE>.
> Test DUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (MORE K V) = (MULTIPLE-VALUE- 
> LIST (CURSOR-FIRST CURS)) THEN (MULTIPLE-VALUE-LIST (CURSOR-NEXT-DUP  
> CURS)) WHILE MORE COLLECT V)))
> Expected value: (0 -1 -2 -3 -4 -5 -6 -7 -8 -9)
> Actual value: #<UNBOUND-VARIABLE #x665B1D6>.
> Test NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V) = (MULTIPLE-VALUE-LIST  
> (CURSOR-NEXT-NODUP CURS)) FOR I FROM 0 DOWNTO -9990 BY 10 WHILE M  
> ALWAYS (= V I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x6668D3E>.
> Test PREV-NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V) =  
> (MULTIPLE-VALUE-LIST (CURSOR-PREV-NODUP CURS)) FOR I FROM -9999 TO -9  
> BY 10 WHILE M ALWAYS (= V I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x6676486>.
> Test PNODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V P) = (MULTIPLE-VALUE-LIST  
> (CURSOR-PNEXT-NODUP CURS)) FOR I FROM 0 TO 9990 BY 10 WHILE M ALWAYS  
> (= P I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x6683656>.
> Test PPREV-NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V P) =  
> (MULTIPLE-VALUE-LIST (CURSOR-PPREV-NODUP CURS)) FOR I FROM 9999  
> DOWNTO 9 BY 10 WHILE M ALWAYS (= P I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x66906A6>.
> Test CUR-DEL1 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
> ((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
> IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
> (SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
> LAST C) (CURSOR-DELETE C)) (EQUAL (LIST (GET-VALUE 4 IBT) (GET-VALUE  
> 5 IBT) (GET-VALUE 9 IBT) (GET-VALUE 10 IBT)) '(16 25 NIL 100))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
> VECTOR  #x66BF17E> .>.
> Test INDEXED-DELETE failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS)  
> (CURSOR-DELETE CURS))))
> Expected value: T
> Actual value: NIL.
> Test TEST-DELETED failed
> Form: (VALUES (GET-VALUE 10000 INDEXED2) (GET-VALUE 1000 INDEX3))
> Expected values: NIL
>                   NIL
> Actual value: #<UNBOUND-VARIABLE #x66C982E>.
> Test INDEXED-DELETE2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-FIRST CURS)  
> (CURSOR-NEXT-DUP CURS) (CURSOR-DELETE CURS))))
> Expected value: T
> Actual value: NIL.
> Test TEST-DELETED2 failed
> Form: (VALUES (GET-VALUE 0 INDEXED2) (GET-VALUE 0 INDEX3) (GET-VALUE  
> 1 INDEXED2) (WITH-BTREE-CURSOR (C INDEX3) (CURSOR-FIRST C) (MULTIPLE- 
> VALUE-BIND (M K V) (CURSOR-NEXT C) V)))
> Expected values: 0
>                   0
>                   NIL
>                   -2
> Actual value: #<UNBOUND-VARIABLE #x66D59E6>.
> Test CUR-DEL2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
> ((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
> IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
> (SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
> FIRST C) (CURSOR-NEXT-DUP C) (CURSOR-DELETE C)) (EQUAL (LIST (GET- 
> VALUE 1 ID1) (GET-VALUE 0 ID1)) '(1 0))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR Array index 4 out of bounds for #<SLOT- 
> VECTOR  #x66E4BEE> .>.
> Test GET-BOTH failed
> Form: (WITH-BTREE-CURSOR (C INDEXED2) (CURSOR-GET-BOTH C 200 -200))
> Expected values: T
>                   200
>                   -200
> Actual value: NIL.
> Test PGET-BOTH failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P)  
> (CURSOR-PGET-BOTH C 10 107) (VALUES K V P)))
> Expected values: 10
>                   -107
>                   107
> Actual value: #<UNBOUND-VARIABLE #x66E061E>.
> Test PGET-BOTH-RANGE failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P)  
> (CURSOR-PGET-BOTH-RANGE C 10 106.5) (VALUES K V P)))
> Expected values: 10
>                   -107
>                   107
> Actual value: #<UNBOUND-VARIABLE #x66FE846>.
> Test PCURSOR failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (VALUES (PCURSOR-PKEY (CURSOR- 
> PFIRST C)) (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR- 
> PNEXT-NODUP C)) (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY  
> (CURSOR-PPREV C)) (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY  
> (CURSOR-PLAST C)) (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY  
> (CURSOR-PSET-RANGE C 199.5)) (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10  
> 101)) (PCURSOR-PKEY (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
> Expected values: 0
>                   2
>                   10
>                   11
>                   10
>                   9
>                   9999
>                   3000
>                   2000
>                   101
>                   112
> Actual value: #<UNBOUND-VARIABLE #x66F67E6>.
> Test NEWINDEX failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (SETQ INDEX4 (ADD-INDEX INDEXED2 :INDEX-NAME  
> 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T))))
> Expected value: T
> Actual value: NIL.
> Test PCURSOR2 failed
> Form: (WITH-BTREE-CURSOR (C INDEX4) (VALUES (PCURSOR-PKEY (CURSOR- 
> PFIRST C)) (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR- 
> PNEXT-NODUP C)) (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY  
> (CURSOR-PPREV C)) (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY  
> (CURSOR-PLAST C)) (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY  
> (CURSOR-PSET-RANGE C 199.5)) (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10  
> 101)) (PCURSOR-PKEY (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
> Expected values: 0
>                   2
>                   10
>                   11
>                   10
>                   9
>                   9999
>                   3000
>                   2000
>                   101
>                   112
> Actual value: #<UNBOUND-VARIABLE #x662B3AE>.
> ADD-GET-REMOVE ADD-GET-REMOVE-SYMBOL EXISTSP PREPARES-SLEEPYCAT TEST- 
> SEQ1
> TEST-SEQ2 CLEANSUP-SLEEPYCAT REMOVE-ELEMENT
> Test MIGRATE1 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
> COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
> STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
> (ADD-TO-ROOT "x" "y" :STORE-CONTROLLER SC1) (COPY-FROM-KEY "x" SC1  
> SC2) (SETF RV (EQUAL (GET-FROM-ROOT "x" :STORE-CONTROLLER SC1) (GET- 
> FROM-ROOT "x" :STORE-CONTROLLER SC2)))) (PROGN (SETQ *STORE- 
> CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
> Expected value: T
> Actual value: #<SIMPLE-ERROR Strategy resolution for this spec  
> completely failed!>.
> Test MIGRATE2 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
> COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
> STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
> (LET ((IBT (BUILD-BTREE SC1))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET- 
> VALUE I IBT) (* I I))) (LET ((MIG (MIGRATE SC2 IBT))) (BTREE-DIFFER  
> IBT MIG)))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO- 
> COMMIT* *PREV-COMMIT*))))
> Expected value: NIL
> Actual value: #<SIMPLE-ERROR Strategy resolution for this spec  
> completely failed!>.
> Test MIGRATE3 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
> COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
> STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
> (LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX (ADD-INDEX  
> IBT :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T))) (LOOP FOR I  
> FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I I))) (LET* ((MIG  
> (MIGRATE SC2 IBT)) (NINDEX (GETHASH 'CRUNCH (INDICES IBT)))) (LOOP  
> FOR I FROM 0 TO 10 DO (IF (NOT (EQUAL (GET-VALUE I INDEX) (GET-VALUE  
> I NINDEX))) (PROGN (FORMAT T "YIKES ~A ~%" I)))) (SETF RV (NOT (BTREE- 
> DIFFER IBT MIG))))))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE)  
> (SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
> Expected value: T
> Actual value: #<SIMPLE-ERROR Strategy resolution for this spec  
> completely failed!>.
> Test MIGRATE4 failed
> Form: (FINISHES (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT*  
> *AUTO-COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET*  
> ((SC1 (OPEN-STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH- 
> SECONDARY*))) (LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX  
> (ADD-INDEX IBT :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T)) (X  
> 0)) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I I))))))  
> (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV- 
> COMMIT*)))))
> Expected value: T
> Actual value: NIL.
> MIGRATE5
> 52 out of 106 total tests failed: UPDATE-CLASS, CHANGE-CLASS, CHANGE- 
> CLASS2, CHANGE-CLASS3, ADD-INDICES,
>     TEST-INDICES, INDEXED-PUT, INDEXED-GET, SIMPLE-SLOT-GET,
>     INDEXED-GET-FROM-SLOT1, INDEXED-GET-FROM-SLOT2, NO-KEY-NOR-INDICES,
>     REMOVE-KV-FROM-SLOT1, NO-KEY-NOR-INDICES-SLOT1, REMOVE-KV-FROM- 
> SLOT2,
>     NO-KEY-NOR-INDICES-SLOT2, MAP-INDEXED, GET-FIRST, GET-FIRST2, GET- 
> LAST,
>     GET-LAST2, SET, SET2, SET-RANGE, SET-RANGE2, REM-KV, REM-IDEXKV,
>     ADD-INDICES2, PUT-INDEXED2, GET-INDEXED2, GET-FROM-INDEX3, DUP-TEST,
>     NODUP-TEST, PREV-NODUP-TEST, PNODUP-TEST, PPREV-NODUP-TEST, CUR- 
> DEL1,
>     INDEXED-DELETE, TEST-DELETED, INDEXED-DELETE2, TEST-DELETED2, CUR- 
> DEL2,
>     GET-BOTH, PGET-BOTH, PGET-BOTH-RANGE, PCURSOR, NEWINDEX,  
> PCURSOR2, MIGRATE1,
>     MIGRATE2, MIGRATE3, MIGRATE4.
> NIL
> ELE-TESTS>
> 
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060111/5d3ad3d2/attachment.html>

From read at robertlread.net  Wed Jan 11 12:41:49 2006
From: read at robertlread.net (Robert L. Read)
Date: Wed, 11 Jan 2006 06:41:49 -0600
Subject: [elephant-devel] secondary indices
In-Reply-To: <1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
Message-ID: <1136983310.9533.80.camel@localhost.localdomain>

Well the test should all pass.

However, this is not as bad as it seems; the 52 failures don't represent
52 SEPARATE failures,
but a failure, probably something simple, but hard-to-find, in the
indexing, which causes all the 
following tests to fail.


On Tue, 2006-01-10 at 23:26 -0500, Waldo Rubinstein wrote:

> I ran the tests and here is what I got (Note: I never ran this  
> before, so not sure what I'm looking for):
> 
> Welcome to OpenMCL Version 1.0 (DarwinPPC32)!
> ? (time (do-all-tests-spec *test-path-primary*))
> Doing 106 pending tests of 106 tests total.
> FIXNUMS FIXNUM-TYPE-1 BIGNUMS FLOATS RATIONALS BASE-STRINGS STRINGS  
> SYMBOLS
> CHARS PATHNAMES CONSES HASH-TABLES-1 HASH-TABLES-2 ARRAYS-1 ARRAYS-2
> TEST-DEEP-EQUALP OBJECTS CIRCULAR PERSISTENT NON-TRANSIENT-CLASS-SLOT-1
> NON-TRANSIENT-CLASS-SLOT-2 TRANSIENT-CLASS-SLOT CLASS-DEFINERS BAD- 
> INHERITENCE
> MIXES MIXES-RIGHT-SLOTS INHERIT INHERIT-RIGHT-SLOTS INITFORM-CLASSES
> INITFORM-TEST INITARG-TEST NO-EVAL-INITFORM REDEFCLASS MAKUNBOUND
> Test UPDATE-CLASS failed
> Form: (PROGN (DEFCLASS UPDATE-CLASS NIL ((SLOT1 :INITFORM 1 :ACCESSOR  
> SLOT1)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
> (FOO (MAKE-INSTANCE 'UPDATE-CLASS :SC *STORE-CONTROLLER*))) (DEFCLASS  
> UPDATE-CLASS NIL ((SLOT2 :INITFORM 2 :ACCESSOR SLOT2)) (:METACLASS  
> PERSISTENT-METACLASS)) (VALUES (SLOT2 FOO) (SIGNALS-CONDITION (SLOT1  
> FOO)))))
> Expected values: 2
>                   T
> Actual value: #<UNBOUND-SLOT #x8570FF6>.
> Test CHANGE-CLASS failed
> Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :INITFORM 1 :ACCESSOR  
> SLOT1)) (:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL  
> ((SLOT1 :INITFORM 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR  
> SLOT2)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
> (FOO (MAKE-INSTANCE 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE- 
> CLASS FOO (FIND-CLASS 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
> Expected values: 1
>                   2
> Actual value: #<UNBOUND-SLOT #x858BBAE>.
> Test CHANGE-CLASS2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET  
> ((FOO (BUILD-BTREE *STORE-CONTROLLER*))) (CHANGE-CLASS FOO (FIND- 
> CLASS (IF (TYPEP *STORE-CONTROLLER* 'BDB-STORE-CONTROLLER) 'BDB- 
> INDEXED-BTREE 'SQL-INDEXED-BTREE))) (IS-NOT-NULL (INDICES FOO))))
> Expected value: T
> Actual value: #<UNBOUND-SLOT #x859B756>.
> Test CHANGE-CLASS3 failed
> Form: (PROGN (DEFCLASS CLASS-ONE NIL ((SLOT1 :ACCESSOR SLOT1))  
> (:METACLASS PERSISTENT-METACLASS)) (DEFCLASS CLASS-TWO NIL  
> ((SLOT1 :INITFORM 0 :ACCESSOR SLOT1) (SLOT2 :INITFORM 2 :ACCESSOR  
> SLOT2)) (:METACLASS PERSISTENT-METACLASS)) (LET* ((*AUTO-COMMIT* T)  
> (FOO (MAKE-INSTANCE 'CLASS-ONE :SC *STORE-CONTROLLER*))) (CHANGE- 
> CLASS FOO (FIND-CLASS 'CLASS-TWO)) (VALUES (SLOT1 FOO) (SLOT2 FOO))))
> Expected values: 0
>                   2
> Actual value: #<UNBOUND-SLOT #x8596516>.
> BASICPERSISTENCE TESTOID BTREE-MAKE BTREE-PUT BTREE-GET REMOVE-KV  
> REMOVED
> MAP-BTREE INDEXED-BTREE-MAKE
> Test ADD-INDICES failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (SETF INDEX1 (ADD-INDEX INDEXED :INDEX-NAME 'SLOT1 :KEY- 
> FORM 'KEY-MAKER)) (SETF INDEX2 (ADD-INDEX INDEXED :INDEX-NAME  
> 'SLOT2 :KEY-FORM '(LAMBDA (S KEY VALUE) (DECLARE (IGNORE S KEY))  
> (VALUES T (SLOT2 VALUE)))))))
> Expected value: T
> Actual value: NIL.
> Test TEST-INDICES failed
> Form: (VALUES (= (HASH-TABLE-COUNT (INDICES INDEXED)) 2) (EQ INDEX1  
> (GETHASH 'SLOT1 (INDICES INDEXED))) (EQ INDEX2 (GETHASH 'SLOT2  
> (INDICES INDEXED))))
> Expected values: T
>                   T
>                   T
> Actual value: #<UNBOUND-VARIABLE #x8628626>.
> Test INDEXED-PUT failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (LOOP FOR OBJ IN OBJS FOR KEY IN KEYS DO (SETF (GET- 
> VALUE KEY INDEXED) OBJ))))
> Expected value: T
> Actual value: NIL.
> Test INDEXED-GET failed
> Form: (LOOP FOR KEY IN KEYS FOR I FROM 1 TO 1000 FOR OBJ = (GET-VALUE  
> KEY INDEXED) ALWAYS (AND (= (SLOT1 OBJ) I) (= (SLOT2 OBJ) (* I 100))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x864A6A6>.
> Test SIMPLE-SLOT-GET failed
> Form: (PROGN (SETF (GET-VALUE (NTH 0 KEYS) INDEXED) (NTH 0 OBJS))  
> (LET ((OBJ (GET-VALUE 1 INDEX1))) (AND (= (SLOT1 OBJ) 1) (= (SLOT2  
> OBJ) (* 1 100)))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x8648FCE>.
> Test INDEXED-GET-FROM-SLOT1 failed
> Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT1) FOR I FROM 1 TO  
> 1000 FOR OBJ = (GET-VALUE I INDEX) ALWAYS (= (SLOT1 OBJ) I))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x8645B76>.
> Test INDEXED-GET-FROM-SLOT2 failed
> Form: (LOOP WITH INDEX = (GET-INDEX INDEXED 'SLOT2) FOR I FROM 1 TO  
> 1000 FOR OBJ = (GET-VALUE (* I 100) INDEX) ALWAYS (= (SLOT2 OBJ) (* I  
> 100)))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x8641FBE>.
> REMOVE-KV-INDEXED
> Test NO-KEY-NOR-INDICES failed
> Form: (VALUES (GET-VALUE FIRST-KEY INDEXED) (GET-PRIMARY-KEY 1  
> INDEX1) (GET-PRIMARY-KEY 100 INDEX2))
> Expected values: NIL
>                   NIL
>                   NIL
> Actual value: #<UNBOUND-VARIABLE #x855DEAE>.
> Test REMOVE-KV-FROM-SLOT1 failed
> Form: (FINISHES (REMOVE-KV 2 INDEX1))
> Expected value: T
> Actual value: NIL.
> Test NO-KEY-NOR-INDICES-SLOT1 failed
> Form: (VALUES (GET-VALUE (SECOND KEYS) INDEXED) (GET-PRIMARY-KEY 2  
> INDEX1) (GET-PRIMARY-KEY 200 INDEX2))
> Expected values: NIL
>                   NIL
>                   NIL
> Actual value: #<UNBOUND-VARIABLE #x8559DAE>.
> Test REMOVE-KV-FROM-SLOT2 failed
> Form: (FINISHES (REMOVE-KV 300 INDEX2))
> Expected value: T
> Actual value: NIL.
> Test NO-KEY-NOR-INDICES-SLOT2 failed
> Form: (VALUES (GET-VALUE (THIRD KEYS) INDEXED) (GET-PRIMARY-KEY 3  
> INDEX1) (GET-PRIMARY-KEY 300 INDEX2))
> Expected values: NIL
>                   NIL
>                   NIL
> Actual value: #<UNBOUND-VARIABLE #x8555CA6>.
> Test MAP-INDEXED failed
> Form: (LET ((KS NIL) (VS NIL)) (FLET ((MAPPER (K V) (PUSH K KS) (PUSH  
> V VS))) (MAP-BTREE #'MAPPER INDEXED)) (VALUES (AND (SUBSETP KS (CDDDR  
> KEYS) :TEST #'EQUALP) (SUBSETP (CDDDR KEYS) KS :TEST #'EQUALP))))
> Expected value: T
> Actual value: NIL.
> Test GET-FIRST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST  
> C) (DECLARE (IGNORE HAS V)) (= K 4))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85728C6>.
> Test GET-FIRST2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-FIRST  
> C) (DECLARE (IGNORE HAS V)) (= K 400))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x8582076>.
> Test GET-LAST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST  
> C) (DECLARE (IGNORE HAS V)) (= K 1000))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x8591826>.
> Test GET-LAST2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-LAST  
> C) (DECLARE (IGNORE HAS V)) (= K 100000))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85A0D76>.
> Test SET failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C  
> 200) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85CFE4E>.
> Test SET2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET C  
> 500) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 500))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85DF0BE>.
> Test SET-RANGE failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX1) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET- 
> RANGE C 199.5) (DECLARE (IGNORE HAS K)) (= (SLOT1 V) 200))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85EE326>.
> Test SET-RANGE2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (C INDEX2) (MULTIPLE-VALUE-BIND (HAS K V) (CURSOR-SET- 
> RANGE C 501) (DECLARE (IGNORE HAS K)) (= (SLOT2 V) 600))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85FD54E>.
> Test REM-KV failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET  
> ((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*))) (LOOP FOR I FROM 0  
> TO 10 DO (SETF (GET-VALUE I IBT) (* I I))) (REMOVE-KV 0 IBT) (REMOVE- 
> KV 1 IBT) (REMOVE-KV 10 IBT) (EQUAL (LIST (GET-VALUE 0 IBT) (GET- 
> VALUE 1 IBT) (GET-VALUE 10 IBT) (GET-VALUE 5 IBT)) '(NIL NIL NIL 25))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x8609BAE>.
> Test REM-IDEXKV failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
> ((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
> IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
> (SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
> FIRST C) (DOTIMES (I 10) (MULTIPLE-VALUE-BIND (HAS KEY VALUE) (CURSOR- 
> NEXT C)))) (REMOVE-KV 4 IBT) (REMOVE-KV 5 IBT) (EQUAL (LIST (GET- 
> VALUE 4 IBT) (GET-VALUE 5 IBT) (GET-VALUE 6 IBT) (WITH-BTREE-CURSOR  
> (C IBT) (CURSOR-FIRST C) (DOTIMES (I 4) (MULTIPLE-VALUE-BIND (HAS KEY  
> VALUE) (CURSOR-NEXT C) VALUE)) (MULTIPLE-VALUE-BIND (HAS KEY VALUE)  
> (CURSOR-NEXT C) VALUE))) '(NIL NIL 36 49))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x862FAB6>.
> MAKE-INDEXED2
> Test ADD-INDICES2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (SETQ INDEX3 (ADD-INDEX INDEXED2 :INDEX-NAME  
> 'CRUNCH :KEY-FORM 'CRUNCH))))
> Expected value: T
> Actual value: NIL.
> Test PUT-INDEXED2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (LOOP FOR I FROM 0 TO 10000 DO (SETF (GET-VALUE I  
> INDEXED2) (- I)))))
> Expected value: T
> Actual value: NIL.
> Test GET-INDEXED2 failed
> Form: (LOOP FOR I FROM 0 TO 10000 ALWAYS (= (- I) (GET-VALUE I  
> INDEXED2)))
> Expected value: T
> Actual value: #<TYPE-ERROR #x856D096>.
> Test GET-FROM-INDEX3 failed
> Form: (LOOP FOR I FROM 0 TO 1000 ALWAYS (= (* I -10) (GET-VALUE I  
> INDEX3)))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x856A34E>.
> Test DUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (MORE K V) = (MULTIPLE-VALUE- 
> LIST (CURSOR-FIRST CURS)) THEN (MULTIPLE-VALUE-LIST (CURSOR-NEXT-DUP  
> CURS)) WHILE MORE COLLECT V)))
> Expected value: (0 -1 -2 -3 -4 -5 -6 -7 -8 -9)
> Actual value: #<UNBOUND-VARIABLE #x859571E>.
> Test NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V) = (MULTIPLE-VALUE-LIST  
> (CURSOR-NEXT-NODUP CURS)) FOR I FROM 0 DOWNTO -9990 BY 10 WHILE M  
> ALWAYS (= V I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85A16AE>.
> Test PREV-NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V) =  
> (MULTIPLE-VALUE-LIST (CURSOR-PREV-NODUP CURS)) FOR I FROM -9999 TO -9  
> BY 10 WHILE M ALWAYS (= V I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85CD1AE>.
> Test PNODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (LOOP FOR (M K V P) = (MULTIPLE-VALUE-LIST  
> (CURSOR-PNEXT-NODUP CURS)) FOR I FROM 0 TO 9990 BY 10 WHILE M ALWAYS  
> (= P I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85D86DE>.
> Test PPREV-NODUP-TEST failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (WITH- 
> BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS) (LOOP FOR (M K V P) =  
> (MULTIPLE-VALUE-LIST (CURSOR-PPREV-NODUP CURS)) FOR I FROM 9999  
> DOWNTO 9 BY 10 WHILE M ALWAYS (= P I))))
> Expected value: T
> Actual value: #<UNBOUND-VARIABLE #x85E3BE6>.
> Test CUR-DEL1 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
> ((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
> IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
> (SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
> LAST C) (CURSOR-DELETE C)) (EQUAL (LIST (GET-VALUE 4 IBT) (GET-VALUE  
> 5 IBT) (GET-VALUE 9 IBT) (GET-VALUE 10 IBT)) '(16 25 NIL 100))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x860F77E>.
> Test INDEXED-DELETE failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-LAST CURS)  
> (CURSOR-DELETE CURS))))
> Expected value: T
> Actual value: NIL.
> Test TEST-DELETED failed
> Form: (VALUES (GET-VALUE 10000 INDEXED2) (GET-VALUE 1000 INDEX3))
> Expected values: NIL
>                   NIL
> Actual value: #<UNBOUND-VARIABLE #x8619B2E>.
> Test INDEXED-DELETE2 failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (WITH-BTREE-CURSOR (CURS INDEX3) (CURSOR-FIRST CURS)  
> (CURSOR-NEXT-DUP CURS) (CURSOR-DELETE CURS))))
> Expected value: T
> Actual value: NIL.
> Test TEST-DELETED2 failed
> Form: (VALUES (GET-VALUE 0 INDEXED2) (GET-VALUE 0 INDEX3) (GET-VALUE  
> 1 INDEXED2) (WITH-BTREE-CURSOR (C INDEX3) (CURSOR-FIRST C) (MULTIPLE- 
> VALUE-BIND (M K V) (CURSOR-NEXT C) V)))
> Expected values: 0
>                   0
>                   NIL
>                   -2
> Actual value: #<UNBOUND-VARIABLE #x86245D6>.
> Test CUR-DEL2 failed
> Form: (WITH-TRANSACTION (:STORE-CONTROLLER *STORE-CONTROLLER*) (LET*  
> ((IBT (BUILD-INDEXED-BTREE *STORE-CONTROLLER*)) (ID1 (ADD-INDEX  
> IBT :INDEX-NAME 'IDX1 :KEY-FORM 'ODD))) (LOOP FOR I FROM 0 TO 10 DO  
> (SETF (GET-VALUE I IBT) (* I I))) (WITH-BTREE-CURSOR (C ID1) (CURSOR- 
> FIRST C) (CURSOR-NEXT-DUP C) (CURSOR-DELETE C)) (EQUAL (LIST (GET- 
> VALUE 1 ID1) (GET-VALUE 0 ID1)) '(1 0))))
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x863093E>.
> Test GET-BOTH failed
> Form: (WITH-BTREE-CURSOR (C INDEXED2) (CURSOR-GET-BOTH C 200 -200))
> Expected values: T
>                   200
>                   -200
> Actual value: NIL.
> Test PGET-BOTH failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P)  
> (CURSOR-PGET-BOTH C 10 107) (VALUES K V P)))
> Expected values: 10
>                   -107
>                   107
> Actual value: #<UNBOUND-VARIABLE #x864D7B6>.
> Test PGET-BOTH-RANGE failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (MULTIPLE-VALUE-BIND (M K V P)  
> (CURSOR-PGET-BOTH-RANGE C 10 106.5) (VALUES K V P)))
> Expected values: 10
>                   -107
>                   107
> Actual value: #<UNBOUND-VARIABLE #x864B9FE>.
> Test PCURSOR failed
> Form: (WITH-BTREE-CURSOR (C INDEX3) (VALUES (PCURSOR-PKEY (CURSOR- 
> PFIRST C)) (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR- 
> PNEXT-NODUP C)) (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY  
> (CURSOR-PPREV C)) (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY  
> (CURSOR-PLAST C)) (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY  
> (CURSOR-PSET-RANGE C 199.5)) (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10  
> 101)) (PCURSOR-PKEY (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
> Expected values: 0
>                   2
>                   10
>                   11
>                   10
>                   9
>                   9999
>                   3000
>                   2000
>                   101
>                   112
> Actual value: #<UNBOUND-VARIABLE #x8643776>.
> Test NEWINDEX failed
> Form: (FINISHES (WITH-TRANSACTION (:STORE-CONTROLLER *STORE- 
> CONTROLLER*) (SETQ INDEX4 (ADD-INDEX INDEXED2 :INDEX-NAME  
> 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T))))
> Expected value: T
> Actual value: NIL.
> Test PCURSOR2 failed
> Form: (WITH-BTREE-CURSOR (C INDEX4) (VALUES (PCURSOR-PKEY (CURSOR- 
> PFIRST C)) (PCURSOR-PKEY (CURSOR-PNEXT C)) (PCURSOR-PKEY (CURSOR- 
> PNEXT-NODUP C)) (PCURSOR-PKEY (CURSOR-PNEXT-DUP C)) (PCURSOR-PKEY  
> (CURSOR-PPREV C)) (PCURSOR-PKEY (CURSOR-PPREV-NODUP C)) (PCURSOR-PKEY  
> (CURSOR-PLAST C)) (PCURSOR-PKEY (CURSOR-PSET C 300)) (PCURSOR-PKEY  
> (CURSOR-PSET-RANGE C 199.5)) (PCURSOR-PKEY (CURSOR-PGET-BOTH C 10  
> 101)) (PCURSOR-PKEY (CURSOR-PGET-BOTH-RANGE C 11 111.4))))
> Expected values: 0
>                   2
>                   10
>                   11
>                   10
>                   9
>                   9999
>                   3000
>                   2000
>                   101
>                   112
> Actual value: #<UNBOUND-VARIABLE #x8666A06>.
> ADD-GET-REMOVE ADD-GET-REMOVE-SYMBOL EXISTSP PREPARES-SLEEPYCAT TEST- 
> SEQ1
> TEST-SEQ2 CLEANSUP-SLEEPYCAT REMOVE-ELEMENT
> Test MIGRATE1 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
> COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
> STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
> (ADD-TO-ROOT "x" "y" :STORE-CONTROLLER SC1) (COPY-FROM-KEY "x" SC1  
> SC2) (SETF RV (EQUAL (GET-FROM-ROOT "x" :STORE-CONTROLLER SC1) (GET- 
> FROM-ROOT "x" :STORE-CONTROLLER SC2)))) (PROGN (SETQ *STORE- 
> CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x85779EE>.
> Test MIGRATE2 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
> COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
> STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
> (LET ((IBT (BUILD-BTREE SC1))) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET- 
> VALUE I IBT) (* I I))) (LET ((MIG (MIGRATE SC2 IBT))) (BTREE-DIFFER  
> IBT MIG)))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO- 
> COMMIT* *PREV-COMMIT*))))
> Expected value: NIL
> Actual value: #<SIMPLE-ERROR #x85ACC46>.
> Test MIGRATE3 failed
> Form: (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT* *AUTO- 
> COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET ((SC1 (OPEN- 
> STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH-SECONDARY*)))  
> (LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX (ADD-INDEX  
> IBT :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T))) (LOOP FOR I  
> FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I I))) (LET* ((MIG  
> (MIGRATE SC2 IBT)) (NINDEX (GETHASH 'CRUNCH (INDICES IBT)))) (LOOP  
> FOR I FROM 0 TO 10 DO (IF (NOT (EQUAL (GET-VALUE I INDEX) (GET-VALUE  
> I NINDEX))) (PROGN (FORMAT T "YIKES ~A ~%" I)))) (SETF RV (NOT (BTREE- 
> DIFFER IBT MIG))))))) (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE)  
> (SETQ *AUTO-COMMIT* *PREV-COMMIT*))) RV)
> Expected value: T
> Actual value: #<SIMPLE-ERROR #x85A511E>.
> Test MIGRATE4 failed
> Form: (FINISHES (LET ((OLD-STORE *STORE-CONTROLLER*) (*PREV-COMMIT*  
> *AUTO-COMMIT*) (*AUTO-COMMIT* T) (RV NIL)) (UNWIND-PROTECT (LET*  
> ((SC1 (OPEN-STORE *TEST-PATH-PRIMARY*)) (SC2 (OPEN-STORE *TEST-PATH- 
> SECONDARY*))) (LET* ((IBT (BUILD-INDEXED-BTREE SC1))) (LET ((INDEX  
> (ADD-INDEX IBT :INDEX-NAME 'CRUNCH :KEY-FORM 'CRUNCH :POPULATE T)) (X  
> 0)) (LOOP FOR I FROM 0 TO 10 DO (SETF (GET-VALUE I IBT) (* I I))))))  
> (PROGN (SETQ *STORE-CONTROLLER* OLD-STORE) (SETQ *AUTO-COMMIT* *PREV- 
> COMMIT*)))))
> Expected value: T
> Actual value: NIL.
> MIGRATE5
> 52 out of 106 total tests failed: UPDATE-CLASS, CHANGE-CLASS, CHANGE- 
> CLASS2, CHANGE-CLASS3, ADD-INDICES,
>     TEST-INDICES, INDEXED-PUT, INDEXED-GET, SIMPLE-SLOT-GET,
>     INDEXED-GET-FROM-SLOT1, INDEXED-GET-FROM-SLOT2, NO-KEY-NOR-INDICES,
>     REMOVE-KV-FROM-SLOT1, NO-KEY-NOR-INDICES-SLOT1, REMOVE-KV-FROM- 
> SLOT2,
>     NO-KEY-NOR-INDICES-SLOT2, MAP-INDEXED, GET-FIRST, GET-FIRST2, GET- 
> LAST,
>     GET-LAST2, SET, SET2, SET-RANGE, SET-RANGE2, REM-KV, REM-IDEXKV,
>     ADD-INDICES2, PUT-INDEXED2, GET-INDEXED2, GET-FROM-INDEX3, DUP-TEST,
>     NODUP-TEST, PREV-NODUP-TEST, PNODUP-TEST, PPREV-NODUP-TEST, CUR- 
> DEL1,
>     INDEXED-DELETE, TEST-DELETED, INDEXED-DELETE2, TEST-DELETED2, CUR- 
> DEL2,
>     GET-BOTH, PGET-BOTH, PGET-BOTH-RANGE, PCURSOR, NEWINDEX,  
> PCURSOR2, MIGRATE1,
>     MIGRATE2, MIGRATE3, MIGRATE4.
> (DO-ALL-TESTS-SPEC *TEST-PATH-PRIMARY*) took 8,681 milliseconds  
> (8.681 seconds) to run.
> Of that, 6,746 milliseconds (6.746 seconds) were spent in user mode
>           706 milliseconds (0.706 seconds) were spent in system mode
>           1,229 milliseconds (1.229 seconds) were spent executing  
> other OS processes.
> 738 milliseconds (0.738 seconds) was spent in GC.
> 372,045,168 bytes of memory allocated.
> NIL
> 
> - Waldo
> 
> On Jan 10, 2006, at 5:06 PM, Robert L. Read wrote:
> 
> > Dear Kilian,
> >
> >     Thanks for trying this on openmcl.  This code works for me  
> > under SBCL 9.5.  As we
> > all know, CLOS-related stuff like this tends to be more dissimilar  
> > between list implementations
> > than other parts of LISP.  Since this works under SBCL and has not  
> > been reported under
> > any other implementation, it might be an implementation-specific  
> > problem.  Unfortunately,
> > I don't have a PPC machine on which to attempt to duplicate this  
> > problem.
> >
> >     I will be happy to help you debug this, if you are interested  
> > (Waldo Rubinstein maybe
> > even more interested!)
> >     As a first step, we need to establish what the test regime does  
> > under your OpenMCL.
> > As you may know, Elephant comes with 106 tests (some of which  
> > involve data migration).
> > I wouldn't claim they are exhaustive but they are far better than  
> > nothing.  If you are
> > willing to take a few extra minutes, please execute this and send  
> > me the transcript:
> >
> > (asdf:operate 'asdf:load-op :elephant)
> > (asdf:operate 'asdf:load-op :ele-bdb)
> > (asdf:operate 'asdf:load-op :elephant-tests)
> > (in-package "ELEPHANT-TESTS")
> > (setq *test-path-primary* *testdb-path*)
> > (setq *test-path-secondary* nil)
> > (do-all-tests-spec *test-path-primary*)
> >
> > (on my computer, the actual test execution takes 31 seconds.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060111/446aa90f/attachment.html>

From waldo at trianet.net  Fri Jan 13 04:37:26 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Thu, 12 Jan 2006 23:37:26 -0500
Subject: [elephant-devel] secondary indices
In-Reply-To: <1136983155.9533.77.camel@localhost.localdomain>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
	<1136983155.9533.77.camel@localhost.localdomain>
Message-ID: <8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>

Robert,

I've been trying to understand the inners of indexed-btree. I have to  
admit, this exercise has helped me greatly in my quest for learning  
Lisp.

So far, I have been able to narrow down the problem to the transient  
slot INDICES-CACHE of the BDB-INDEXED-BTREE class.

Even though this slot has the same :initform as the non-transient  
version (e.g. make-hash-table), after you make an instance of indexed- 
btree, you can query (indices *instance*) and it returns the hash  
table object. However, when you (indices-cache *instance*) you get the:

Array index 4 out of bounds for #<SLOT-VECTOR  #x89BFD86> .
    [Condition of type SIMPLE-ERROR]

What I can't figure out yet is that in collections.lisp, you have:

(defmethod shared-initialize :after ((instance indexed-btree) slot-names
				     &rest rest)
   (declare (ignore slot-names rest))
   (setf (indices-cache instance) (indices instance)))

which, to my understanding, should set the value of the transient  
slot to point to the same hash-table that the INDICES slot points to.  
However, this doesn't seem to do anything. Even if you inspect  
(indices instance) before the setf, you get the hash-table, but for  
some reason, whatever (indices-cache instance) points to is not  
accepting the change.

I tried tracing it back into classes.lisp, into the:

(defmethod shared-initialize :around ((instance persistent-object)  
slot-names &rest initargs &key &allow-other-keys)
   "Initializes the persistent slots via initargs or forms.
This seems to be necessary because it is typical for
implementations to optimize setting the slots via initforms
and initargs in such a way that slot-value-using-class et al
aren't used.  Calls the next method for the transient slots."

In this method, it seems to initialize the persistent and non- 
persistent slots. At the end of the method, (indices instance) is  
properly pointing to a hash table. However, the (indices-cache  
instance) in that method is still returning the same error. Now, at  
the end of the defmethod, you have:

	  ;; let the implementation initialize the transient slots
	  (apply #'call-next-method instance transient-slot-inits  
initargs))))))

which is where I'm a bit at a loss. I would assume it's trying to  
call the next-method during the initialization process. However,  
transient-slot-inits is internally defined (flet) within this  
defmethod and it doesn't seem to be doing anything. Also, after  
inspecting all the values of the parameters passed to this defmethod,  
there is no reference to the INDICES-CACHE slot. If you inspect SLOT- 
NAMES in this method, it only has (INDICES). Could this be the reason?

I hope this may trigger something that could allow you or others to  
solve the problem or at least help me further in diagnosing it.

Anyway, I will continue hacking as much as I can.

Thanks,
Waldo

On Jan 11, 2006, at 7:39 AM, Robert L. Read wrote:

> OK, thank you.
>
> If we can get Elephant into a state that you don't have to be a  
> LISP hacker to use it,
> we will be greatly expanding its usefulness.
>
> I will try to work with Waldo Rubinstein to solve this problem for  
> openmcl.
>


From read at robertlread.net  Fri Jan 13 15:16:35 2006
From: read at robertlread.net (Robert L. Read)
Date: Fri, 13 Jan 2006 09:16:35 -0600
Subject: [elephant-devel] secondary indices
In-Reply-To: <8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
	<1136983155.9533.77.camel@localhost.localdomain>
	<8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>
Message-ID: <1137165395.13294.67.camel@localhost.localdomain>

Dear Waldo,
     Thanks for you excellent message.  This problem reaches
the border of my knowledge, but hopefully this answer will 
be somewhat helpful.


> Robert,
> 
> I've been trying to understand the inners of indexed-btree. I have
to  
> admit, this exercise has helped me greatly in my quest for learning  
> Lisp.

I also taught (and is teaching) me a great deal.
> 
> So far, I have been able to narrow down the problem to the transient  
> slot INDICES-CACHE of the BDB-INDEXED-BTREE class.
> 
Good.

> Even though this slot has the same :initform as the non-transient  
> version (e.g. make-hash-table), after you make an instance of
indexed- 
> btree, you can query (indices *instance*) and it returns the hash  
> table object. However, when you (indices-cache *instance*) you get
the:
> 
> Array index 4 out of bounds for #<SLOT-VECTOR  #x89BFD86> .
>     [Condition of type SIMPLE-ERROR]

This must be some kind of direct OpenMCL error.  I don't understand it;
but at least we can use it to debug this problem.

> 
> What I can't figure out yet is that in collections.lisp, you have:
> 
> (defmethod shared-initialize :after ((instance indexed-btree) slot-
names
>                                      &rest rest)
>    (declare (ignore slot-names rest))
>    (setf (indices-cache instance) (indices instance)))
> 
> which, to my understanding, should set the value of the transient  
> slot to point to the same hash-table that the INDICES slot points
to.  
> However, this doesn't seem to do anything. Even if you inspect  
> (indices instance) before the setf, you get the hash-table, but for  
> some reason, whatever (indices-cache instance) points to is not  
> accepting the change.

OK, I think you have found the heart of the problem.  

> 
> I tried tracing it back into classes.lisp, into the:
> 
> (defmethod shared-initialize :around ((instance persistent-object)  
> slot-names &rest initargs &key &allow-other-keys)
>    "Initializes the persistent slots via initargs or forms.
> This seems to be necessary because it is typical for
> implementations to optimize setting the slots via initforms
> and initargs in such a way that slot-value-using-class et al
> aren't used.  Calls the next method for the transient slots."
> 
> In this method, it seems to initialize the persistent and non- 
> persistent slots. At the end of the method, (indices instance) is  
> properly pointing to a hash table. However, the (indices-cache  
> instance) in that method is still returning the same error. Now, at  
> the end of the defmethod, you have:
> 
>           ;; let the implementation initialize the transient slots
>           (apply #'call-next-method instance transient-slot-inits  
> initargs))))))

I didn't write this routine (It's possible by that my extraction 
of the class bdb-indexed-btree is breaking it.

> 
> which is where I'm a bit at a loss. I would assume it's trying to  
> call the next-method during the initialization process. However,  
> transient-slot-inits is internally defined (flet) within this  
> defmethod and it doesn't seem to be doing anything. Also, after  

I think "transient-slot-inits" is not a function but a list of 
the slot-names that are transient that will need to be initialized.
However, you are correct that "initialize-from-initarg" is defined
by an flet here, but it does do something --- it tries to 
set certain slots.  Whether it is correct or not, I don't know.

> inspecting all the values of the parameters passed to this
defmethod,  
> there is no reference to the INDICES-CACHE slot. If you inspect SLOT- 
> NAMES in this method, it only has (INDICES). Could this be the reason?

I can think of two explanations for this:  Possibly it is bad to 
name the accessor for the slot named "indices-cache" the same as the 
slot itself (but I dobut it).  More likely at the time you are 
examinining it you are in the contect of indexed-btree or bdb-btree
and NOT the class bdb-indexed-btree.  (I'm not too sure of this...
it may be implementation dependent, and I'm not a MOP/CLOS expert.)


> 
> I hope this may trigger something that could allow you or others to  
> solve the problem or at least help me further in diagnosing it.

I think you have basically identified the approach I would use
to debugging this problem:  we know invoking indices-cache at
various times gives the above error, and we can't allow that.

So although it is quite complicated, we can potentially sprinkle
"print" statements into things like shared-initialize till we 
identified the precise point where indices-cache is legitimate
and then becomes illegitimate.  I think you have already started
this process---I suggest you just continue it.

I also suggest you organize your code into a specific piece
of test code, something like this:
(asdf:operate 'asdf:load-op :elephant)
(asdf:operate 'asdf:load-op :ele-bdb)
(asdf:operate 'asdf:load-op :elephant-tests)
(in-package "ELEPHANT-TESTS")
(open-store *testdb-path*)
(setq indexed (build-indexed-btree *store-controller*))
(elephant::indices-cache indexed)

For example, this code works for me under SBCL; if it fails
under OpenMCL on the last line, then what we need to 
do is to push something like (format t "IC = ~A~%" (indices-cache X))
into the shared-initialize and other routines until we understand 
it more deeply.

By producing a reproducible piece of text like this, we might 
not solve the problem:  Possibly there is a bug in OpenMCL, or 
more likely I have misunderstood CLOS and MOP in some way that
won't have a one-liner fix.  But if we can get it down to  clear,
reproducible piece of code, I suspect Andrew Blumberg would be
able to solve it from that point, or we could even ask for 
help from some OpenMCL forum.

Clearly, SBCL and OpenMCL are behaving differently; if we can
track down this difference very clearly, we have best hope of 
solving it.

Ross Overbeek once told me:

Momements of confusion are to be cherished, because they
precede moments of enlightment.

I apologize for not understanding this more deeply; like you,
I am still learning.

> 
> Anyway, I will continue hacking as much as I can.
> (asdf:operate 'asdf:load-op :elephant)

> Thanks,
> Waldo
> 
> On Jan 11, 2006, at 7:39 AM, Robert L. Read wrote:
> 
> > OK, thank you.
> >
> > If we can get Elephant into a state that you don't have to be a  
> > LISP hacker to use it,
> > we will be greatly expanding its usefulness.
> >
> > I will try to work with Waldo Rubinstein to solve this problem for  
> > openmcl.
> >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060113/8bb49c17/attachment.html>

From blumberg at math.uchicago.edu  Fri Jan 13 18:40:58 2006
From: blumberg at math.uchicago.edu (Andrew Blumberg)
Date: Fri, 13 Jan 2006 12:40:58 -0600 (CST)
Subject: [elephant-devel] secondary indices
In-Reply-To: <1137165395.13294.67.camel@localhost.localdomain>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
	<1136983155.9533.77.camel@localhost.localdomain>
	<8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>
	<1137165395.13294.67.camel@localhost.localdomain>
Message-ID: <Pine.LNX.4.61.0601131238580.5001@math.uchicago.edu>

hi robert and waldo,

 	before going too far in debugging the secondary indices part of 
the problem, i think it's worth trying to figure out the CLOS problem --- 
i suspect that the rest of the issue might be a symptom of that, and 
"fixing" the secondary indices stuff might just paper over the real 
problem.  if you're going to spend time instrumenting code, better to 
investigate the CLOS test failures first.

 								- andrew


From waldo at trianet.net  Fri Jan 13 19:10:50 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Fri, 13 Jan 2006 14:10:50 -0500
Subject: [elephant-devel] secondary indices
In-Reply-To: <Pine.LNX.4.61.0601131238580.5001@math.uchicago.edu>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
	<1136983155.9533.77.camel@localhost.localdomain>
	<8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>
	<1137165395.13294.67.camel@localhost.localdomain>
	<Pine.LNX.4.61.0601131238580.5001@math.uchicago.edu>
Message-ID: <667E973B-B795-4623-BE63-84FCA854392F@trianet.net>

Thanks Andrew. Robert, how do you want to proceed. Where should I  
look for potential CLOS problems?

- Waldo

On Jan 13, 2006, at 1:40 PM, Andrew Blumberg wrote:

> hi robert and waldo,
>
> 	before going too far in debugging the secondary indices part of  
> the problem, i think it's worth trying to figure out the CLOS  
> problem --- i suspect that the rest of the issue might be a symptom  
> of that, and "fixing" the secondary indices stuff might just paper  
> over the real problem.  if you're going to spend time instrumenting  
> code, better to investigate the CLOS test failures first.
>
> 								- andrew



From waldo at trianet.net  Fri Jan 13 19:15:30 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Fri, 13 Jan 2006 14:15:30 -0500
Subject: [elephant-devel] secondary indices
In-Reply-To: <1137165395.13294.67.camel@localhost.localdomain>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
	<1136983155.9533.77.camel@localhost.localdomain>
	<8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>
	<1137165395.13294.67.camel@localhost.localdomain>
Message-ID: <D29BFAF1-9AE6-44BD-8C23-DF07D5C13877@trianet.net>

Well, I renamed the slot to a different name leaving the accessor the  
same and still presents the same problem.

- Waldo

On Jan 13, 2006, at 10:16 AM, Robert L. Read wrote:

> I can think of two explanations for this:  Possibly it is bad to
> name the accessor for the slot named "indices-cache" the same as the
> slot itself (but I dobut it).  More likely at the time you are
> examinining it you are in the contect of indexed-btree or bdb-btree
> and NOT the class bdb-indexed-btree.  (I'm not too sure of this...
> it may be implementation dependent, and I'm not a MOP/CLOS expert.)



From read at robertlread.net  Fri Jan 13 19:32:25 2006
From: read at robertlread.net (Robert L. Read)
Date: Fri, 13 Jan 2006 13:32:25 -0600
Subject: [elephant-devel] secondary indices
In-Reply-To: <667E973B-B795-4623-BE63-84FCA854392F@trianet.net>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
	<1136983155.9533.77.camel@localhost.localdomain>
	<8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>
	<1137165395.13294.67.camel@localhost.localdomain>
	<Pine.LNX.4.61.0601131238580.5001@math.uchicago.edu>
	<667E973B-B795-4623-BE63-84FCA854392F@trianet.net>
Message-ID: <1137180746.13294.77.camel@localhost.localdomain>

It's not clear to me that it matters.

Going either direction will likely lead to a solution.  I would take
Andrew's advice, 
since his hunch may have vailidity, and also because it is the first
failure in the test suite.

But the course of action would be the same:

1)  Take the test that fails, copy the code out of it into a separate
file where the RT 
code won't obscure the problem.
2)  Instrument the code to find the earliest possible error; in this
case, that may be 
the failure of that instantiation code to let you get to that slot, or
maybe some other slot.
3)  Whenever you want, I will take your test code and run it under SBCL.

In this way we should be able to relatively quickly determine the
important difference
between SBCL and OpenMCL.  Depending on what you find, I will likely
need Andrew's
advice on how to fix it.



On Fri, 2006-01-13 at 14:10 -0500, Waldo Rubinstein wrote:

> Thanks Andrew. Robert, how do you want to proceed. Where should I  
> look for potential CLOS problems?
> 
> - Waldo
> 
> On Jan 13, 2006, at 1:40 PM, Andrew Blumberg wrote:
> 
> > hi robert and waldo,
> >
> > 	before going too far in debugging the secondary indices part of  
> > the problem, i think it's worth trying to figure out the CLOS  
> > problem --- i suspect that the rest of the issue might be a symptom  
> > of that, and "fixing" the secondary indices stuff might just paper  
> > over the real problem.  if you're going to spend time instrumenting  
> > code, better to investigate the CLOS test failures first.
> >
> > 								- andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060113/fd749cb6/attachment.html>

From waldo at trianet.net  Fri Jan 13 23:49:58 2006
From: waldo at trianet.net (Waldo Rubinstein)
Date: Fri, 13 Jan 2006 18:49:58 -0500
Subject: [elephant-devel] secondary indices
In-Reply-To: <1137180746.13294.77.camel@localhost.localdomain>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
	<1136983155.9533.77.camel@localhost.localdomain>
	<8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>
	<1137165395.13294.67.camel@localhost.localdomain>
	<Pine.LNX.4.61.0601131238580.5001@math.uchicago.edu>
	<667E973B-B795-4623-BE63-84FCA854392F@trianet.net>
	<1137180746.13294.77.camel@localhost.localdomain>
Message-ID: <B88DEFAC-9D2D-447F-8C58-30C364BF2F8C@trianet.net>

Robert,

I've tried to tackle this to no avail. Maybe my knowledge does not  
extend that far yet :(

Little more help would be needed. Maybe someone else's participation  
could help.

Thanks,
Waldo

On Jan 13, 2006, at 2:32 PM, Robert L. Read wrote:

> It's not clear to me that it matters.
>
> Going either direction will likely lead to a solution.  I would  
> take Andrew's advice,
> since his hunch may have vailidity, and also because it is the  
> first failure in the test suite.
>
> But the course of action would be the same:
>
> 1)  Take the test that fails, copy the code out of it into a  
> separate file where the RT
> code won't obscure the problem.
> 2)  Instrument the code to find the earliest possible error; in  
> this case, that may be
> the failure of that instantiation code to let you get to that slot,  
> or maybe some other slot.
> 3)  Whenever you want, I will take your test code and run it under  
> SBCL.
>
> In this way we should be able to relatively quickly determine the  
> important difference
> between SBCL and OpenMCL.  Depending on what you find, I will  
> likely need Andrew's
> advice on how to fix it.
>
>
>
> On Fri, 2006-01-13 at 14:10 -0500, Waldo Rubinstein wrote:
>> Thanks Andrew. Robert, how do you want to proceed. Where should I
>> look for potential CLOS problems?
>>
>> - Waldo
>>
>> On Jan 13, 2006, at 1:40 PM, Andrew Blumberg wrote:
>>
>> > hi robert and waldo,
>> >
>> > 	before going too far in debugging the secondary indices part of
>> > the problem, i think it's worth trying to figure out the CLOS
>> > problem --- i suspect that the rest of the issue might be a symptom
>> > of that, and "fixing" the secondary indices stuff might just paper
>> > over the real problem.  if you're going to spend time instrumenting
>> > code, better to investigate the CLOS test failures first.
>> >
>> > 								- andrew

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060113/65f4bf18/attachment.html>

From read at robertlread.net  Fri Jan 13 23:57:41 2006
From: read at robertlread.net (Robert L. Read)
Date: Fri, 13 Jan 2006 17:57:41 -0600
Subject: [elephant-devel] secondary indices
In-Reply-To: <B88DEFAC-9D2D-447F-8C58-30C364BF2F8C@trianet.net>
References: <14E1B8D1-B853-4097-BD47-E122CE7BB7D2@onlinehome.de>
	<1136930803.9533.19.camel@localhost.localdomain>
	<1BBAA284-3547-43F3-99B0-A4F0B3A1D84B@trianet.net>
	<996C9366-3F05-46B4-96B8-18DB8595C775@onlinehome.de>
	<1136983155.9533.77.camel@localhost.localdomain>
	<8E55EBCD-BDFB-4FEF-A7B0-96388821CDC6@trianet.net>
	<1137165395.13294.67.camel@localhost.localdomain>
	<Pine.LNX.4.61.0601131238580.5001@math.uchicago.edu>
	<667E973B-B795-4623-BE63-84FCA854392F@trianet.net>
	<1137180746.13294.77.camel@localhost.localdomain>
	<B88DEFAC-9D2D-447F-8C58-30C364BF2F8C@trianet.net>
Message-ID: <1137196662.13294.89.camel@localhost.localdomain>

OK; I will review your emails and try to find a specific piece of code
that we can run.

I maybe able to instrument the code sufficiently for you to run it and
get something valuable from it.





On Fri, 2006-01-13 at 18:49 -0500, Waldo Rubinstein wrote:

> Robert,
> 
> 
> I've tried to tackle this to no avail. Maybe my knowledge does not
> extend that far yet :(
> 
> 
> Little more help would be needed. Maybe someone else's participation
> could help.
> 
> 
> Thanks,
> Waldo
> 
> 
> On Jan 13, 2006, at 2:32 PM, Robert L. Read wrote:
> 
> 
> > It's not clear to me that it matters.
> > 
> > Going either direction will likely lead to a solution.  I would take
> > Andrew's advice, 
> > since his hunch may have vailidity, and also because it is the first
> > failure in the test suite.
> > 
> > But the course of action would be the same:
> > 
> > 1)  Take the test that fails, copy the code out of it into a
> > separate file where the RT 
> > code won't obscure the problem.
> > 2)  Instrument the code to find the earliest possible error; in this
> > case, that may be 
> > the failure of that instantiation code to let you get to that slot,
> > or maybe some other slot.
> > 3)  Whenever you want, I will take your test code and run it under
> > SBCL.
> > 
> > In this way we should be able to relatively quickly determine the
> > important difference
> > between SBCL and OpenMCL.  Depending on what you find, I will likely
> > need Andrew's
> > advice on how to fix it.
> > 
> > 
> > 
> > On Fri, 2006-01-13 at 14:10 -0500, Waldo Rubinstein wrote: 
> > 
> > > Thanks Andrew. Robert, how do you want to proceed. Where should I  
> > > look for potential CLOS problems?
> > > 
> > > - Waldo
> > > 
> > > On Jan 13, 2006, at 1:40 PM, Andrew Blumberg wrote:
> > > 
> > > > hi robert and waldo,
> > > >
> > > > 	before going too far in debugging the secondary indices part of  
> > > > the problem, i think it's worth trying to figure out the CLOS  
> > > > problem --- i suspect that the rest of the issue might be a symptom  
> > > > of that, and "fixing" the secondary indices stuff might just paper  
> > > > over the real problem.  if you're going to spend time instrumenting  
> > > > code, better to investigate the CLOS test failures first.
> > > >
> > > > 								- andrew
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060113/89043924/attachment.html>

From read at robertlread.net  Sat Jan 14 16:22:10 2006
From: read at robertlread.net (Robert L. Read)
Date: Sat, 14 Jan 2006 10:22:10 -0600
Subject: [elephant-devel] Re: more allegro edits to avoid compiler warnings
In-Reply-To: <200601102315.k0ANFuUm029921@blombos.isi.edu>
References: <200601102315.k0ANFuUm029921@blombos.isi.edu>
Message-ID: <1137255730.13294.103.camel@localhost.localdomain>

Thank you, Andrew.

This seems very relevant to a problem that Waldo Rubinstein is trying to
solve under OpenMCL.

The following code breaks on OpenMCL at the first accessor try:

(asdf:operate 'asdf:load-op :elephant)
(asdf:operate 'asdf:load-op :ele-bdb)
(asdf:operate 'asdf:load-op :elephant-tests)
(in-package "ELEPHANT-TESTS")

(open-store *testdb-path*)



      (defclass update-class () 
((slot1 :initform 1 :accessor slot1))
(:metaclass persistent-metaclass))
(setq *auto-commit* t)
(setq foo (make-instance 'update-class :sc *store-controller*))
(slot1 foo)
(defclass update-class ()
  ((slot2 :initform 2 :accessor slot2))
  (:metaclass persistent-metaclass))
(slot2 foo)
(slot1 foo)


Waldo, execute the above code under OpenMCL.
It should fail on the last statement, but I suspect it will fail on
(slot2 foo).
Send me the transcript.

I wonder if what that does under ACL?  I will attempt to run the code
you sent me today under SBCL.

It is clear that there are some implementation-specific differences that
are tripping us up on these 
features (which, after all, are very advanced; one cannot contemplate
dynamically changing a class
definition under Java!)  

On Tue, 2006-01-10 at 15:15 -0800, Andrew Philpot wrote:

> I wrote:
> 
>   Finally, I think things are working enough to prototype making my
>   knowledge base persistent.  Yet, I still get lots of segmentation
>   faults and seeming configuration level errors such as "The slot SLOT1
>   is missing from the object #<PFOO @ #x720bb22a> of class
>   #<PERSISTENT-METACLASS PFOO> during operation SETF."  I'm mostly
>   unclear as to why code which works inside a RT:DEFTEST doesn't always
>   work at top level, or in my client code.  I seem to do a lot of
>   deleting the database, stopping lisp, rebuilding elephant, and trying
>   my example again.  I fear I'm corrupting either the BDB store, the
>   local Lisp connection to it, and/or CLOS with my missteps.  Symptoms
>   are things like [sic]
> 
> I left out the symptoms before.  Sorry.
> 
> I've since gone further, and so I have a sample case that I would like
> to show you.  I would be interested in whether this happens on other
> platforms as well.  Again, if it comes down a question of why doesn't
> YYY work on ACL as it does on SBCL et al, I can submit to Franz.
> 
> Essentially, I have found that reDEFCLASSing persistent classes tends
> to stir things up in a way that slots are no longer visible after the
> redefinition.  Mostly by accident, I also found that explicitly
> calling CLOS::FINALIZE-INHERITANCE after the redefinition and before
> any use of the class usually does the right thing.
> 
> See the attached code and transcript, abstracted from my real
> application.  If you set the variable user::*EAGERLY-FINALIZE* to
> non-NIL, then at COMPILE/LOAD/EVAL it will try to finalize the
> definition (if such already exists).  I'm trying to be
> hyperconservative about when to define things, so that's why it does
> OPEN-STORE/CLOSE-STORE so globally before tinkering with the
> definitions.
> 
> #Andrew
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060114/a3ecd47e/attachment.html>

From philpot at ISI.EDU  Sat Jan 14 20:09:35 2006
From: philpot at ISI.EDU (Andrew Philpot)
Date: Sat, 14 Jan 2006 12:09:35 -0800
Subject: [elephant-devel] one experiential data point
Message-ID: <200601142009.k0EK9ZiZ008127@blombos.isi.edu>


Richard asked if I was still using Elephant.  I thought I would send a
quick note describing my experience using Elephant to persistently
store a lisp-based ontology with multi-lingual lexical attachments.

The "native" lisp implementation is pretty simple.  Essentially, it is
a semantic network implemented in Lisp using symbols for nodes and
using property lists for (binary) relations.  One additional wrinkle
is that a given string might name more than one ontological element,
most typically a concept and a possibly related, possibly unrelated
English lexical item ("word").  Accordingly, symbols are interned in
different CL packages according to their linguistic/ontological role
(concept, lexical item, sense, and a few others); all packages
containing a given fragment of the ontology are named per a convention
allowing modular mixing and matching.

A small schematic example:

FRAG1-CONCEPT::|bank<side|
  :DEFINITION "side of a road or river"
  :HAS-SENSE (FRAG1-SENSE::|bank<side:EN:bank| FRAG1-SENSE::|bank<side:ES:banco
|)
  :SUPER FRAG1-CONCEPT::|side<part|
  :SUB FRAG1-INSTANCE::|Left Bank|
  :SOURCE :WORDNET
  :PART-OF-SPEECH :NOUN
  :HAS-SUBJECT-DOMAIN DOM-CONCEPT::|engineering|

FRAG1-SENSE::|bank<side:EN:bank|
  :HAS-CONCEPT FRAG1-CONCEPT:|bank<side|
  :HAS-LEXICAL-ITEM FRAG1-EN::|bank|
  :SOURCE :WORDNET
  :DERIVATION FRAG1-SENSE::|banked<modified::EN:banked|

FRAG1-EN::|bank|
  :HAS-SENSE FRAG1-SENSE::|bank<side:EN:bank|
  :LANGUAGE :EN
  :SOURCE :WORDNET

etc. etc.  All in all there are roughly 600-700K of these named
objects in my core ontology base, with millions more instances linked
in below as well as other kinds of ancillary attachments.

The above model using symbols for objects and and property lists for
relations is not very fancy nor "modern", but it's very convenient and
relatively lightweight for an in-core implementation.  However,
interned symbols don't get GC'd even when no one cares about them
anymore, so the Lisp address size limits what I can do.  Hence
persistence a la Elephant.

My current model for implementing this in Elephant:

A. objects are 3-lists which serve as keys.  FRAG1-EN::|bank| becomes
(:frag1 :en "bank").  Algebraic in the sense that I can easily
construct them a view them as a composition of smaller bits; nothing
that can't be GC'd if needed (and the cardinality of namespaces and
language keywords are both expected to be small).

B. Relation network are hash-tables (EQL) keyed by the relation name.

So the first one above is 

(add-from-root '(:frag1 :concept "bank<side")
   (let ((ht (make-hash-table :test #'eql)))
     (setf (gethash :DEFINITION ht) "side of a road or river")
     (setf (gethash :HAS-SENSE ht) 
       (list '(:FRAG1 :SENSE "bank<side:EN:bank")
             '(:FRAG1 :SENSE "bank<side:ES:banco")))
     (setf (gethash :SUPER ht) '(:FRAG1 :CONCEPT "side<part"))
     (setf (gethash :SUB ht) '(:FRAG1 :INSTANCE "Left Bank"))
     (setf (gethash :SOURCE ht) :WORDNET)
     (setf (gethash :PART-OF-SPEECH ht) :NOUN)
     (setf (gethash :HAS-SUBJECT-DOMAIN ht) '(:DOM :CONCEPT "engineering"))))

I'm reasonably happy with this implementation, but it does have some
shortcomings:

1. Much slower than I expected.  To load the above 600K entities into
   Lisp takes about 10 minutes; to load into elephant (presumably a
   one-time operation) takes about a day!  Maybe transactions or wiser
   use of cursors or store controllers or what have you is called for,
   but I didn't expect this.

2. Takes a lot of space.  Roughly 350 MB of Lisp source files turn
   into 3.6 GB of disk space.  Again, maybe I'm doing something
   terribly wrong.

I don't have data at this point about access times.  I hope they don't
reflect the load time.

I'm still going forward on this.  At this point, elephant is a long
shot contender with a home grown RDBMS serialization back-ended in
MySQL, which works but makes data updates laborious and installation a
chore as well.  To replicate the functionality of my other
implementations, I will need some auxiliary indexing, in particular so
I can ask for all objects whose names match a substring; and I will
need some mechanism to handle relation inverses -- I want to say
(:super A B) and be able to ask (:sub B A).

It seems that a CLOS-based model might address some of the issues I
just mentioned.  I am kind of worried that it would be even slower.
And of course, I've had some troubles with MOP.  I'm not ruling this
out.

Any suggestions anyone might have would be welcome.  When I have a
web-based demo, I will also circulate it here.

Thanks,
Andrew Philpot




From read at robertlread.net  Sat Jan 14 20:45:54 2006
From: read at robertlread.net (Robert L. Read)
Date: Sat, 14 Jan 2006 14:45:54 -0600
Subject: [elephant-devel] one experiential data point
In-Reply-To: <200601142009.k0EK9ZiZ008127@blombos.isi.edu>
References: <200601142009.k0EK9ZiZ008127@blombos.isi.edu>
Message-ID: <1137271555.13294.137.camel@localhost.localdomain>

Thanks, Andrew, this is very interesting.

I would mention with respect to speed:

1)  BerkeleyDB is much faster than the SQL implementations right now,
2)  My base64-based serialization for the SQL backend could obviously 
be improved tremendously.

Improving my serialization would could down on disk space and time in
the 
SQL-back-end version a great deal.  Even the BerkeleyDB serializer could
be
improved, although it is much better than what I am doing for the SQL-
back end.

I hope you will keep us informed as to your experience and your choices;
if 
you choose not to use Elephant, that will be interesting information as
well.

For the substring searching, are you planning to use a "like" query in
MySQL
in your homegrown implementation?   I doubt that Elephant can compete
with 
that; although it can produce functional indexes that technically have
that power,
it would be pretty sophisticated programming to do it.



On Sat, 2006-01-14 at 12:09 -0800, Andrew Philpot wrote:

> Richard asked if I was still using Elephant.  I thought I would send a
> quick note describing my experience using Elephant to persistently
> store a lisp-based ontology with multi-lingual lexical attachments.
> 
> The "native" lisp implementation is pretty simple.  Essentially, it is
> a semantic network implemented in Lisp using symbols for nodes and
> using property lists for (binary) relations.  One additional wrinkle
> is that a given string might name more than one ontological element,
> most typically a concept and a possibly related, possibly unrelated
> English lexical item ("word").  Accordingly, symbols are interned in
> different CL packages according to their linguistic/ontological role
> (concept, lexical item, sense, and a few others); all packages
> containing a given fragment of the ontology are named per a convention
> allowing modular mixing and matching.
> 
> A small schematic example:
> 
> FRAG1-CONCEPT::|bank<side|
>   :DEFINITION "side of a road or river"
>   :HAS-SENSE (FRAG1-SENSE::|bank<side:EN:bank| FRAG1-SENSE::|bank<side:ES:banco
> |)
>   :SUPER FRAG1-CONCEPT::|side<part|
>   :SUB FRAG1-INSTANCE::|Left Bank|
>   :SOURCE :WORDNET
>   :PART-OF-SPEECH :NOUN
>   :HAS-SUBJECT-DOMAIN DOM-CONCEPT::|engineering|
> 
> FRAG1-SENSE::|bank<side:EN:bank|
>   :HAS-CONCEPT FRAG1-CONCEPT:|bank<side|
>   :HAS-LEXICAL-ITEM FRAG1-EN::|bank|
>   :SOURCE :WORDNET
>   :DERIVATION FRAG1-SENSE::|banked<modified::EN:banked|
> 
> FRAG1-EN::|bank|
>   :HAS-SENSE FRAG1-SENSE::|bank<side:EN:bank|
>   :LANGUAGE :EN
>   :SOURCE :WORDNET
> 
> etc. etc.  All in all there are roughly 600-700K of these named
> objects in my core ontology base, with millions more instances linked
> in below as well as other kinds of ancillary attachments.
> 
> The above model using symbols for objects and and property lists for
> relations is not very fancy nor "modern", but it's very convenient and
> relatively lightweight for an in-core implementation.  However,
> interned symbols don't get GC'd even when no one cares about them
> anymore, so the Lisp address size limits what I can do.  Hence
> persistence a la Elephant.
> 
> My current model for implementing this in Elephant:
> 
> A. objects are 3-lists which serve as keys.  FRAG1-EN::|bank| becomes
> (:frag1 :en "bank").  Algebraic in the sense that I can easily
> construct them a view them as a composition of smaller bits; nothing
> that can't be GC'd if needed (and the cardinality of namespaces and
> language keywords are both expected to be small).
> 
> B. Relation network are hash-tables (EQL) keyed by the relation name.
> 
> So the first one above is 
> 
> (add-from-root '(:frag1 :concept "bank<side")
>    (let ((ht (make-hash-table :test #'eql)))
>      (setf (gethash :DEFINITION ht) "side of a road or river")
>      (setf (gethash :HAS-SENSE ht) 
>        (list '(:FRAG1 :SENSE "bank<side:EN:bank")
>              '(:FRAG1 :SENSE "bank<side:ES:banco")))
>      (setf (gethash :SUPER ht) '(:FRAG1 :CONCEPT "side<part"))
>      (setf (gethash :SUB ht) '(:FRAG1 :INSTANCE "Left Bank"))
>      (setf (gethash :SOURCE ht) :WORDNET)
>      (setf (gethash :PART-OF-SPEECH ht) :NOUN)
>      (setf (gethash :HAS-SUBJECT-DOMAIN ht) '(:DOM :CONCEPT "engineering"))))
> 
> I'm reasonably happy with this implementation, but it does have some
> shortcomings:
> 
> 1. Much slower than I expected.  To load the above 600K entities into
>    Lisp takes about 10 minutes; to load into elephant (presumably a
>    one-time operation) takes about a day!  Maybe transactions or wiser
>    use of cursors or store controllers or what have you is called for,
>    but I didn't expect this.
> 
> 2. Takes a lot of space.  Roughly 350 MB of Lisp source files turn
>    into 3.6 GB of disk space.  Again, maybe I'm doing something
>    terribly wrong.
> 
> I don't have data at this point about access times.  I hope they don't
> reflect the load time.
> 
> I'm still going forward on this.  At this point, elephant is a long
> shot contender with a home grown RDBMS serialization back-ended in
> MySQL, which works but makes data updates laborious and installation a
> chore as well.  To replicate the functionality of my other
> implementations, I will need some auxiliary indexing, in particular so
> I can ask for all objects whose names match a substring; and I will
> need some mechanism to handle relation inverses -- I want to say
> (:super A B) and be able to ask (:sub B A).
> 
> It seems that a CLOS-based model might address some of the issues I
> just mentioned.  I am kind of worried that it would be even slower.
> And of course, I've had some troubles with MOP.  I'm not ruling this
> out.
> 
> Any suggestions anyone might have would be welcome.  When I have a
> web-based demo, I will also circulate it here.
> 
> Thanks,
> Andrew Philpot
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060114/37d18320/attachment.html>

From blumberg at math.uchicago.edu  Wed Jan 18 03:26:56 2006
From: blumberg at math.uchicago.edu (Andrew Blumberg)
Date: Tue, 17 Jan 2006 21:26:56 -0600 (CST)
Subject: [elephant-devel] Re: more allegro edits to avoid compiler warnings
In-Reply-To: <1137255730.13294.103.camel@localhost.localdomain>
References: <200601102315.k0ANFuUm029921@blombos.isi.edu>
	<1137255730.13294.103.camel@localhost.localdomain>
Message-ID: <Pine.LNX.4.61.0601172124190.16493@math.uchicago.edu>

hi andrew and robert,

 	i'm going to start trying to debug the problems reported below. 
the class updating code was intended to force a finalization before any 
access occurred (by obsoleting the instances), but obviously it isn't 
working correctly.  i'll keep you posted on my efforts to correct this 
problem.  thanks for the detailed transcript.

 								- andrew

> On Tue, 2006-01-10 at 15:15 -0800, Andrew Philpot wrote:
>
>> I left out the symptoms before.  Sorry.
>>
>> I've since gone further, and so I have a sample case that I would like
>> to show you.  I would be interested in whether this happens on other
>> platforms as well.  Again, if it comes down a question of why doesn't
>> YYY work on ACL as it does on SBCL et al, I can submit to Franz.
>>
>> Essentially, I have found that reDEFCLASSing persistent classes tends
>> to stir things up in a way that slots are no longer visible after the
>> redefinition.  Mostly by accident, I also found that explicitly
>> calling CLOS::FINALIZE-INHERITANCE after the redefinition and before
>> any use of the class usually does the right thing.
>>
>> See the attached code and transcript, abstracted from my real
>> application.  If you set the variable user::*EAGERLY-FINALIZE* to
>> non-NIL, then at COMPILE/LOAD/EVAL it will try to finalize the
>> definition (if such already exists).  I'm trying to be
>> hyperconservative about when to define things, so that's why it does
>> OPEN-STORE/CLOSE-STORE so globally before tinkering with the
>> definitions.
>>
>> #Andrew
>>
>


From root at common-lisp.net  Wed Jan 18 16:23:29 2006
From: root at common-lisp.net (root)
Date: Wed, 18 Jan 2006 10:23:29 -0600 (CST)
Subject: [elephant-devel] Auto-nag: elephant link verifier failed
Message-ID: <20060118162329.D808D239C6@common-lisp.net>

You are being nagged on because your project's
name showed up in the nightly Checkbot run.  Checkbot checks
for broken links etc.  This probably means that you are 
either pointing to a broken link or that someone is pointing
to a broken link on your site.  Or something else, it check's
a bunch of stuff.  Update your webpages or you shall be nagged
again next week.

To find out what's wrong with your webpages, please consult:

  http://common-lisp.net/checkbot/checkbot-common-lisp.net.html

Any questions?  You can reach the author of this cronjob at
admin at common-lisp.net

Thanks!


From philpot at ISI.EDU  Wed Jan 18 18:22:43 2006
From: philpot at ISI.EDU (Andrew Philpot)
Date: Wed, 18 Jan 2006 10:22:43 -0800
Subject: [elephant-devel] standard routine or idiom to clear BDB from within
	Lisp?
Message-ID: <200601181822.k0IIMhFq008105@ubirr.isi.edu>


The following tends to hang.

(defun clear-from-root (&key (store-path *dbpath*)
                             (verbose t))
  (let ((remcount 0)
	(errcount 0))
    (with-open-store (store-path)
      (let ((btree (controller-root *store-controller*)))
	(with-btree-cursor (cursor btree)
	  (loop (multiple-value-bind (more k v)
		    (ignore-errors (cursor-next cursor))
		  (declare (ignorable v))
		  (if (typep k 'error)
		      (progn
			(incf errcount)
			(when verbose
			  (warn "Error was ~A" k)))
		    (progn
		      (unless more (return nil))
		      (when verbose
			(format *debug-io* "~%Dropped ~S -> ~S" k v))
		      (remove-kv k btree)
		      (incf remcount))))))))
    (values remcount
	    errcount)))


From read at robertlread.net  Wed Jan 18 13:32:50 2006
From: read at robertlread.net (Robert L. Read)
Date: Wed, 18 Jan 2006 07:32:50 -0600
Subject: [elephant-devel] Re: Common lisp transition issues?
In-Reply-To: <43CD5BA4.5030407@csail.mit.edu>
References: <43CD5BA4.5030407@csail.mit.edu>
Message-ID: <1137591171.4657.50.camel@localhost.localdomain>

Hi, Ian, and welcome.  It sounds like you will be able to make some nice
contributions.
I am the maintainer, so I am the main person to coordinate with.  I will
be out of email range
today, but we can discuss it more tomorrow.  The original authors are on
the "devel" list,
and we can re-open the licensing conversation with them.

There are some known PPC problems; Andrew Blumberg and others are
working on that,
and I'm sure they would welcome your help.

Can you give me the link about elephant being down?  I don't know what
that means
and would like to find it and repair it.  As far as I know, elephant has
not been down,
and I know of no reason why you shouldn't be able to sign up to the
list.

On Tue, 2006-01-17 at 16:03 -0500, Ian Eslick wrote:

> Aside from the common-lisp.net pointer to elephant being down (.org 
> works) I have been unable to sign up to the developers list and have 
> some patches to make the Allegro bdb version of elephant on Mac OS X 
> more robust (at least in my particular setup) that I would like to submit.
> 
> I'm planning to use elephant for a large data collection effort and 
> expect I'll need/desire to contribute to the system as I proceed in 
> using it for this application so wanted to find out who are the key 
> people to coordinate with for checking in code and get on the list so I 
> can stay up to date with development.
> 
> I'm using the recent 0.4.0 release with version 4.3 of sleepycat under 
> Allegro 7.0 on a PPC G4 PowerBook running Mac OS X 10.4.  The approach 
> to serialization of strings in 0.4.0 didn't work for me.  I suspect it's 
> a byte ordering problem in the unicode encoding.  On my PPC under 
> allegro the native-to-string failed to work (returned all '???' - a good 
> indication of byte order problems) with the default copy in serializing 
> 2-byte characters.  Also in my version of allegro strings are stored 
> internall as 1-bit entities but the :ics features is still set so :ics 
> does not categorically imply 2-byte unicode.  Not sure if this is a 
> Franz feature or a bug.  I also added a little bit the build process to 
> support .dylib builds under Mac OS X and some vendor variances in 
> *features* (Allegro doesn't provide the :darwin feature, only :macosx).  
> etc, etc.  I have little patches for all this.
> 
> My setup passes all tests on bdb except migration because I haven't 
> installed Postgres yet.  I'll do that at some point and perhaps check 
> out MySQL.
> 
> I also want to discuss the LLGPL license vs. the GPL license for systems 
> based on the code.  Now that we can backend to different SQL platforms 
> there it would be nice to have more flexibility on using the elephant 
> lisp code which LLGPL would provide.  I saw a discussion in November on 
> this but notice that the license hasn't been update.
> 
> Many thanks,
> Ian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060118/236c3bd1/attachment.html>

From midfield at gmail.com  Thu Jan 19 23:58:06 2006
From: midfield at gmail.com (Ben)
Date: Thu, 19 Jan 2006 17:58:06 -0600
Subject: [elephant-devel] standard routine or idiom to clear BDB from
	within Lisp?
In-Reply-To: <200601181822.k0IIMhFq008105@ubirr.isi.edu>
References: <200601181822.k0IIMhFq008105@ubirr.isi.edu>
Message-ID: <9157df230601191558o1bb64711nfae5dfe750d58e74@mail.gmail.com>

i would recommend using whatever backend functions there are for
dropping btrees (or for SQL, tables, presumably.)  i seem to remember
adding a hook for such a thing for the BDB back end, it's somewhere in
the sleepycat code.  should be relatively trivial to access it from
within elephant.

B

On 1/18/06, Andrew Philpot <philpot at isi.edu> wrote:
>
> The following tends to hang.
>
> (defun clear-from-root (&key (store-path *dbpath*)
>                              (verbose t))
>   (let ((remcount 0)
>         (errcount 0))
>     (with-open-store (store-path)
>       (let ((btree (controller-root *store-controller*)))
>         (with-btree-cursor (cursor btree)
>           (loop (multiple-value-bind (more k v)
>                     (ignore-errors (cursor-next cursor))
>                   (declare (ignorable v))
>                   (if (typep k 'error)
>                       (progn
>                         (incf errcount)
>                         (when verbose
>                           (warn "Error was ~A" k)))
>                     (progn
>                       (unless more (return nil))
>                       (when verbose
>                         (format *debug-io* "~%Dropped ~S -> ~S" k v))
>                       (remove-kv k btree)
>                       (incf remcount))))))))
>     (values remcount
>             errcount)))
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>


From root at common-lisp.net  Mon Jan 23 08:15:18 2006
From: root at common-lisp.net (root)
Date: Mon, 23 Jan 2006 02:15:18 -0600 (CST)
Subject: [elephant-devel] Auto-nag: elephant link verifier failed
Message-ID: <20060123081518.F29461E21E@common-lisp.net>

You are being nagged on because your project's
name showed up in the nightly Checkbot run.  Checkbot checks
for broken links etc.  This probably means that you are 
either pointing to a broken link or that someone is pointing
to a broken link on your site.  Or something else, it check's
a bunch of stuff.  Update your webpages or you shall be nagged
again next week.

To find out what's wrong with your webpages, please consult:

  http://common-lisp.net/checkbot/checkbot-common-lisp.net.html

Any questions?  You can reach the author of this cronjob at
clo-devel at common-lisp.net

Thanks!


From read at robertlread.net  Mon Jan 23 18:11:25 2006
From: read at robertlread.net (Robert L. Read)
Date: Mon, 23 Jan 2006 12:11:25 -0600
Subject: [elephant-devel] Elephant doesn't work with Berkeley DB 4.4 right
	now...
Message-ID: <1138039885.8907.195.camel@localhost.localdomain>


It works with 4.3.  I'm not sure when 4.4 came out (fairly recently,
however.)

Use elephant with 4.3.

Currently Andrew and Waldo are helping me with some bugs for OpenMCL;
I will probably create a new release once all the tests are completely
green there.

At present, Elephant partially works with OpenMCL but not all features
work
and its use with that implementation is discouraged until Andrew fixes
this
problem.

Many thanks to Andrew and Waldo for their continued effort.

----
Robert L. Read, PhD                                     read &T
robertlread.net
Consider visiting Progressive Engineering:
http://robertlread.net/pe
In Austin: 912-8593                                        "Think
globally, Act locally." -- RBF


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060123/5c1b23a1/attachment.html>

From philpot at ISI.EDU  Mon Jan 23 18:41:51 2006
From: philpot at ISI.EDU (Andrew Philpot)
Date: Mon, 23 Jan 2006 10:41:51 -0800
Subject: [elephant-devel] stack overflow during deserialization
Message-ID: <200601231841.k0NIfpBM003052@ubirr.isi.edu>


Hi, All.

Allegro 6.2 on Linux.  FWIW, the feature combination (and allegro ics)
returns true in this session.

After a very long (10+ hours) load of data into elephant (1.2 GB of
data loaded), I have encountered a stack overflow error in elephant in
(CL:LABELS ELE::DESERIALIZE ELE::%DESERIALIZE), i.e., the LABELS
function named ELE::%DESERIALIZE inside ELE::DESERIALIZE.  

The original call was to asking for (the relatively benign):

  (ELE:GET-FROM-ROOT '(:WNVFRM :CORE "Frame:08"))

whose return value should be a hash table mapping Lisp keywords to
lists, or may not yet exist.

I couldn't do much hacking here in the error handler, but one thing
that I did see was that when ELE::DESERIALIZE was run interpreted, I
did not get any stack overflow, but when compiled, I sometimes did.  I
notice that the definition has highly aggressive optimization
declarations.  I'm going to take them out, recompile and retry my data
load.

Any suggestions or insights would be appreciated.

Andrew P.



From eslick at csail.mit.edu  Tue Jan 24 04:09:47 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Mon, 23 Jan 2006 23:09:47 -0500
Subject: [elephant-devel] Add indexed persistent class slots to elephant?
Message-ID: <43D5A88B.2080604@csail.mit.edu>

While diving into the elephant code to understand it better I started to 
think about my normal usage model and that one common model is to lookup 
objects by slot value or a range of slot values.  This seems like a very 
common operation and that adding an initarg ':indexed' to the metaclass 
would allow for some simple default functionality:

low-level interface:
- define cursors over persistent-class slots as well as btrees and 
secondary indices
- make it easy to iterate over duplicate class+slot and class+slot+value 
keys
- we get an index of every persistent-object of a given class if we 
implement
   the right comparison operation.

mid-level interface:
- grab sets of objects based on slot-name and slot-value or range of 
slot values

high-level interface:
- a simple constraint language with boolean combinators that selects 
instances
  based on various combinations of slot ranges or values
- it becomes easier to compile constraints when the class contains 
information
  directly that tells you what indexes exist so you can do optimize the 
query ahead
  of time.

Supporting this requires adding an additional around method to (setf 
slot-value-using-class) on
persistent-slots to specialize on indexed slots and update the slot 
index and then potentially
adding an additional layer of cursor operators.  This is optional 
functionality that will only slow down write, not read, operations and 
will be backwards compatible.  It should be easy to add SQL support.  
The benefit will be to add some nice default behavior that makes the 
database aspect of the low-level interfaces much more directly 
accessible to new users. 

On my local copy I've implemented the metaclass support, overloading and 
a good chunk of the constraint language and still pass all of the 
current tests.  I think I understand the problem well enough now to 
query the user community for advice and buy-in.  I have yet to support 
all the unpleasant details related to changing classes, but the 
implications of dropping or adding an indexed slot is rather 
straightforward so I think that finishing the implementation and writing 
the appropriate tests isn't too much work.

The first question is whether the primary developers and users are open 
to the addition of this feature.

If so, the big design question I'm facing at present is:

1) Reuse the current btree infrastructure to create a btree for each 
class that maps oids to persistent-objects and instantiate a secondary 
index for each indexed slot using the slot accessor functions.  This is 
the easisest to implement, but might provide somewhat poor performance 
on create & writes.

2) Create another underlying DB with string keys 
"class-name+slot-name+value" => "oid"? 

2a) - Is it better to point to oid's or directly to serialized 
persistent-objects?  The nice thing about oid's is that later I can 
implement join-like operations in the query language using oids without 
having to deserialize and cache persistent objects.  Persistent-objects 
are perhaps more convenient for direct use, however.

Comments would be greatly appreciated.  I especially invite debate if 
others feel this is the wrong level of abstraction to work at (i.e. 
instead write a new def macro for indexed classes and a related protocol 
that accomplishes the same result by reusing primary and secondary 
btrees).  The proposal above seems in good taste to me and I've already 
invested some quality time in it, but since I'll be touching a fair bit 
of the system to put this in I want to make sure there is support.

Ian





From eslick at csail.mit.edu  Tue Jan 24 04:45:22 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Mon, 23 Jan 2006 23:45:22 -0500
Subject: [elephant-devel] License Discussion
Message-ID: <43D5B0E2.6050304@csail.mit.edu>

I am writing to see if I can restart the license discussion I saw a 
little of in the archives late last year.  With the SQL backends it 
seems more appropriate to release the next version of elephant as an 
LLGPL code base to encourage wider adoption by folks who work for or 
consult for other companies and can't use the existing elephant code 
with a SQL backend in commercial software.  It certainly makes it more 
motivating for me to contribute to the code base as my lab (sponsored by 
a bunch of companies) has expressed a preference for LLGPL in my own 
library projects.  I've had to strip all the GPL'ed outside lisp 
libraries but am holding for the time being to see if elephant will 
indeed reconsider its license.  :) 

Regardless it's a wonderful system already and the first shot in awhile 
for the lisp community to settle on a PODB.  It's probably also helpful 
that the code is maturing at the same time as AllegroCache is generating 
some broad interest.

Please consider this my request for the LLGPL.  
http://opensource.franz.com/preamble.html

Regards,
Ian


From read at robertlread.net  Tue Jan 24 05:23:37 2006
From: read at robertlread.net (Robert L. Read)
Date: Mon, 23 Jan 2006 23:23:37 -0600
Subject: [elephant-devel] License Discussion
In-Reply-To: <43D5B0E2.6050304@csail.mit.edu>
References: <43D5B0E2.6050304@csail.mit.edu>
Message-ID: <1138080217.8907.212.camel@localhost.localdomain>

For the code that I have written, I am completely in accordance with
this
suggestion to use the LLGPL.

However, Ben and Andrew and others are really the primary authors;
unless
we reach consensus with them, and perhaps others, I don't see how we 
can make the switch.

On Mon, 2006-01-23 at 23:45 -0500, Ian Eslick wrote:

> I am writing to see if I can restart the license discussion I saw a 
> little of in the archives late last year.  With the SQL backends it 
> seems more appropriate to release the next version of elephant as an 
> LLGPL code base to encourage wider adoption by folks who work for or 
> consult for other companies and can't use the existing elephant code 
> with a SQL backend in commercial software.  It certainly makes it more 
> motivating for me to contribute to the code base as my lab (sponsored by 
> a bunch of companies) has expressed a preference for LLGPL in my own 
> library projects.  I've had to strip all the GPL'ed outside lisp 
> libraries but am holding for the time being to see if elephant will 
> indeed reconsider its license.  :) 
> 
> Regardless it's a wonderful system already and the first shot in awhile 
> for the lisp community to settle on a PODB.  It's probably also helpful 
> that the code is maturing at the same time as AllegroCache is generating 
> some broad interest.
> 
> Please consider this my request for the LLGPL.  
> http://opensource.franz.com/preamble.html
> 
> Regards,
> Ian
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060123/fd56f8ac/attachment.html>

From read at robertlread.net  Tue Jan 24 15:15:01 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 24 Jan 2006 09:15:01 -0600
Subject: [elephant-devel] Add indexed persistent class slots to elephant?
In-Reply-To: <43D5A88B.2080604@csail.mit.edu>
References: <43D5A88B.2080604@csail.mit.edu>
Message-ID: <1138115702.8907.246.camel@localhost.localdomain>

Dear Ian,
    Wow.  It sounds like you are indeed deeply in to it.  I am the
maintainer, and not a primary 
author; in some ways I do not even consider myself qualified to answer.
I will happily accept
and your code, as long as the test are green and are you are expanding
the possibilities for
the users, rather than constraining them.  And documentation would be
nice.
    As to your technical questions, about the approaches to implementing
the indexing, in my
experience indexing is one of the most fluid aspects of an object space;
it is a tunable issue
that tends to change as usage pattern changes even more than the
underlying data structure.
So from my point of view I would much prefer macros or simple functions
that let you easily
add, and discard, indexes.  Of course, your observation that one does
typically index a slot
is quite correct, and it would be very convenient if such an index could
be added and dropped
very easily; if one could also implement boolean operators based on
those indexes, so much
the better.  As you state, one should be able to somehow introspect the
class and discover
the existing indexing structure; I'm not sure of the best way to do
that.
    Although I am a "Smug Lisp Weanie" and fairly well educated, I have
to admit that all
of this direct hacking of CLOS and MOP makes my head spin a little.
That is the stuff that
creates the most problems in trying to work with multiple platforms.  We
seem now to 
work with ACL and SBCL, and Andrew Blumberg is helping me solve and
OpenMCL problem
right now.  As a maintainer, I'm a little loathe to recommend even more
complexity in that part.
    However, I always believe interfaces are more important than
implementations.  The real
question is: are you conveniently (elegantly?) expanding the API to make
elephant more useful?
Whether this is done with a simple function or metaclass keywords might
actually be a side 
issue.  The code that will utilize the slot-based indexes is more
important than the code that
will create them.
    If I have a right to insist on anything, I will insist that the
indexes be easily droppable.

On Mon, 2006-01-23 at 23:09 -0500, Ian Eslick wrote:

> While diving into the elephant code to understand it better I started to 
> think about my normal usage model and that one common model is to lookup 
> objects by slot value or a range of slot values.  This seems like a very 
> common operation and that adding an initarg ':indexed' to the metaclass 
> would allow for some simple default functionality:
> 
> low-level interface:
> - define cursors over persistent-class slots as well as btrees and 
> secondary indices
> - make it easy to iterate over duplicate class+slot and class+slot+value 
> keys
> - we get an index of every persistent-object of a given class if we 
> implement
>    the right comparison operation.
> 
> mid-level interface:
> - grab sets of objects based on slot-name and slot-value or range of 
> slot values
> 
> high-level interface:
> - a simple constraint language with boolean combinators that selects 
> instances
>   based on various combinations of slot ranges or values
> - it becomes easier to compile constraints when the class contains 
> information
>   directly that tells you what indexes exist so you can do optimize the 
> query ahead
>   of time.
> 
> Supporting this requires adding an additional around method to (setf 
> slot-value-using-class) on
> persistent-slots to specialize on indexed slots and update the slot 
> index and then potentially
> adding an additional layer of cursor operators.  This is optional 
> functionality that will only slow down write, not read, operations and 
> will be backwards compatible.  It should be easy to add SQL support.  
> The benefit will be to add some nice default behavior that makes the 
> database aspect of the low-level interfaces much more directly 
> accessible to new users. 
> 
> On my local copy I've implemented the metaclass support, overloading and 
> a good chunk of the constraint language and still pass all of the 
> current tests.  I think I understand the problem well enough now to 
> query the user community for advice and buy-in.  I have yet to support 
> all the unpleasant details related to changing classes, but the 
> implications of dropping or adding an indexed slot is rather 
> straightforward so I think that finishing the implementation and writing 
> the appropriate tests isn't too much work.
> 
> The first question is whether the primary developers and users are open 
> to the addition of this feature.
> 
> If so, the big design question I'm facing at present is:
> 
> 1) Reuse the current btree infrastructure to create a btree for each 
> class that maps oids to persistent-objects and instantiate a secondary 
> index for each indexed slot using the slot accessor functions.  This is 
> the easisest to implement, but might provide somewhat poor performance 
> on create & writes.
> 
> 2) Create another underlying DB with string keys 
> "class-name+slot-name+value" => "oid"? 
> 
> 2a) - Is it better to point to oid's or directly to serialized 
> persistent-objects?  The nice thing about oid's is that later I can 
> implement join-like operations in the query language using oids without 
> having to deserialize and cache persistent objects.  Persistent-objects 
> are perhaps more convenient for direct use, however.
> 
> Comments would be greatly appreciated.  I especially invite debate if 
> others feel this is the wrong level of abstraction to work at (i.e. 
> instead write a new def macro for indexed classes and a related protocol 
> that accomplishes the same result by reusing primary and secondary 
> btrees).  The proposal above seems in good taste to me and I've already 
> invested some quality time in it, but since I'll be touching a fair bit 
> of the system to put this in I want to make sure there is support.
> 
> Ian
> 
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel

----
Robert L. Read, PhD                                     read &T
robertlread.net
Consider visiting Progressive Engineering:
http://robertlread.net/pe
In Austin: 912-8593                                        "Think
globally, Act locally." -- RBF


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060124/96d5c964/attachment.html>

From eslick at csail.mit.edu  Tue Jan 24 15:46:25 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Tue, 24 Jan 2006 10:46:25 -0500
Subject: [elephant-devel] Add indexed persistent class slots to	elephant?
In-Reply-To: <1138115702.8907.246.camel@localhost.localdomain>
References: <43D5A88B.2080604@csail.mit.edu>
	<1138115702.8907.246.camel@localhost.localdomain>
Message-ID: <43D64BD1.1030505@csail.mit.edu>

Robert,

Thanks for the comments.  You make some good points that I'll have to 
think about a bit.  I make a few comments below.

> author; in some ways I do not even consider myself qualified to 
> answer.  I will happily accept
> and your code, as long as the test are green and are you are expanding 
> the possibilities for
> the users, rather than constraining them.  And documentation would be 
> nice.

Ok, I'll stick myself on CVS so I can push things over incrementally and 
back things out more easily in case of damage (plus keep up to date on 
change you and others are making).  Did I hear that someone was working 
on GC for persistent objects?

>     As to your technical questions, about the approaches to 
> implementing the indexing, in my
> experience indexing is one of the most fluid aspects of an object 
> space; it is a tunable issue
> that tends to change as usage pattern changes even more than the 
> underlying data structure.
> So from my point of view I would much prefer macros or simple 
> functions that let you easily
> add, and discard, indexes.  Of course, your observation that one does 
> typically index a slot
> is quite correct, and it would be very convenient if such an index 
> could be added and dropped
> very easily; if one could also implement boolean operators based on 
> those indexes, so much
> the better.  As you state, one should be able to somehow introspect 
> the class and discover
> the existing indexing structure; I'm not sure of the best way to do that.

I think the big convenience feature is that object slot-writes will 
automatically update the respective slot index.  This isn't something 
you can do with an API layers on top.  In fact, doing this sort of 
indexing isn't easy at all.  As I understand it to update a slot index 
which is defined as a primary index I have to do a separate call to 
(setf get-value) to write the new value as key and object as value.  As 
a secondary index I have to do a write to the primary btree to update 
the secondary index which has the effect of calling all the secondary 
index functions on that object.  If I do separate btrees for each slot 
then each application needs to write it's own :after method on accessors 
to handle index updating.  This makes a simple add/delete of indexes 
less convenient and 'built-in' to the persistent language.

The changes to the MOP itself are actually fairly trivial.  One change 
parallels the existing transient slot definition semantics and the other 
is a single function that wraps the :around method on (setf 
slot-value-by-class).  Everything else is or should be separable. 

I will go ahead and make an API that uses primary btrees and just have 
the MOP extension use that.  If you redefine the class to remove the 
indexed keyword, the index will be dropped and if you add it then the 
new slot btree will be created and brought up to date.  That is the 
cleanest way to support add/drop of indexed slots using the metaclass 
approach.

You can also choose not to use the :indexed slot keyword and use the 
separate API instead.  You could also wrap your persistent object's 
accessors to automatically keep the index up to date so there will be a 
relatively simple way to bypass any dependence on the MOP.  Someone 
could then write a macro for defining indexed persistent classes that 
does the wrapping automatically.

I have to think about a common way to keep track of slot btrees so the 
two access methods play nicely.  I'll send some documentation on the 
whole things when I've got it working.

>     Although I am a "Smug Lisp Weanie" and fairly well educated, I 
> have to admit that all
> of this direct hacking of CLOS and MOP makes my head spin a little.  
> That is the stuff that
> creates the most problems in trying to work with multiple platforms.  
> We seem now to
> work with ACL and SBCL, and Andrew Blumberg is helping me solve and 
> OpenMCL problem
> right now.  As a maintainer, I'm a little loathe to recommend even 
> more complexity in that part.

This stuff always makes my head hurt as well.  I try to think of it like 
a good mental workout!  I'll try to make sure that the changes are as 
orthogonal as possible so there aren't any additional issues across 
versions.

>     However, I always believe interfaces are more important than 
> implementations.  The real
> question is: are you conveniently (elegantly?) expanding the API to 
> make elephant more useful?
> Whether this is done with a simple function or metaclass keywords 
> might actually be a side
> issue.  The code that will utilize the slot-based indexes is more 
> important than the code that
> will create them.
>     If I have a right to insist on anything, I will insist that the 
> indexes be easily droppable.

Please insist!  Any constraints provided by others means a smaller 
design space for me to get lost in!  Let me know if my proposal above 
satisfies your request.

Regards,
Ian



From read at robertlread.net  Tue Jan 24 16:23:20 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 24 Jan 2006 10:23:20 -0600
Subject: [elephant-devel] Add indexed persistent class slots to	elephant?
In-Reply-To: <43D64BD1.1030505@csail.mit.edu>
References: <43D5A88B.2080604@csail.mit.edu>
	<1138115702.8907.246.camel@localhost.localdomain>
	<43D64BD1.1030505@csail.mit.edu>
Message-ID: <1138119800.8907.267.camel@localhost.localdomain>

On Tue, 2006-01-24 at 10:46 -0500, Ian Eslick wrote:

> Ok, I'll stick myself on CVS so I can push things over incrementally
> and 
> back things out more easily in case of damage (plus keep up to date
> on 
> change you and others are making).  Did I hear that someone was
> working 
> on GC for persistent objects?


I don't think any serious work has begun on GC for persistent objects; I
think the idea has lauded.
I think it would be nice.

Currently only anonymous CVS access is allowed which does not allow
commits, I think.

My preferred way of doing CVS management would be to give a separate,
named branch
for your work, which, when ready, can be merged back into the trunk.

Perhaps someone can remind me how I would give Ian write permission on
the CVS repository?


----
Robert L. Read, PhD                                     read &T
robertlread.net
Consider visiting Progressive Engineering:
http://robertlread.net/pe
In Austin: 912-8593                                        "Think
globally, Act locally." -- RBF


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060124/c9703acc/attachment.html>

From eslick at csail.mit.edu  Tue Jan 24 16:43:16 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Tue, 24 Jan 2006 11:43:16 -0500
Subject: [elephant-devel] Add indexed persistent class slots	to	elephant?
In-Reply-To: <1138119800.8907.267.camel@localhost.localdomain>
References: <43D5A88B.2080604@csail.mit.edu>	
	<1138115702.8907.246.camel@localhost.localdomain>	
	<43D64BD1.1030505@csail.mit.edu>
	<1138119800.8907.267.camel@localhost.localdomain>
Message-ID: <43D65924.6090208@csail.mit.edu>


Robert L. Read wrote:

> On Tue, 2006-01-24 at 10:46 -0500, Ian Eslick wrote:
>
>> Ok, I'll stick myself on CVS so I can push things over incrementally and
>> back things out more easily in case of damage (plus keep up to date on
>> change you and others are making).  Did I hear that someone was working
>> on GC for persistent objects?
>
>
> I don't think any serious work has begun on GC for persistent objects; 
> I think the idea has lauded.
> I think it would be nice.

But probably not critical just now.  GC is a fair bit of work and the 
benefit is only realized when it starts to become a real problem for 
people.  What I'm doing will interact with GC as reclaimed objects need 
to have the associated key/value pairs dropped from slot-indices.  
Slot-indices should be like weak hashes...but if they're stored in the 
main btree table that may not be so easy depending on how GC is 
implemented.  It might make sense to add a weak option to btrees when it 
comes time to get around to GC...maybe lazily drop reclaimed instances 
when referenced.

That reminds me.  Without GC and/or renaming oids we'll eventually run 
out of OID address space, again when the DB gets enough use.  I haven't 
seen any provisions for fixnum wrap-around.  Am I missing something or 
is this correct?

Anyway, if I use existing btrees as a common base on which to create the 
two ways to use slot indexes, I'll need a canonical way to associate a 
persistent class with its btree indices.  I can store a separate root 
index in the controller and have the accessors indirect through that to 
find the appropriate slot-index but how do I bootstrap this new 
persistent root?  The root is bootstrapped through :from-oid -1.  I 
could have another default btree at oid = -2 for the special persistent 
slot indices.  Alternatively I could pollute the root by storing the 
slot-index in a very long symbol stored as a key in the root index. 

Thoughts?

Ian




From xach at xach.com  Tue Jan 24 17:13:40 2006
From: xach at xach.com (Zach Beane)
Date: Tue, 24 Jan 2006 12:13:40 -0500
Subject: [elephant-devel] Add indexed persistent class slots to elephant?
In-Reply-To: <43D65924.6090208@csail.mit.edu>
References: <43D5A88B.2080604@csail.mit.edu>
	<1138115702.8907.246.camel@localhost.localdomain>
	<43D64BD1.1030505@csail.mit.edu>
	<1138119800.8907.267.camel@localhost.localdomain>
	<43D65924.6090208@csail.mit.edu>
Message-ID: <20060124171340.GW4450@xach.com>

On Tue, Jan 24, 2006 at 11:43:16AM -0500, Ian Eslick wrote:

> That reminds me.  Without GC and/or renaming oids we'll eventually run 
> out of OID address space, again when the DB gets enough use.  I haven't 
> seen any provisions for fixnum wrap-around.  Am I missing something or 
> is this correct?

This isn't a direct comment, but I recently came across this book:

   http://www.amazon.com/gp/product/0136298338/102-6097685-4704118

It has a concise but interesting chapter on the design and
implementation of the Statice database. You can also find it on
Abebooks for $1.

I don't have it in front of me, but IIRC it described Statice OIDs as
96 bit and intended to be unique across space and time. It also said
that the database was *never* garbage collected.

Zach



From read at robertlread.net  Tue Jan 24 18:40:19 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 24 Jan 2006 12:40:19 -0600
Subject: [elephant-devel] Improved test examples checked into the HEAD....
Message-ID: <1138128019.8907.296.camel@localhost.localdomain>

Andrew Blumberg is very kindly and patiently working on some subtle 
bugs in the latest version when run on OpenMCL.

His work led me to understand that the test suite, which I consider to 
be critical to the success of a project of this size, although
fundamentally
very useful is a bit confusing to use when you try to use it in a
BerkeleyD-only
or a SQL-only installation.

I have therefore just checked in three files that I hope clarify
matters:

BerkeleyDB-tests.lisp --- an example of running the tests in a
BerkeleyDB-only installation
SQLDB-test.lisp --- an example of running the tests in a SQL (e.g.
Postgres or SQLite3)-only
installation.  
MigrationTests.lisp --- invocation of the "migration" tests, which test
moving data from
one back-end to another.  Since one can use any combination nation or
repositories ---
including BDB -> BDB, this necessarily requires the user to configure
for their own 
installation.

I will document all those on the project website and in the Installation
notes.


* * *

I would particularly like to thank Andrew Blumberg and Waldo Rubinstein
for their
work getting OpenMCL working; a subtle, MOP-ish difference is breaking
things there.

Personally, I think the momentum of this project is building; we are
seeing a lot of 
problems (relatively speaking) reported because we are expanding the
usage of the 
system.  I hope I'm not just fooling myself.

I was recently asked how many people use Elephant --- I really have no
idea.
If you read this and are using Elephant, or tried and abandoned it for
some reason,
please send me an email at "read at robertlread.net" or even to this
list.

I would really like to know if the number of users is closer to 10 or
100; if I 
had to guess I would say 15 today.


----
Robert L. Read, PhD                                     read &T
robertlread.net
Consider visiting Progressive Engineering:
http://robertlread.net/pe
In Austin: 912-8593                                        "Think
globally, Act locally." -- RBF


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060124/e399d78c/attachment.html>

From read at robertlread.net  Tue Jan 24 18:54:04 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 24 Jan 2006 12:54:04 -0600
Subject: [elephant-devel] Add indexed persistent class slots to elephant?
In-Reply-To: <20060124171340.GW4450@xach.com>
References: <43D5A88B.2080604@csail.mit.edu>
	<1138115702.8907.246.camel@localhost.localdomain>
	<43D64BD1.1030505@csail.mit.edu>
	<1138119800.8907.267.camel@localhost.localdomain>
	<43D65924.6090208@csail.mit.edu>  <20060124171340.GW4450@xach.com>
Message-ID: <1138128845.8907.307.camel@localhost.localdomain>

I think, in particular with respect to the oid issue, it is worth
reposting 
the note that I have added to the project as "BEN-RETIRING-TODO.txt".

Some of the things he lists have actually been done (whoopy!)


Hi all,

Thanks for your continued interest in Elephant.  Unfortunately I have
some bad news.  My day job has become all-consuming (and it
unfortunately does not involve working with Elephant at all.)  As such
I feel I need to step down as maintainer.  Andrew has volunteered to
maintain this project, but he will also probably have limited cycles
to contribute coding too.

While it seems Elephant is in a semi-useful state, it does need more
work.  So, if you like Elephant and would like to make it better,
there are plenty of things to work on!  I will stay on these email
lists and try to contribute what knowledge I have.  Andrew will try to
coordinate.  I've outlined here various projects in priority order, as
I see them.

At some point in the future I hope to come back!

High-priority projects:

1) we are storing persistent objects incorrectly.  they should be
stored only as OIDs, and we should have a separate OID->class table.
this way change-class can be handled correctly.

2) GC.  Gabor has started on this -- thank you!

3) New release.  Update docs, webpage to reflect new stuff.
(Sleepycat 4.3, SBCL+unicode, new MOP stuff.)

4) Metering and understanding locking issues.  Large transactions seem
to use a lot of locks.  In general understanding how to use Sleepycat
efficiently seems like a good thing.

5) Tests tests tests!

6) LispWorks, Allegro 7.0 and OpenMCL compatibility.  There are some
patches which have been sent in (check the logs.)  Also someone sent
in Solaris patches a while ago.

Difficult, but important projects:

7) callbacks for btree sorting: right now the sorted btree sorting
function is a bit of nasty C code.  i have code somewhere which did
callbacks instead, i can try to revive that.  the issue is that the
sorter needs to know the underlying lisp representation.  this is a
screw for unicode strings and bignums.  the C code is a hack for 16
bit unicode strings for allegro / lispworks (using IBMs ICU stuff),
and for the sbcl unicode stuff (using glibc wchar stuff.)  also bignum
sorting is by approximation via floats, which fails after some point.
but in hindsight maintaining all this is probably a lose.

8) Other backends.  Sleepycat is nice but has license issues.
Calling to C is nice but not nice.  Elephant is fairly modular:
serializer + MOP stuff + backend.  Therefore: write different
backends.  Possibly: Embedded Firebird, SQLLite (seems to not be that
great.)  Or, write an All-Lisp backend.  Copy what ZODB does (maybe
DirectoryStorage instead of FileStorage.)  Ask me for details.  With
SQL backends, there are possibility of using SQL types to solve some
of the btree sorting problems (and break others!)

9) init/reinit object protocol: Dan Knapp suggests (or I infer) an
initialization / reinitialization protocol might be nice, since there
is a difference between objects which are freshly created and those
which are loaded from the DB.

Med-priority projects:

10) equality joins have to be done on the lisp side, since many lisp
btrees share the same sleepycat btree.

11) single-user-mode: SUM = cache all slot values.  used to be high
priority, but i no longer want it, so unless someone wants it...

Low-priority projects:

12) class / slot to ID performance hack.  create a table which
contains slot definitions, probably cache this in memory on the class.
store slots as OID + slot ID -- avoid the symbol lookup.

13) typed arrays.  some lisps can pass things like arrays of 32 bit
signed integers directly to C.  right now we destructure and
restructure.  invalidated by 11).

14) bignum fixes.  OpenMCL: check that ldb is non-consing (i think it
is), look at %ldb-fixnum-from-bignum.  profile %bignum-ref on CMUCL /
SBCL.  or, figure out a great way to serialize bignums!

15) peter b's patch to non-persistent objs.  check the list.

16) dynamic-extent for transactions on SBCL/CMUCL.  i've tried to get
with-transaction to not cons.....

Impossible projects:

17) Serialize lambdas, closures, functions.  Seems to require
implementation support.  Also serialization of packages.

Take care, B

Seems to require
implementation support.  Also serialization of packages.

Take care, B

[read at localhost elephant]$





On Tue, 2006-01-24 at 12:13 -0500, Zach Beane wrote:

> On Tue, Jan 24, 2006 at 11:43:16AM -0500, Ian Eslick wrote:
> 
> > That reminds me.  Without GC and/or renaming oids we'll eventually run 
> > out of OID address space, again when the DB gets enough use.  I haven't 
> > seen any provisions for fixnum wrap-around.  Am I missing something or 
> > is this correct?
> 
> This isn't a direct comment, but I recently came across this book:
> 
>    http://www.amazon.com/gp/product/0136298338/102-6097685-4704118
> 
> It has a concise but interesting chapter on the design and
> implementation of the Statice database. You can also find it on
> Abebooks for $1.
> 
> I don't have it in front of me, but IIRC it described Statice OIDs as
> 96 bit and intended to be unique across space and time. It also said
> that the database was *never* garbage collected.
> 
> Zach
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060124/8bf9af4c/attachment.html>

From eslick at csail.mit.edu  Tue Jan 24 21:29:55 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Tue, 24 Jan 2006 16:29:55 -0500
Subject: [elephant-devel] Statice and FramerD
In-Reply-To: <20060124171340.GW4450@xach.com>
References: <43D5A88B.2080604@csail.mit.edu>	<1138115702.8907.246.camel@localhost.localdomain>	<43D64BD1.1030505@csail.mit.edu>	<1138119800.8907.267.camel@localhost.localdomain>	<43D65924.6090208@csail.mit.edu>
	<20060124171340.GW4450@xach.com>
Message-ID: <43D69C53.7060302@csail.mit.edu>

I just was talking with Howie Shrobe today, a colleague who actually 
hacked on Statice back when he was at Symbolics.  It sounds like they 
did default object indexing (all objects had a class index that referred 
to them) and was pretty hairy under the hood due to some technical 
politics.  Zach's comments are consistent with what Howie told me. 

We talked a bit about GC and he said that there is a real problem 
defining reachability in a multi-user DB.  GC might make more sense in 
solo mode, but requires hooking into the running image to determine what 
references are in memory that aren't yet in the root index.  The only 
obvious way to be sure you can GC is to disconnect everyone from the DB 
(as if you were doing recovery) and do reachability from all root 
indices to mark objects as live.

I also had a chance encounter with Ken Haase, the author of FramerD - an 
object database intended for doing knowledge representation and he 
brought up a few design issues that he found important:

1) Derived feature indices (Elephant's secondary indices provide good 
support for this)
2) Appending values.  Sleepycat allows duplicate keys in secondary 
indices so
    if we want to index on values and two values share the same date, 
for instance, then
    there are two btree key/value entries.  In FramerD he had unique 
keys and, much like in
    a hash collision, recorded the new value by appending it to an 
array/list within the btree
    without having to update the BTrees.  Without this duplicate keys 
can be expensive.  His
    worry was having to read the whole list that collided, add an 
element, and write it back.
3) Lightweight objects (Elephant does this right by keeping only oids in 
memory and doing
    slot read/writes via the btrees)

I'll try to pick his brain on query methods and optimization the next 
time he drops in.

Ian

Zach Beane wrote:

>On Tue, Jan 24, 2006 at 11:43:16AM -0500, Ian Eslick wrote:
>
>  
>
>>That reminds me.  Without GC and/or renaming oids we'll eventually run 
>>out of OID address space, again when the DB gets enough use.  I haven't 
>>seen any provisions for fixnum wrap-around.  Am I missing something or 
>>is this correct?
>>    
>>
>
>This isn't a direct comment, but I recently came across this book:
>
>   http://www.amazon.com/gp/product/0136298338/102-6097685-4704118
>
>It has a concise but interesting chapter on the design and
>implementation of the Statice database. You can also find it on
>Abebooks for $1.
>
>I don't have it in front of me, but IIRC it described Statice OIDs as
>96 bit and intended to be unique across space and time. It also said
>that the database was *never* garbage collected.
>
>Zach
>
>_______________________________________________
>elephant-devel site list
>elephant-devel at common-lisp.net
>http://common-lisp.net/mailman/listinfo/elephant-devel
>  
>


From read at robertlread.net  Wed Jan 25 01:17:19 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 24 Jan 2006 19:17:19 -0600
Subject: [elephant-devel] License Discussion
In-Reply-To: <9157df230601241716y4fec781off43ae34c5ba1e53@mail.gmail.com>
References: <43D5B0E2.6050304@csail.mit.edu>
	<1138080217.8907.212.camel@localhost.localdomain>
	<9157df230601241716y4fec781off43ae34c5ba1e53@mail.gmail.com>
Message-ID: <1138151839.8907.351.camel@localhost.localdomain>

But I have separated the need for sleepycat out.

So a user need not be anyway encumbered by sleepycat, using no code 
from them whatsoever, if they choose not to install it.

On Tue, 2006-01-24 at 19:16 -0600, Ben wrote:

> it's fine with me and andrew, so i'm happy with LLGPL.  on the other
> hand, there are some licensing issues with using sleepycat itself, so
> that should be taken into account.....
> 
> B
> 
> On 1/23/06, Robert L. Read <read at robertlread.net> wrote:
> >  For the code that I have written, I am completely in accordance with this
> >  suggestion to use the LLGPL.
> >
> >  However, Ben and Andrew and others are really the primary authors; unless
> >  we reach consensus with them, and perhaps others, I don't see how we
> >  can make the switch.
> >
> >
> >  On Mon, 2006-01-23 at 23:45 -0500, Ian Eslick wrote:
> >  I am writing to see if I can restart the license discussion I saw a
> > little of in the archives late last year. With the SQL backends it
> > seems more appropriate to release the next version of elephant as an
> > LLGPL code base to encourage wider adoption by folks who work for or
> > consult for other companies and can't use the existing elephant code
> > with a SQL backend in commercial software. It certainly makes it more
> > motivating for me to contribute to the code base as my lab (sponsored by
> > a bunch of companies) has expressed a preference for LLGPL in my own
> > library projects. I've had to strip all the GPL'ed outside lisp
> > libraries but am holding for the time being to see if elephant will
> > indeed reconsider its license. :)
> >
> > Regardless it's a wonderful system already and the first shot in awhile
> > for the lisp community to settle on a PODB. It's probably also helpful
> > that the code is maturing at the same time as AllegroCache is generating
> > some broad interest.
> >
> > Please consider this my request for the LLGPL.
> > http://opensource.franz.com/preamble.html
> >
> > Regards,
> > Ian
> > _
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060124/fd05c55e/attachment.html>

From eslick at csail.mit.edu  Wed Jan 25 01:30:36 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Tue, 24 Jan 2006 20:30:36 -0500
Subject: [elephant-devel] License Discussion
In-Reply-To: <9157df230601241716y4fec781off43ae34c5ba1e53@mail.gmail.com>
References: <43D5B0E2.6050304@csail.mit.edu>	
	<1138080217.8907.212.camel@localhost.localdomain>
	<9157df230601241716y4fec781off43ae34c5ba1e53@mail.gmail.com>
Message-ID: <43D6D4BC.6030309@csail.mit.edu>

That's great to hear!  As Robert indicated, I think the real benefit is 
to users of the SQL backend.  Also, if we write a lisp backend using the 
current front-end metaclass and serializer then we'll (hopefully) have a 
fully LLGPL library solution for persistent objects.  I think that makes 
it much easier to build user community momentum.  License issues 
ultimately kept much work from proceeding on PLOB and a couple of other 
potential lisp persistent databases.

Ben & Andrew, thank you for your great work on the system to date!

Cheers,
Ian

Ben wrote:

>it's fine with me and andrew, so i'm happy with LLGPL.  on the other
>hand, there are some licensing issues with using sleepycat itself, so
>that should be taken into account.....
>
>B
>
>On 1/23/06, Robert L. Read <read at robertlread.net> wrote:
>  
>
>> For the code that I have written, I am completely in accordance with this
>> suggestion to use the LLGPL.
>>
>> However, Ben and Andrew and others are really the primary authors; unless
>> we reach consensus with them, and perhaps others, I don't see how we
>> can make the switch.
>>
>>
>> On Mon, 2006-01-23 at 23:45 -0500, Ian Eslick wrote:
>> I am writing to see if I can restart the license discussion I saw a
>>little of in the archives late last year. With the SQL backends it
>>seems more appropriate to release the next version of elephant as an
>>LLGPL code base to encourage wider adoption by folks who work for or
>>consult for other companies and can't use the existing elephant code
>>with a SQL backend in commercial software. It certainly makes it more
>>motivating for me to contribute to the code base as my lab (sponsored by
>>a bunch of companies) has expressed a preference for LLGPL in my own
>>library projects. I've had to strip all the GPL'ed outside lisp
>>libraries but am holding for the time being to see if elephant will
>>indeed reconsider its license. :)
>>
>>Regardless it's a wonderful system already and the first shot in awhile
>>for the lisp community to settle on a PODB. It's probably also helpful
>>that the code is maturing at the same time as AllegroCache is generating
>>some broad interest.
>>
>>Please consider this my request for the LLGPL.
>>http://opensource.franz.com/preamble.html
>>
>>Regards,
>>Ian
>>_______________________________________________
>>elephant-devel site list
>>elephant-devel at common-lisp.net
>>http://common-lisp.net/mailman/listinfo/elephant-devel
>>
>>
>>_______________________________________________
>>elephant-devel site list
>>elephant-devel at common-lisp.net
>>http://common-lisp.net/mailman/listinfo/elephant-devel
>>
>>
>>    
>>


From eslick at csail.mit.edu  Wed Jan 25 01:31:27 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Tue, 24 Jan 2006 20:31:27 -0500
Subject: [elephant-devel] Secondary indices
Message-ID: <43D6D4EF.5040600@csail.mit.edu>

If I remove a secondary index from an indexed-btree, what happens to the 
underlying Berkeley DB storage?  It appears that any secondary values 
written just stick around rather than being removed via remove-kv or 
something like it.  Is that a correct assessment?  I'm just thinking 
about what happens if a user explicitely wants to drop a secondary 
database and reclaim any related storage.

Thank you,
Ian




From midfield at gmail.com  Wed Jan 25 01:33:37 2006
From: midfield at gmail.com (Ben)
Date: Tue, 24 Jan 2006 19:33:37 -0600
Subject: [elephant-devel] Secondary indices
In-Reply-To: <43D6D4EF.5040600@csail.mit.edu>
References: <43D6D4EF.5040600@csail.mit.edu>
Message-ID: <9157df230601241733g38f689bdn2715d652dfb4bcd3@mail.gmail.com>

it certainly appears that way.  in the back of my mind i think i had a
copying garbage collector in mind.....oh boy.

B

On 1/24/06, Ian Eslick <eslick at csail.mit.edu> wrote:
> If I remove a secondary index from an indexed-btree, what happens to the
> underlying Berkeley DB storage?  It appears that any secondary values
> written just stick around rather than being removed via remove-kv or
> something like it.  Is that a correct assessment?  I'm just thinking
> about what happens if a user explicitely wants to drop a secondary
> database and reclaim any related storage.
>
> Thank you,
> Ian
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>


From midfield at gmail.com  Wed Jan 25 01:34:16 2006
From: midfield at gmail.com (Ben)
Date: Tue, 24 Jan 2006 19:34:16 -0600
Subject: [elephant-devel] License Discussion
In-Reply-To: <43D6D4BC.6030309@csail.mit.edu>
References: <43D5B0E2.6050304@csail.mit.edu>
	<1138080217.8907.212.camel@localhost.localdomain>
	<9157df230601241716y4fec781off43ae34c5ba1e53@mail.gmail.com>
	<43D6D4BC.6030309@csail.mit.edu>
Message-ID: <9157df230601241734o6e6aceb9xa9f6a0bb0c8a1693@mail.gmail.com>

well i think kudos should go to robert for managing this beast,
keeping it alive and keeping andrew and me honest about our code!

B

On 1/24/06, Ian Eslick <eslick at csail.mit.edu> wrote:
> That's great to hear!  As Robert indicated, I think the real benefit is
> to users of the SQL backend.  Also, if we write a lisp backend using the
> current front-end metaclass and serializer then we'll (hopefully) have a
> fully LLGPL library solution for persistent objects.  I think that makes
> it much easier to build user community momentum.  License issues
> ultimately kept much work from proceeding on PLOB and a couple of other
> potential lisp persistent databases.
>
> Ben & Andrew, thank you for your great work on the system to date!
>
> Cheers,
> Ian
>
> Ben wrote:
>
> >it's fine with me and andrew, so i'm happy with LLGPL.  on the other
> >hand, there are some licensing issues with using sleepycat itself, so
> >that should be taken into account.....
> >
> >B
> >
> >On 1/23/06, Robert L. Read <read at robertlread.net> wrote:
> >
> >
> >> For the code that I have written, I am completely in accordance with this
> >> suggestion to use the LLGPL.
> >>
> >> However, Ben and Andrew and others are really the primary authors; unless
> >> we reach consensus with them, and perhaps others, I don't see how we
> >> can make the switch.
> >>
> >>
> >> On Mon, 2006-01-23 at 23:45 -0500, Ian Eslick wrote:
> >> I am writing to see if I can restart the license discussion I saw a
> >>little of in the archives late last year. With the SQL backends it
> >>seems more appropriate to release the next version of elephant as an
> >>LLGPL code base to encourage wider adoption by folks who work for or
> >>consult for other companies and can't use the existing elephant code
> >>with a SQL backend in commercial software. It certainly makes it more
> >>motivating for me to contribute to the code base as my lab (sponsored by
> >>a bunch of companies) has expressed a preference for LLGPL in my own
> >>library projects. I've had to strip all the GPL'ed outside lisp
> >>libraries but am holding for the time being to see if elephant will
> >>indeed reconsider its license. :)
> >>
> >>Regardless it's a wonderful system already and the first shot in awhile
> >>for the lisp community to settle on a PODB. It's probably also helpful
> >>that the code is maturing at the same time as AllegroCache is generating
> >>some broad interest.
> >>
> >>Please consider this my request for the LLGPL.
> >>http://opensource.franz.com/preamble.html
> >>
> >>Regards,
> >>Ian
> >>_______________________________________________
> >>elephant-devel site list
> >>elephant-devel at common-lisp.net
> >>http://common-lisp.net/mailman/listinfo/elephant-devel
> >>
> >>
> >>_______________________________________________
> >>elephant-devel site list
> >>elephant-devel at common-lisp.net
> >>http://common-lisp.net/mailman/listinfo/elephant-devel
> >>
> >>
> >>
> >>
>


From read at robertlread.net  Wed Jan 25 02:16:06 2006
From: read at robertlread.net (Robert L. Read)
Date: Tue, 24 Jan 2006 20:16:06 -0600
Subject: [elephant-devel] License Discussion
In-Reply-To: <9157df230601241734o6e6aceb9xa9f6a0bb0c8a1693@mail.gmail.com>
References: <43D5B0E2.6050304@csail.mit.edu>
	<1138080217.8907.212.camel@localhost.localdomain>
	<9157df230601241716y4fec781off43ae34c5ba1e53@mail.gmail.com>
	<43D6D4BC.6030309@csail.mit.edu>
	<9157df230601241734o6e6aceb9xa9f6a0bb0c8a1693@mail.gmail.com>
Message-ID: <1138155366.8907.361.camel@localhost.localdomain>

So, based on the idea that Ben Lee and Andrew Blumenberg and myself
are far and away the largest code contributers so far, and that the
three of us 
agree to a an LLGLP release, it seems appropriate for me to plan
the next release as a LLGLP release.

It will take me some time to properly place the copyright notices and so
forth.
My plan right now is to iron out the current OpenMCL bugs, and when 
we are all green on OpenMCL, ACL (if I can get it tested there) and SBCL
then I will make a new release.  This will probably be numbered 0.5.0 to
represent the license change, although the code and feature changes will
be relatively small.

And, obviously, this does not revoke the previously releases; anyone can
continue to use them and even fork the codebase if they prefer.

If anyone else has contributed code to the project and does not want
their code to be part of the LLGPL release, please state so publicly 
here and we'll decide what to do about it then.






On Tue, 2006-01-24 at 19:34 -0600, Ben wrote:

> well i think kudos should go to robert for managing this beast,
> keeping it alive and keeping andrew and me honest about our code!
> 
> B
> 
> On 1/24/06, Ian Eslick <eslick at csail.mit.edu> wrote:
> > That's great to hear!  As Robert indicated, I think the real benefit is
> > to users of the SQL backend.  Also, if we write a lisp backend using the
> > current front-end metaclass and serializer then we'll (hopefully) have a
> > fully LLGPL library solution for persistent objects.  I think that makes
> > it much easier to build user community momentum.  License issues
> > ultimately kept much work from proceeding on PLOB and a couple of other
> > potential lisp persistent databases.
> >
> > Ben & Andrew, thank you for your great work on the system to date!
> >
> > Cheers,
> > Ian
> >
> > Ben wrote:
> >
> > >it's fine with me and andrew, so i'm happy with LLGPL.  on the other
> > >hand, there are some licensing issues with using sleepycat itself, so
> > >that should be taken into account.....
> > >
> > >B
> > >
> > >On 1/23/06, Robert L. Read <read at robertlread.net> wrote:
> > >
> > >
> > >> For the code that I have written, I am completely in accordance with this
> > >> suggestion to use the LLGPL.
> > >>
> > >> However, Ben and Andrew and others are really the primary authors; unless
> > >> we reach consensus with them, and perhaps others, I don't see how we
> > >> can make the switch.
> > >>
> > >>
> > >> On Mon, 2006-01-23 at 23:45 -0500, Ian Eslick wrote:
> > >> I am writing to see if I can restart the license discussion I saw a
> > >>little of in the archives late last year. With the SQL backends it
> > >>seems more appropriate to release the next version of elephant as an
> > >>LLGPL code base to encourage wider adoption by folks who work for or
> > >>consult for other companies and can't use the existing elephant code
> > >>with a SQL backend in commercial software. It certainly makes it more
> > >>motivating for me to contribute to the code base as my lab (sponsored by
> > >>a bunch of companies) has expressed a preference for LLGPL in my own
> > >>library projects. I've had to strip all the GPL'ed outside lisp
> > >>libraries but am holding for the time being to see if elephant will
> > >>indeed reconsider its license. :)
> > >>
> > >>Regardless it's a wonderful system already and the first shot in awhile
> > >>for the lisp community to settle on a PODB. It's probably also helpful
> > >>that the code is maturing at the same time as AllegroCache is generating
> > >>some broad interest.
> > >>
> > >>Please consider this my request for the LLGPL.
> > >>http://opensource.franz.com/preamble.html
> > >>
> > >>Regards,
> > >>Ian
> > >>_______________________________________________
> > >>elephant-devel site list
> > >>elephant-devel at common-lisp.net
> > >>http://common-lisp.net/mailman/listinfo/elephant-devel
> > >>
> > >>
> > >>_______________________________________________
> > >>elephant-devel site list
> > >>elephant-devel at common-lisp.net
> > >>http://common-lisp.net/mailman/listinfo/elephant-devel
> > >>
> > >>
> > >>
> > >>
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060124/07435acf/attachment.html>

From midfield at gmail.com  Wed Jan 25 01:16:00 2006
From: midfield at gmail.com (Ben)
Date: Tue, 24 Jan 2006 19:16:00 -0600
Subject: [elephant-devel] License Discussion
In-Reply-To: <1138080217.8907.212.camel@localhost.localdomain>
References: <43D5B0E2.6050304@csail.mit.edu>
	<1138080217.8907.212.camel@localhost.localdomain>
Message-ID: <9157df230601241716y4fec781off43ae34c5ba1e53@mail.gmail.com>

it's fine with me and andrew, so i'm happy with LLGPL.  on the other
hand, there are some licensing issues with using sleepycat itself, so
that should be taken into account.....

B

On 1/23/06, Robert L. Read <read at robertlread.net> wrote:
>  For the code that I have written, I am completely in accordance with this
>  suggestion to use the LLGPL.
>
>  However, Ben and Andrew and others are really the primary authors; unless
>  we reach consensus with them, and perhaps others, I don't see how we
>  can make the switch.
>
>
>  On Mon, 2006-01-23 at 23:45 -0500, Ian Eslick wrote:
>  I am writing to see if I can restart the license discussion I saw a
> little of in the archives late last year. With the SQL backends it
> seems more appropriate to release the next version of elephant as an
> LLGPL code base to encourage wider adoption by folks who work for or
> consult for other companies and can't use the existing elephant code
> with a SQL backend in commercial software. It certainly makes it more
> motivating for me to contribute to the code base as my lab (sponsored by
> a bunch of companies) has expressed a preference for LLGPL in my own
> library projects. I've had to strip all the GPL'ed outside lisp
> libraries but am holding for the time being to see if elephant will
> indeed reconsider its license. :)
>
> Regardless it's a wonderful system already and the first shot in awhile
> for the lisp community to settle on a PODB. It's probably also helpful
> that the code is maturing at the same time as AllegroCache is generating
> some broad interest.
>
> Please consider this my request for the LLGPL.
> http://opensource.franz.com/preamble.html
>
> Regards,
> Ian
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>
>


From aycan.irican at core.gen.tr  Wed Jan 25 18:45:06 2006
From: aycan.irican at core.gen.tr (Aycan iRiCAN)
Date: Wed, 25 Jan 2006 20:45:06 +0200
Subject: [elephant-devel] unable to use persistent-class
Message-ID: <87irs8kuz1.fsf@core.gen.tr>


Hello,

I'm trying to use elephant in one of our projects. It's a simple UCW
based web application. I set up elephant properly and run
tests. Started sbcl with slime and evaluate a few lines

(asdf:oos 'asdf:load-op :elephant)
(asdf:oos 'asdf:load-op :ele-bdb)
(asdf:oos 'asdf:load-op :elephant-tests)
(do-all-tests-spec *testdb-path*)

The result is:

5 out of 106 total tests failed: BASICPERSISTENCE, MIGRATE1, MIGRATE2,
   MIGRATE3, MIGRATE4.

After these results, I think that I can try to use it with my
application. I just added :elephant to dependencies of my system
definition. 

<snip>
   :depends-on (:ucw :elephant)
</snip>

And also added it to my package definition.

<snip>
    (:use :common-lisp :it.bese.ucw :elephant)
</snip>

I have a simple content class. So I defined its metaclass.

(defclass content ()
    ((title :accessor title
            :initarg :title
            :initform "Sample Title"
            :documentation "Title for content")
     (data :accessor data
           :initarg :data
           :initform "Data"
           :documentation "Data for content"))
  (:metaclass persistent-metaclass))

After these steps, I tried to load my app with asdf but got this
error:

debugger invoked on a SIMPLE-ERROR in thread
#<THREAD "initial thread" {90033C9}>:
  Error during processing of --eval option (LOAD #P"bin/make-image.lisp"):

  There is no applicable method for the generic function
    #<STANDARD-GENERIC-FUNCTION ELEPHANT::NEXT-OID (1)>
  when called with arguments
    (NIL).

Is this problem related with Waldo Rubinstein's post? 
    http://common-lisp.net/pipermail/elephant-devel/2006-January/000212.html

Can you give me a temporary solution?

Best Regards.

-- 
Aycan iRiCAN
C0R3 Computer Security Group
http://www.core.gen.tr
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060125/eeb19c29/attachment.sig>

From read at robertlread.net  Wed Jan 25 19:31:35 2006
From: read at robertlread.net (Robert L. Read)
Date: Wed, 25 Jan 2006 13:31:35 -0600
Subject: [elephant-devel] unable to use persistent-class
In-Reply-To: <87irs8kuz1.fsf@core.gen.tr>
References: <87irs8kuz1.fsf@core.gen.tr>
Message-ID: <1138217495.8907.432.camel@localhost.localdomain>

You will also have to depend on :ele-bdb.

Yes, it looks to me like it is the same problem as Waldo's that you
mention;
you have to call:

(open-store "pathtotheBerkeleyDBdatabasedirectory")

before you invoke defclass.

I would try executing that, and then your class definition,
directly in a  REPL or listener by hand.

If that works, you might have to do the open-store inside an "eval-when"
clause,
depending on how you have organized your code.

Does that make sense?



On Wed, 2006-01-25 at 20:45 +0200, Aycan iRiCAN wrote:

> Hello,
> 
> I'm trying to use elephant in one of our projects. It's a simple UCW
> based web application. I set up elephant properly and run
> tests. Started sbcl with slime and evaluate a few lines
> 
> (asdf:oos 'asdf:load-op :elephant)
> (asdf:oos 'asdf:load-op :ele-bdb)
> (asdf:oos 'asdf:load-op :elephant-tests)
> (do-all-tests-spec *testdb-path*)
> 
> The result is:
> 
> 5 out of 106 total tests failed: BASICPERSISTENCE, MIGRATE1, MIGRATE2,
>    MIGRATE3, MIGRATE4.
> 
> After these results, I think that I can try to use it with my
> application. I just added :elephant to dependencies of my system
> definition. 
> 
> <snip>
>    :depends-on (:ucw :elephant)
> </snip>
> 
> And also added it to my package definition.
> 
> <snip>
>     (:use :common-lisp :it.bese.ucw :elephant)
> </snip>
> 
> I have a simple content class. So I defined its metaclass.
> 
> (defclass content ()
>     ((title :accessor title
>             :initarg :title
>             :initform "Sample Title"
>             :documentation "Title for content")
>      (data :accessor data
>            :initarg :data
>            :initform "Data"
>            :documentation "Data for content"))
>   (:metaclass persistent-metaclass))
> 
> After these steps, I tried to load my app with asdf but got this
> error:
> 
> debugger invoked on a SIMPLE-ERROR in thread
> #<THREAD "initial thread" {90033C9}>:
>   Error during processing of --eval option (LOAD #P"bin/make-image.lisp"):
> 
>   There is no applicable method for the generic function
>     #<STANDARD-GENERIC-FUNCTION ELEPHANT::NEXT-OID (1)>
>   when called with arguments
>     (NIL).
> 
> Is this problem related with Waldo Rubinstein's post? 
>     http://common-lisp.net/pipermail/elephant-devel/2006-January/000212.html
> 
> Can you give me a temporary solution?
> 
> Best Regards.
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060125/afd47980/attachment.html>

From aycan.irican at core.gen.tr  Wed Jan 25 20:58:52 2006
From: aycan.irican at core.gen.tr (Aycan iRiCAN)
Date: Wed, 25 Jan 2006 22:58:52 +0200
Subject: [elephant-devel] unable to use persistent-class
In-Reply-To: <1138217495.8907.432.camel@localhost.localdomain> (Robert
	L. Read's message of "Wed, 25 Jan 2006 13:31:35 -0600")
References: <87irs8kuz1.fsf@core.gen.tr>
	<1138217495.8907.432.camel@localhost.localdomain>
Message-ID: <87ek2wkos3.fsf@core.gen.tr>


Thank you,

I opened the store before my class declaration. Could this be changed
in the future? Why we need to open the store first?

Now I have another problem. I have a web action that handles a form
post. This action creates an instace of content class and fills the
data slot.

Since I am using utf-8 encoding with sbcl, when I list the content
instances with a web page, I'm having strange characters from data
slot.

Is there any problem with utf-8? I didn't tried to persist the
instance yet. Just created an instance and read back from it.

Best Regards.

"Robert L. Read" <read at robertlread.net> writes:

> You will also have to depend on :ele-bdb.
>
> Yes, it looks to me like it is the same problem as Waldo's that you mention;
> you have to call:
>
> (open-store "pathtotheBerkeleyDBdatabasedirectory")
>
> before you invoke defclass.
>
> I would try executing that, and then your class definition,
> directly in a  REPL or listener by hand.
>
> If that works, you might have to do the open-store inside an "eval-when" clause,
> depending on how you have organized your code.
>
> Does that make sense?
>
> On Wed, 2006-01-25 at 20:45 +0200, Aycan iRiCAN wrote:
>
>     Hello,
>
>     I'm trying to use elephant in one of our projects. It's a simple UCW
>     based web application. I set up elephant properly and run
>     tests. Started sbcl with slime and evaluate a few lines
>
>     (asdf:oos 'asdf:load-op :elephant)
>     (asdf:oos 'asdf:load-op :ele-bdb)
>     (asdf:oos 'asdf:load-op :elephant-tests)
>     (do-all-tests-spec *testdb-path*)
>
>     The result is:
>
>     5 out of 106 total tests failed: BASICPERSISTENCE, MIGRATE1, MIGRATE2,
>        MIGRATE3, MIGRATE4.
>
>     After these results, I think that I can try to use it with my
>     application. I just added :elephant to dependencies of my system
>     definition. 
>
>     <snip>
>        :depends-on (:ucw :elephant)
>     </snip>
>
>     And also added it to my package definition.
>
>     <snip>
>         (:use :common-lisp :it.bese.ucw :elephant)
>     </snip>
>
>     I have a simple content class. So I defined its metaclass.
>
>     (defclass content ()
>         ((title :accessor title
>                 :initarg :title
>                 :initform "Sample Title"
>                 :documentation "Title for content")
>          (data :accessor data
>                :initarg :data
>                :initform "Data"
>                :documentation "Data for content"))
>       (:metaclass persistent-metaclass))
>
>     After these steps, I tried to load my app with asdf but got this
>     error:
>
>     debugger invoked on a SIMPLE-ERROR in thread
>     #<THREAD "initial thread" {90033C9}>:
>       Error during processing of --eval option (LOAD #P"bin/make-image.lisp"):
>
>       There is no applicable method for the generic function
>         #<STANDARD-GENERIC-FUNCTION ELEPHANT::NEXT-OID (1)>
>       when called with arguments
>         (NIL).
>
>     Is this problem related with Waldo Rubinstein's post? 
>         http://common-lisp.net/pipermail/elephant-devel/2006-January/000212.html
>
>     Can you give me a temporary solution?
>
>     Best Regards.
>
>     _______________________________________________
>     elephant-devel site list
>     elephant-devel at common-lisp.net
>     http://common-lisp.net/mailman/listinfo/elephant-devel

-- 
Aycan iRiCAN
C0R3 Computer Security Group
http://www.core.gen.tr
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060125/1a654824/attachment.sig>

From read at robertlread.net  Wed Jan 25 22:20:35 2006
From: read at robertlread.net (Robert L. Read)
Date: Wed, 25 Jan 2006 16:20:35 -0600
Subject: [elephant-devel] unable to use persistent-class
In-Reply-To: <87ek2wkos3.fsf@core.gen.tr>
References: <87irs8kuz1.fsf@core.gen.tr>
	<1138217495.8907.432.camel@localhost.localdomain>
	<87ek2wkos3.fsf@core.gen.tr>
Message-ID: <1138227635.8907.452.camel@localhost.localdomain>

On Wed, 2006-01-25 at 22:58 +0200, Aycan iRiCAN wrote:

> Thank you,
> 
> I opened the store before my class declaration. Could this be changed
> in the future? Why we need to open the store first?


Off the top of my head, it would not be easy to change it.  The store
must be opened for 
something to be persisted into it; defining the class as a minimum sets
up some identifiers
in preparation for the first instance being created.


> 
> Now I have another problem. I have a web action that handles a form
> post. This action creates an instace of content class and fills the
> data slot.
> 
> Since I am using utf-8 encoding with sbcl, when I list the content
> instances with a web page, I'm having strange characters from data
> slot.
> 
> Is there any problem with utf-8? I didn't tried to persist the
> instance yet. Just created an instance and read back from it.


I use SBCL 0.9.5 with the unicode option turned off and I persist
unicode strings there often
and have not had a problem.  I have no reason to believe there is a bug,
but of course there
might be.

That question can really only be answered by a small test example.  In
my experience, 
there will be at least as many problems encountered in the rendering of
the characters 
and the character encoding as the actual storage.  If you can reduce the
problem to 
a very small test program, we can construct a test from that a debug it.

I have in fact only stored characters which happen to be from the
Esperanto 
character set, which is ISO-8859-3, but it it is using the utf-8
encoding.

I specifically had to make some changes to make sure it works with SBCL
when 
the unicode feature is turned on.

If things look wonky, you might try converting to bytes and looking at
the actual 
byte values.


> 
> Best Regards.
> 
> "Robert L. Read" <read at robertlread.net> writes:
> 
> > You will also have to depend on :ele-bdb.
> >
> > Yes, it looks to me like it is the same problem as Waldo's that you mention;
> > you have to call:
> >
> > (open-store "pathtotheBerkeleyDBdatabasedirectory")
> >
> > before you invoke defclass.
> >
> > I would try executing that, and then your class definition,
> > directly in a  REPL or listener by hand.
> >
> > If that works, you might have to do the open-store inside an "eval-when" clause,
> > depending on how you have organized your code.
> >
> > Does that make sense?
> >
> > On Wed, 2006-01-25 at 20:45 +0200, Aycan iRiCAN wrote:
> >
> >     Hello,
> >
> >     I'm trying to use elephant in one of our projects. It's a simple UCW
> >     based web application. I set up elephant properly and run
> >     tests. Started sbcl with slime and evaluate a few lines
> >
> >     (asdf:oos 'asdf:load-op :elephant)
> >     (asdf:oos 'asdf:load-op :ele-bdb)
> >     (asdf:oos 'asdf:load-op :elephant-tests)
> >     (do-all-tests-spec *testdb-path*)
> >
> >     The result is:
> >
> >     5 out of 106 total tests failed: BASICPERSISTENCE, MIGRATE1, MIGRATE2,
> >        MIGRATE3, MIGRATE4.
> >
> >     After these results, I think that I can try to use it with my
> >     application. I just added :elephant to dependencies of my system
> >     definition. 
> >
> >     <snip>
> >        :depends-on (:ucw :elephant)
> >     </snip>
> >
> >     And also added it to my package definition.
> >
> >     <snip>
> >         (:use :common-lisp :it.bese.ucw :elephant)
> >     </snip>
> >
> >     I have a simple content class. So I defined its metaclass.
> >
> >     (defclass content ()
> >         ((title :accessor title
> >                 :initarg :title
> >                 :initform "Sample Title"
> >                 :documentation "Title for content")
> >          (data :accessor data
> >                :initarg :data
> >                :initform "Data"
> >                :documentation "Data for content"))
> >       (:metaclass persistent-metaclass))
> >
> >     After these steps, I tried to load my app with asdf but got this
> >     error:
> >
> >     debugger invoked on a SIMPLE-ERROR in thread
> >     #<THREAD "initial thread" {90033C9}>:
> >       Error during processing of --eval option (LOAD #P"bin/make-image.lisp"):
> >
> >       There is no applicable method for the generic function
> >         #<STANDARD-GENERIC-FUNCTION ELEPHANT::NEXT-OID (1)>
> >       when called with arguments
> >         (NIL).
> >
> >     Is this problem related with Waldo Rubinstein's post? 
> >         http://common-lisp.net/pipermail/elephant-devel/2006-January/000212.html
> >
> >     Can you give me a temporary solution?
> >
> >     Best Regards.
> >
> >     _______________________________________________
> >     elephant-devel site list
> >     elephant-devel at common-lisp.net
> >     http://common-lisp.net/mailman/listinfo/elephant-devel
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060125/caa36cda/attachment.html>

From read at robertlread.net  Wed Jan 25 22:27:04 2006
From: read at robertlread.net (Robert L. Read)
Date: Wed, 25 Jan 2006 16:27:04 -0600
Subject: [elephant-devel] unable to use persistent-class
In-Reply-To: <1138227635.8907.452.camel@localhost.localdomain>
References: <87irs8kuz1.fsf@core.gen.tr>
	<1138217495.8907.432.camel@localhost.localdomain>
	<87ek2wkos3.fsf@core.gen.tr>
	<1138227635.8907.452.camel@localhost.localdomain>
Message-ID: <1138228025.8907.458.camel@localhost.localdomain>

On Wed, 2006-01-25 at 16:20 -0600, Robert L. Read wrote:

> I use SBCL 0.9.5 with the unicode option turned off and I persist
> unicode strings there often
> and have not had a problem.  I have no reason to believe there is a
> bug, but of course there
> might be.


Sorry - this should read with it "turned on".

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060125/dc688e3c/attachment.html>

From eslick at csail.mit.edu  Thu Jan 26 19:10:42 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Thu, 26 Jan 2006 14:10:42 -0500
Subject: [elephant-devel] Lisp locks up with BDB
Message-ID: <43D91EB2.4020602@csail.mit.edu>

Has anyone else had the problem where, when setting a slot, the
underlying lisp occasionally locks up without any indication.  This may
be caused by some interactions that happen when I update the secondary
DB during persistent slot writes, but it's expensive to track down the
problem as I have to restart lisp each time I run into the problem.  At
least my data persists! 

If anyone else has clues as to what causes lockups, it would be
immensely helpful in focusing my instrumentation in tracking this down.

Ian


From eslick at csail.mit.edu  Thu Jan 26 23:37:56 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Thu, 26 Jan 2006 18:37:56 -0500
Subject: [elephant-devel] Outer transaction required? 
Message-ID: <43D95D54.6030204@csail.mit.edu>

As a follow-up to my earlier post...

It appears that many outer destructive operations need to wrap its calls
in an outer transaction and that the with-transaction statements within
functions like remove-kv for indexed-btrees are not sufficient.  Does
anyone know why this is so?  I kept finding my lisp crashing in
db-delete-buffered even with auto-commit set to t.  Very distressing.  :)

Is there any way to turn on an error-checking mode for development that
catches side-effecting operations that are not inside a transaction? 

Ian


From eslick at csail.mit.edu  Sat Jan 28 19:37:21 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Sat, 28 Jan 2006 14:37:21 -0500
Subject: [elephant-devel] Secondary DB behavior
Message-ID: <43DBC7F1.6010108@csail.mit.edu>

This was not clear in the documentation, but secondary db's appear to
have the following behavior.

btree - an indexed btree
inverted - an secondary index of btree
fn - A function that computes the secondary key given the value given to
btree

Write pkey/value1 into btree
inverted has fn(value1)/pkey

Change the association of pkey by writing
pkey/value2 into btree

inverted now has:
fn(value1)/pkey
fn(value2)/pkey

Namely, each secondary index contains the entire history of the values
of pkey and associates
them with pkey.  If you are sorting and searching objects by using
secondary indices this is
not very useful. 

Is this the intended functionality?  This could be a setup problem on my
end or the way secondary
indices were intended to work.  In which case it makes sense for me to
make sure that when
a secondary value changes, the original secondary key/value pair is
deleted from the secondary
index.  I just discovered this and the uniqueness of mappings to pkey in
the above example is
crucial for my application.

Thank you,
Ian


From read at robertlread.net  Sat Jan 28 22:30:52 2006
From: read at robertlread.net (Robert L. Read)
Date: Sat, 28 Jan 2006 16:30:52 -0600
Subject: [elephant-devel] Secondary DB behavior
In-Reply-To: <43DBC7F1.6010108@csail.mit.edu>
References: <43DBC7F1.6010108@csail.mit.edu>
Message-ID: <1138487453.8907.581.camel@localhost.localdomain>

I seriously doubt that is the intended behavior.

It seems to me that is a very serious bug (possibly a relatively recent
one, as well.)

It is unfortunate that we don't have  test that would detect that.  If 
you have the energy, please write such a test (probably based on code
you
already have.)

If you can't do it, I will be able to do it, but not until later this
week.


On Sat, 2006-01-28 at 14:37 -0500, Ian Eslick wrote:

> This was not clear in the documentation, but secondary db's appear to
> have the following behavior.
> 
> btree - an indexed btree
> inverted - an secondary index of btree
> fn - A function that computes the secondary key given the value given to
> btree
> 
> Write pkey/value1 into btree
> inverted has fn(value1)/pkey
> 
> Change the association of pkey by writing
> pkey/value2 into btree
> 
> inverted now has:
> fn(value1)/pkey
> fn(value2)/pkey
> 
> Namely, each secondary index contains the entire history of the values
> of pkey and associates
> them with pkey.  If you are sorting and searching objects by using
> secondary indices this is
> not very useful. 
> 
> Is this the intended functionality?  This could be a setup problem on my
> end or the way secondary
> indices were intended to work.  In which case it makes sense for me to
> make sure that when
> a secondary value changes, the original secondary key/value pair is
> deleted from the secondary
> index.  I just discovered this and the uniqueness of mappings to pkey in
> the above example is
> crucial for my application.
> 
> Thank you,
> Ian
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060128/d1430604/attachment.html>

From eslick at csail.mit.edu  Sun Jan 29 00:31:51 2006
From: eslick at csail.mit.edu (Ian Eslick)
Date: Sat, 28 Jan 2006 19:31:51 -0500
Subject: [elephant-devel] Secondary DB behavior
In-Reply-To: <1138487453.8907.581.camel@localhost.localdomain>
References: <43DBC7F1.6010108@csail.mit.edu>
	<1138487453.8907.581.camel@localhost.localdomain>
Message-ID: <43DC0CF7.4010104@csail.mit.edu>

I looked into the code further and the current bdb-controller is
manually maintaining the association between the btree database and the
indices secondary database.  (It's opened with
sleepycat::db-fake-associate).  This means that lisp code is determining
what values are written into the secondary indices.  sleepycat still
maintains the association for reads against the indices-assoc db so you
get the primary btree key/value pair on reads to a secondary index and
proper cursor read behavior, but it doesn't take care of cleaning out
prior references to the primary key in the updated secondary index. 

(I assume this change was to avoid allowing people to write arbitrary
lisp code that was called via a callback from the berkeley code?)

What we really want is an inverted index where the value is guaranteed
to be unique, but not the keys so we can rapidly index to the primary
value via the non-unique keys.  I couldn't find any place in the
sleepycat docs where it clarified this.  Without having the ability to
lookup entries by the value (inverted index) I don't know if this is
easily possible or not...secondary indices would have to be dual
structures, one containing the key-ordered btree and another, perhaps a
hash, that maps primary keys onto btree entries.

I worked around this for now by just removing the primary key-value pair
and then rewriting it after updating with the new slot values so that
all the secondary indices delete their references based on the prior
value and then update with the new value, but this is an expensive
solution in the long term.  I can do better writing a custom version of
(setf get-value) for my class indices, but fixing the underlying problem
seems like a better solution.

I'm not sure exactly how to integrate with the RT code yet, but while
I'm figuring that out for my index tests here is a test case.

(defun mul-key (skey pkey val)
    (if (numberp val)
        (values t (* val 2))
        (values nil)))

(defun test-secondary-index ()
    (let* ((primary (make-indexed-btree))
             (secondary (add-index primary :index-name 'test
                                                          :key-form 'mul-key
                                                          :populate nil)))
        (setf (get-value 'key primary) 2)
        (assert (eq (get-value 4 secondary) 2))  ;; this should work
        (setf (get-value 'key primary) 4)
        (assert (eq (get-value 8 secondary) 4))  ;; good so far, this
should also work
        (assert (eq (get-value 4 secondary) nil)) ;; ** bug **  this
assert should succeed, but doesn't
         ))

I reproduced this phenomenon on the current HEAD (0.5.0 release
candidate). 

This is only an issue when you update a secondary index based on a
varying value in the primary index value.  For example, if I compute a
secondary index key based on a slot value of an object in the primary
index, then I overwrite that value, I get the duplicated key-pkey pairs
in the secondary index.  I think this is a feature because the current
code clearly has no provision for this behavior.  However it's a feature
I think we'd like to think about carefully and document clearly if we
don't change it.

Ian

Robert L. Read wrote:
> I seriously doubt that is the intended behavior.
>
> It seems to me that is a very serious bug (possibly a relatively
> recent one, as well.)
>
> It is unfortunate that we don't have  test that would detect that.  If
> you have the energy, please write such a test (probably based on code you
> already have.)
>
> If you can't do it, I will be able to do it, but not until later this
> week.
>
>
> On Sat, 2006-01-28 at 14:37 -0500, Ian Eslick wrote:
>> This was not clear in the documentation, but secondary db's appear to
>> have the following behavior.
>>
>> btree - an indexed btree
>> inverted - an secondary index of btree
>> fn - A function that computes the secondary key given the value given to
>> btree
>>
>> Write pkey/value1 into btree
>> inverted has fn(value1)/pkey
>>
>> Change the association of pkey by writing
>> pkey/value2 into btree
>>
>> inverted now has:
>> fn(value1)/pkey
>> fn(value2)/pkey
>>
>> Namely, each secondary index contains the entire history of the values
>> of pkey and associates
>> them with pkey.  If you are sorting and searching objects by using
>> secondary indices this is
>> not very useful. 
>>
>> Is this the intended functionality?  This could be a setup problem on my
>> end or the way secondary
>> indices were intended to work.  In which case it makes sense for me to
>> make sure that when
>> a secondary value changes, the original secondary key/value pair is
>> deleted from the secondary
>> index.  I just discovered this and the uniqueness of mappings to pkey in
>> the above example is
>> crucial for my application.
>>
>> Thank you,
>> Ian
>> _______________________________________________
>> elephant-devel site list
>> elephant-devel at common-lisp.net <mailto:elephant-devel at common-lisp.net>
>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>     


From root at common-lisp.net  Mon Jan 30 08:15:15 2006
From: root at common-lisp.net (root)
Date: Mon, 30 Jan 2006 02:15:15 -0600 (CST)
Subject: [elephant-devel] Auto-nag: elephant link verifier failed
Message-ID: <20060130081515.A53FE510E@common-lisp.net>

You are being nagged on because your project's name showed
up in the nightly Checkbot run.  Checkbot checks for broken links etc.
This probably means that you are either pointing to a broken link or
that someone is pointing to a broken link on your site.  Or something
else, it check's a bunch of stuff.  Update your webpages or you shall
be nagged again next week.

To find out what's wrong with your webpages, please consult:

  http://common-lisp.net/checkbot/checkbot-common-lisp.net.html

Any questions?  You got this email but your webpage doesn't seem to be
listed on the checkbot webpage?  You can reach the author of this
cronjob at clo-devel at common-lisp.net

Thanks!