From florian.obser at t-online.de  Mon Nov  7 16:40:22 2005
From: florian.obser at t-online.de (Florian Obser)
Date: Mon, 07 Nov 2005 17:40:22 +0100
Subject: [elephant-devel] DB_NOTFOUND and Unknown error: -5
Message-ID: <436F8376.3030001@t-online.de>

hi,
I tried the tutorial on
http://common-lisp.net/project/elephant/doc/Tutorial.html with SBCL
0.8.16 and libdb4.2 from debian sarge and elephant 0.2.1.

Opening the store results in:

CL-USER> (open-store "/tmp/testdb1")
-----------------------------------------
Berkeley DB error: DB_NOTFOUND: No matching key/data pair found
   [Condition of type SLEEPYCAT:DB-ERROR]

Restarts:
  0: [ABORT] Abort handling SLIME request.
  1: [ABORT] Reduce debugger level (leaving debugger, returning to
toplevel).
  2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.

Backtrace:
  0: ((OPEN-CONTROLLER (STORE-CONTROLLER)) #<unavailable argument>
#<unavailable argument> #<unavailable argument> #<unavailable argument>)
  1: (SB-INT:EVAL-IN-LEXENV 2 (OPEN-STORE "/tmp/testdb1")
#S(SB-KERNEL:LEXENV :FUNS NIL :VARS NIL :BLOCKS NIL :TAGS NIL
:TYPE-RESTRICTIONS NIL :LAMBDA NIL :CLEANUP NIL :HANDLED-CONDITIONS NIL
:DISABLED-PACKAGE-LOCKS NIL :POLICY ((SPEED . 1) (SPACE . 1) (SAFETY .
1) (INHIBIT-WARNINGS . 1) (DEBUG . 1) (COMPILATION-SPEED . 1))))[:EXTERNAL]
  2: ("hairy arg processor for SWANK::EVAL-REGION" "(open-store
\"/tmp/testdb1\")
" T)
  3: ("#'(LAMBDA NIL (MULTIPLE-VALUE-BIND # # ...))")
[...]
-----------------------------------------
(The backtrace continues after this point but I assume it's only slime
related)
I tried different locations for the store directory (current working
directory or /tmp/ in this case. I found the suggestion that this libdb
error message might be related to permissions on the directory so I
tried a chmod 777. In all cases after running the open-store using a
fresh empty directory the following files are in the directory:
__db.001
__db.002
__db.003
__db.004
__db.005
%ELEPHANT
log.0000000001

I aborted (using 0) the backtrace and tried to continue the tutorial.
Inserting and retrieving of strings works fine, but when I tried to
create a persistence-class instance I get a new error:
CL-USER> (defclass my-persistent-class ()
                ((slot1 :accessor slot1)
	         (slot2 :accessor slot2))
		         (:metaclass persistent-metaclass))

CL-USER> (setq foo (make-instance 'my-persistent-class))
-----------------------------------------------------------
Berkeley DB error: Unknown error: -5
   [Condition of type SLEEPYCAT:DB-ERROR]

Restarts:
  0: [ABORT] Abort handling SLIME request.
  1: [ABORT] Reduce debugger level (leaving debugger, returning to
toplevel).
  2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.

Backtrace:
  0: (SLEEPYCAT::NEXT-COUNTER #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
#X0827A1E8 :TYPE (* T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X0827AAB8
:TYPE (* T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X00000000 :TYPE (*
T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X08278FF0 :TYPE (* (SIGNED
8))> 12 #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X0827A1D0 :TYPE (*
(SIGNED 8))> 16)
  1: ((INITIALIZE-INSTANCE :BEFORE (PERSISTENT)) #<unavailable argument>
#<unavailable argument> #<MY-PERSISTENT-CLASS {939EA59}> NIL)
  2: ("#'(LAMBDA (SB-PCL::.PV-CELL. SB-PCL::.NEXT-METHOD-CALL.
SB-PCL::.ARG0. ...) (DECLARE #) ...)" #<unavailable argument>
#<unavailable argument> #<MY-PERSISTENT-CLASS {939EA59}> NIL)
  3: ((MAKE-INSTANCE (CLASS)) #<unavailable argument> #<unavailable
argument> #<PERSISTENT-METACLASS MY-PERSISTENT-CLASS> NIL)
-----------------------------------------------------------

and after this I can't continue working with the object:
CL-USER> (add-to-root "foo" foo)
---------------------
The variable FOO is unbound.
   [Condition of type UNBOUND-VARIABLE]

Restarts:
  0: [ABORT] Abort handling SLIME request.
  1: [ABORT] Reduce debugger level (leaving debugger, returning to
toplevel).
  2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.

Backtrace:
  0: (SB-INT:EVAL-IN-LEXENV 2 FOO #S(SB-KERNEL:LEXENV :FUNS NIL :VARS
NIL :BLOCKS NIL :TAGS NIL :TYPE-RESTRICTIONS NIL :LAMBDA NIL :CLEANUP
NIL :HANDLED-CONDITIONS NIL :DISABLED-PACKAGE-LOCKS NIL :POLICY ((SPEED
. 1) (SPACE . 1) (SAFETY . 1) (INHIBIT-WARNINGS . 1) (DEBUG . 1)
(COMPILATION-SPEED . 1))))[:EXTERNAL]
  1: (SB-INT:EVAL-IN-LEXENV 2 (ADD-TO-ROOT "foo" FOO)
---------------------

Any suggestions what might be going wrong here? (I think I understand
the third backtrace, but what about the first two?)
- Florian
-- 
If c++ had garbage collection most programs would delete themself upon
execution.


From read at robertlread.net  Mon Nov  7 19:29:07 2005
From: read at robertlread.net (Robert L. Read)
Date: Mon, 07 Nov 2005 13:29:07 -0600
Subject: [elephant-devel] DB_NOTFOUND and Unknown error: -5
In-Reply-To: <436F8376.3030001@t-online.de>
References: <436F8376.3030001@t-online.de>
Message-ID: <1131391747.5391.69.camel@localhost.localdomain>

Dear Florian,

    I just tried this on:
1) SBCL 8.18
2) Fedora Core 3
3) libdb4.2
4) elephant0.2.1

I didn't have any problem, so I gained no information from that.

The backtrace information you provide does not give me any hints, but 
then, even though I am now the maintainer, I might not have enough
experience with it.

I'd like to be as helpful as possible.  Are you dedicated to your
particular version setup?
For example, would you be willing to try the cvs branch SQL-BACK-END on
SBCL 9.5? That
is what I have mostly been working on, and will become Elephant 0.3.0
unless someone
finds a problem with it by next week.   It would be MUCH easier for me
to debug this
problem if you were on that set-up.  However, I know in some way that is
a big change.

As an alternative, you could run the tests, and send me the output of
that; that might 
provide some help.  

The tests for version 0.2.1 can be run via:
(asdf:operate 'asdf:load-op :elephant-tests)
(in-package "ELEPHANT-TESTS")
(do-all-tests)

Perhaps this is obvious, but the 0.2.1 system does work, at least under
some 
configurations.

Also, when you untar it, in the "tests" directory there is aleady a
directory "testdb".
I wonder if you would have this problem if you used that directory?

As a further guess as to what might be wrong based on the backtraces,
the third
one is just because the "setq foo" failed due to the second failure.
The first and 
second failures look like they are indeed failures in the BerkeleyDB
system or 
right at the interface to it.  I don't mean by that that there is a bug
in BerkeleyDB, just
that this problem has something to do with the way BerkeleyDB is set up
compared
to your installation, rather than being a simple bug somewhere in
Elephant LISP code.

Hoping that somehow helps,



On Mon, 2005-11-07 at 17:40 +0100, Florian Obser wrote:

> hi,
> I tried the tutorial on
> http://common-lisp.net/project/elephant/doc/Tutorial.html with SBCL
> 0.8.16 and libdb4.2 from debian sarge and elephant 0.2.1.
> 
> Opening the store results in:
> 
> CL-USER> (open-store "/tmp/testdb1")
> -----------------------------------------
> Berkeley DB error: DB_NOTFOUND: No matching key/data pair found
>    [Condition of type SLEEPYCAT:DB-ERROR]
> 
> Restarts:
>   0: [ABORT] Abort handling SLIME request.
>   1: [ABORT] Reduce debugger level (leaving debugger, returning to
> toplevel).
>   2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
> 
> Backtrace:
>   0: ((OPEN-CONTROLLER (STORE-CONTROLLER)) #<unavailable argument>
> #<unavailable argument> #<unavailable argument> #<unavailable argument>)
>   1: (SB-INT:EVAL-IN-LEXENV 2 (OPEN-STORE "/tmp/testdb1")
> #S(SB-KERNEL:LEXENV :FUNS NIL :VARS NIL :BLOCKS NIL :TAGS NIL
> :TYPE-RESTRICTIONS NIL :LAMBDA NIL :CLEANUP NIL :HANDLED-CONDITIONS NIL
> :DISABLED-PACKAGE-LOCKS NIL :POLICY ((SPEED . 1) (SPACE . 1) (SAFETY .
> 1) (INHIBIT-WARNINGS . 1) (DEBUG . 1) (COMPILATION-SPEED . 1))))[:EXTERNAL]
>   2: ("hairy arg processor for SWANK::EVAL-REGION" "(open-store
> \"/tmp/testdb1\")
> " T)
>   3: ("#'(LAMBDA NIL (MULTIPLE-VALUE-BIND # # ...))")
> [...]
> -----------------------------------------
> (The backtrace continues after this point but I assume it's only slime
> related)
> I tried different locations for the store directory (current working
> directory or /tmp/ in this case. I found the suggestion that this libdb
> error message might be related to permissions on the directory so I
> tried a chmod 777. In all cases after running the open-store using a
> fresh empty directory the following files are in the directory:
> __db.001
> __db.002
> __db.003
> __db.004
> __db.005
> %ELEPHANT
> log.0000000001
> 
> I aborted (using 0) the backtrace and tried to continue the tutorial.
> Inserting and retrieving of strings works fine, but when I tried to
> create a persistence-class instance I get a new error:
> CL-USER> (defclass my-persistent-class ()
>                 ((slot1 :accessor slot1)
> 	         (slot2 :accessor slot2))
> 		         (:metaclass persistent-metaclass))
> 
> CL-USER> (setq foo (make-instance 'my-persistent-class))
> -----------------------------------------------------------
> Berkeley DB error: Unknown error: -5
>    [Condition of type SLEEPYCAT:DB-ERROR]
> 
> Restarts:
>   0: [ABORT] Abort handling SLIME request.
>   1: [ABORT] Reduce debugger level (leaving debugger, returning to
> toplevel).
>   2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
> 
> Backtrace:
>   0: (SLEEPYCAT::NEXT-COUNTER #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
> #X0827A1E8 :TYPE (* T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X0827AAB8
> :TYPE (* T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X00000000 :TYPE (*
> T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X08278FF0 :TYPE (* (SIGNED
> 8))> 12 #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X0827A1D0 :TYPE (*
> (SIGNED 8))> 16)
>   1: ((INITIALIZE-INSTANCE :BEFORE (PERSISTENT)) #<unavailable argument>
> #<unavailable argument> #<MY-PERSISTENT-CLASS {939EA59}> NIL)
>   2: ("#'(LAMBDA (SB-PCL::.PV-CELL. SB-PCL::.NEXT-METHOD-CALL.
> SB-PCL::.ARG0. ...) (DECLARE #) ...)" #<unavailable argument>
> #<unavailable argument> #<MY-PERSISTENT-CLASS {939EA59}> NIL)
>   3: ((MAKE-INSTANCE (CLASS)) #<unavailable argument> #<unavailable
> argument> #<PERSISTENT-METACLASS MY-PERSISTENT-CLASS> NIL)
> -----------------------------------------------------------
> 
> and after this I can't continue working with the object:
> CL-USER> (add-to-root "foo" foo)
> ---------------------
> The variable FOO is unbound.
>    [Condition of type UNBOUND-VARIABLE]
> 
> Restarts:
>   0: [ABORT] Abort handling SLIME request.
>   1: [ABORT] Reduce debugger level (leaving debugger, returning to
> toplevel).
>   2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
> 
> Backtrace:
>   0: (SB-INT:EVAL-IN-LEXENV 2 FOO #S(SB-KERNEL:LEXENV :FUNS NIL :VARS
> NIL :BLOCKS NIL :TAGS NIL :TYPE-RESTRICTIONS NIL :LAMBDA NIL :CLEANUP
> NIL :HANDLED-CONDITIONS NIL :DISABLED-PACKAGE-LOCKS NIL :POLICY ((SPEED
> . 1) (SPACE . 1) (SAFETY . 1) (INHIBIT-WARNINGS . 1) (DEBUG . 1)
> (COMPILATION-SPEED . 1))))[:EXTERNAL]
>   1: (SB-INT:EVAL-IN-LEXENV 2 (ADD-TO-ROOT "foo" FOO)
> ---------------------
> 
> Any suggestions what might be going wrong here? (I think I understand
> the third backtrace, but what about the first two?)
> - Florian

----
Robert L. Read, PhD                                     read &T
robertlread.net
Consider visiting Progressive Engineering:
http://robertlread.net/pe
In Austin: 912-8593                                        "Think
globally, Act locally." -- RBF


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051107/7dd1f6fb/attachment.html>

From read at robertlread.net  Mon Nov  7 21:05:34 2005
From: read at robertlread.net (Robert L. Read)
Date: Mon, 07 Nov 2005 15:05:34 -0600
Subject: [elephant-devel] DB_NOTFOUND and Unknown error: -5
In-Reply-To: <436FBCBE.5030808@t-online.de>
References: <436F8376.3030001@t-online.de>
	<1131391747.5391.69.camel@localhost.localdomain>
	<436FBCBE.5030808@t-online.de>
Message-ID: <1131397534.5391.81.camel@localhost.localdomain>

On Mon, 2005-11-07 at 21:44 +0100, Florian Obser wrote:


> Will sbcl 0.9.5 be a requirement for elephant 0.3.0 or do you think i
> can stick with 0.8.16? Are the unicode-issues resolved?
> I'm going to try the cvs-version as soon as I find the time, probably
> next weekend.

No, 0.3.0 should still work with SBCL 0.8.X.  In fact thanks to Juho
Snellman I recently committed
the necessary changes to the SQL-BACK-END branch to deal with these.

The SBCL team obsoleted a low-level memory function between 8 and 9; I'm
not sure why but
you have to test what version you are in and call the correct thing.  In
theory I have done it correctly
so it will work under both versions.


> 
> > 
> > As an alternative, you could run the tests, and send me the output of
> > that; that might
> > provide some help. 
> > 
> > The tests for version 0.2.1 can be run via:
> > (asdf:operate 'asdf:load-op :elephant-tests)
> > (in-package "ELEPHANT-TESTS")
> > (do-all-tests)
> 
> A quick test failed in the first line:
> ------------------------------
> component "elephant-tests" not found
>    [Condition of type ASDF:MISSING-COMPONENT]
> ------------------------------

This is probably because you have not placed a symbolic link to
blahblah/elephant-0.2.1/elephant-tests.asd in
the "asdf::*central-repository*" directory.

I personally use  "/usr/local/share/lisp" for this, and have the
following three lines in my /etc/sbclrc file,
to make sure SBCL knows where to find my ASDF systems:

(defparameter *ASDF-PACKAGE-HOME* "/usr/local/share/lisp/")
(require 'asdf)
(setq asdf:*central-registry* (cons *asdf-package-home* asdf:*central-
registry*))


> 
> This probably means that i messed up the elephant setup. If you're
> really interested in the output I can try to get it running, but if not
> I'm going to ignore the problem and switch to the cvs version and sql.

I'd rather deal with the CVS version; that does NOT mean that you should
use the SQL version.
The SQL-BACK-END branch works equally well with the BerkeleyDB or the
SQL version; and 
you can elect which to use.  In fact the BerkeleyDB is about 5 times
faster than the SQL version.

Nonetheless, the 0.3.0 version will be the latest version and will be
better maintained.

No, the Unicode and UTF-8 issues are not resolved yet.  I know they
concern the low-level
serializer, that constructs a byte stream serialization of the lisp
objects.  Unfortunately in
some way this is sensitive to the character definition.  So I understand
the general problem,
but have not had time to fix it; I intend to do it fairly soon, as I
need Unicode stuff for my
own projects.



> Probably. When I searched google for "DB_NOTFOUND: No matching key/data
> pair found" I got many hits related to many different projects so I
> think that it's not that easy to set up libdb correctly.
> 
> > 
> > Hoping that somehow helps,
> 
> Yes, I think so. Thank you for the quick response.
> 
> - Florian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051107/8fb6117f/attachment.html>

From florian.obser at t-online.de  Mon Nov  7 20:44:46 2005
From: florian.obser at t-online.de (Florian Obser)
Date: Mon, 07 Nov 2005 21:44:46 +0100
Subject: [elephant-devel] DB_NOTFOUND and Unknown error: -5
In-Reply-To: <1131391747.5391.69.camel@localhost.localdomain>
References: <436F8376.3030001@t-online.de>
	<1131391747.5391.69.camel@localhost.localdomain>
Message-ID: <436FBCBE.5030808@t-online.de>

Dear Robert,

Robert L. Read wrote:
> Dear Florian,
[...]
>  
> I'd like to be as helpful as possible.  Are you dedicated to your
> particular version setup?
> For example, would you be willing to try the cvs branch SQL-BACK-END on
> SBCL 9.5? That
> is what I have mostly been working on, and will become Elephant 0.3.0
> unless someone
> finds a problem with it by next week.   It would be MUCH easier for me
> to debug this
> problem if you were on that set-up.  However, I know in some way that is
> a big change.

I'm not really dedicated to my setup, but I like to get as much software
out of the debian repository. This is by no means mission critical and I
consider my work with elephant and lisp recreational.
I considered switching to the sql backend myself but since the tutorial
mentioned that the sleepycat backend is better tested I tried that first.

Will sbcl 0.9.5 be a requirement for elephant 0.3.0 or do you think i
can stick with 0.8.16? Are the unicode-issues resolved?
I'm going to try the cvs-version as soon as I find the time, probably
next weekend.

> 
> As an alternative, you could run the tests, and send me the output of
> that; that might
> provide some help. 
> 
> The tests for version 0.2.1 can be run via:
> (asdf:operate 'asdf:load-op :elephant-tests)
> (in-package "ELEPHANT-TESTS")
> (do-all-tests)

A quick test failed in the first line:
------------------------------
component "elephant-tests" not found
   [Condition of type ASDF:MISSING-COMPONENT]
------------------------------

This probably means that i messed up the elephant setup. If you're
really interested in the output I can try to get it running, but if not
I'm going to ignore the problem and switch to the cvs version and sql.

> 
> Perhaps this is obvious, but the 0.2.1 system does work, at least under
> some
> configurations.
> 
> Also, when you untar it, in the "tests" directory there is aleady a
> directory "testdb".
> I wonder if you would have this problem if you used that directory?

That doesn't change anything, probably because the directory is empty
(except for the CVS directory).

> 
> As a further guess as to what might be wrong based on the backtraces,
> the third
> one is just because the "setq foo" failed due to the second failure. 

Yes that's what I guessed.

> The first and
> second failures look like they are indeed failures in the BerkeleyDB
> system or
> right at the interface to it.  I don't mean by that that there is a bug
> in BerkeleyDB, just
> that this problem has something to do with the way BerkeleyDB is set up
> compared
> to your installation, rather than being a simple bug somewhere in
> Elephant LISP code.

Probably. When I searched google for "DB_NOTFOUND: No matching key/data
pair found" I got many hits related to many different projects so I
think that it's not that easy to set up libdb correctly.

> 
> Hoping that somehow helps,

Yes, I think so. Thank you for the quick response.

- Florian


From read at robertlread.net  Tue Nov  8 21:04:48 2005
From: read at robertlread.net (Robert L. Read)
Date: Tue, 08 Nov 2005 15:04:48 -0600
Subject: [elephant-devel] DB_NOTFOUND and Unknown error: -5
In-Reply-To: <437110C3.6020603@t-online.de>
References: <436F8376.3030001@t-online.de>
	<1131391747.5391.69.camel@localhost.localdomain>
	<436FBCBE.5030808@t-online.de>
	<1131397534.5391.81.camel@localhost.localdomain>
	<437110C3.6020603@t-online.de>
Message-ID: <1131483888.5391.189.camel@localhost.localdomain>

On Tue, 2005-11-08 at 21:55 +0100, Florian Obser wrote:

> I found the time for some more tests today. Now it works!
> 

Excellent!

> 
> I first build libdb 4.2 from sources and tried the CVS version. After
> editing the makefile gcc produced a lot of warnings and errors about
> missing symbols. Since the makefile hinted that you (or whoever checked
> in the makefile) was using libdb 4.3 I changed to the latest libdb 4.3
> and suddenly everything worked.

Darn --- I hope I haven't done something that requires libdb 4.3 without
knowing it, although I 
suppose it is good that it works with the latest version.  

Do you mean that you are using 0.2.1 or the SQL-BACK-END branch of the
CVS repository
or the latest version (on the "trunk", as opposed to the "branch") of
the CVS repository for Elephant?


----
Robert L. Read, PhD                                     read &T
robertlread.net
Consider visiting Progressive Engineering:
http://robertlread.net/pe
In Austin: 912-8593                                        "Think
globally, Act locally." -- RBF


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051108/b8da8208/attachment.html>

From read at robertlread.net  Tue Nov  8 21:23:19 2005
From: read at robertlread.net (Robert L. Read)
Date: Tue, 08 Nov 2005 15:23:19 -0600
Subject: [elephant-devel] DB_NOTFOUND and Unknown error: -5
In-Reply-To: <437115DF.2010505@t-online.de>
References: <436F8376.3030001@t-online.de>
	<1131391747.5391.69.camel@localhost.localdomain>
	<436FBCBE.5030808@t-online.de>
	<1131397534.5391.81.camel@localhost.localdomain>
	<437110C3.6020603@t-online.de>
	<1131483888.5391.189.camel@localhost.localdomain>
	<437115DF.2010505@t-online.de>
Message-ID: <1131484999.5391.204.camel@localhost.localdomain>

That gets you the latest version on the trunk (which might have a few
bug fixes over 0.2.1).

To get the SQL-BACK-END branch, you would to the same, except:

co -r SQL-BACK-END elephant

which colloquially means "Check out the revision name by SQL-BACK-END
and set the sticky tags
to the that branch so that this directory will point to that branch
forever or until I change it with a CVS command."

On Tue, 2005-11-08 at 22:17 +0100, Florian Obser wrote:

> Robert L. Read wrote:
> 
> > Do you mean that you are using 0.2.1 or the SQL-BACK-END branch of the
> > CVS repository
> > or the latest version (on the "trunk", as opposed to the "branch") of
> > the CVS repository for Elephant?
> 
> cvs -z3 -d
> :pserver:anonymous:anonymous at common-lisp.net:/project/elephant/cvsroot
> co elephant
> 
> you tell me :)
> 
> - florian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051108/2bee5e9e/attachment.html>

From florian.obser at t-online.de  Tue Nov  8 21:17:19 2005
From: florian.obser at t-online.de (Florian Obser)
Date: Tue, 08 Nov 2005 22:17:19 +0100
Subject: [elephant-devel] DB_NOTFOUND and Unknown error: -5
In-Reply-To: <1131483888.5391.189.camel@localhost.localdomain>
References: <436F8376.3030001@t-online.de>	
	<1131391747.5391.69.camel@localhost.localdomain>	
	<436FBCBE.5030808@t-online.de>	
	<1131397534.5391.81.camel@localhost.localdomain>	
	<437110C3.6020603@t-online.de>
	<1131483888.5391.189.camel@localhost.localdomain>
Message-ID: <437115DF.2010505@t-online.de>

Robert L. Read wrote:

> Do you mean that you are using 0.2.1 or the SQL-BACK-END branch of the
> CVS repository
> or the latest version (on the "trunk", as opposed to the "branch") of
> the CVS repository for Elephant?

cvs -z3 -d
:pserver:anonymous:anonymous at common-lisp.net:/project/elephant/cvsroot
co elephant

you tell me :)

- florian


From florian.obser at t-online.de  Tue Nov  8 20:55:31 2005
From: florian.obser at t-online.de (Florian Obser)
Date: Tue, 08 Nov 2005 21:55:31 +0100
Subject: [elephant-devel] DB_NOTFOUND and Unknown error: -5
In-Reply-To: <1131397534.5391.81.camel@localhost.localdomain>
References: <436F8376.3030001@t-online.de>	
	<1131391747.5391.69.camel@localhost.localdomain>	
	<436FBCBE.5030808@t-online.de>
	<1131397534.5391.81.camel@localhost.localdomain>
Message-ID: <437110C3.6020603@t-online.de>

I found the time for some more tests today. Now it works!

Robert L. Read wrote:
> On Mon, 2005-11-07 at 21:44 +0100, Florian Obser wrote:
> 
>>Will sbcl 0.9.5 be a requirement for elephant 0.3.0 or do you think i
>>can stick with 0.8.16? Are the unicode-issues resolved?
>>I'm going to try the cvs-version as soon as I find the time, probably
>>next weekend.
>>
> No, 0.3.0 should still work with SBCL 0.8.X.  In fact thanks to Juho
> Snellman I recently committed
> the necessary changes to the SQL-BACK-END branch to deal with these.
> 
> The SBCL team obsoleted a low-level memory function between 8 and 9; I'm
> not sure why but
> you have to test what version you are in and call the correct thing.  In
> theory I have done it correctly
> so it will work under both versions.

I first build libdb 4.2 from sources and tried the CVS version. After
editing the makefile gcc produced a lot of warnings and errors about
missing symbols. Since the makefile hinted that you (or whoever checked
in the makefile) was using libdb 4.3 I changed to the latest libdb 4.3
and suddenly everything worked.

> 
>>
[...]
>>
>>A quick test failed in the first line:
>>------------------------------
>>component "elephant-tests" not found
>>   [Condition of type ASDF:MISSING-COMPONENT]
>>------------------------------
>>
> This is probably because you have not placed a symbolic link to
> blahblah/elephant-0.2.1/elephant-tests.asd in
> the "asdf::*central-repository*" directory.
> 
> I personally use  "/usr/local/share/lisp" for this, and have the
> following three lines in my /etc/sbclrc file,
> to make sure SBCL knows where to find my ASDF systems:
> 
> (defparameter *ASDF-PACKAGE-HOME* "/usr/local/share/lisp/")
> (require 'asdf)
> (setq asdf:*central-registry* (cons *asdf-package-home*
> asdf:*central-registry*))

Yes, of course. I missed that one last night, but it was too late in the
night. I tried that one too, before switching to libdb 4.3 and cvs (so
in fact I was using the debian packets and elephant-0.2.1.tar.gz) but it
failed with DB_NOTFOUND

- Florian


From dankna at accela.net  Tue Nov 22 09:27:51 2005
From: dankna at accela.net (Dan Knapp)
Date: Tue, 22 Nov 2005 04:27:51 -0500
Subject: [elephant-devel] SQLite 3 support - fixed
Message-ID: <F1010D3E-1547-4877-8405-AC47F77B5920@accela.net>

   Enclosed, find my patch against SQL-BACK-END to make Elephant work  
with SQLite 3.  I have also attached the detailed notes I took while  
debugging, since they'll probably be of interest to Robert.  There  
may be a couple small changes in the patch not explained in the  
notes... ask me if it's unclear.  The notes are stream-of- 
consciousness stuff, so don't expect them to be concise or direct.

   The main problem was a bug in how slot values were decoded.  The  
actual connecting to the database and exchanging sql with it went  
without a hitch.

   I needed the Berkely DB support to continue working, so that I  
could migrate my data, so I fixed one small bug there and changed the  
library path to where fink installs it - I used the "db43*" packages.

   I tested this all on OS X:

Darwin mt-kukulu.local 8.2.0 Darwin Kernel Version 8.2.0: Fri Jun 24  
17:46:54 PDT 2005; root:xnu-792.2.4.obj~3/RELEASE_PPC Power Macintosh  
powerpc

   I used clsql-3.4.6, uffi-1.5.7, and the version of sqlite which  
came with the OS, 3.0.8.6.

   Tell me how it works for you!


??

-- Dan Knapp

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051122/6fa798fa/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: diff.txt
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051122/6fa798fa/attachment.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051122/6fa798fa/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dnk-changes.txt
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051122/6fa798fa/attachment-0001.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051122/6fa798fa/attachment-0002.html>

From read at robertlread.net  Tue Nov 22 13:13:06 2005
From: read at robertlread.net (Robert L. Read)
Date: Tue, 22 Nov 2005 07:13:06 -0600
Subject: [elephant-devel] SQLite 3 support - fixed
In-Reply-To: <F1010D3E-1547-4877-8405-AC47F77B5920@accela.net>
References: <F1010D3E-1547-4877-8405-AC47F77B5920@accela.net>
Message-ID: <1132665187.5301.184.camel@localhost.localdomain>

Dear Dan,
    Thanks!  This is excellent!  If we can get this to work, it will be
significant step forward in the 
overall value of the Elephant package.
    I will make this my highest priority upon returning from the
Thanksgiving holiday next week.
In fact, I had meant to release SQL-BACK-END as Elephant 0.3 by now;
this give me further
incentive to do that, although I will evaluate your patch first, as it
would be very nice to 
release Postgres and Sql-Lite support at the same time.

    


On Tue, 2005-11-22 at 04:27 -0500, Dan Knapp wrote:

>   Enclosed, find my patch against SQL-BACK-END to make Elephant work
> with SQLite 3.  I have also attached the detailed notes I took while
> debugging, since they'll probably be of interest to Robert.  There may
> be a couple small changes in the patch not explained in the notes...
> ask me if it's unclear.  The notes are stream-of-consciousness stuff,
> so don't expect them to be concise or direct.
> 
> 
>   The main problem was a bug in how slot values were decoded.  The
> actual connecting to the database and exchanging sql with it went
> without a hitch.
> 
> 
>   I needed the Berkely DB support to continue working, so that I could
> migrate my data, so I fixed one small bug there and changed the
> library path to where fink installs it - I used the "db43*" packages.
> 
> 
>   I tested this all on OS X:
> 
> 
> Darwin mt-kukulu.local 8.2.0 Darwin Kernel Version 8.2.0: Fri Jun 24
> 17:46:54 PDT 2005; root:xnu-792.2.4.obj~3/RELEASE_PPC Power Macintosh
> powerpc
> 
> 
>   I used clsql-3.4.6, uffi-1.5.7, and the version of sqlite which came
> with the OS, 3.0.8.6.
> 
> 
>   Tell me how it works for you!
> 
> 
> 
> 
> 
> 
> 
> -- Dan Knapp
> 
> 
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051122/68b02f1f/attachment.html>

From read at robertlread.net  Tue Nov 22 15:50:33 2005
From: read at robertlread.net (Robert L. Read)
Date: Tue, 22 Nov 2005 09:50:33 -0600
Subject: [elephant-devel] SQLite 3 support - fixed
In-Reply-To: <F1010D3E-1547-4877-8405-AC47F77B5920@accela.net>
References: <F1010D3E-1547-4877-8405-AC47F77B5920@accela.net>
Message-ID: <1132674633.5301.194.camel@localhost.localdomain>

Dear Dan,
    Your patch works for me.
    By that I mean:
    1)  It doesn't break any of the existing tests,
    2)  It passes all of the tests except the migration tests (and that
is a bug in the way 
I wrote the tests, almost certainly.)
    3)  Although I have not specific test for it, the problem of
treating "nil"s as unbound 
was one that I had noticed and wondered about, it is very nice that you
have fixed it.

Unfortunately, your patch has revealed some weaknesses in my tests, and
also in 
the way that I construct the controllers (as you no doubt noted, I had
hard-wired them to Postgres
with a comment, but your successful use of SQLite 3 now makes it
essential that I make 
that more completely configurable.  Also, I have not yet gone through
your patch line-by-line,
but thanks to your debugging notes, I have every confidence that I will
accept them.

I have some questions for you:
1) I assume from you email that you did in fact migrate data from BDB to
SQLite 3?
2) Which LISP are you using?
3) Although I can reconstruct it, I would like a few lines that exhibit
the null/unbound 
problem as you tested it, so that I can build a test case around it.
(This will save me 
some time and give me more confidence.)

So again I thank you for this patch, and since I am on a roll I hope to
continue working 
on this until I can get the branch properly released.



On Tue, 2005-11-22 at 04:27 -0500, Dan Knapp wrote:

>   Enclosed, find my patch against SQL-BACK-END to make Elephant work
> with SQLite 3.  I have also attached the detailed notes I took while
> debugging, since they'll probably be of interest to Robert.  There may
> be a couple small changes in the patch not explained in the notes...
> ask me if it's unclear.  The notes are stream-of-consciousness stuff,
> so don't expect them to be concise or direct.
> 
> 
>   The main problem was a bug in how slot values were decoded.  The
> actual connecting to the database and exchanging sql with it went
> without a hitch.
> 
> 
>   I needed the Berkely DB support to continue working, so that I could
> migrate my data, so I fixed one small bug there and changed the
> library path to where fink installs it - I used the "db43*" packages.
> 
> 
>   I tested this all on OS X:
> 
> 
> Darwin mt-kukulu.local 8.2.0 Darwin Kernel Version 8.2.0: Fri Jun 24
> 17:46:54 PDT 2005; root:xnu-792.2.4.obj~3/RELEASE_PPC Power Macintosh
> powerpc
> 
> 
>   I used clsql-3.4.6, uffi-1.5.7, and the version of sqlite which came
> with the OS, 3.0.8.6.
> 
> 
>   Tell me how it works for you!
> 
> 
> 
> 
> 
> 
> 
> -- Dan Knapp
> 
> 
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051122/e36b71ac/attachment.html>

From read at robertlread.net  Wed Nov 23 20:09:41 2005
From: read at robertlread.net (Robert L. Read)
Date: Wed, 23 Nov 2005 14:09:41 -0600
Subject: [elephant-devel] Elephant 0.3.0 released
Message-ID: <1132776582.5301.227.camel@localhost.localdomain>

Elephant 0.3.0 is now released.

The following capsule has been added to the introduction section of the
documentation, which
has alarge section describing the extensions in this release:

This release, Elephant 0.3, also provides support for relational
backends. It has been tested with Postgres and SQLite 3. It is back-
compatible with any code that ran against previous versions of Elephant,
but also supports simultaneous multi-repository operation and convenient
migration of data between repositories. This hopefully allows decisions
about the prefered back-end storage mechanism to be delayed and changed,
even after an application that uses Elephant is mature.

* * *

This is the work that was done in the branch SQL-BACK-END, which I have
now merged into 
the trunk of the projects CVS repository and tagged ELEPHANT-0-3-0.  (I
have also created a 
tar file and reference in the project homepage:

http://common-lisp.net/project/elephant/


If you choose to download and try this release, please send me a note,
whether you are 
successful or have problems.  At present I am capable of being very
responsive to problems
with this system.


----
Robert L. Read, PhD                                     read &T
robertlread.net
Consider visiting Progressive Engineering:
http://robertlread.net/pe
In Austin: 912-8593                                        "Think
globally, Act locally." -- RBF


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051123/aa6e342b/attachment.html>

From dankna at accela.net  Thu Nov 24 02:14:15 2005
From: dankna at accela.net (Dan Knapp)
Date: Wed, 23 Nov 2005 21:14:15 -0500
Subject: [elephant-devel] Elephant 0.3.0 released
In-Reply-To: <1132776582.5301.227.camel@localhost.localdomain>
References: <1132776582.5301.227.camel@localhost.localdomain>
Message-ID: <2F188632-25DB-492A-8490-4424CD0FF624@accela.net>

> This is the work that was done in the branch SQL-BACK-END, which I  
> have now merged into
> the trunk of the projects CVS repository and tagged  
> ELEPHANT-0-3-0.  (I have also created a
> tar file and reference in the project homepage:

   I hate to point this out, but you actually tagged it  
ELEPHNAT-0-3-0.  Anyway!  I am trying it out
now.  There seem to be some efficiency issues... caching the result  
of make-form-key could actually
save 10% of cpu time in my setup... so I'm going to take a look at  
them once I get it running.

-- Dan Knapp

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051123/c577ae7b/attachment.html>

From dankna at accela.net  Thu Nov 24 02:14:15 2005
From: dankna at accela.net (Dan Knapp)
Date: Wed, 23 Nov 2005 21:14:15 -0500
Subject: [elephant-devel] Elephant 0.3.0 released
In-Reply-To: <1132776582.5301.227.camel@localhost.localdomain>
References: <1132776582.5301.227.camel@localhost.localdomain>
Message-ID: <2F188632-25DB-492A-8490-4424CD0FF624@accela.net>

> This is the work that was done in the branch SQL-BACK-END, which I  
> have now merged into
> the trunk of the projects CVS repository and tagged  
> ELEPHANT-0-3-0.  (I have also created a
> tar file and reference in the project homepage:

   I hate to point this out, but you actually tagged it  
ELEPHNAT-0-3-0.  Anyway!  I am trying it out
now.  There seem to be some efficiency issues... caching the result  
of make-form-key could actually
save 10% of cpu time in my setup... so I'm going to take a look at  
them once I get it running.

-- Dan Knapp

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051123/c577ae7b/attachment-0001.html>

From read at robertlread.net  Sun Nov 27 14:52:21 2005
From: read at robertlread.net (Robert L. Read)
Date: Sun, 27 Nov 2005 08:52:21 -0600
Subject: [elephant-devel] Elephant 0.3.0 released
In-Reply-To: <2F188632-25DB-492A-8490-4424CD0FF624@accela.net>
References: <1132776582.5301.227.camel@localhost.localdomain>
	<2F188632-25DB-492A-8490-4424CD0FF624@accela.net>
Message-ID: <1133103141.5301.252.camel@localhost.localdomain>



>   I hate to point this out, but you actually tagged it ELEPHNAT-0-3-0.
> Anyway!  I am trying it out


Dohh!!!

Sorry about that; I will leave that tag there are put a properly spelled
one on the same set of files
later today.



> now.  There seem to be some efficiency issues... caching the result of
> make-form-key could actually
> save 10% of cpu time in my setup... so I'm going to take a look at
> them once I get it running.

Thank you.  I am generally of the school of not optimizing early; my own
application which drives 
this has never broken a noticable sweat; perhaps just because it is
unpopular.

If you are using a relational database back-end, a huge improvement
would be to modify the 
existing serializer to produce a character-writable stream rather than a
byte stream and eliminate
the base64 encoding that I put in.  The the base64 encoding/decoding
itself is not a problem; 
but it probably doubles or triples the number of bytes that have to be
stored in the database,
and that I/O is probably the dominant cost in the system.

I am not cavalier about this, but I have to choose my battles wisely.
For example, I consider
getting Elephant to work the Unicode based SBCL far more important than
even a 3-fold 
improvement in performance at this point.

However, I encourage you to do more great work, like the bug fix you
made, and I will attempt
to incorporate it promptly.

I also would like to know other peoples experience with the 0.3.0
release.  I tested it thoroughly
in my machine; but that is only one setup.


> 
> 
> 
> -- Dan Knapp
> 
> 
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20051127/038461c2/attachment.html>