[elephant-devel] Class indexes dropped after database restart?
Red Daly
reddaly at gmail.com
Wed Dec 30 05:42:34 UTC 2009
On Tue, Dec 29, 2009 at 5:53 PM, Ian Eslick <eslick at media.mit.edu> wrote:
> Hmmm....I can't reproduce this locally with my copy of latest.
>
> Your test code passes on a fresh DB as well. It's possible you're catching an error having to do with opening non 1.0 databases.
>
> A few quick sanity checks:
> 1) Did you clean out the test repository before running? (tests/delscript.sh)
> 2) Also, are you using elephant-1.0 or the defunct elephant-dev darcs repository?
I ran the test in a different, non-elephant directory with a fresh
darcs repository pulled from the elephant-1.0 tree. Here's a copy of
the my-config.sexp:
((:compiler . :gcc)
(:berkeley-db-version . "4.7")
(:berkeley-db-include-dir . "/usr/local/BerkeleyDB.4.7/include/")
(:berkeley-db-lib-dir . "/usr/local/BerkeleyDB.4.7/lib/")
(:berkeley-db-lib . "/usr/local/BerkeleyDB.4.7/lib/libdb-4.7.so")
(:berkeley-db-deadlock . "/usr/local/BerkeleyDB.4.7/bin/db_deadlock")
(:berkeley-db-cachesize . 20971520)
(:berkeley-db-max-locks . 2000)
(:berkeley-db-max-objects . 2000)
(:berkeley-db-max-transactions . 1000)
(:berkeley-db-map-degree2 . t)
(:berkeley-db-mvcc . nil)
(:clsql-lib-paths . nil)
(:prebuilt-libraries . nil))
Did you try to reproduce this using Linux/SBCL? I don't know why it
would make a difference.
>
> Thank you,
> Ian
Best,
Red
>
> On Dec 29, 2009, at 12:52 PM, Red Daly wrote:
>
>> This bug is easy to reproduce with the latest version of elephant.
>> Here is some code I used to reproduce the bug:
>>
>> (require :elephant)
>>
>> (defpackage :ele-bug
>> (:use :ele :cl))
>>
>> (in-package :ele-bug)
>>
>> (defclass a ()
>> ((x :initarg :x :index t :initform 0 :accessor x))
>> (:metaclass ele:persistent-metaclass))
>>
>> (defun create-instances (n &optional (class 'a))
>> (dotimes (i n)
>> (make-instance class :x i)))
>>
>> (defun stats ()
>> (dolist (class-sym '(a b))
>> (format t "~A: ~A instances; ~A with value 10~%"
>> class-sym
>> (length (get-instances-by-class class-sym))
>> (length (get-instances-by-value class-sym 'x 10)))))
>>
>> (defun open-close-test ()
>> (declare (optimize (debug 3)))
>> (macrolet ((is (form)
>> `(assert ,form)))
>> (flet ((open-it ()
>> (ele:open-store '(:bdb "db2/")))
>> (test-class-indexes (n-class n-10)
>> (is (= n-class (length (get-instances-by-class 'a))))
>> (is (= n-10 (length (get-instances-by-value 'a 'x 10))))))
>> (with-open-store ('(:bdb "db2/"))
>> (create-instances 100 'a)
>> (test-class-indexes 100 1))
>> (with-open-store ('(:bdb "db2/"))
>> (test-class-indexes 100 1)))))
>>
>> (open-close-test)
>>
>> I was able to fix the problem with a minor patch, but I doubt the
>> correctness of my solution. Here is the diff:
>>
>> diff -rN old-elephant-1.0/src/elephant/controller.lisp
>> new-elephant-1.0/src/elephant/controller.lisp
>> 711c711,712
>> < :key-form 'schema-classname-keyform))
>> ---
>>> :key-form 'schema-classname-keyform
>>> :populate t))
>> 716c717,718
>> < :key-form 'instance-cidx-keyform))))
>> ---
>>> :key-form 'instance-cidx-keyform
>>> :populate t))))
>>
>>
>> I essentially just added a :populate t argument to the open-controller
>> method. This seems to indicate that the 'BY-NAME secondary indices
>> that map a class indices to OIDs, and class names to class indices, do
>> not persist. Is this expected behavior? I'm not familiar enough with
>> the internals to tell.
>>
>> Thanks,
>> Red
>>
>>
>>
>>
>>
>> On Wed, Sep 23, 2009 at 3:27 PM, Red Daly <reddaly at gmail.com> wrote:
>>> This is the 1.0 repository. I will see if I can reproduce the problem in a
>>> test case.
>>> Red
>>>
>>>
>>>
>>>
>>> On Sep 23, 2009, at 9:50 AM, Ian Eslick <eslick at media.mit.edu> wrote:
>>>
>>> Which darcs repository? There is the old 0.91 release and the new
>>> elephant-1.0 branch...
>>> Ian
>>> On Sep 22, 2009, at 8:41 PM, Red Daly wrote:
>>>
>>> I should also note that I am using the version of elephant from darcs. And
>>> I discovered that when the program went down it was due to a segfault (I
>>> believe in emacs). However, all instances had been created under many
>>> different transactions that did not abort.
>>>
>>> Here is some more fiddling around:
>>>
>>> CL-USER> (lisp-implementation-version)
>>> "1.0.29.54.rc1"
>>> BOOKS> (ele::get-db-schemas *ele-db* 'material)
>>> NIL
>>> BOOKS> (ele::controller-instance-class-index *ele-db*)
>>> #<BDB-BTREE-INDEX
>>> oid:13701>
>>> BOOKS> (ele:map-btree #'(lambda (key
>>> val)
>>> (format t "~S => ~S~%" key
>>> val))
>>> (ele::controller-instance-class-index *ele-db*))
>>> 307 =>
>>> 307
>>> NIL
>>>
>>>
>>> Red
>>>
>>> On Tue, Sep 22, 2009 at 8:08 PM, Red Daly <reddaly at gmail.com> wrote:
>>>>
>>>> I noticed there was a disparity between get-instances-by-class and
>>>> map-inverted-index. I will let the code speak for itself:
>>>>
>>>> BOOKS> (length (ele:map-inverted-index #'(lambda (isbn obj) obj) 'material
>>>> 'isbn :collect
>>>> t))
>>>>
>>>> 1945
>>>> BOOKS> (length (ele:get-instances-by-class 'material))
>>>> 0
>>>> BOOKS> (defparameter f (make-instance 'material :title "Fake Book"))
>>>>
>>>> F
>>>> BOOKS> (length (ele:get-instances-by-class 'material))
>>>>
>>>> 1
>>>> BOOKS> (length (ele:map-inverted-index #'(lambda (isbn obj) obj) 'material
>>>> 'isbn :collect
>>>> t))
>>>> 1946
>>>>
>>>> There is a similar problem with the university class.
>>>>
>>>> BOOKS> (length (ele:get-instances-by-class 'university))
>>>> 0
>>>> BOOKS> *stanford-university*
>>>> #<UNIVERSITY
>>>> oid:204>
>>>> BOOKS> (official-name *stanford-university*)
>>>> "Stanford University"
>>>>
>>>> All of the instances in the database were created in a prior session.
>>>> Then I shut down the DB and restarted Lisp. Now MAP-CLASS and
>>>> GET-INSTANCES-BY-CLASS return nil for all classes. It seems like new
>>>> instances get added to the "class index," but older instances are not
>>>> found. However, these classes persist in the inverted slot indexes. Can
>>>> anybody diagnose what's going on?
>>>>
>>>> here are the definitions of the classes:
>>>>
>>>> (defclass department ()
>>>> ((university :initarg :university :initform nil :accessor
>>>> department-university
>>>> :index t)
>>>> (official-name :initarg :official-name :initform nil :accessor
>>>> official-name)
>>>> (common-name :initarg :common-name :initform nil :accessor common-name)
>>>> (abbreviation :initarg :abbreviation :initform nil :accessor
>>>> abbreviation
>>>> :index t))
>>>> (:metaclass ele:persistent-metaclass)
>>>> (:documentation "A department has a bunch of course listings."))
>>>>
>>>> (defclass material ()
>>>> ((title :initarg :title :initform nil :accessor material-title
>>>> :index t)
>>>> (authors :initarg :authors :initform nil :accessor material-authors) ;;
>>>> a list of
>>>> authors
>>>> (edition :initarg :edition :initform nil :accessor material-edition)
>>>> (isbn :initarg :isbn :initform nil :accessor material-isbn
>>>> :index t)
>>>> (asin :initarg :asin :initform nil :accessor material-asin
>>>> :index t
>>>> :documentation "An Amazon.com identifier that uniquely identifies
>>>> the material")
>>>> (image-uri :initarg :image-uri :initform nil :accessor
>>>> material-image-uri))
>>>> (:metaclass ele:persistent-metaclass))
>>>>
>>>> Thanks,
>>>> Red
>>>>
>>>
>>> _______________________________________________
>>> elephant-devel site list
>>> elephant-devel at common-lisp.net
>>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>>
>>> _______________________________________________
>>> elephant-devel site list
>>> elephant-devel at common-lisp.net
>>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>
>> _______________________________________________
>> elephant-devel site list
>> elephant-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/elephant-devel
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>
More information about the elephant-devel
mailing list