From aslakg at gmail.com  Mon Apr 13 09:05:03 2009
From: aslakg at gmail.com (Aslak Gronflaten)
Date: Mon, 13 Apr 2009 17:05:03 +0800
Subject: [elephant-devel] How to model a tag cloud?
Message-ID: <57870af40904130205mf6aa6d1sb9ce220d83cf085a@mail.gmail.com>

Hi,

using elephant with a berkeley backend, how should I go about modelling tag
cloud like
functionality?

If I have a number of persistent classes, f.ex.  Things and Items and I wish
to be
able to tag each Thing and each Item with one or more tags (let's use
keywords f.ex.)

The functionality I want is to be able to efficiently:
 - retrieve all Things (or Items ) tagged with any of a given set of
tags/keyword
   as in  (give-me-all-tagged 'Thing :any-of '(:foo :bar :baz))
 - retrive all Things (or Items) tagged with all of a given set of tags, as
in
   (give-my-all-tagged 'Thing :all-of '(:foo :baz))

For now I've been using b-trees with the tags as keys and done some set
logic in code
on top of that, but not sure how that'll work with large datasets (thousands
of Things
in existence).

Any suggestions?

Regards,
   Aslak
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090413/4b8606d0/attachment.html>

From reddaly at gmail.com  Wed Apr 15 00:02:41 2009
From: reddaly at gmail.com (Red Daly)
Date: Tue, 14 Apr 2009 17:02:41 -0700
Subject: [elephant-devel] Close all stores on exit
In-Reply-To: <64887.88.73.230.97.1234342299.squirrel@mail.stardawn.org>
References: <61298.88.73.197.249.1234277671.squirrel@mail.stardawn.org>
	<51F633EB-75B1-4B02-897F-FABEFE4CFD1B@media.mit.edu>
	<64887.88.73.230.97.1234342299.squirrel@mail.stardawn.org>
Message-ID: <f6700f0c0904141702p13f0a42dmab952edac811aef2@mail.gmail.com>

On Wed, Feb 11, 2009 at 1:51 AM, Leslie P. Polzer
<sky at viridian-project.de>wrote:

>
> > That sounds like a good idea.  I've never had problems with recovery
> > or recovery time in my sites, but it is good practice to shut down
> > cleanly.
>
> It's more of a problem in a multi-process environment:
>
> As soon as one process initiates recovery then the other process has
> to be restarted (the stores cannot be closed either when it's come
> to that).


Just for the record, whenever there is a power-outage or other crash my
recoveries take about 30 minutes.  Since this only happens every few months,
it is no big deal.  I am able to avoid long recoveries by running
db_checkpoint every minute or so on the database.  I also run db_deadlock to
prevent deadlocks.

This enhancement is appreciated. thanks

Red
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090414/1e61d6d9/attachment.html>

From franks-muc at web.de  Sat Apr 18 22:10:08 2009
From: franks-muc at web.de (Frank Schorr)
Date: Sun, 19 Apr 2009 00:10:08 +0200
Subject: [elephant-devel] deserialization error with LispWorks on win32 and
	postmodern
Message-ID: <1667660991@web.de>

This is with LispWorks on win32 and postmodern

I'm still using the old version 0.9 and tried upgrading now since I would like to use the new snapshot feature.
pulled the latest version.
open-store with a fresh db results in an error as shown below.

Best regards

Frank



Elephant deserialization error:
NIL
  1 (continue) retry db-postmodern transaction
  2 DB-POSTMODERN::ABORT-TRANSACTION
  3 (abort) Abort job 19 :(BIND-STANDARD-STREAMS-AND-EXECUTE #<EDITOR::EDITOR-OUTPUT-STREAM #<EDITOR:BUFFER Background Output> 21DBF89F> (FUNCALL-BACKGROUND-JOB-AUX #<CAPI:EDITOR-PANE CAPI:EDITOR-PANE  21C41623> BACKGROUND-REGION-EVAL (# # # # T NIL NIL)))

Type :b for backtrace, :c <option number> to proceed,  or :? for other options

ELE-TESTS 1 : 1 > :b :verbose
Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION (LABELS ELEPHANT-SERIALIZER2::%DESERIALIZE) ELEPHANT-SERIALIZER2::DESERIALIZE) (offset 2877)

Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION (LABELS ELEPHANT-SERIALIZER2::%DESERIALIZE) ELEPHANT-SERIALIZER2::DESERIALIZE) (offset 472)

Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION (LABELS ELEPHANT-SERIALIZER2::%DESERIALIZE) ELEPHANT-SERIALIZER2::DESERIALIZE) (offset 1520)

Call to ELEPHANT-SERIALIZER2::DESERIALIZE (offset 239)

Call to DESERIALIZE (offset 292)
  ELEPHANT::BS                   : #S(ELEPHANT-MEMUTIL:BUFFER-STREAM :BUFFER #<Pointer to type (:UNSIGNED :CHAR) = #x00943F48> :SIZE 45 :POSITION 2844 :LENGTH 80)
  ELEPHANT::SC                   : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  ELEPHANT::OID-ONLY             : NIL
  DBG::G                         : NIL
  DBG::G                         : #<Closure (DESERIALIZE . 1) 21A4D7A2>
  CONDITIONS::*HANDLER-CLUSTERS* : (((ERROR . #<Closure # 21A4D7A2>)) ((CL-POSTGRES:DATABASE-ERROR . #<Function # 22187C3A>)) ((WARNING . EDITOR::EDITOR-EVALUATION-WARNING)) ((END-OF-FILE . #<Closure # 2273CD92>) (READER-ERROR . #<Closure # 21A4DE42>)))

Call to DB-POSTMODERN::DESERIALIZE-FROM-DATABASE (offset 63)
  DB-POSTMODERN::X         : #(17 0 0 0 0 13 9 3 0 0 0 69 81 76 9 11 0 0 0 67 79 77 77 79 78 45 76 73 83 80 4 0 0 0 64 4 0 0 128 63 ...)
  DB-POSTMODERN::SC        : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  DB-POSTMODERN::OIDS-ONLY : NIL
  DB-POSTMODERN::OTHER     : #S(ELEPHANT-MEMUTIL:BUFFER-STREAM :BUFFER #<Pointer to type (:UNSIGNED :CHAR) = #x00943F48> :SIZE 45 :POSITION 2844 :LENGTH 80)

Call to (METHOD DB-POSTMODERN::INTERNAL-GET-VALUE (T DB-POSTMODERN::PM-BTREE)) (offset 369)
  DB-POSTMODERN::KEY          : "3 INDICES"
  DB-POSTMODERN::BT           : #<DB-POSTMODERN::PM-BTREE 200BBAFF>
  CLOS::.ISL.                 : :DONT-KNOW
  DB-POSTMODERN::VALUE        : NIL
  DB-POSTMODERN::EXISTS-P     : NIL
  DB-POSTMODERN::*SC*         : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  DB-POSTMODERN::*CONNECTION* : #<CL-POSTGRES:DATABASE-CONNECTION 225F9F3B>
  DB-POSTMODERN::RESULT       : #(17 0 0 0 0 13 9 3 0 0 0 69 81 76 9 11 0 0 0 67 79 77 77 79 78 45 76 73 83 80 4 0 0 0 64 4 0 0 128 63 ...)

Call to (METHOD GET-VALUE (T DB-POSTMODERN::PM-BTREE)) (offset 20)
  DB-POSTMODERN::KEY : "3 INDICES"
  DB-POSTMODERN::BT  : #<DB-POSTMODERN::PM-BTREE 200BBAFF>
  CLOS::.ISL.        : :DONT-KNOW

Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION 2 (METHOD GET-VALUE (T DB-POSTMODERN::PM-BTREE-WRAPPER))) (offset 137)
  DB-POSTMODERN::*CONNECTION* : #<CL-POSTGRES:DATABASE-CONNECTION 225F9F3B>
  DB-POSTMODERN::*SC*         : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  DB-POSTMODERN::KEY          : "3 INDICES"
  DB-POSTMODERN::BT           : #<PM-SPECIAL-BTREE-WRAPPER oid:2>

Call to (METHOD ELEPHANT::PERSISTENT-SLOT-READER (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER T T)) (offset 78)
  DB-POSTMODERN::SC        : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  DB-POSTMODERN::INSTANCE  : #<PM-SPECIAL-INDEXED-BTREE-WRAPPER oid:3>
  DB-POSTMODERN::NAME      : DB-POSTMODERN::INDICES
  DB-POSTMODERN::OIDS-ONLY : :DONT-KNOW
  CLOS::.ISL.              : :DONT-KNOW
  DB-POSTMODERN::*SC*      : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>

Call to CLOS::METHOD-COMBINATION-TEMPLATE (offset 45)

Call to (METHOD CLOS::DYNAMIC-SLOT-VALUE-USING-SLOTD (T T)) (offset 120)
  CLOS::INSTANCE : #<PM-SPECIAL-INDEXED-BTREE-WRAPPER oid:3>
  CLOS::SLOTD    : #<ELEPHANT::PERSISTENT-EFFECTIVE-SLOT-DEFINITION DB-POSTMODERN::INDICES 225D046F>

Call to (METHOD GET-INDEX (DB-POSTMODERN::PM-INDEXED-BTREE-WRAPPER T)) (offset 16)
  DB-POSTMODERN::BT         : #<PM-SPECIAL-INDEXED-BTREE-WRAPPER oid:3>
  DB-POSTMODERN::INDEX-NAME : ELEPHANT::BY-NAME
  CLOS::.ISL.               : :DONT-KNOW

Call to (METHOD ELEPHANT::ENSURE-INDEX (INDEXED-BTREE T)) (offset 27)

Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION 2 (METHOD ELEPHANT::OPEN-CONTROLLER :AFTER (STORE-CONTROLLER))) (offset 65)

Call to DB-POSTMODERN::EXECUTE-TRANSACTION-ONE-TRY (offset 240)
  DB-POSTMODERN::SC                : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  DB-POSTMODERN::TXN-FN            : #<Closure ((METHOD ELEPHANT::OPEN-CONTROLLER :AFTER (STORE-CONTROLLER)) . 2) 200BA252>
  DB-POSTMODERN::ALWAYS-ROLLBACK   : NIL
  DB-POSTMODERN::TRAN              : #<POSTMODERN::TRANSACTION-HANDLE 200BA17F>
  DB-POSTMODERN::COMMITED          : NIL
  DB-POSTMODERN::*TXN-VALUE-CACHE* : NIL
  ELEPHANT::*CURRENT-TRANSACTION*  : (#<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)> NIL NIL)
  *STORE-CONTROLLER*               : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>

Call to (METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER T)) (offset 949)
  DB-POSTMODERN::SC                  : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  DB-POSTMODERN::TXN-FN              : #<Closure ((METHOD ELEPHANT::OPEN-CONTROLLER :AFTER (STORE-CONTROLLER)) . 2) 200BA252>
  DBG::G                             : :DONT-KNOW
  DB-POSTMODERN::ALWAYS-ROLLBACK     : NIL
  DB-POSTMODERN::RETRY-CLEANUP-FN    : NIL
  DB-POSTMODERN::RETRIES             : 200
  CLOS::.ISL.                        : :DONT-KNOW
  POSTMODERN:*DATABASE*              : #<CL-POSTGRES:DATABASE-CONNECTION 225F9F3B>
  DB-POSTMODERN::FIX-BEFORE-RETRY-FN : NIL
  DB-POSTMODERN::TRY                 : 200
  DBG::G                             : #<Closure ((METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER T)) . 5) 200AA532>
  DBG::G                             : #<Function ((METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER T)) . 4) 22187D7A>
  DBG::G                             : #<Closure ((METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER T)) . 3) 200AA54A>
  CONDITIONS::*RESTART-CLUSTERS*     : ((#<RESTART ABORT 21A4CF0F>) (#<RESTART ABORT 21A4D177>) (#<RESTART DB-POSTMODERN::RETRY-TRANSACTION 200B550B> #<RESTART DB-POSTMODERN::ABORT-TRANSACTION 200B54DB>) (#<RESTART ABORT 200B549F>) (#<RESTART ABORT 219CA8AF>))
  DBG::G                             : #<Function ((METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER T)) . 8) 22187C3A>
  CONDITIONS::*HANDLER-CLUSTERS*     : (((CL-POSTGRES:DATABASE-ERROR . #<Function # 22187C3A>)) ((WARNING . EDITOR::EDITOR-EVALUATION-WARNING)) ((END-OF-FILE . #<Closure # 2273CD92>) (READER-ERROR . #<Closure # 21A4DE42>)))

Call to CLOS::METHOD-COMBINATION-TEMPLATE (offset 146)

Call to CLOS::NEXT-METHOD-CALL-2 (offset 87)
  CONS               : (METHOD ELEPHANT::OPEN-CONTROLLER :AROUND (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER))
  CLOS::NEXT-METHODS : (#<Closure CLOS::METHOD-COMBINATION-TEMPLATE 225C668A>)
  #:G174431          : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  #:G174432          : NIL

Call to (METHOD ELEPHANT::OPEN-CONTROLLER :AROUND (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER)) (offset 61)
  DB-POSTMODERN::SC          : #<POSTMODERN-STORE-CONTROLLER (POSTGRESQL 127.0.0.1 elepm postgres %postgres%)>
  DBG::G                     : NIL
  CLOS::.ISL.                : :DONT-KNOW
  DBG::G                     : (#<Closure CLOS::METHOD-COMBINATION-TEMPLATE 225C668A>)
  DB-POSTMODERN:*CACHE-MODE* : NIL

Call to CLOS::METHOD-COMBINATION-TEMPLATE (offset 42)

Call to OPEN-STORE (offset 66)

Call to EVAL (offset 113)
  EXP : (OPEN-STORE *DB-SPEC*)

Call to EDITOR::EDITOR-EVAL (offset 177)
  EDITOR:BUFFER  : :DONT-KNOW
  EDITOR::SEXP   : (OPEN-STORE *DB-SPEC*)
  EDITOR::ERRORP : T

Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION 3 (HARLEQUIN-COMMON-LISP:SUBFUNCTION 1 EDITOR::REGION-LISP-EVAL)) (offset 26)
  EDITOR::FORM  : (OPEN-STORE *DB-SPEC*)
  EDITOR:BUFFER : #<EDITOR:BUFFER work.lisp>

Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION 1 EDITOR::REGION-LISP-EVAL) (offset 624)
  #:BUFFER41963          : #<EDITOR:BUFFER work.lisp>
  EDITOR:BUFFER          : #<EDITOR:BUFFER work.lisp>
  PRINT                  : T
  EDITOR::START          : #<EDITOR::POINT "work.lisp" 0 offset 6054 2273DD0B>
  EDITOR::END            : #<EDITOR::POINT "work.lisp" 0 offset 6076 2273DCE3>
  DBG::|block-catch-tag| : (NIL)
  DBG::|block-catch-tag| : (NIL)

Call to EDITOR::WITH-COMPILATION-ENVIRONMENT-AT-POINT-FN (offset 266)
  EDITOR::POINT         : #<EDITOR::POINT "work.lisp" 0 offset 6054 2273DD0B>
  EDITOR::START-MESSAGE : "Evaluating..."
  EDITOR::END-MESSAGE   : "Finished evaluating"
  EDITOR::BODY-FN       : #<Closure (EDITOR::REGION-LISP-EVAL . 1) 2273DA42>

Call to EDITOR::REGION-LISP-EVAL (offset 204)
  EDITOR:BUFFER : #<EDITOR:BUFFER work.lisp>
  EDITOR::START : #<EDITOR::POINT "work.lisp" 0 offset 6054 2273DD0B>
  EDITOR::END   : #<EDITOR::POINT "work.lisp" 0 offset 6076 2273DCE3>
  PRINT         : T

Call to EDITOR::BACKGROUND-REGION-EVAL (offset 103)
  EDITOR::WINDOW         : #<EDITOR::WM-WINDOW "work.lisp" 2212AC47>
  EDITOR:BUFFER          : #<EDITOR:BUFFER work.lisp>
  EDITOR::START          : #<EDITOR::POINT "work.lisp" 0 offset 6054 2273DD0B>
  EDITOR::END            : #<EDITOR::POINT "work.lisp" 0 offset 6076 2273DCE3>
  PRINT                  : T
  EDITOR::WARNINGS       : NIL
  EDITOR::AFTER-FUNCTION : NIL

Call to CAPI::FUNCALL-BACKGROUND-JOB-AUX (offset 50)
  CAPI::PANE : #<CAPI:EDITOR-PANE CAPI:EDITOR-PANE  21C41623>
  FUNCTION   : EDITOR::BACKGROUND-REGION-EVAL
  CAPI::ARGS : (#<EDITOR::WM-WINDOW "work.lisp" 2212AC47> #<EDITOR:BUFFER work.lisp> #<EDITOR::POINT "work.lisp" 0 offset 6054 2273DD0B> #<EDITOR::POINT "work.lisp" 0 offset 6076 2273DCE3> T NIL NIL)

Call to CAPI::BIND-STANDARD-STREAMS-AND-EXECUTE (offset 204)
  STREAM          : #<EDITOR::EDITOR-OUTPUT-STREAM #<EDITOR:BUFFER Background Output> 21DBF89F>
  CAPI::FUNC-ARGS : (CAPI::FUNCALL-BACKGROUND-JOB-AUX #<CAPI:EDITOR-PANE CAPI:EDITOR-PANE  21C41623> EDITOR::BACKGROUND-REGION-EVAL (#<EDITOR::WM-WINDOW "work.lisp" 2212AC47> #<EDITOR:BUFFER work.lisp> #<EDITOR::POINT "work.lisp" 0 offset 6054 2273DD0B> #<EDITOR::POINT "work.lisp" 0 offset 6076 2273DCE3> T NIL NIL))

Call to MP::BACKGROUND-EXECUTE-LOOP (offset 629)

Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION MP::PROCESS-SG-FUNCTION MP::INITIALIZE-PROCESS-STACK) (offset 301)
  MP::IGNORED : :DONT-KNOW
  VALUES      : NIL
  MP:PROCESS  : #<MP:PROCESS Name "Background execute 1" Priority 50000000 State "Running">
  MP::SYMBOLS : NIL


ELE-TESTS 2 : 1 > 
______________________________________________________
GRATIS f?r alle WEB.DE-Nutzer: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://movieflat.web.de




From tomoyuki28jp at gmail.com  Sun Apr 19 10:26:50 2009
From: tomoyuki28jp at gmail.com (Tomo Matsumoto)
Date: Sun, 19 Apr 2009 19:26:50 +0900
Subject: [elephant-devel] error: "sorry,
	too many clients already has occurred."
Message-ID: <780e7d140904190326m303a9e72p9955fe8625aed94f@mail.gmail.com>

Hello everyone,

This is my first post here.

I use elephant with postgres, and I get the error like this.
While trying to connect to database localhost/test/postgres
  using database-type POSTGRESQL:
  Error 1 / FATAL:  sorry, too many clients already
  has occurred.

I use it with hunchentoot. Do I need to call close-store every time
after processing a request?

This happened when I test my web applications in my local developing
environment, so there weren't so much requests to the DB. Why does
this happen and how can I avoid the error?

Thanks in advice,
Tomo



From sky at viridian-project.de  Sun Apr 19 11:49:27 2009
From: sky at viridian-project.de (Leslie P. Polzer)
Date: Sun, 19 Apr 2009 13:49:27 +0200 (CEST)
Subject: [elephant-devel] error: "sorry,
 too many clients already has occurred."
In-Reply-To: <780e7d140904190326m303a9e72p9955fe8625aed94f@mail.gmail.com>
References: <780e7d140904190326m303a9e72p9955fe8625aed94f@mail.gmail.com>
Message-ID: <2d61c2e8e4a3c22b726463737773e4db.squirrel@mail.stardawn.org>


> I use elephant with postgres, and I get the error like this.
> While trying to connect to database localhost/test/postgres
>   using database-type POSTGRESQL:
>   Error 1 / FATAL:  sorry, too many clients already
>   has occurred.
>
> I use it with hunchentoot. Do I need to call close-store every time
> after processing a request?

No, you shouldn't.


> This happened when I test my web applications in my local developing
> environment, so there weren't so much requests to the DB. Why does
> this happen and how can I avoid the error?

What Lisp are you using?

  Leslie

-- 
LinkedIn Profile: http://www.linkedin.com/in/polzer
Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
Blog: http://blog.viridian-project.de/




From tomoyuki28jp at gmail.com  Sun Apr 19 12:03:07 2009
From: tomoyuki28jp at gmail.com (Tomo Matsumoto)
Date: Sun, 19 Apr 2009 21:03:07 +0900
Subject: [elephant-devel] error: "sorry,
	too many clients already has 	occurred."
In-Reply-To: <2d61c2e8e4a3c22b726463737773e4db.squirrel@mail.stardawn.org>
References: <780e7d140904190326m303a9e72p9955fe8625aed94f@mail.gmail.com>
	<2d61c2e8e4a3c22b726463737773e4db.squirrel@mail.stardawn.org>
Message-ID: <780e7d140904190503n4a6abe78ke7a73e29cc9f72b0@mail.gmail.com>

Thanks for your quick response, Leslie.

> What Lisp are you using?

I use following versions of softwares.
- Elephant 0.9
- SBCL: 1.0.18
- OS: Ubuntu Linux 8.1.0 32bit

I am trying to upgrade elephant to the Version 1.0 and see how it works.

Tomo

On Sun, Apr 19, 2009 at 8:49 PM, Leslie P. Polzer
<sky at viridian-project.de> wrote:
>
>> I use elephant with postgres, and I get the error like this.
>> While trying to connect to database localhost/test/postgres
>> ? using database-type POSTGRESQL:
>> ? Error 1 / FATAL: ?sorry, too many clients already
>> ? has occurred.
>>
>> I use it with hunchentoot. Do I need to call close-store every time
>> after processing a request?
>
> No, you shouldn't.
>
>
>> This happened when I test my web applications in my local developing
>> environment, so there weren't so much requests to the DB. Why does
>> this happen and how can I avoid the error?
>
> What Lisp are you using?
>
> ?Leslie
>
> --
> LinkedIn Profile: http://www.linkedin.com/in/polzer
> Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
> Blog: http://blog.viridian-project.de/
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>



From sky at viridian-project.de  Sun Apr 19 15:11:55 2009
From: sky at viridian-project.de (Leslie P. Polzer)
Date: Sun, 19 Apr 2009 17:11:55 +0200 (CEST)
Subject: [elephant-devel] error: "sorry,
 too many clients already has 	occurred."
In-Reply-To: <780e7d140904190503n4a6abe78ke7a73e29cc9f72b0@mail.gmail.com>
References: <780e7d140904190326m303a9e72p9955fe8625aed94f@mail.gmail.com>
	<2d61c2e8e4a3c22b726463737773e4db.squirrel@mail.stardawn.org>
	<780e7d140904190503n4a6abe78ke7a73e29cc9f72b0@mail.gmail.com>
Message-ID: <3f754fe932cda816c8dc56ac0f03dcf0.squirrel@mail.stardawn.org>


> I use following versions of softwares.
> - Elephant 0.9
> - SBCL: 1.0.18
> - OS: Ubuntu Linux 8.1.0 32bit
>
> I am trying to upgrade elephant to the Version 1.0 and see how it works.

This problem is likely to go away then. I have added code to close
PG connections past 0.9.1.

  Leslie

-- 
LinkedIn Profile: http://www.linkedin.com/in/polzer
Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
Blog: http://blog.viridian-project.de/




From aslakg at gmail.com  Mon Apr 20 07:23:55 2009
From: aslakg at gmail.com (Aslak Gronflaten)
Date: Mon, 20 Apr 2009 15:23:55 +0800
Subject: [elephant-devel] Database trouble
Message-ID: <57870af40904200023h47912102s137bcdff901d6bb5@mail.gmail.com>

Hi,

I've been using elephant 0.91 for quite a while now, with a berkeley 4.6
backend, and it's been a joy.
First of all my thanks to all contributors on this project!
Recently however, as usage has gone up, some problems have started occuring.
One big problem has been that when I run (get-instances-by-class 'foo) I get
a list of foo, but with some nil's occuring as well.
Is this a sign of database corruption? How do I remedy this? I've tried
running both the recover and checkpoint tools on the db, but no change.
I've experienced deadlocks as well, and had to kill my sbcl - wondering if
that (non clean shutdown) is what caused the problems in the first place.
Or is this something to do with indexes - could they be remade somehow?  I
defined my foo class like this:


  (DEFPCLASS FOO ()
    ((FIELD1 :ACCESSOR FIELD1 :INITARG :FIELD1 :INITFORM NIL :INDEX T)
     (FIELD2 :ACCESSOR FIELD2 :INITARG :FIELD2 :INITFORM NIL :INDEX
         NIL)
     (FIELD2 :ACCESSOR FIELD2 :INITARG :FIELD2 :INITFORM NIL :INDEX
         NIL))
    (:INDEX T))
  (DEFUN GET-FOO-LIST () (GET-INSTANCES-BY-CLASS 'FOO))
  (DEFUN GET-FOO-BY-FIELD1 (DB::VALUE)
    (GET-INSTANCE-BY-VALUE 'FOO 'FIELD1 DB::VALUE)))


Appreciate any help,
   Aslak
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090420/d9b26ba0/attachment.html>

From sky at viridian-project.de  Thu Apr 23 09:36:44 2009
From: sky at viridian-project.de (Leslie P. Polzer)
Date: Thu, 23 Apr 2009 11:36:44 +0200 (CEST)
Subject: [elephant-devel] Database trouble
In-Reply-To: <57870af40904200023h47912102s137bcdff901d6bb5@mail.gmail.com>
References: <57870af40904200023h47912102s137bcdff901d6bb5@mail.gmail.com>
Message-ID: <fb1d27b435201d820752762b53e45485.squirrel@mail.stardawn.org>


> I've been using elephant 0.91 for quite a while now, with a berkeley 4.6
> backend, and it's been a joy.

Great! :)


> Recently however, as usage has gone up, some problems have started occuring.
> One big problem has been that when I run (get-instances-by-class 'foo) I get
> a list of foo, but with some nil's occuring as well.
> Is this a sign of database corruption?

Not necessarily, I'd rather guess at a bug in the indexing mechanism
that has been solved by now.


> How do I remedy this?

Some possible solutions:

  * upgrade your Elephant

  * rebuild your indices

  * wrap your get-instances stuff in a function that removes
    the NIL entries


> I've tried running both the recover and checkpoint tools on the db, but
> no change.

No, that won't help. :)


> I've experienced deadlocks as well, and had to kill my sbcl

FYI you can prevent deadlocks from appearing.


> - wondering if that (non clean shutdown) is what caused the problems
> in the first place.

I don't think so.


> Or is this something to do with indexes - could they be remade somehow?

I think there was an easy way but I can't remember it right now...

  Cheers,

    Leslie

-- 
LinkedIn Profile: http://www.linkedin.com/in/polzer
Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
Blog: http://blog.viridian-project.de/




From aslakg at gmail.com  Thu Apr 23 10:23:25 2009
From: aslakg at gmail.com (Aslak Gronflaten)
Date: Thu, 23 Apr 2009 18:23:25 +0800
Subject: [elephant-devel] Database trouble
In-Reply-To: <fb1d27b435201d820752762b53e45485.squirrel@mail.stardawn.org>
References: <57870af40904200023h47912102s137bcdff901d6bb5@mail.gmail.com>
	<fb1d27b435201d820752762b53e45485.squirrel@mail.stardawn.org>
Message-ID: <57870af40904230323x5460660eke9ae4623a73d13@mail.gmail.com>

Hi,

thanks for the reply!
I've tried wrapping my get-instances, that's fine and will work for now.
I've tried ugrading elephant - that did not work at all - unable to open the
db at all - but I don't have the details here, I'll try a bit harder, and if
I get stuck I might post separately about this.
Rebuilding indices sounds interesting - I'll investigate in how to do this!

Thanks ,
  Aslak

On Thu, Apr 23, 2009 at 5:36 PM, Leslie P. Polzer
<sky at viridian-project.de>wrote:

>
> > I've been using elephant 0.91 for quite a while now, with a berkeley 4.6
> > backend, and it's been a joy.
>
> Great! :)
>
>
> > Recently however, as usage has gone up, some problems have started
> occuring.
> > One big problem has been that when I run (get-instances-by-class 'foo) I
> get
> > a list of foo, but with some nil's occuring as well.
> > Is this a sign of database corruption?
>
> Not necessarily, I'd rather guess at a bug in the indexing mechanism
> that has been solved by now.
>
>
> > How do I remedy this?
>
> Some possible solutions:
>
>  * upgrade your Elephant
>
>  * rebuild your indices
>
>  * wrap your get-instances stuff in a function that removes
>    the NIL entries
>
>
> > I've tried running both the recover and checkpoint tools on the db, but
> > no change.
>
> No, that won't help. :)
>
>
> > I've experienced deadlocks as well, and had to kill my sbcl
>
> FYI you can prevent deadlocks from appearing.
>
>
> > - wondering if that (non clean shutdown) is what caused the problems
> > in the first place.
>
> I don't think so.
>
>
> > Or is this something to do with indexes - could they be remade somehow?
>
> I think there was an easy way but I can't remember it right now...
>
>  Cheers,
>
>    Leslie
>
> --
> LinkedIn Profile: http://www.linkedin.com/in/polzer
> Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
> Blog: http://blog.viridian-project.de/
>
>


-- 
Mvh Aslak
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090423/7fb6021b/attachment.html>

From sky at viridian-project.de  Thu Apr 23 10:56:07 2009
From: sky at viridian-project.de (Leslie P. Polzer)
Date: Thu, 23 Apr 2009 12:56:07 +0200 (CEST)
Subject: [elephant-devel] Database trouble
In-Reply-To: <57870af40904230323x5460660eke9ae4623a73d13@mail.gmail.com>
References: <57870af40904200023h47912102s137bcdff901d6bb5@mail.gmail.com>
	<fb1d27b435201d820752762b53e45485.squirrel@mail.stardawn.org>
	<57870af40904230323x5460660eke9ae4623a73d13@mail.gmail.com>
Message-ID: <f036fa9a9237e6c1853ef55c9b65d954.squirrel@mail.stardawn.org>


> I've tried wrapping my get-instances, that's fine and will work for now.
> I've tried ugrading elephant - that did not work at all - unable to open the
> db at all - but I don't have the details here, I'll try a bit harder, and if
> I get stuck I might post separately about this.

Yes, please do so.


> Rebuilding indices sounds interesting - I'll investigate in how to do this!

Just did a quick look and found it right away, check out

  (describe 'ele::rebuild-slot-index)


About the deadlock stuff:

You can run the external detector (or let Elephant start it) or explicitly
enable the BDB built-in detector. The latter works best and is the least
hassle -- I've posted how to do this on the list some months ago, look for it.

  HTH :)

    Leslie

-- 
LinkedIn Profile: http://www.linkedin.com/in/polzer
Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
Blog: http://blog.viridian-project.de/




From eslick at media.mit.edu  Thu Apr 23 14:15:11 2009
From: eslick at media.mit.edu (Ian Eslick)
Date: Thu, 23 Apr 2009 07:15:11 -0700
Subject: [elephant-devel] Database trouble
In-Reply-To: <fb1d27b435201d820752762b53e45485.squirrel@mail.stardawn.org>
References: <57870af40904200023h47912102s137bcdff901d6bb5@mail.gmail.com>
	<fb1d27b435201d820752762b53e45485.squirrel@mail.stardawn.org>
Message-ID: <324CE945-6A6C-435A-B7FD-D6296F3D6DDB@media.mit.edu>

You can remove/add btree indexes manually.  There is a rebuild-slot- 
indices function (or something similar) in 1.0

Sent from my iPhone

On Apr 23, 2009, at 2:36 AM, "Leslie P. Polzer" <sky at viridian- 
project.de> wrote:

>
>> I've been using elephant 0.91 for quite a while now, with a  
>> berkeley 4.6
>> backend, and it's been a joy.
>
> Great! :)
>
>
>> Recently however, as usage has gone up, some problems have started  
>> occuring.
>> One big problem has been that when I run (get-instances-by-class  
>> 'foo) I get
>> a list of foo, but with some nil's occuring as well.
>> Is this a sign of database corruption?
>
> Not necessarily, I'd rather guess at a bug in the indexing mechanism
> that has been solved by now.
>
>
>> How do I remedy this?
>
> Some possible solutions:
>
>  * upgrade your Elephant
>
>  * rebuild your indices
>
>  * wrap your get-instances stuff in a function that removes
>    the NIL entries
>
>
>> I've tried running both the recover and checkpoint tools on the db,  
>> but
>> no change.
>
> No, that won't help. :)
>
>
>> I've experienced deadlocks as well, and had to kill my sbcl
>
> FYI you can prevent deadlocks from appearing.
>
>
>> - wondering if that (non clean shutdown) is what caused the problems
>> in the first place.
>
> I don't think so.
>
>
>> Or is this something to do with indexes - could they be remade  
>> somehow?
>
> I think there was an easy way but I can't remember it right now...
>
>  Cheers,
>
>    Leslie
>
> -- 
> LinkedIn Profile: http://www.linkedin.com/in/polzer
> Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
> Blog: http://blog.viridian-project.de/
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel



From tomoyuki28jp at gmail.com  Thu Apr 23 20:59:31 2009
From: tomoyuki28jp at gmail.com (Tomo Matsumoto)
Date: Fri, 24 Apr 2009 05:59:31 +0900
Subject: [elephant-devel] unique object IDs and OIDs
Message-ID: <780e7d140904231359m3ee9f949xb5e011707ef26fb7@mail.gmail.com>

Hello,

http://common-lisp.net/pipermail/elephant-devel/2008-November/004534.html
> PE> If using OID values makes sense for this purpose, is there a user-API
> PE> way to get an object by its OID?
>
>yep, via class index:
>
>(get-value oid (find-class-index 'classname))

It seems like the function named find-class-index disappeared in Elephant 1.0?
How can I do the same thing with version 1.0?

What I want to do is getting a instance by a class name and an oid.

Thanks in advance.

Tomo

> PE> I'm wondering if it is appropriate to use the OIDs that elephant
> PE> assigns objects for this purpose.
>
>yep, i think it's fine
>
> PE>  Will these values change during the lifetime of the associated object?
>
>they shouldn't.
>
>oids are not preserved across export/import operations (by design of these
>export/import operations), but typically it's not a problem.
>
> PE> If using OID values makes sense for this purpose, is there a user-API
> PE> way to get an object by its OID?
>
>yep, via class index:
>
>(get-value oid (find-class-index 'classname))
>
>an internal function get-cached-instance does it in more straightforward
>way, though.
>
> PE>  Should I instead add another indexed field on my object that contains
> PE> the OID value?
>
>definitely not, unless you want to use your own ids.



From killerstorm at newmail.ru  Fri Apr 24 17:04:19 2009
From: killerstorm at newmail.ru (Alex Mizrahi)
Date: Fri, 24 Apr 2009 20:04:19 +0300
Subject: [elephant-devel] gp-export
Message-ID: <gssrfc$6k4$1@ger.gmane.org>

helo

(Short introduction for those who does not know what is this about: 
gp-export is a tool
to export/import databases in human-readable (SEXP) format, with 
serialization based on
s-serialization of cl-prevalence, meant to work with different databases. In 
previous discussions
it was considered as a path of migration between version 0.9 and 1.0 of 
elephant/postmodern.)

Just a few monthes passed, and I've made gp-export that is suitable for 
migration from 0.9 to 1.0. :)
The problem was that there was no support for btrees whatsoever, so I had to 
refactor it badly.
Anyway, now it seems to work and it even has some tests.

However, for now gp-export does not back up whole store -- it only iterates 
through all objects
it finds in index and dumps them together with all their slots etc. It is 
such because it was meant
to be working with various stores. (But it supports only elephant/postmodern 
so far.)
But probably if we also backup store root as some fake object instance, it 
would make full
elephant store backup.

Also, there are some backend-specific pieces in exporter, such as what slots 
to filter out.
I've only made support for postmodern, if one wants it to work with BDB, he 
needs to add support.
Also, migration is backend-specific (as it needs to map between btree 
classes), maybe it needs
some refactoring to be unified.

So, the questions are:
   Is there a need for such thing?
   Are backups described above full/good enough?
   Should it be somehow bundled with elephant or maintained separately?
   Are there people who'd like to hack it to add support for different 
stuff?
   (Well, I can do this, but I'm not very fast :) )

By the way, besides gp-export, I've considered using elephant's migration 
for 0.9 to 1.0 upgrade:

postmodern-0.9 --migration--> bdb-0.9 --upgrade--> bdb-1.0 --migration--> 
postmodern-1.0

However, migration does not work right in version 0.9... And I'm not very 
enthusiastic about
digging old code. I've asked about this some time ago, as nobody replied I'm 
assuming nobody
cares, so I'm officially giving up.

with best regards, Alex 'killerstorm' Mizrahi. 






From sky at viridian-project.de  Sat Apr 25 12:19:02 2009
From: sky at viridian-project.de (Leslie P. Polzer)
Date: Sat, 25 Apr 2009 14:19:02 +0200 (CEST)
Subject: [elephant-devel] gp-export
In-Reply-To: <gssrfc$6k4$1@ger.gmane.org>
References: <gssrfc$6k4$1@ger.gmane.org>
Message-ID: <20483cdde4bf994a4f95da59784955c0.squirrel@mail.stardawn.org>


> helo

220 Hi there


> Just a few monthes passed, and I've made gp-export that is suitable for
> migration from 0.9 to 1.0. :)
> The problem was that there was no support for btrees whatsoever, so I had to
> refactor it badly.
> Anyway, now it seems to work and it even has some tests.

Let's put the source up somewhere.


> However, for now gp-export does not back up whole store -- it only iterates
> through all objects
> it finds in index and dumps them together with all their slots etc. It is
> such because it was meant
> to be working with various stores. (But it supports only elephant/postmodern
> so far.)

Hmm... but it will walk all class indices of the store?


> But probably if we also backup store root as some fake object instance, it
> would make full
> elephant store backup.

Yes, I think so.


> I've only made support for postmodern, if one wants it to work with BDB, he
> needs to add support.

Is this abstracted well enough?


> So, the questions are:
>    Is there a need for such thing?

Yes, considering that upgrading doesn't work too well in some cases,
needs to be written anew for every change and clutters the core code.


>    Are backups described above full/good enough?

What would be missing?


>    Should it be somehow bundled with elephant or maintained separately?

It depends on how well its components are separated from each other.

If the Elephant-specific parts can be separated easily from the rest
then we should put up the latter.


>    Are there people who'd like to hack it to add support for different
> stuff?

Like, yes. Have time for it, rather not. :/

For me it also depends on how extensible your version of gp-export is.

I'd appreciate to have the opportunity to look at the code.


>    (Well, I can do this, but I'm not very fast :) )
>
> By the way, besides gp-export, I've considered using elephant's migration
> for 0.9 to 1.0 upgrade:
>
> postmodern-0.9 --migration--> bdb-0.9 --upgrade--> bdb-1.0 --migration-->
> postmodern-1.0
>
> However, migration does not work right in version 0.9...

It does not? What errors/incorrect data did you get? Sorry if you
already posted this...

  Cheers!

    Leslie

-- 
LinkedIn Profile: http://www.linkedin.com/in/polzer
Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
Blog: http://blog.viridian-project.de/




From eslick at media.mit.edu  Sun Apr 26 11:40:51 2009
From: eslick at media.mit.edu (Ian Eslick)
Date: Sun, 26 Apr 2009 07:40:51 -0400
Subject: [elephant-devel] gp-export
In-Reply-To: <gssrfc$6k4$1@ger.gmane.org>
References: <gssrfc$6k4$1@ger.gmane.org>
Message-ID: <86776217-FA9B-4CF9-8F03-E11751A13895@media.mit.edu>

gp-export sounds very useful.  I actually implemented something  
similar to help convert databases from one lisp to another (string  
serialization is not compatible across lisp implementations).  This  
would be a more general solution to my hack.  I'll look at it when I  
get a chance.

Cheers,
Ian

On Apr 24, 2009, at 1:04 PM, Alex Mizrahi wrote:

> helo
>
> (Short introduction for those who does not know what is this about:
> gp-export is a tool
> to export/import databases in human-readable (SEXP) format, with
> serialization based on
> s-serialization of cl-prevalence, meant to work with different  
> databases. In
> previous discussions
> it was considered as a path of migration between version 0.9 and 1.0  
> of
> elephant/postmodern.)
>
> Just a few monthes passed, and I've made gp-export that is suitable  
> for
> migration from 0.9 to 1.0. :)
> The problem was that there was no support for btrees whatsoever, so  
> I had to
> refactor it badly.
> Anyway, now it seems to work and it even has some tests.
>
> However, for now gp-export does not back up whole store -- it only  
> iterates
> through all objects
> it finds in index and dumps them together with all their slots etc.  
> It is
> such because it was meant
> to be working with various stores. (But it supports only elephant/ 
> postmodern
> so far.)
> But probably if we also backup store root as some fake object  
> instance, it
> would make full
> elephant store backup.
>
> Also, there are some backend-specific pieces in exporter, such as  
> what slots
> to filter out.
> I've only made support for postmodern, if one wants it to work with  
> BDB, he
> needs to add support.
> Also, migration is backend-specific (as it needs to map between btree
> classes), maybe it needs
> some refactoring to be unified.
>
> So, the questions are:
>   Is there a need for such thing?
>   Are backups described above full/good enough?
>   Should it be somehow bundled with elephant or maintained separately?
>   Are there people who'd like to hack it to add support for different
> stuff?
>   (Well, I can do this, but I'm not very fast :) )
>
> By the way, besides gp-export, I've considered using elephant's  
> migration
> for 0.9 to 1.0 upgrade:
>
> postmodern-0.9 --migration--> bdb-0.9 --upgrade--> bdb-1.0 -- 
> migration-->
> postmodern-1.0
>
> However, migration does not work right in version 0.9... And I'm not  
> very
> enthusiastic about
> digging old code. I've asked about this some time ago, as nobody  
> replied I'm
> assuming nobody
> cares, so I'm officially giving up.
>
> with best regards, Alex 'killerstorm' Mizrahi.
>
>
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel




From eslick at media.mit.edu  Sun Apr 26 11:48:07 2009
From: eslick at media.mit.edu (Ian Eslick)
Date: Sun, 26 Apr 2009 07:48:07 -0400
Subject: [elephant-devel] gp-export
In-Reply-To: <20483cdde4bf994a4f95da59784955c0.squirrel@mail.stardawn.org>
References: <gssrfc$6k4$1@ger.gmane.org>
	<20483cdde4bf994a4f95da59784955c0.squirrel@mail.stardawn.org>
Message-ID: <B2FFBC27-B666-4EC8-BB5E-122C9201E95F@media.mit.edu>

>> However, for now gp-export does not back up whole store -- it only  
>> iterates
>> through all objects
>> it finds in index and dumps them together with all their slots etc.  
>> It is
>> such because it was meant
>> to be working with various stores. (But it supports only elephant/ 
>> postmodern
>> so far.)
>
> Hmm... but it will walk all class indices of the store?
>

The manual defines reachability as:
1) All instances in the class indices
2) All objects in the root tree
3) All objects pointed to by objects in #1 and #2

The root btree may contain an array which then points to standard  
objects.  All should be copied.
References from arrays, standard objects, etc to persistent objects  
should also be retained.

Does gp-export handle arrays and other aggregates, reference integrity  
and circularity detection?

>>   Should it be somehow bundled with elephant or maintained  
>> separately?
>
> It depends on how well its components are separated from each other.
>
> If the Elephant-specific parts can be separated easily from the rest
> then we should put up the latter.

I suspect that a proper solution is likely to be elephant-specific.   
If it is used and useful for other lisp-based OODBs then I see no  
reason it shouldn't be maintained separately and made a dependency for  
elephant.

>
>>   Are there people who'd like to hack it to add support for different
>> stuff?
>
>

I might for my own tasks.


Ian