From ben at medianstrip.net  Sun Oct  3 01:12:23 2004
From: ben at medianstrip.net (Ben)
Date: Sat, 2 Oct 2004 21:12:23 -0400 (EDT)
Subject: [elephant-devel] Re: Elephant 0.2
In-Reply-To: <20040922195213.D38182@contarex.medianstrip.net>
References: <20040922203611.21978.qmail@web52309.mail.yahoo.com>
	<20040922195213.D38182@contarex.medianstrip.net>
Message-ID: <20041002211110.P50518@contarex.medianstrip.net>

hi bill --

thanks for the blog post about the tutorial.  if you have time i'd
like to try to help you work out your win32 problems.  i see no reason
why it shouldn't work, though i'm no win32 expert.

what lisp and C compiler are you using?  where does the failure occur?

take care, B

On Wed, 22 Sep 2004, Ben wrote:

> what fails?  I assume you're using Allegro?  what's your C compiler?
>
> win32 is not a platform i test on as i don't personally own a win32
> machine.  it'd be interesting to see if this would work.  probably the
> way i've written the C code wants to be compile by GCC as opposed to
> VC++ or something, though i'm no expert on these matters.
>
> B
>
> On Wed, 22 Sep 2004, Bill Clementson wrote:
>
>> I tried to build on Win32 but failed. Has anyone
>> successfully built elephant on win32?
>> 
>> - Bill
>> 
>> --- Ben <ben at medianstrip.net> wrote:
>> 
>>> 
>>> Elephant 0.2 was released on September 19th, 2004.
>>> This is an BETA
>>> release. New features:
>>> 
>>>      * Secondary indices and cursors
>>>      * PPC Darwin OpenMCL / SBCL
>>>      * Doc strings and improved documentation
>>>      * An RT-based test suite
>>>      * many bugfixes
>>> 
>>> This release has been tested on CMUCL 19a, SBCL
>>> 0.8.14 and Allegro 6.2
>>> on x86 Linux and FreeBSD, and OpenMCL 0.14.2-p1 and
>>> SBCL 0.8.14 on PPC
>>> Darwin.
>>> 
>>> Tarballs (and now HTML docs) can be found on the
>>> website:
>>> 
>>> http://www.common-lisp.net/project/elephant
>>> 
>>> This release should be a lot more stable than the
>>> last.  Enjoy!
>>> 
>>> Ben and Andrew
>>> 
>>> PS Nicholas, this release has a map-btree function,
>>> in addition to
>>> more general cursors.
>>> 
>> 
>> 
>> 
>> 
>> _______________________________
>> Do you Yahoo!?
>> Declare Yourself - Register online to vote today!
>> http://vote.yahoo.com
>> 
>



From ben at medianstrip.net  Mon Oct  4 02:25:53 2004
From: ben at medianstrip.net (Ben)
Date: Sun, 3 Oct 2004 22:25:53 -0400 (EDT)
Subject: [elephant-devel] Re: Elephant 0.2
In-Reply-To: <20041003231711.21219.qmail@web52302.mail.yahoo.com>
References: <20041003231711.21219.qmail@web52302.mail.yahoo.com>
Message-ID: <20041003202157.F65722@contarex.medianstrip.net>

over the next few days i'll try to get my hands on a computer with
vc++, and try to get it to run.  i don't know where, exactly.....

B

On Sun, 3 Oct 2004, Bill Clementson wrote:

> Hi Ben,
>
>> thanks for the blog post about the tutorial.
>
> thank you for creating elephant and making it
> available to others!
>
>> if you
>> have time i'd
>> like to try to help you work out your win32
>> problems.  i see no reason
>> why it shouldn't work, though i'm no win32 expert.
>>
>> what lisp and C compiler are you using?  where does
>> the failure occur?
>
> I have the win32 versions of acl, lw and clisp on my
> pc. I don't use gcc on win32, I use MS VC++. I used
> the MS VC++ compiler to compile the Berkeley DB
> without any problems. I tried to compile
> src/libsleepycat.c specifying the
> C:\bin\db-4.2.52.NC\build_win32\Release directory as
> the library directory (there wasn't a /lib/ directory
> as seemed to be indicated in your makefile) and
> C:\bin\db-4.2.52.NC\dbinc as the include directory
> (there wasn't a /include/ directory either). The
> compile produced a lot of missing dependencies so I
> just gave up as I didn't have time to search around
> for the missing things. I also didn't know how to
> translate the gcc compile & link options for the MSVC
> compiler.
>
> I'm going to be at a conference and in training most
> of the next 2 weeks, so won't have much time to try
> alternative compile options; however, if you have any
> ideas fire them at me and I'll try them when I get the
> opportunity.
>
> Cheers,
> Bill
>
>
>
>
> __________________________________
> Do you Yahoo!?
> New and Improved Yahoo! Mail - 100MB free storage!
> http://promotions.yahoo.com/new_mail
>



From ben at medianstrip.net  Mon Oct  4 23:25:21 2004
From: ben at medianstrip.net (Ben)
Date: Mon, 4 Oct 2004 19:25:21 -0400 (EDT)
Subject: [elephant-devel] Re: Elephant 0.2
In-Reply-To: <20041004031651.50833.qmail@web52310.mail.yahoo.com>
References: <20041004031651.50833.qmail@web52310.mail.yahoo.com>
Message-ID: <20041004190502.O76573@contarex.medianstrip.net>

i've managed to get the libsleepycat.dll to compile.  however i can't
test it with allegro / lispworks b/c i'm not allowed to install
software on the windows boxes i have access to.

i can either send you instructions / patched files which let you test
this, and you can test it, or you can wait for me to find a windows
box i can install allegro on.

B

On Sun, 3 Oct 2004, Bill Clementson wrote:

> That will be neat if you can get it working on win32.
>
> - Bill
>
> --- Ben <ben at medianstrip.net> wrote:
>
>> over the next few days i'll try to get my hands on a
>> computer with
>> vc++, and try to get it to run.  i don't know where,
>> exactly.....
>>
>> B
>>
>> On Sun, 3 Oct 2004, Bill Clementson wrote:
>>
>>> Hi Ben,
>>>
>>>> thanks for the blog post about the tutorial.
>>>
>>> thank you for creating elephant and making it
>>> available to others!
>>>
>>>> if you
>>>> have time i'd
>>>> like to try to help you work out your win32
>>>> problems.  i see no reason
>>>> why it shouldn't work, though i'm no win32
>> expert.
>>>>
>>>> what lisp and C compiler are you using?  where
>> does
>>>> the failure occur?
>>>
>>> I have the win32 versions of acl, lw and clisp on
>> my
>>> pc. I don't use gcc on win32, I use MS VC++. I
>> used
>>> the MS VC++ compiler to compile the Berkeley DB
>>> without any problems. I tried to compile
>>> src/libsleepycat.c specifying the
>>> C:\bin\db-4.2.52.NC\build_win32\Release directory
>> as
>>> the library directory (there wasn't a /lib/
>> directory
>>> as seemed to be indicated in your makefile) and
>>> C:\bin\db-4.2.52.NC\dbinc as the include directory
>>> (there wasn't a /include/ directory either). The
>>> compile produced a lot of missing dependencies so
>> I
>>> just gave up as I didn't have time to search
>> around
>>> for the missing things. I also didn't know how to
>>> translate the gcc compile & link options for the
>> MSVC
>>> compiler.
>>>
>>> I'm going to be at a conference and in training
>> most
>>> of the next 2 weeks, so won't have much time to
>> try
>>> alternative compile options; however, if you have
>> any
>>> ideas fire them at me and I'll try them when I get
>> the
>>> opportunity.
>>>
>>> Cheers,
>>> Bill
>>>
>>>
>>>
>>>
>>> __________________________________
>>> Do you Yahoo!?
>>> New and Improved Yahoo! Mail - 100MB free storage!
>>> http://promotions.yahoo.com/new_mail
>>>
>>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>



From ben at medianstrip.net  Tue Oct  5 04:47:08 2004
From: ben at medianstrip.net (Ben)
Date: Tue, 5 Oct 2004 00:47:08 -0400 (EDT)
Subject: [elephant-devel] Re: Elephant 0.2
In-Reply-To: <20041005023643.23358.qmail@web52305.mail.yahoo.com>
References: <20041005023643.23358.qmail@web52305.mail.yahoo.com>
Message-ID: <20041005004124.G2017@contarex.medianstrip.net>

the relevant files are in

http://common-lisp.net/project/elephant/win32/

create a DLL project / solution.  add the two files (one is a modified
libsleepycat.c, the other a .def file for dll export.)  point the
project / solution at the berkeley db headers.  tell the project to
link with libdb42.lib (should be in the releases dir.)  build.

if that works, now point sleepycat.lisp at the appropriate files (you
may only need the libsleepycat.dll since we statically linked vs
libdb42.lib.)

thanks for the help and take care, B

On Mon, 4 Oct 2004, Bill Clementson wrote:

> Wow, that was quick.
>
> If you like, you can send me the instructions. If I
> get the opportunity, I'll try it over the next couple
> of days. As I mentioned, I'm out of town this week
> but, if I can, I'll try compiling/testing in the
> evenings if I can.
>
> - Bill
>
> --- Ben <ben at medianstrip.net> wrote:
>
>> i've managed to get the libsleepycat.dll to compile.
>>  however i can't
>> test it with allegro / lispworks b/c i'm not allowed
>> to install
>> software on the windows boxes i have access to.
>>
>> i can either send you instructions / patched files
>> which let you test
>> this, and you can test it, or you can wait for me to
>> find a windows
>> box i can install allegro on.
>>
>> B
>>
>> On Sun, 3 Oct 2004, Bill Clementson wrote:
>>
>>> That will be neat if you can get it working on
>> win32.
>>>
>>> - Bill
>>>
>>> --- Ben <ben at medianstrip.net> wrote:
>>>
>>>> over the next few days i'll try to get my hands
>> on a
>>>> computer with
>>>> vc++, and try to get it to run.  i don't know
>> where,
>>>> exactly.....
>>>>
>>>> B
>>>>
>>>> On Sun, 3 Oct 2004, Bill Clementson wrote:
>>>>
>>>>> Hi Ben,
>>>>>
>>>>>> thanks for the blog post about the tutorial.
>>>>>
>>>>> thank you for creating elephant and making it
>>>>> available to others!
>>>>>
>>>>>> if you
>>>>>> have time i'd
>>>>>> like to try to help you work out your win32
>>>>>> problems.  i see no reason
>>>>>> why it shouldn't work, though i'm no win32
>>>> expert.
>>>>>>
>>>>>> what lisp and C compiler are you using?  where
>>>> does
>>>>>> the failure occur?
>>>>>
>>>>> I have the win32 versions of acl, lw and clisp
>> on
>>>> my
>>>>> pc. I don't use gcc on win32, I use MS VC++. I
>>>> used
>>>>> the MS VC++ compiler to compile the Berkeley DB
>>>>> without any problems. I tried to compile
>>>>> src/libsleepycat.c specifying the
>>>>> C:\bin\db-4.2.52.NC\build_win32\Release
>> directory
>>>> as
>>>>> the library directory (there wasn't a /lib/
>>>> directory
>>>>> as seemed to be indicated in your makefile) and
>>>>> C:\bin\db-4.2.52.NC\dbinc as the include
>> directory
>>>>> (there wasn't a /include/ directory either). The
>>>>> compile produced a lot of missing dependencies
>> so
>>>> I
>>>>> just gave up as I didn't have time to search
>>>> around
>>>>> for the missing things. I also didn't know how
>> to
>>>>> translate the gcc compile & link options for the
>>>> MSVC
>>>>> compiler.
>>>>>
>>>>> I'm going to be at a conference and in training
>>>> most
>>>>> of the next 2 weeks, so won't have much time to
>>>> try
>>>>> alternative compile options; however, if you
>> have
>>>> any
>>>>> ideas fire them at me and I'll try them when I
>> get
>>>> the
>>>>> opportunity.
>>>>>
>>>>> Cheers,
>>>>> Bill
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> __________________________________
>>>>> Do you Yahoo!?
>>>>> New and Improved Yahoo! Mail - 100MB free
>> storage!
>>>>> http://promotions.yahoo.com/new_mail
>>>>>
>>>>
>>>
>>>
>>> __________________________________________________
>>> Do You Yahoo!?
>>> Tired of spam?  Yahoo! Mail has the best spam
>> protection around
>>> http://mail.yahoo.com
>>>
>>
>
>
>
>
> _______________________________
> Do you Yahoo!?
> Declare Yourself - Register online to vote today!
> http://vote.yahoo.com
>



From ben at medianstrip.net  Wed Oct  6 20:59:13 2004
From: ben at medianstrip.net (Ben)
Date: Wed, 6 Oct 2004 16:59:13 -0400 (EDT)
Subject: [elephant-devel] Re: Elephant 0.2
In-Reply-To: <20041006200514.42976.qmail@web52301.mail.yahoo.com>
References: <20041006200514.42976.qmail@web52301.mail.yahoo.com>
Message-ID: <20041006165245.J10792@contarex.medianstrip.net>

i compiled with visual studio.net.  i don't know if this should make a
difference.

what happens if you move the #include <string.h> to the first line of
the file?

are you linking with a C runtime library?

B

ps obviously, i know nothing about win32 C programming!

On Wed, 6 Oct 2004, Bill Clementson wrote:

> Hi Ben,
>
> I followed your instructions and got the following:
>
> --------------------Configuration: elephant - Win32
> Debug--------------------
> Compiling...
> libsleepycat.c
> c:\usr\home\lisp\elephant-0.2\src\libsleepycat.c(62) :
> warning C4013: 'memcpy' undefined; assuming extern
> returning int
> c:\program files\microsoft visual
> studio\vc98\include\string.h(101) : error C2040:
> 'memcpy' : 'void *(void *,const void *,unsigned int )'
> differs in levels of indirection from 'void *(void
> *,const void *,unsigned int )'
> c:\usr\home\lisp\elephant-0.2\src\libsleepycat.c(273)
> : warning C4013: 'case_cmp' undefined; assuming extern
> returning int
> c:\usr\home\lisp\elephant-0.2\src\libsleepycat.c(277)
> : warning C4013: 'utf16_cmp' undefined; assuming
> extern returning int
> c:\usr\home\lisp\elephant-0.2\src\libsleepycat.c(279)
> : warning C4013: 'lex_cmp' undefined; assuming extern
> returning int
> Error executing cl.exe.
>
> elephant.dll - 1 error(s), 4 warning(s)
>



From ben at medianstrip.net  Thu Oct  7 01:38:30 2004
From: ben at medianstrip.net (Ben)
Date: Wed, 6 Oct 2004 21:38:30 -0400 (EDT)
Subject: [elephant-devel] Re: Elephant 0.2
In-Reply-To: <20041006234919.43301.qmail@web52302.mail.yahoo.com>
References: <20041006234919.43301.qmail@web52302.mail.yahoo.com>
Message-ID: <20041006213719.W34845@contarex.medianstrip.net>

great -- i'll commit some CVS changes so that other win32 folk can get
it to work too.  let me know if you run into other problems.

take care, B

On Wed, 6 Oct 2004, Bill Clementson wrote:

> It works! I'm using vc++ 6 and moving the #include to
> the top of the file worked. I also had to make a few
> minor mods to sleepycat.lisp (file locations) but then
> everything built and I was able to run some of the
> intial tests from your web site without problem. I'm
> doing this in between meetings, so it might be a while
> before I can try things out properly.
>
> Thanks for your help in getting this going. I'll let
> you know how I get on with testing it.
>
> Cheers,
> Bill
>
> --- Ben <ben at medianstrip.net> wrote:
>
>> i compiled with visual studio.net.  i don't know if
>> this should make a
>> difference.
>>
>> what happens if you move the #include <string.h> to
>> the first line of
>> the file?
>>
>> are you linking with a C runtime library?
>>
>> B
>>
>> ps obviously, i know nothing about win32 C
>> programming!
>>
>> On Wed, 6 Oct 2004, Bill Clementson wrote:
>>
>>> Hi Ben,
>>>
>>> I followed your instructions and got the
>> following:
>>>
>>> --------------------Configuration: elephant -
>> Win32
>>> Debug--------------------
>>> Compiling...
>>> libsleepycat.c
>>>
>> c:\usr\home\lisp\elephant-0.2\src\libsleepycat.c(62)
>> :
>>> warning C4013: 'memcpy' undefined; assuming extern
>>> returning int
>>> c:\program files\microsoft visual
>>> studio\vc98\include\string.h(101) : error C2040:
>>> 'memcpy' : 'void *(void *,const void *,unsigned
>> int )'
>>> differs in levels of indirection from 'void *(void
>>> *,const void *,unsigned int )'
>>>
>>
> c:\usr\home\lisp\elephant-0.2\src\libsleepycat.c(273)
>>> : warning C4013: 'case_cmp' undefined; assuming
>> extern
>>> returning int
>>>
>>
> c:\usr\home\lisp\elephant-0.2\src\libsleepycat.c(277)
>>> : warning C4013: 'utf16_cmp' undefined; assuming
>>> extern returning int
>>>
>>
> c:\usr\home\lisp\elephant-0.2\src\libsleepycat.c(279)
>>> : warning C4013: 'lex_cmp' undefined; assuming
>> extern
>>> returning int
>>> Error executing cl.exe.
>>>
>>> elephant.dll - 1 error(s), 4 warning(s)
>>>
>>
>
>
>
>
> _______________________________
> Do you Yahoo!?
> Declare Yourself - Register online to vote today!
> http://vote.yahoo.com
>



From ben at medianstrip.net  Fri Oct  8 00:40:23 2004
From: ben at medianstrip.net (Ben)
Date: Thu, 7 Oct 2004 20:40:23 -0400 (EDT)
Subject: [elephant-devel] Re: Elephant 0.2
In-Reply-To: <20041007164132.68273.qmail@web52307.mail.yahoo.com>
References: <20041007164132.68273.qmail@web52307.mail.yahoo.com>
Message-ID: <20041007203833.M49336@contarex.medianstrip.net>

thanks again.  i'm rolling another tarball for the win32 fixes
(amongst a few other things.)  let me know if you run into other
problems.

i've taken the liberty of quoting bits of your weblog in the
installation instructions.  i hope this is ok.  (you're listed in the
CREDITS now too.)

take care, B

On Thu, 7 Oct 2004, Bill Clementson wrote:

> Hi Ben,
>
> I've posted on my weblog about elephant today:
> http://home.comcast.net/~bc19191/blog/041007.html
>
> Maybe you'll get a few more win32 guys trying out
> elephant. I mentioned that the modified libsleepycat.c
> and libsleepycat.def files should be in cvs.
> Presumably you've either checked them in or will be
> checking them in soon?
>
> Thanks again for your help.
>
> Cheers,
> Bill



From ben at medianstrip.net  Fri Oct  8 02:34:30 2004
From: ben at medianstrip.net (Ben)
Date: Thu, 7 Oct 2004 22:34:30 -0400 (EDT)
Subject: [elephant-devel] Elephant 0.2.1
Message-ID: <20041007223250.X59717@contarex.medianstrip.net>

I've released Elephant 0.2.1.  Tarballs are on the webpage.  This is a
minor bugfix release.  Thanks to Bill Clementson for help in getting
Elephant to compile on Win32 / VC++ -- instructions and supporting
files are part of the tarball now.

Take care, B



From peter at peterbengtson.com  Wed Oct 20 12:37:00 2004
From: peter at peterbengtson.com (Peter Bengtson)
Date: Wed, 20 Oct 2004 14:37:00 +0200
Subject: [elephant-devel] Transient slots in non-persistent objects
Message-ID: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>

Hi,

I'm developing a server application in OpenMCL where I'm using Elephant 
to store one single (large) object, read into memory at startup and 
dumped back to the DB at shutdown. Thus, I do not use persistent CLOS 
objects, only standard ones.

The data structure is heavily circular, and contains CLOS objects with 
slots pointing to OS resources, such as sockets. I don't wish to save 
these slots - they are transient.

Is there a way to specify this behaviour? I guess only the serializer 
needs to be modified. It is of course possible work around it with an 
:AROUND method on the slots in question and a special variable which 
will make the accessor functions simply return NIL, but this is kludgy 
and adds unnecessary overhead.

	/ Peter




From ben at medianstrip.net  Wed Oct 20 15:42:04 2004
From: ben at medianstrip.net (Ben)
Date: Wed, 20 Oct 2004 11:42:04 -0400 (EDT)
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
Message-ID: <20041020111514.I48634@contarex.medianstrip.net>

At the moment, there is no way to make transient slots on ordinary
objects.  On first glance, I don't know how to support non-standard
slot options in the standard slot metaclass.  The serializer uses
"slot-value", not the accessors, so I think unless OpenMCL calls
"slot-value-using-class" (which I don't remember off the top of my
head) you'll have a hard time overriding that.

There are some other kludges I can think of, but they are just
kludges, roughly equivalent to the one you've suggested.  One thing
you can do is keep track of the objects which hold OS resources.  For
example, you can put something in initialize-instance which adds the
instance to a collection stored somewhere on the class object.  (EQ
hashtables are probably good for this.)  Then you can have a method on
the class object which walks the collection and NULLs the offending
slots.  Call this before shutdown / save.

Honestly, your usage pattern wasn't one I had in mind when Elephant
was designed.  Some of your concerns might be fixed by
"single-user-mode", but unfortunately given time constraints that
probably won't get done until at least the end of November.  Can I ask
what is wrong with using persistent objects?  Is it caching (DB hits
on read), DB hits on write, or lazy loading?

You might consider flipping various flags on the DB for performance,
like DB_PRIVATE, DB_TXN_NO_SYNC, etc.

take care, B

On Wed, 20 Oct 2004, Peter Bengtson wrote:

> Hi,
>
> I'm developing a server application in OpenMCL where I'm using Elephant to 
> store one single (large) object, read into memory at startup and dumped back 
> to the DB at shutdown. Thus, I do not use persistent CLOS objects, only 
> standard ones.
>
> The data structure is heavily circular, and contains CLOS objects with slots 
> pointing to OS resources, such as sockets. I don't wish to save these slots - 
> they are transient.
>
> Is there a way to specify this behaviour? I guess only the serializer needs 
> to be modified. It is of course possible work around it with an :AROUND 
> method on the slots in question and a special variable which will make the 
> accessor functions simply return NIL, but this is kludgy and adds unnecessary 
> overhead.
>
> 	/ Peter
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>



From peter at peterbengtson.com  Wed Oct 20 16:22:20 2004
From: peter at peterbengtson.com (Peter Bengtson)
Date: Wed, 20 Oct 2004 18:22:20 +0200
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <20041020111514.I48634@contarex.medianstrip.net>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020111514.I48634@contarex.medianstrip.net>
Message-ID: <37678BE8-22B4-11D9-83D2-000D932E263E@peterbengtson.com>

Hi Ben,

There is nothing wrong with using persistent objects throughout the 
application - I may indeed go that way in the end - but at the moment 
I'm trying not to tie the database model to the application to closely. 
It should, ideally, be possible to use any one of a number of 
persistent storage methods.

The database will probably be kept in memory - I'll rely on the OS to 
handle the memory issues through VM - so the storage overhead of 
caching OIDs and keeping buffers for detecting circularities, etc, is 
something I wish to avoid. And objects and their slot valus need to be 
compared through identity.

There are two ways to go: either follow the minimalist approach, or 
embrace persistence entirely and use it throughout. Or, in other words, 
use a memory-based or a disk-based approach. For now, I want to keep to 
the memory-based approach, but this may well change later.

A thought struck me: looking at #'slots-and-values in serializer.lisp, 
I see you are using #'compute-slots, a generic function, to obtain the 
names of the slots to be saved. Wouldn't it be possible to simply 
specialize on #'compute-slots to obtain the effect I'm after?

	/ Peter


2004-10-20 kl. 17.42 skrev Ben:

> At the moment, there is no way to make transient slots on ordinary
> objects.  On first glance, I don't know how to support non-standard
> slot options in the standard slot metaclass.  The serializer uses
> "slot-value", not the accessors, so I think unless OpenMCL calls
> "slot-value-using-class" (which I don't remember off the top of my
> head) you'll have a hard time overriding that.
>
> There are some other kludges I can think of, but they are just
> kludges, roughly equivalent to the one you've suggested.  One thing
> you can do is keep track of the objects which hold OS resources.  For
> example, you can put something in initialize-instance which adds the
> instance to a collection stored somewhere on the class object.  (EQ
> hashtables are probably good for this.)  Then you can have a method on
> the class object which walks the collection and NULLs the offending
> slots.  Call this before shutdown / save.
>
> Honestly, your usage pattern wasn't one I had in mind when Elephant
> was designed.  Some of your concerns might be fixed by
> "single-user-mode", but unfortunately given time constraints that
> probably won't get done until at least the end of November.  Can I ask
> what is wrong with using persistent objects?  Is it caching (DB hits
> on read), DB hits on write, or lazy loading?
>
> You might consider flipping various flags on the DB for performance,
> like DB_PRIVATE, DB_TXN_NO_SYNC, etc.
>
> take care, B




From peter at peterbengtson.com  Wed Oct 20 17:28:15 2004
From: peter at peterbengtson.com (Peter Bengtson)
Date: Wed, 20 Oct 2004 19:28:15 +0200
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <20041020111514.I48634@contarex.medianstrip.net>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020111514.I48634@contarex.medianstrip.net>
Message-ID: <6CB66D30-22BD-11D9-83D2-000D932E263E@peterbengtson.com>

Would this work?

;;;
;;; Patch Elephant to allow specification of transient slots for 
non-persistent objects
;;;

(in-package :elephant)

(defun slots-and-values (o)
   (declare (optimize (speed 3) (safety 0)))
   (let ((class (class-of o)))
     (loop for sd in (compute-slots class)
	  for slot-name = (slot-definition-name sd)
	  with ret = ()
	  do
	  (when (and (persistent-slot-p class slot-name)
	             (slot-boundp o slot-name)
		         (eq :instance (slot-definition-allocation sd)))
	    (push (slot-value o slot-name) ret)
	    (push slot-name ret))
	  finally (return ret))))


(defmethod persistent-slot-p ((class t) (slot-name t))
   t)


(export 'persistent-slot-p)


;;;
;;; This would be used in the application in the following manner, 
assuming there is
;;; a class called CLIENT with a slot called CLIENT-SOCKET containing 
transient data:

(in-package :cl-user)

(defmethod persistent-slot-p ((class client) (slot-name (eql 
'client-socket)))
   nil)


	/ Peter




From ben at medianstrip.net  Wed Oct 20 17:58:37 2004
From: ben at medianstrip.net (Ben)
Date: Wed, 20 Oct 2004 13:58:37 -0400 (EDT)
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <37678BE8-22B4-11D9-83D2-000D932E263E@peterbengtson.com>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020111514.I48634@contarex.medianstrip.net>
	<37678BE8-22B4-11D9-83D2-000D932E263E@peterbengtson.com>
Message-ID: <20041020135516.O61332@contarex.medianstrip.net>

> Hi Ben,
>
> There is nothing wrong with using persistent objects throughout the 
> application - I may indeed go that way in the end - but at the moment I'm 
> trying not to tie the database model to the application to closely. It 
> should, ideally, be possible to use any one of a number of persistent storage 
> methods.

fair enough.  You can always use macros to encapsulate declarations of
your classes.

> The database will probably be kept in memory - I'll rely on the OS to handle 
> the memory issues through VM - so the storage overhead of caching OIDs and 
> keeping buffers for detecting circularities, etc, is something I wish to 
> avoid. And objects and their slot valus need to be compared through identity.
>
> There are two ways to go: either follow the minimalist approach, or embrace 
> persistence entirely and use it throughout. Or, in other words, use a 
> memory-based or a disk-based approach. For now, I want to keep to the 
> memory-based approach, but this may well change later.

Sleepycat uses shared memory, so supposedly a lot of database
operations don't touch the disk.  As far as object identity is
concerned, it is hard to maintain that for any persistence system
across application invocations without some sort of OID system.  This
is one of the reasons to use persistent classes.

> A thought struck me: looking at #'slots-and-values in serializer.lisp, I see 
> you are using #'compute-slots, a generic function, to obtain the names of the 
> slots to be saved. Wouldn't it be possible to simply specialize on 
> #'compute-slots to obtain the effect I'm after?

yes, though doing that may be difficult cross-lisp.  If you're using
just OpenMCL, it might be not so hard.

take care, B



From ben at medianstrip.net  Wed Oct 20 18:07:58 2004
From: ben at medianstrip.net (Ben)
Date: Wed, 20 Oct 2004 14:07:58 -0400 (EDT)
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <6CB66D30-22BD-11D9-83D2-000D932E263E@peterbengtson.com>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020111514.I48634@contarex.medianstrip.net>
	<6CB66D30-22BD-11D9-83D2-000D932E263E@peterbengtson.com>
Message-ID: <20041020140310.F61332@contarex.medianstrip.net>

that's an intriguing option.  the MOP way to do this (not that it's
particularly better) would be to subclass the standard slot definition
class, and have the persistent-slot-p method specialize on class, not
name.  in fact we already do this for slots in persistent classes, so
there is an opportunity for integration here.  however getting
defclass to use non-standard slot definition objects might be just as
much work as using persistent classes.

in any case, i'll take a look at merging your patch soon and let you
know.

take care, B

On Wed, 20 Oct 2004, Peter Bengtson wrote:

> Would this work?
>
> ;;;
> ;;; Patch Elephant to allow specification of transient slots for 
> non-persistent objects
> ;;;
>
> (in-package :elephant)
>
> (defun slots-and-values (o)
>  (declare (optimize (speed 3) (safety 0)))
>  (let ((class (class-of o)))
>    (loop for sd in (compute-slots class)
> 	  for slot-name = (slot-definition-name sd)
> 	  with ret = ()
> 	  do
> 	  (when (and (persistent-slot-p class slot-name)
> 	             (slot-boundp o slot-name)
> 		         (eq :instance (slot-definition-allocation sd)))
> 	    (push (slot-value o slot-name) ret)
> 	    (push slot-name ret))
> 	  finally (return ret))))
>
>
> (defmethod persistent-slot-p ((class t) (slot-name t))
>  t)
>
>
> (export 'persistent-slot-p)
>
>
> ;;;
> ;;; This would be used in the application in the following manner, assuming 
> there is
> ;;; a class called CLIENT with a slot called CLIENT-SOCKET containing 
> transient data:
>
> (in-package :cl-user)
>
> (defmethod persistent-slot-p ((class client) (slot-name (eql 
> 'client-socket)))
>  nil)
>
>
> 	/ Peter
>



From dankna at accela.net  Wed Oct 20 23:43:14 2004
From: dankna at accela.net (Dan Knapp)
Date: Wed, 20 Oct 2004 19:43:14 -0400
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <20041020140310.F61332@contarex.medianstrip.net>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020111514.I48634@contarex.medianstrip.net>
	<6CB66D30-22BD-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020140310.F61332@contarex.medianstrip.net>
Message-ID: <CF9B4883-22F1-11D9-88C5-000A95CBEDB2@accela.net>

> that's an intriguing option.  the MOP way to do this (not that it's
> particularly better) would be to subclass the standard slot definition
> class, and have the persistent-slot-p method specialize on class, not
> name.  in fact we already do this for slots in persistent classes, so
> there is an opportunity for integration here.  however getting
> defclass to use non-standard slot definition objects might be just as
> much work as using persistent classes.

   Why do you need to subclass it?  Can't you just define an :around
method for standard-slot (or whatever it's called)?  Then defclass
wouldn't need to do anything special.  I haven't thought this through,
it's just an idea.

   Hi, by the way!  I decided a couple days ago to give Elephant a try.
It was slightly annoying to install the prerequisites (I wish 
asdf-install
were integrated with fink), but I got everything working.  It's a lot
more impressive than I thought it would be.

   While I'm writing, might as well ask my question.  It's not clear
from the manual:  Is there any way, kludgey or otherwise, to
redefine a persistent class and have all instances of it be updated?

-- Dan Knapp




From ben at medianstrip.net  Thu Oct 21 18:44:04 2004
From: ben at medianstrip.net (Ben)
Date: Thu, 21 Oct 2004 14:44:04 -0400 (EDT)
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <CF9B4883-22F1-11D9-88C5-000A95CBEDB2@accela.net>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020111514.I48634@contarex.medianstrip.net>
	<6CB66D30-22BD-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020140310.F61332@contarex.medianstrip.net>
	<CF9B4883-22F1-11D9-88C5-000A95CBEDB2@accela.net>
Message-ID: <20041021141936.L72634@contarex.medianstrip.net>

>  Why do you need to subclass it?  Can't you just define an :around
> method for standard-slot (or whatever it's called)?  Then defclass
> wouldn't need to do anything special.  I haven't thought this through,
> it's just an idea.

an around method specializing on the standard slot-definition would
affect all slots in all classes, unless there's something clever i'm
not seeing.

>  Hi, by the way!  I decided a couple days ago to give Elephant a try.
> It was slightly annoying to install the prerequisites (I wish asdf-install
> were integrated with fink), but I got everything working.  It's a lot
> more impressive than I thought it would be.

hello!  yes, i wish there was a way for a quicker install.  but
there's a lot of C code.....

>  While I'm writing, might as well ask my question.  It's not clear
> from the manual:  Is there any way, kludgey or otherwise, to
> redefine a persistent class and have all instances of it be updated?

it depends on what you want.  things that should "just work": adding
slots without new default / computed values; removing slots (though
this may leave some garbage in the DB.)

basically, as explained in the tutorial, there is no "table layout"
for persistent objects, so there's usually no need for things like
"schema update."  objects are put into the database, one row per slot,
keyed by OID and slot-name.  unbound slots are "empty rows."
therefore if you add a slot to a class, every instance will get that
slot, it will just be unbound.  if you remove a slot, there will be a
bunch of garbage rows in the DB (and hope you don't add that slot back
or else you might get confused!)  eventually when a garbage collector
is written, those rows will be collected.

if you want to do something fancy with default values /
reinitialize-instance, you may need some finesse.  if you know where
all instances of the affected class are, you can of course manually
load them and do whatever you want to them.  the problem with the
CLOS-standard change-class / redefinition machinery is that they
assume all instances of the object are "in memory", which of course is
false for any persistence mechanism.  i'm thinking that i might want
to add a bit to the MOP in that we might need a protocol which
distinguishes between creating a new instance and creating an instance
which is already in the database.  on the TODO list.

thanks for the interest, B



From peter at peterbengtson.com  Fri Oct 22 10:52:37 2004
From: peter at peterbengtson.com (Peter Bengtson)
Date: Fri, 22 Oct 2004 12:52:37 +0200
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <20041020140310.F61332@contarex.medianstrip.net>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020111514.I48634@contarex.medianstrip.net>
	<6CB66D30-22BD-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020140310.F61332@contarex.medianstrip.net>
Message-ID: <7CB7DAE6-2418-11D9-AA55-000D932E263E@peterbengtson.com>

The patch should of course look like this, not as previously posted - 
the object itself (o) should be passed to #'persistent-slot-p rather 
than the class object. However, you might of course decide to implement 
this in a different manner. Anyway:

(in-package :elephant)

(defun slots-and-values (o)
   (declare (optimize (speed 3) (safety 0)))
   (loop for sd in (compute-slots (class-of o))
	  for slot-name = (slot-definition-name sd)
	  with ret = ()
	  do
	  (when (and (persistent-slot-p o slot-name)
	             (slot-boundp o slot-name)
		         (eq :instance (slot-definition-allocation sd)))
	    (push (slot-value o slot-name) ret)
	    (push slot-name ret))
	  finally (return ret)))


(defmethod persistent-slot-p ((class t) (slot-name t))
   t)


(export 'persistent-slot-p)


		/ Peter




From dankna at accela.net  Fri Oct 22 22:27:51 2004
From: dankna at accela.net (Dan Knapp)
Date: Fri, 22 Oct 2004 18:27:51 -0400
Subject: [elephant-devel] Transient slots in non-persistent objects
In-Reply-To: <20041021141936.L72634@contarex.medianstrip.net>
References: <BCBF63C2-2294-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020111514.I48634@contarex.medianstrip.net>
	<6CB66D30-22BD-11D9-83D2-000D932E263E@peterbengtson.com>
	<20041020140310.F61332@contarex.medianstrip.net>
	<CF9B4883-22F1-11D9-88C5-000A95CBEDB2@accela.net>
	<20041021141936.L72634@contarex.medianstrip.net>
Message-ID: <9C80C38E-2479-11D9-88C5-000A95CBEDB2@accela.net>

> an around method specializing on the standard slot-definition would
> affect all slots in all classes, unless there's something clever i'm
> not seeing.

   Okay, I figured out what I was grasping at.  You define a method 
along these
lines:

(defmethod :around direct-slot-definition-class ((class standard-class) 
&rest initargs)
     (if (member :transient initargs)
         #.(find-class 'the-elephant-slot-definition-class)
         (call-next-method)))

   Then any classes which need to will use the appropriate class for
their slots rather than the standard one.  This could also be a primary
method, which is often easier than an :around method because loading
:around methods isn't idempotent; see the docs for
direct-slot-definition-class for details.

> it depends on what you want.  things that should "just work": adding
> slots without new default / computed values; removing slots (though
> this may leave some garbage in the DB.)

   This makes sense.  Glad to hear it!

-- Dan Knapp