[elephant-devel] Status of Leslie's question and Sean's patch?

[Ian Eslick]

On Dec 21, 2007, at 12:53 PM, Leslie P. Polzer wrote:

>
>> Please correct me if I'm wrong, but isn't this exactly what Lisp
>> does?  If you assign the wrong type to a slot under SBCL, it doesn't
>> complain so in this sense Elephant behaves in the same way as Lisp
>> does with in-memory objects.  At some point it is our responsibility
>> not to shoot ourselves in the foot!
>
> Sure, but I'm not talking about preventing that automatically, but  
> about
> giving the programmer an opportunity to do agile development by
> specifying what should be done when data types have changed.
>
> I'm so after this not only because it'd be very useful for me, but  
> because
> this is exactly the way the rest of Lisp and esp. CLOS/MOP works:
> if “do what I mean” won't work, give the programmer an opportunity
> to clarify.

>
>> I would like to improve the default behavior and error
>> handling of these cases, but the interaction between MOP functions
>> such as update-instance-for-redefined-class method and persistent
>> instances is pretty complicated.  For example, if you don't map all
>> instances of an object after class redefinition, then some objects  
>> are
>> in-line with an old schema, and some with the new.  That's not a
>> problem if the update method is called on class initialization, but
>> there is no guarantee from lisp that this will happen because you
>> could exit and restart before you've loaded every instance.
>
> Not really a problem. I haven't looked at how the MOP plays into
> this, but the following will work:
>
> 1) slot of type A
> 2) slot of type B; data store partly updated
> 3) slot of type C
>
> as long as *both* the transitions A->C and B->C are defined.
> Of course, this way one has to provide a quadratic number of  
> transitions,
> but: first, we assume a schema doesn't change *that* often that this
> becomes a problem; and if it does, the programmer could still employ
> map-root at some point to update them all to the latest schema.
> Second, we could at some point in the future provide a protocol
> that lets us record schema changes so we can do just A->B->C
> whereby a linear number of transitions is achieved.

You are quite right.  I should probably have said that the benefits of  
making this sort of thing work cleanly better has yet to justify the  
cost to support in my mind.  Schemas and schema evolution solve quite  
a few problems (evolution of non-persistent objects, etc), but fitting  
that into the MOP and the existing Elephant is a significant piece of  
work touching most of the Elephant code base in one way or another.


>
>> Lazy redefinition can't be guaranteed to be consistent.
>
> With the above and the assumption that the programmer doesn't shoot
> herself in the foot, it can be guaranteed (correct me if I'm wrong).

It probably can be guaranteed, but the assumption set necessary to  
ensure it is critical.  We can talk about this after you've looked at  
the problem some more.

For example.  In order to convert an instance of A to an instance of B  
we have to have a persistent function.  The only way to store  
functions today are to store forms that we can compile into  
functions.  These forms will require external dependencies to be  
maintained.  If someone is working in a live Agile system and an A->B  
form is stored somewhere and a function it depends on gets redefined,  
it is not necessarily obvious that A->B is now performing a new  
function A->B'.  If an archived object is read from the DB and now  
must go through A->B->C->D, You compound the A->B' error with B->C and  
C->D  you can end up with a D' that is incorrect, with no knowledge  
why it happened because it was buried in a function you wrote, assumed  
was applied and never looked at again.

Perhaps rather than an implicit feature, we provide a toolkit for  
making user schemas easy to build.    This could make it easy to  
create, compare and perform operations on schemas from classes and to  
hook into the MOP to extract them and use them in class redefinition  
and creation.  We could provide an inherited slot for class instances  
that keeps a pointer to the schema.  Users that are afraid of this  
functionality can just choose to update manually so that consistence  
is guaranteed.  Trading off performance for a strong guarantee strikes  
me as the better default behavior.

>
>> I'm very open to suggestions (although I'm unlikely to implement this
>> feature anytime soon).  There has been alot of discussion around this
>> area (see Pierre and I discussing class deserialization and
>> initialization, part of which should be solved by Sean's patch) that
>> the initialization and reinitialization on deserialization protocol
>> should probably be rethought.
>
> I will take a look at Sean's patch and the discussions you mentioned,
> and try to hack up some more if I see the need for it.

You might want to look through the Trac database and Wiki where all of  
the existing proposals for improving Elephant are documented.  I  
believe schema evolution is on the roadmap for 0.9.x, but I'm only bug  
fixing these days and don't have the bandwidth to take on the major  
projects I had lined up for 0.9.x before I got pulled away.

I'm happy to support this effort if you or someone else are interested  
in taking it on, though.


>  Thanks,
>
>    Leslie
>
> -- 
> My personal blog: http://blog.viridian-project.de/
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel