[cl-prevalence-devel] simple-array serialization patch

[Sven Van Caekenberghe]

Mike,

On 05 Apr 2007, at 17:19, Michael J. Forster wrote:

> I don't know if you or anyone else is interested, but I have  
> implemented
> xml and sexp serialization/deserialization of simple arrays -- I  
> needed it
> for an app that uses cl-prevalence.  I've attached the patch.

The patch is OK in terms of code (I guess it is working fine in your  
situation), but I am not sure that it is conceptually correct (but  
maybe I am wrong).

According to my reading of CLHS the type simple-array on itself does  
not guarantee a (what I would call) homogeneous array (an array with  
the same type of element everywhere). The typespecs '(simple-array *)  
and '(simple-array <element-type>) would refer to this, but I don't  
know whether you can use them in method signatures.

Even so, the array-element-type could very well be too general, like  
T or cons or array. In that case, your serialization code fails to  
take shared and circular references into account (you are effectively  
assuming more primitive, non-shared, non-circural element-types -  
which probably works in the way you are using CL-PREVALENCE).

So, as I see and understand it now, your code would be OK, if we  
further qualify it with a test that the array-element-type is  
somewhat 'primitive'. But I am not sure how to express that in the  
method signature or how to test/enforce it in code, maybe we need a  
custom type predicate ?

Also, it would be very helpful if we had unit tests covering your  
extended serialization special cases.

Anyway, your patch would be an important optimalization for better/ 
faster serialization in some important cases!

> BTW, I would like to say that cl-prevalence is fantastic.  We've  
> been using
> it for five non-trivial (>25 classes, avg. 3000 instances per  
> class) webapps
> without a hitch for almost a year now.

That is very nice to hear: could you give some more details, like:

- what CL implementation you are using ?
- what serialization you are using ?
- the typical sizes of you transaction and snapshot files ?
- total number of objects under prevalence, 75000 ?
- rate of change (transaction log growth per day or so) ?
- size of the image ?
- machine details ?
- do you have any GC problems ?
- anything else you want to share

Regards,

Sven