From Roger at common-lisp.net Thu May 1 14:32:52 2008 From: Roger at common-lisp.net (Roger at common-lisp.net) Date: Thu, 01 May 2008 17:32:52 +0300 Subject: [elephant-devel] Greetings from Roger J. Garang, Message-ID: Greetings from Roger J. Garang, Due to security reasons and the kind of place I am in BENIN REPUBLIC at themoment, I have to contact you in this manner with apology. I am Roger J. Garang, Son of late Dr. General John Garang the former rebel leader of Sudan who letter became the vice president of Sudan,before he was killed by a planned deal through a plane crash, which took place on Monday August 1st, 2005 in mountains located in southern Sudan. http://news.bbc.co.uk/2/hi/in_pictures/4735725.stm He was buried on Saturday August 6th, 2005. http://news.bbc.co.uk/2/hi/africa/4126370.stm After the death of my mother Mrs. Rebecca Garang on the 15th December 2005, the president of Sudan placed embargo on my late father's assets during his speech on January 2nd, 2006, which made me to flee from my country through road to another country where the United Nations granted me refugee under political asylum camp in Republic of BENIN Meanwhile, I am currently seeking to invest in small and large business enterprises in your country under partnership but I am preferred to deal directly with legitimate owners who can be able to handle my project rather than brokers or middlemen. Therefore, I would like to request full contact details where I can reach the Chief Executive Officer of your organization to enable us discuss further details on this project. However, I have a stake of USD$107,000.000.00 One hundred and fifty-Seven Million Dollars available for investment. So I would be honored if this is treated with Utmost Urgency and do not fail to include your direct telephone number for me to reach you easily for more imformation regarding this project. You can contact me only on my altanative email address (garangroger at yahoo.it) where i can be reach any time in the asylum camp. Sincerely Yours, Roger Garang, (On behalf of the Family) From leslie.polzer at gmx.net Fri May 2 19:41:31 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Fri, 2 May 2008 21:41:31 +0200 (CEST) Subject: [elephant-devel] Postmodern, Act II Message-ID: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org> I suppose we have fixed a very large percentage of bugs in the vanilla code base. Let's move on to the (global) sync cache. With this enabled I sometimes get Database error 55000: currval of sequence "txn_id" is not yet defined in this session After a new request it works. I seem to get these errors only after certain periods of idleness (probably 10 to 30 minutes). This should be easy to fix, either by using NEXTVAL all the time (not sure whether this will work or whether this is good at all) or by ensuring that at least one NEXTVAL precedes usage of CURRVAL. I'll look into it. Leslie From killerstorm at newmail.ru Fri May 2 20:21:36 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Fri, 2 May 2008 23:21:36 +0300 Subject: [elephant-devel] Re: Postmodern, Act II References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org> Message-ID: LPP> With this enabled I sometimes get LPP> Database error 55000: currval of sequence "txn_id" is not yet LPP> defined in this session can you catch a backtrace? it seems to be quite relevant here LPP> This should be easy to fix, either by using NEXTVAL all the time LPP> (not sure whether this will work or whether this is good at all) yep, that will just break logic LPP> or by ensuring that at least one NEXTVAL precedes usage of CURRVAL. that's the way it is meant to work: NEXTVAL is called in ensure-cache-up-to-date function, and ensure-cache-up-to-date is called before any operation on cache -- and before any write to DB. the only code path i'm aware of is (setf internal-get-value) on btree -> ins_upd_btree -> notify_btree_update -> currval but, before calling ins_upd_btree, (setf internal-get-value) calls txn-cache-set-value, that ensures cache is up to date, calling NEXTVAL so this code path seems to be correct -- NEXTVAL is always called before CURRVAL. so only reason i can think of at moment (before looking into backtrace, at least) is misconfiguration -- some processes/threads you launch have global sync cache enabled, while others have not. and those who don't have it enabled call CURRVAL without initializing it with NEXTVAL (because they are not initializing cache). From leslie.polzer at gmx.net Sat May 3 07:03:48 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sat, 3 May 2008 09:03:48 +0200 (CEST) Subject: [elephant-devel] Re: Postmodern, Act II In-Reply-To: References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org> Message-ID: <64660.88.73.200.178.1209798228.squirrel@mail.stardawn.org> > LPP> With this enabled I sometimes get > > LPP> Database error 55000: currval of sequence "txn_id" is not yet LPP> defined in this session > > can you catch a backtrace? it seems to be quite relevant here 6: (LAMBDA NIL) At /home/sky/projects/mystic/hg.beta1/packages/elephant/src/db-postmodern/pm-sql.lisp:157 7: (LAMBDA NIL) At /home/sky/projects/mystic/hg.beta1/packages/elephant/src/db-postmodern/pm-btree.lisp:263 8: (SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER T)) At /home/sky/projects/mystic/hg.beta1/packages/elephant/src/db-postmodern/pm-transaction.lisp: 60 Local variables: ARG-0 = 8 ARG-3 = # ARG-4 = # 9: (SB-PCL::FAST-METHOD (SETF DB-POSTMODERN::INTERNAL-GET-VALUE) (T T DB-POSTMODERN::PM-BTREE)) At /home/sky/projects/mystic/hg.beta1/packages/elephant/src/db-postmodern/pm-btree.lisp:263 Local variables: ARG-2 = # ARG-3 = # ARG-1 = # ARG-2 = NIL 13: (SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER T)) At /home/sky/projects/mystic/hg.beta1/packages/elephant/src/db-postmodern/pm-transaction.lisp:60 Local variables: ARG-0 = 8 ARG-3 = # ARG-4 = # And here's the PostgreSQL point of view: May 2 22:36:57 wintermute postgres[3702]: [16-1] ERROR: currval of sequence "txn_id" is not yet defined in this session May 2 22:36:57 wintermute postgres[3702]: [16-2] CONTEXT: SQL statement "INSERT INTO update_log (txn_id, id, key) VALUES (currval('txn_id'), $1 , $2 )" May 2 22:36:57 wintermute postgres[3702]: [16-3] PL/pgSQL function "notify_btree_update" line 2 at SQL statement May 2 22:36:57 wintermute postgres[3702]: [16-4] SQL statement "SELECT notify_btree_update(2, $1 )" May 2 22:36:57 wintermute postgres[3702]: [16-5] PL/pgSQL function "ins_upd_slots" line 11 at perform May 2 22:36:57 wintermute postgres[3702]: [16-6] STATEMENT: select ins_upd_slots($1,$2) May 2 22:36:57 wintermute postgres[3702]: [17-1] ERROR: current transaction is aborted, commands ignored until end of transaction block May 2 22:36:57 wintermute postgres[3702]: [18-1] ERROR: current transaction is aborted, commands ignored until end of transaction block May 2 22:36:57 wintermute postgres[3702]: [19-1] LOG: execute : ABORT > so this code path seems to be correct -- NEXTVAL is always called before CURRVAL. > > so only reason i can think of at moment (before looking into backtrace, at least) is misconfiguration -- some processes/threads you launch have global sync cache enabled, while others have not. It might be that I forgot to enable the sync cache in those sessions; I need to check this out. If this is the case we should detect it and provide a meaningful error message. Leslie From klists at saphor.de Sat May 3 07:56:15 2008 From: klists at saphor.de (Marc) Date: Sat, 03 May 2008 09:56:15 +0200 Subject: [elephant-devel] Newbie question: Index on collections In-Reply-To: <47E78550.4000704@saphor.de> References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu> <47E78550.4000704@saphor.de> Message-ID: <481C1A9F.8050100@saphor.de> Marc wrote: > Ian Eslick wrote: >> Great! Actually in writing the following e-mail, I realized that >> this had forced me to think through a bunch of things and so I wrote >> a version of the functionality in associations.lisp and just checked >> it in. >> >> Start playing with, reading and commenting on what's wrong with it. >> For example, I'd like a better API for adding/removing items from >> slots than overloading (setf (accessor instance) related-instance). >> I left a commented API proposal in the source code for you to look at. >> >> There are still bugs that I haven't thought through that will crop up >> when you change classes, redefine classes, etc. Let me know what you >> think. > Thanks for the file and the documentation in this mail. Starting from > this I'll now begin to dig into it and try to grope my way around. > > As to timing: this week is somewhat busy due to a conference, but I > still hope for some time in between sessions. I fear that the delay has been much longer than anticipated --- it has been very busy ever after the conference ---, but no w I hope finally to find some time --- provided that the issue is not long obsolete. Best regards, Marc P. S.: I've noticed that there have been few changes in the unstable branch lately, but many more in the stable one. Which one is the one to work on for this right now? > >> >> ======================== >> >> Download the elephant-unstable branch: >> >> darcs get >> http://www.common-lisp.net/project/elephant/darcs/elephant-unstable >> >> It's probably best to spend a little time reading and experimenting, >> then write up a proposal so we can discuss it a bit. The support for >> defining association slots (but not implementing them) is already >> implemented in metaclasses.lisp. >> >> The idea of an association is that instances of class A can find >> instances of class B that are 'associated' with it. Association >> relations can be undirected or directed, typically called >> many-to-many or 1-to-many. 1-to-many is a kind of inverse indexing >> and many-to-many requires a separate btree/class/table to maintain >> the undirected relation. Hopefully the following description will >> help clarify what I mean. >> >> >> One-to-Many associations: >> >> If class B has a slot that stores a single instance of A, then class >> A would like to have a method that returns the set of instances of B >> that refer to the instance of A. >> >> If A is person and B is job, then job->owner is a reference to A and >> A->jobs is a method (no slot storage) that refers to an index on >> job->owner. The index keys are A oids. >> >> If you look in indexed-slots.lisp, you'll get a sense of how indices >> work. set-valued-slots.lisp is also useful as it puts a btree in >> each instance's slot to store sets of objects using the pset >> abstraction in pset.lisp. Reading these three files should help alot >> to see how we use btrees and dup-btrees to maintain sets of objects >> and relations. >> >> >> One challenge is how to specify an association implicitly in slot >> definitions. Ideally you want it to be on the object that you are >> most often going to use to add and remove relations: >> >> (defpclass person () >> ((jobs :accessor jobs :association (job owner)))) >> >> (defpclass job () >> ((title :accessor title ...) >> (company :accessor company ...))) >> >> But this means that person has to change the definition of job and >> that can be problematic. So you could require agreement... >> >> (defpclass person () >> ((jobs :accessor jobs :association (job owner))) >> >> (defpclass job () >> ((title ...) >> (company ...) >> (owner :accessor owner :association person))) >> >> This means that job has a slot owner who's values are instances of >> the class person. This slot is also effectively indexed. A person >> has a method jobs which looks up the index in (job owner) and returns >> the list of instances of job that have the instance of A in the owner >> slot. >> >> >> Many-to-Many: >> >> To add many-to-many relations, any A->B relation also requires that >> we can find the relation from the other direction B->A. Typically >> this requires a special table to keep track of pairs of related >> objects. You can then query on either column to get the set of >> related objects. >> >> many-to-many association: A<->B >> (A1 B1) >> (A1 B2) >> (A2 B1) >> (A2 B3) >> >> (get-related assoc A1)->B1,B2 >> (get-related assoc B1)->A1,A2 >> >> We can simply stick with the index model and just ensure that there >> are indexes on both classes instead of one. When we add an element >> to one side, we also add it to the other. (This is what is >> implemented now) >> >> >> Optimizations/Alternatives: >> >> One optimization for one-to-many is to return a standard object >> implementing the pset protocol that just keeps a reference to the >> index and supports map operations over the appropriate subset of the >> objects in the index. >> >> Regards, >> Ian >> >> On Mar 19, 2008, at 2:42 AM, Marc wrote: >>> Hi Ian, >>>> >>>> My best guess is weeks, not months. We're hoping for April >>>> sometime, but it depends on a number of factors. Elephant is >>>> getting quite complex to test so the last few bugs can take some time. >>>> >>>> You can speed this process up if you want to help develop and test >>>> the association facility (most of the hooks are there so it >>>> shouldn't be too hard, even if you don't fully understand the >>>> elephant internals). If not, we have plenty of new features that >>>> need tests written against them! >>> Since we need this feature anyway for our application, I'm quite >>> prepared to join in the work on the association facility. Any >>> pointer on what would be the best entry point into that task? >>> >>> Best regards, >>> >>> Marc >>>> >>>> On Mar 18, 2008, at 12:50 AM, Marc wrote: >>>> >>>>> Ian Eslick wrote: >>>>>> The pset is probably your best bet. Other collection objects >>>>>> have more performance impact. With a pset you can ask: >>>>>> >>>>>> is X a friend of Y? (find-item X (friends Y)) => X | nil >>>>>> >>>>>> all people that X is friends of: >>>>>> >>>>>> (remove-nulls >>>>>> (map-class 'people (lambda (y) >>>>>> (when (find-item X (friends Y)) >>>>>> Y)) >>>>>> :collect t)) >>>>>> >>>>>> This requires walking all person objects. For larger DB's, you >>>>>> can build your own 'many-to-many' relation table or waiting for >>>>>> the next release which should do this for you. >>>>> Is there a time line (however preliminary) when this release might be >>>>> available? >>>>>> >>>>>> (defpclass friend-relation () >>>>>> ((first :accessor first :initarg :first :index t) >>>>>> (second :accessor second :initarg :second :index t))) >>>>>> >>>>>> (defmethod add-friend-relation (x y) >>>>>> (make-instance 'friend-relation :first x :second y)) >>>>>> >>>>>> (defmethod friends-of (x) >>>>>> (union (mapcar #'second (get-instances-by-slot 'friend-relation >>>>>> 'first x)) >>>>>> (mapcar #'first (get-instances-by-slot 'friend-relation >>>>>> 'second x)))) >>>>>> >>>>>> Of course there are ways to do this more efficiently, but I think >>>>>> this is the idea. >>>>>> >>>>> Many thanks for this guidance. That does, indeed, give me an idea >>>>> of the >>>>> best way forward in our application. >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> Marc >>>>> >>>>>> >>>>>> >>>>>> On Mar 16, 2008, at 11:06 PM, Marc wrote: >>>>>> >>>>>>> Hello! >>>>>>> >>>>>>> I fear that this may be a rather stupid newbie question, but >>>>>>> neither from the documentation nor from the mailing list posts I >>>>>>> got an idea as to the best practice for the following type of >>>>>>> searches. Let's use an example akin to the one in the tutorial: >>>>>>> >>>>>>> (defpclass person () >>>>>>> ((name :accessor name :index t) >>>>>>> (friends :accessor friends)) ;collection with names (say as >>>>>>> strings) of the friends of this person >>>>>>> ) >>>>>>> >>>>>>> Given a similar structure, what is the recommended way to handle >>>>>>> queries of the type "find all persons that X is a friend of?" >>>>>>> (i.e. queries on individual entries in slots that contain >>>>>>> collections)? What would be the best data structure for the >>>>>>> friends slot, assuming that the collection remains fairly stable >>>>>>> over time (psets, lists or other collections)? >>>>>>> >>>>>>> Thanks in advance for any hint! >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> Marc >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> elephant-devel site list >>>>>>> elephant-devel at common-lisp.net >>>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>>> >>>>>> _______________________________________________ >>>>>> elephant-devel site list >>>>>> elephant-devel at common-lisp.net >>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> elephant-devel site list >>>>> elephant-devel at common-lisp.net >>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>> >>>> _______________________________________________ >>>> elephant-devel site list >>>> elephant-devel at common-lisp.net >>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>> >>>> >>>> >>> >>> >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> >> > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > > From killerstorm at newmail.ru Sat May 3 07:59:43 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Sat, 3 May 2008 10:59:43 +0300 Subject: [elephant-devel] Re: Postmodern, Act II References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org> <64660.88.73.200.178.1209798228.squirrel@mail.stardawn.org> Message-ID: <708B64FD70804C859F752F4C04FF8EAE@killer> ??>> can you catch a backtrace? it seems to be quite relevant here LPP> 6: (LAMBDA NIL) LPP> At LPP> /home/sky/projects/mystic/hg.beta1/packages/elephant/src/db-postmodern LPP> /pm-sql.lisp:1577: (LAMBDA NIL) LPP> At LPP> /home/sky/projects/mystic/hg.beta1/packages/elephant/src/db-postmodern LPP> /pm-btree.lisp:2638: (SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION LPP> (DB-POSTMODERN::POSTMODERN-STORE-CONTROLLER LPP> T)) LPP> At LPP> /home/sky/projects/mystic/hg.beta1/packages/elephant/src/db-postmodern LPP> /pm-transaction.lisp: 60 LPP> Local variables: LPP> ARG-0 = 8 LPP> ARG-3 = # mystic-world mystic mystic PORT LPP> 5433)> LPP> ARG-4 = # LPP> 9: (SB-PCL::FAST-METHOD (SETF DB-POSTMODERN::INTERNAL-GET-VALUE) (T T LPP> DB-POSTMODERN::PM-BTREE)) it seems you didn't wrap your code into explicit transaction. transaction was started implicitly by (setf internal-get-value) before writing to database, but cache update was done outside transaction, so it was not initialized properly. fix is quite simple -- moving all txn functions inside that implicit transactions. but if you do not start your transactions explicitly, enclosing as many operations as posible, global-sync-cache absolutely makes no sense -- it takes more effort to synchornize changes than to actually load value from database, if that's just a single value. so, maybe, if cache is set into global sync mode, it should signal error if there is no explicit transactions -- because that would be misuse of global sync cache, leading to significant overhead. or you think it makes sense to allow such behaviour? it might make sense in REPL, for example.. From leslie.polzer at gmx.net Sat May 3 08:54:36 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sat, 3 May 2008 10:54:36 +0200 (CEST) Subject: [elephant-devel] Re: Postmodern, Act II In-Reply-To: <708B64FD70804C859F752F4C04FF8EAE@killer> References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org> <64660.88.73.200.178.1209798228.squirrel@mail.stardawn.org> <708B64FD70804C859F752F4C04FF8EAE@killer> Message-ID: <61788.88.73.200.178.1209804876.squirrel@mail.stardawn.org> Dear Alex, > it seems you didn't wrap your code into explicit transaction. Thanks for your quick analysis. > but if you do not start your transactions explicitly, enclosing as many > operations as posible, global-sync-cache absolutely makes no sense -- it > takes more effort to synchornize changes than to actually load value from > database, if that's just a single value. > so, maybe, if cache is set into global sync mode, it should signal error if > there is no explicit transactions -- because that would be misuse of global > sync cache, leading to significant overhead. Can you explain this in a bit more detail? > or you think it makes sense to allow such behaviour? it might make sense in > REPL, for example.. I put transactions only in an explicit transaction block if it makes sense to me, i.e. if there are several successive operations. Why would I put a single operation into a WITH-TRANSACTION block? It clutters the code. And for prototyping/debugging (either directly from the REPL or in files) I can live with implicit txns all the time (except where I need to guarantee ACID). Leslie From killerstorm at newmail.ru Sat May 3 11:31:28 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Sat, 3 May 2008 14:31:28 +0300 Subject: [elephant-devel] Re: Postmodern, Act II References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org><64660.88.73.200.178.1209798228.squirrel@mail.stardawn.org><708B64FD70804C859F752F4C04FF8EAE@killer> <61788.88.73.200.178.1209804876.squirrel@mail.stardawn.org> Message-ID: <6AAE571E5E624EB0856C563EF9067773@killer> ??>> but if you do not start your transactions explicitly, enclosing as ??>> many operations as posible, global-sync-cache absolutely makes no ??>> sense -- it takes more effort to synchornize changes than to actually ??>> load value from database, if that's just a single value. so, maybe, if ??>> cache is set into global sync mode, it should signal error if there is ??>> no explicit transactions -- because that would be misuse of ??>> global sync cache, leading to significant overhead. LPP> Can you explain this in a bit more detail? "global sync cache" works by tracking changes made to btrees in the database -- each write to btree is also written into update_log table. then, at start of each transaction (or more preciously, before first actuall btree read/write operation) cache gets synchronized -- basically it pulls log of all changes since last update, and invalidates cache entries according to what it have read from DB. additionally it does some bookeeping for change tracking. thus, global sync cache only makes sense if you do many (hundreds) database reads in each transaction. if you don't have such situation, don't use it :) ??>> or you think it makes sense to allow such behaviour? it might make ??>> sense in REPL, for example.. LPP> I put transactions only in an explicit transaction block if it LPP> makes sense to me, i.e. if there are several successive operations. LPP> Why would I put a single operation into a WITH-TRANSACTION block? LPP> It clutters the code. this cache mode (and postmodern backend in general) is oriented on webserver-like workload -- each web request always is wrapped into transaction. if request does no DB activity, that's OK -- starting txn overhead is not that significant on scale of typical HTTP request time. but many requests reads lots of values from database -- on thousands scale -- and sync cache makes big difference for this case. even without cache, there is considerable overhead when doing single read outside transaction -- at minimal, postmodern will do BEGIN and COMMIT, which require roundtrips to server, so we have something like 3x overhead here. if we were optimizing for standalone read statements, we could try relying on postgresql implicit transactions -- but that will significantly complicate logic, so we don't use this. but while BEGIN/COMMIT is inevitable evil, cache synchonization overhead can be avoided if not needed, so i thought it's worth giving some kind of warning in case people are using backend in sub-optimal mode From leslie.polzer at gmx.net Sat May 3 12:32:50 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sat, 3 May 2008 14:32:50 +0200 (CEST) Subject: [elephant-devel] Re: Postmodern, Act II In-Reply-To: <6AAE571E5E624EB0856C563EF9067773@killer> References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org><64660.88.73.200.178.1209798228.squirrel@mail.stardawn.org><708B64FD70804C859F752F4C04FF8EAE@killer> <61788.88.73.200.178.1209804876.squirrel@mail.stardawn.org> <6AAE571E5E624EB0856C563EF9067773@killer> Message-ID: <63630.88.73.200.178.1209817970.squirrel@mail.stardawn.org> > this cache mode (and postmodern backend in general) is oriented on > webserver-like workload -- each web request always is wrapped into > transaction. if request does no DB activity, that's OK -- starting txn > overhead is not that significant on scale of typical HTTP request time. but > many requests reads lots of values from database -- on thousands scale -- > and sync cache makes big difference for this case. This is interesting, I'm going to try it. Thanks. I'd still consider the CURRVAL issue a bug that needs to be solved, though. Leslie From eslick at media.mit.edu Sat May 3 12:59:20 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 3 May 2008 08:59:20 -0400 Subject: [elephant-devel] Newbie question: Index on collections In-Reply-To: <481C1A9F.8050100@saphor.de> References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu> <47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> Message-ID: <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> Definitely work elephant-unstable if you want to test the new functionality. Most of the activity in the main branch has been in and around the postmodern interface. The limiting factor in merging the branches is getting support for the new data store API (i.e. for btrees w/ duplicate keys) in CL-SQL and Postmodern. Robert has made some good progress on CL-SQL but we haven't had any work done on this in Postmodern yet. I just merged out everything from the main elephant branch into unstable, so you won't be missing anything; except of course it only works for the BDB data store now. Ian On May 3, 2008, at 3:56 AM, Marc wrote: > Marc wrote: >> Ian Eslick wrote: >>> Great! Actually in writing the following e-mail, I realized that >>> this had forced me to think through a bunch of things and so I >>> wrote a version of the functionality in associations.lisp and just >>> checked it in. >>> >>> Start playing with, reading and commenting on what's wrong with >>> it. For example, I'd like a better API for adding/removing items >>> from slots than overloading (setf (accessor instance) related- >>> instance). I left a commented API proposal in the source code for >>> you to look at. >>> >>> There are still bugs that I haven't thought through that will crop >>> up when you change classes, redefine classes, etc. Let me know >>> what you think. >> Thanks for the file and the documentation in this mail. Starting >> from this I'll now begin to dig into it and try to grope my way >> around. >> >> As to timing: this week is somewhat busy due to a conference, but I >> still hope for some time in between sessions. > I fear that the delay has been much longer than anticipated --- it > has been very busy ever after the conference ---, but no w I hope > finally to find some time --- provided that the issue is not long > obsolete. > > Best regards, > > Marc > > P. S.: I've noticed that there have been few changes in the unstable > branch lately, but many more in the stable one. Which one is the one > to work on for this right now? >> >>> >>> ======================== >>> >>> Download the elephant-unstable branch: >>> >>> darcs get http://www.common-lisp.net/project/elephant/darcs/elephant-unstable >>> >>> It's probably best to spend a little time reading and >>> experimenting, then write up a proposal so we can discuss it a >>> bit. The support for defining association slots (but not >>> implementing them) is already implemented in metaclasses.lisp. >>> >>> The idea of an association is that instances of class A can find >>> instances of class B that are 'associated' with it. Association >>> relations can be undirected or directed, typically called many-to- >>> many or 1-to-many. 1-to-many is a kind of inverse indexing and >>> many-to-many requires a separate btree/class/table to maintain the >>> undirected relation. Hopefully the following description will >>> help clarify what I mean. >>> >>> >>> One-to-Many associations: >>> >>> If class B has a slot that stores a single instance of A, then >>> class A would like to have a method that returns the set of >>> instances of B that refer to the instance of A. >>> >>> If A is person and B is job, then job->owner is a reference to A >>> and A->jobs is a method (no slot storage) that refers to an index >>> on job->owner. The index keys are A oids. >>> >>> If you look in indexed-slots.lisp, you'll get a sense of how >>> indices work. set-valued-slots.lisp is also useful as it puts a >>> btree in each instance's slot to store sets of objects using the >>> pset abstraction in pset.lisp. Reading these three files should >>> help alot to see how we use btrees and dup-btrees to maintain sets >>> of objects and relations. >>> >>> >>> One challenge is how to specify an association implicitly in slot >>> definitions. Ideally you want it to be on the object that you are >>> most often going to use to add and remove relations: >>> >>> (defpclass person () >>> ((jobs :accessor jobs :association (job owner)))) >>> >>> (defpclass job () >>> ((title :accessor title ...) >>> (company :accessor company ...))) >>> >>> But this means that person has to change the definition of job and >>> that can be problematic. So you could require agreement... >>> >>> (defpclass person () >>> ((jobs :accessor jobs :association (job owner))) >>> >>> (defpclass job () >>> ((title ...) >>> (company ...) >>> (owner :accessor owner :association person))) >>> >>> This means that job has a slot owner who's values are instances of >>> the class person. This slot is also effectively indexed. A >>> person has a method jobs which looks up the index in (job owner) >>> and returns the list of instances of job that have the instance of >>> A in the owner slot. >>> >>> >>> Many-to-Many: >>> >>> To add many-to-many relations, any A->B relation also requires >>> that we can find the relation from the other direction B->A. >>> Typically this requires a special table to keep track of pairs of >>> related objects. You can then query on either column to get the >>> set of related objects. >>> >>> many-to-many association: A<->B >>> (A1 B1) >>> (A1 B2) >>> (A2 B1) >>> (A2 B3) >>> >>> (get-related assoc A1)->B1,B2 >>> (get-related assoc B1)->A1,A2 >>> >>> We can simply stick with the index model and just ensure that >>> there are indexes on both classes instead of one. When we add an >>> element to one side, we also add it to the other. (This is what >>> is implemented now) >>> >>> >>> Optimizations/Alternatives: >>> >>> One optimization for one-to-many is to return a standard object >>> implementing the pset protocol that just keeps a reference to the >>> index and supports map operations over the appropriate subset of >>> the objects in the index. >>> >>> Regards, >>> Ian >>> >>> On Mar 19, 2008, at 2:42 AM, Marc wrote: >>>> Hi Ian, >>>>> >>>>> My best guess is weeks, not months. We're hoping for April >>>>> sometime, but it depends on a number of factors. Elephant is >>>>> getting quite complex to test so the last few bugs can take some >>>>> time. >>>>> >>>>> You can speed this process up if you want to help develop and >>>>> test the association facility (most of the hooks are there so it >>>>> shouldn't be too hard, even if you don't fully understand the >>>>> elephant internals). If not, we have plenty of new features >>>>> that need tests written against them! >>>> Since we need this feature anyway for our application, I'm quite >>>> prepared to join in the work on the association facility. Any >>>> pointer on what would be the best entry point into that task? >>>> >>>> Best regards, >>>> >>>> Marc >>>>> >>>>> On Mar 18, 2008, at 12:50 AM, Marc wrote: >>>>> >>>>>> Ian Eslick wrote: >>>>>>> The pset is probably your best bet. Other collection objects >>>>>>> have more performance impact. With a pset you can ask: >>>>>>> >>>>>>> is X a friend of Y? (find-item X (friends Y)) => X | nil >>>>>>> >>>>>>> all people that X is friends of: >>>>>>> >>>>>>> (remove-nulls >>>>>>> (map-class 'people (lambda (y) >>>>>>> (when (find-item X (friends Y)) >>>>>>> Y)) >>>>>>> :collect t)) >>>>>>> >>>>>>> This requires walking all person objects. For larger DB's, >>>>>>> you can build your own 'many-to-many' relation table or >>>>>>> waiting for the next release which should do this for you. >>>>>> Is there a time line (however preliminary) when this release >>>>>> might be >>>>>> available? >>>>>>> >>>>>>> (defpclass friend-relation () >>>>>>> ((first :accessor first :initarg :first :index t) >>>>>>> (second :accessor second :initarg :second :index t))) >>>>>>> >>>>>>> (defmethod add-friend-relation (x y) >>>>>>> (make-instance 'friend-relation :first x :second y)) >>>>>>> >>>>>>> (defmethod friends-of (x) >>>>>>> (union (mapcar #'second (get-instances-by-slot 'friend- >>>>>>> relation 'first x)) >>>>>>> (mapcar #'first (get-instances-by-slot 'friend-relation >>>>>>> 'second x)))) >>>>>>> >>>>>>> Of course there are ways to do this more efficiently, but I >>>>>>> think this is the idea. >>>>>>> >>>>>> Many thanks for this guidance. That does, indeed, give me an >>>>>> idea of the >>>>>> best way forward in our application. >>>>>> >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Marc >>>>>> >>>>>>> >>>>>>> >>>>>>> On Mar 16, 2008, at 11:06 PM, Marc wrote: >>>>>>> >>>>>>>> Hello! >>>>>>>> >>>>>>>> I fear that this may be a rather stupid newbie question, but >>>>>>>> neither from the documentation nor from the mailing list >>>>>>>> posts I got an idea as to the best practice for the following >>>>>>>> type of searches. Let's use an example akin to the one in the >>>>>>>> tutorial: >>>>>>>> >>>>>>>> (defpclass person () >>>>>>>> ((name :accessor name :index t) >>>>>>>> (friends :accessor friends)) ;collection with names (say as >>>>>>>> strings) of the friends of this person >>>>>>>> ) >>>>>>>> >>>>>>>> Given a similar structure, what is the recommended way to >>>>>>>> handle queries of the type "find all persons that X is a >>>>>>>> friend of?" (i.e. queries on individual entries in slots that >>>>>>>> contain collections)? What would be the best data structure >>>>>>>> for the friends slot, assuming that the collection remains >>>>>>>> fairly stable over time (psets, lists or other collections)? >>>>>>>> >>>>>>>> Thanks in advance for any hint! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> Marc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> elephant-devel site list >>>>>>>> elephant-devel at common-lisp.net >>>>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>>>> >>>>>>> _______________________________________________ >>>>>>> elephant-devel site list >>>>>>> elephant-devel at common-lisp.net >>>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> elephant-devel site list >>>>>> elephant-devel at common-lisp.net >>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>> >>>>> _______________________________________________ >>>>> elephant-devel site list >>>>> elephant-devel at common-lisp.net >>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>> >>>>> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> elephant-devel site list >>>> elephant-devel at common-lisp.net >>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>> >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >>> >>> >>> >> >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> >> > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From killerstorm at newmail.ru Sat May 3 13:52:44 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Sat, 3 May 2008 16:52:44 +0300 Subject: [elephant-devel] Re: Postmodern, Act II References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org><64660.88.73.200.178.1209798228.squirrel@mail.stardawn.org><708B64FD70804C859F752F4C04FF8EAE@killer><61788.88.73.200.178.1209804876.squirrel@mail.stardawn.org><6AAE571E5E624EB0856C563EF9067773@killer> <63630.88.73.200.178.1209817970.squirrel@mail.stardawn.org> Message-ID: <6BCC5296F44849D5833E6B0BABE931BF@killer> ??>> this cache mode (and postmodern backend in general) is oriented on ??>> webserver-like workload -- each web request always is wrapped into ??>> transaction. if request does no DB activity, that's OK -- starting txn ??>> overhead is not that significant on scale of typical HTTP request ??>> time. but many requests reads lots of values from database -- on ??>> thousands scale -- and sync cache makes big difference for this case. LPP> This is interesting, I'm going to try it. Thanks. LPP> I'd still consider the CURRVAL issue a bug that needs to be solved, LPP> though. yep, you are right -- it's not good if system breaks in some situations if caching is enabled, even if those situations are deprecated usage. i'll try to fix it when i'll get to it.. From killerstorm at newmail.ru Sat May 3 14:02:01 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Sat, 3 May 2008 17:02:01 +0300 Subject: [elephant-devel] Newbie question: Index on collections References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu><47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> Message-ID: IE> but we haven't had any work done on this in Postmodern yet. by the way, are there any people waiting for postmodern being ported, or is it causing any problems if it's not ported yet (i.e. delaying release or something)? because for now projects i'm working on seem to be working more-or-less good as they are, and there is no big need to move them onto new elephant version -- there are other things to do. but if some people are waiting for it, i can at least check how hard this porting will be, and actually port if it will be easy. From eslick at media.mit.edu Sat May 3 14:21:42 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 3 May 2008 10:21:42 -0400 Subject: [elephant-devel] Newbie question: Index on collections In-Reply-To: References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu><47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> Message-ID: Thanks Alex. The ports of CL-SQL and Postmodern are the primary things holding up the release process right now. If there are major issues there, I may have to refactor my implementation. It would be really helpful to get an ETA on a port and get the main and unstable branches integrated. I just pulled all new patches from the main branch into the unstable branch which, fortunately, wasn't too much work. The BDB store tests pass for me locally. My goal is to build momentum towards a 1.0 release given that the current functionality seems to be getting good traction and is holding up well under real-world use. That will give us a stable platform that people can depend on being kept up to date with bug fixes and also give us some freedom from developing any major new features for awhile. To this end, I'd like to produce a beta candidate (1.0b1) as soon as the back-ends are ported and we've updated the test suite a bit. The upgrade should be pretty easy, I tried to touch the data store API as little as possible. Here are some pointers from my e-mail of 3/23/08: FYI - The backend updates will require: UPDATE STORE CONTROLLER (trivial) - Initialization of store-controller base class slots with btrees that have static oids (schema-table :reader controller-schema-table :documentation "Schema id to schema table, an indexed btree") (instance-table :reader controller-instance-table :documentation "Contains indexed btree of oid to class ids") (index-table :reader controller-index-table :documentation "This is another root for class indexing that is also a data store specific persistent btree instance with a unique OID that persists between sessions. No cache is needed because we cache in the class slots.") BOOKKEEPING FOR SCHEMA IDS (should be easy) - next-cid (sequence of schema id values) - specialize oid->schema-id (accounts for reserved oids for the fixed btrees above) - default-class-id (map class typename to class schema id for reserved types) - default-class-id-type (map class schema id to class type name for reserved types) DUPLICATE BTREES (this is the only part that may be tricky) - Implement duplicate btrees (allow duplicate keys ordered by key, value) - add remove-kv-pair operator to augment remove-kv For examples of all of the above see bdb-controller and bdb-collections. I may be forgetting something, but these are the main items. Thank you, Ian On May 3, 2008, at 10:02 AM, Alex Mizrahi wrote: > IE> but we haven't had any work done on this in Postmodern yet. > > by the way, are there any people waiting for postmodern being > ported, or is it causing any problems if it's not ported yet (i.e. > delaying release or something)? > > because for now projects i'm working on seem to be working more-or- > less good as they are, > and there is no big need to move them onto new elephant version -- > there are other things to do. > > but if some people are waiting for it, i can at least check how hard > this porting will be, and actually port if it will be easy. > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Sat May 3 14:47:56 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 3 May 2008 10:47:56 -0400 Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu> <1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> Message-ID: Hi Alex, You make a good point below. I haven't dealt with these kind of small subsets from large sets using multiple parameters problems in a large system. However, if we take a page from Robert's book about using lisp as the query engine - how would we solve this problem just with lisp objects? Given that in Elephant, the cost of a btree instance is negligable, one way to do this is the just have a set of messages for each user in a user slot (a pset or set-valued slot). Rather than creating a message with a relational connection to a user, we just create a message and add it to the user's collection of messages. Of course to get the full efficiency you want, we'd have to make a sorted-pset that sorts on some fn of the elements. I'd like to think about different metaphors for solving these problems so we don't end up turning Elephant's interface into a less efficient variation of an ORM! Ian On Mar 23, 2008, at 5:27 PM, Alex Mizrahi wrote: > IE> unnecessary. If you have messages indexed by time than you can > simply > IE> walk the index and filter by user until you have a web page > worth of > IE> messages. > > worst case behaviour is very bad -- to prove that there are no > messages, it > will have to scan all of them.. > what's even worse, this worst case is quite frequent -- many users > actually > have empty inboxes. > > IE> A scan, even of an index, may not be fast enough so you could > do an > IE> intersection of user-to values with an ordered list of recent > messages. > > you mean reading oids from user-to index and scanning/filtering > modification-time index with them? > this will mean same worst case. > of course sufficiently smart system can see which list is less, but > still > with thousands of messages and thousands of users this is going to > be quite > slow. > > IE> This should perform similarly to a SQL engine which many ORM > systems > IE> use for queries like this. > > i thought in SQL DBMS expression like "create index tree11_idx on > tree11(qi,value)" > builds ordered list (btree) of pairs pretty much as in derived index > i've > described, and this will make lookups on both values fast. > Quite right. > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Sat May 3 15:40:52 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sat, 3 May 2008 17:40:52 +0200 (CEST) Subject: [elephant-devel] Newbie question: Index on collections In-Reply-To: References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu><47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> Message-ID: <62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> > IE> but we haven't had any work done on this in Postmodern yet. > > by the way, are there any people waiting for postmodern being ported, or is > it causing any problems if it's not ported yet (i.e. delaying release or > something)? I'm waiting for unstable because of the query language, the cached slots, the unbound slot fix and the schema storage. I'd appreciate it if you could look into what's involved. Duplicate keys in btrees should pose no problems with PM, but I'm not sure what else is involved. Leslie From elephant-devel at common-lisp.net Sat May 3 15:51:31 2008 From: elephant-devel at common-lisp.net (info@bank-ghana.org) Date: Sat, 3 May 2008 17:51:31 +0200 Subject: [elephant-devel] Good day. Message-ID: <200805031551.m43FpVCS030906@luna.luna16.de> ATTN: MY NAME IS MR.ADAMS SMITH, THE MANAGER CREDIT AND FOREIGN BILLS OF BANK OF GHANA, ACCRA. I AM WRITING IN RESPECT OF A FOREIGN CUSTOMER OF MY BANK WITH ACCOUNT NUMBER 14-255-2004/utb/t WHO DIED IN A PLANE CRASH [ETHIOPIAN AIRLINE 961] WITH OTHER PASSENGERS ABOARD. SINCE THE DEATH OF THIS CUSTOMER I PERSONALLY HAVE WATCHED WITH KEEN INTEREST TO SEE THE NEXT OF KIN BUT ALL HAS PROVED ABORTIVE AS NO ONE HAS COME TO CLAIM HIS FUNDS OF USD15.5 M, [FIFTEEN MILLION FIVE HUNDRED THOUSAND UNITED STATES DOLLARS] HAS BEEN WITH MY BANK FOR A VERY LONG TIME. ON THIS NOTE I DECIDED TO SEEK FOR WHOM HIS NAME SHALL BE USED AS THE NEXT OF KIN AS NO ONE HAS COME UP TO BE THE NEXT OF KIN, AND THE BANKING ETHICS HERE DOES NOT ALLOW SUCH MONEY TO STAY MORE THAN FOUR YEARS, BECUASE AFTER FOUR YEARS THE ACCOUNT WILL BE DECLARED DORMANT AND THE MONEY FORFEITED TO THE BANK. IN VIEW OF THIS, I DECIDED TO CONTACT YOU TO ACT AS THE NEXT OF KIN. THE REQUEST OF A FOREIGNER AS NEXT OF KIN IN THIS BUSINESS IS OCCASIONED BY THE FACT THAT THE CUSTOMER WAS A FOREIGNER. I AGREED THAT 30% OF THIS MONEY WILL BE FOR YOU AS A FOREIGN PARTNER IN RESPECT OF PROVIDING AN ACCOUNT, WHILE THE REST WILL BE FOR ME. THEREAFTER I WILL VISIT YOUR COUNTRY FOR DISBURSEMENT . THEREFORE TO FACILITATE THE IMMEDIATE TRANSFER OF THIS FUND TO YOUR ACCOUNT AS ARRANGED, YOU MUST APPLY FIRST TO THE BANK AS A RELATION TO THE DECEASED INDICATING YOUR BANK NAME, YOUR BANK ACCOUNT NUMBER, YOUR PRIVATE TELEPHONE NUMBER, AND FAX NUMBER FOR EASY COMMUNICATION AND LOCATION WHERE THE MONEY WILL BE REMITTED. UPON THE RECEIPT OF YOUR REPLY, I WILL SEND YOU BY FAX OR E-MAIL THE APPLICATION AND THE NEXT STEP TO TAKE. I WILL NOT FAIL TO BRING TO YOUR NOTICE THAT THIS BUSINESS IS HITCH FREE AND THAT YOU SHOULD NOT ENTERTAIN ANY FEAR AS THE WHOLE REQUIRED ARRANGEMENT HAS BEEN MADE FOR THE TRANSFER. YOU SHOULD CONTACT ME IMMEDIATELY AS SOON AS YOU RECEIVE THIS LETTER. TRUSTING TO HEAR FROM YOU. YOURS RESPECTFULLY, MR ADAMS SMITH +233244506074 From eslick at media.mit.edu Sat May 3 17:06:23 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 3 May 2008 13:06:23 -0400 Subject: [elephant-devel] Newbie question: Index on collections In-Reply-To: <62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu><47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> <62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> Message-ID: Just FYI - the query language is pretty primitive at present although Daniel has expressed some interest in expanding on the current sketch. Ian On May 3, 2008, at 11:40 AM, Leslie P. Polzer wrote: > >> IE> but we haven't had any work done on this in Postmodern yet. >> >> by the way, are there any people waiting for postmodern being >> ported, or is >> it causing any problems if it's not ported yet (i.e. delaying >> release or >> something)? > > I'm waiting for unstable because of the query language, the cached > slots, > the unbound slot fix and the schema storage. > > I'd appreciate it if you could look into what's involved. > > Duplicate keys in btrees should pose no problems with PM, but I'm > not sure > what else is involved. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Sat May 3 22:04:51 2008 From: read at robertlread.net (Robert L. Read) Date: Sat, 03 May 2008 17:04:51 -0500 Subject: [elephant-devel] Staus of the port of Ian's new functional to CL-SQL backend.... In-Reply-To: <62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu> <47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> <62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> Message-ID: <1209852291.3971.1822.camel@penguin.yourdomain.com> Dear Team, At the moment, as Ian has mentioned, the next major release is dependent on the porting of a big chunk of new functionality that he has written to the CL-SQL and Postmodern backends. I think it is worth a status report on that. I have spent a total of 24 hours of work on it. Unfortunately, only 8 of those were in one block, due to personal time constraints. I have a draft of a solution, which is green except for 15 tests, mostly red around the "association" stuff. However, my solution really needs to be cleaned up before it should be checked in; I was doing a lot of "hack and slash" exploratory programming to get it to work. As I think Ian mentioned in an email, the work really boils down to do issues: implementing some new slots on the controller (easy) and implementing a new kind of btree that supports duplicates (hard). Since Postmodern uses the same kind of cursor control that the CL-SQL backend does, the CL-SQL port and the postmodern port will have at least 50% overlap in the work. I suspect that Alex can do it more qickly than I did; however I would be a little surprised if anyone could do it all in two hours, although the number of lines of code needed to accomplish it all is pretty small. I have actually finally cleared my calendar to work on this tonight and tomorrow; if I can get a clean solution checked in, I suspect that it would be a good guide to Alex, and might allow him to get it done pretty quickly. An ideal solution might be for Alex to wait until I can commit something to elephant-unstable and then look at that. Of course, I wouldn't dissuade Alex from looking at it now---it might be that he can get it done faster than I can, and then I will get the benefit of his work. I apologize for keeping people waiting; I have a full-time job and some family medical problems. We have have planned a little too much for this release; perhaps in the future we should aim more for a time-box style of release management. On Sat, 2008-05-03 at 17:40 +0200, Leslie P. Polzer wrote: > > IE> but we haven't had any work done on this in Postmodern yet. > > > > by the way, are there any people waiting for postmodern being ported, or is > > it causing any problems if it's not ported yet (i.e. delaying release or > > something)? > > I'm waiting for unstable because of the query language, the cached slots, > the unbound slot fix and the schema storage. > > I'd appreciate it if you could look into what's involved. > > Duplicate keys in btrees should pose no problems with PM, but I'm not sure > what else is involved. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From killerstorm at newmail.ru Sun May 4 07:37:05 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Sun, 4 May 2008 10:37:05 +0300 Subject: [elephant-devel] Staus of the port of Ian's new functional toCL-SQL backend.... References: <47DDE029.2090204@saphor.de><16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu><47DF49FF.8070206@saphor.de><47E0B5E4.4000809@saphor.de><86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu><47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de><02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu><62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> <1209852291.3971.1822.camel@penguin.yourdomain.com> Message-ID: RLR> implementing a new kind of btree that supports duplicates (hard). ain't btree with duplicates very similar to btree-index thing, because btree-index also supports duplicates? RLR> Since Postmodern uses the same kind of cursor control that the CL-SQL RLR> backend does, why do you think so? cursor are probably biggest difference between CL-SQL and postmodern -- instead of reading everything and sorting on lisp side, we use sorting done by postgresql, and pick stuff in portions, i.e. cursor-next is something like: SELECT k, v FROM btree11 WHERE ((k = $1) and (v > $2)) OR (k > $1) LIMIT 10 From read at robertlread.net Sun May 4 15:44:38 2008 From: read at robertlread.net (Robert L. Read) Date: Sun, 04 May 2008 10:44:38 -0500 Subject: [elephant-devel] Staus of the port of Ian's new functional toCL-SQL backend.... In-Reply-To: References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu> <47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> <62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> <1209852291.3971.1822.camel@penguin.yourdomain.com> Message-ID: <1209915878.3971.1874.camel@penguin.yourdomain.com> On Sun, 2008-05-04 at 10:37 +0300, Alex Mizrahi wrote: > RLR> implementing a new kind of btree that supports duplicates (hard). > > ain't btree with duplicates very similar to btree-index thing, because > btree-index also supports duplicates? Yes, I thought that too -- my initial strategy was to attempt to subclass that to create the btree-index. Instead, I had to extract a lot of the function for it by hand, and simply subclass sql-btree. So, you are correct, the functionality is in principle similar---but for me it was not so easy to lift it into exactly what is needed. Part of this is that there is only one direct test of dup-btree functionality; I'm trying to rectify that a little right now. > > RLR> Since Postmodern uses the same kind of cursor control that the CL-SQL > RLR> backend does, > > why do you think so? cursor are probably biggest difference between CL-SQL > and postmodern -- instead of reading everything and sorting on lisp side, we > use sorting done by postgresql, and pick stuff in portions, i.e. cursor-next > is something like: > > SELECT k, v FROM btree11 WHERE ((k = $1) and (v > $2)) OR (k > $1) LIMIT 10 I think perhaps I was confused by pm-collections.lisp, which seems actually defines sql-cursor, instead of pm-cursor. I guess that file should probably be removed. > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Mon May 5 02:50:14 2008 From: read at robertlread.net (Robert L. Read) Date: Sun, 04 May 2008 21:50:14 -0500 Subject: [elephant-devel] Staus of the port of Ian's new functional toCL-SQL backend.... In-Reply-To: References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu> <47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> <62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> <1209852291.3971.1822.camel@penguin.yourdomain.com> Message-ID: <1209955814.3971.1901.camel@penguin.yourdomain.com> I have now committed an all-green solution for the CL-SQL backend to elephant-unstable. I don't know that it will help the Postmodern work much --- perhaps just as a pointer for what needs to be done. Nonetheless, it is ready for someone who wants to work on making Postmodern work with elephant-unstable to review if they like. On Sun, 2008-05-04 at 10:37 +0300, Alex Mizrahi wrote: > RLR> implementing a new kind of btree that supports duplicates (hard). > > ain't btree with duplicates very similar to btree-index thing, because > btree-index also supports duplicates? > > RLR> Since Postmodern uses the same kind of cursor control that the CL-SQL > RLR> backend does, > > why do you think so? cursor are probably biggest difference between CL-SQL > and postmodern -- instead of reading everything and sorting on lisp side, we > use sorting done by postgresql, and pick stuff in portions, i.e. cursor-next > is something like: > > SELECT k, v FROM btree11 WHERE ((k = $1) and (v > $2)) OR (k > $1) LIMIT 10 > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Mon May 5 08:01:43 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 5 May 2008 10:01:43 +0200 (CEST) Subject: [elephant-devel] Call SLOT-UNBOUND instead of signaling UNBOUND-SLOT In-Reply-To: <64181.84.157.24.199.1207226818.squirrel@mail.stardawn.org> References: <64027.84.157.24.199.1207221259.squirrel@mail.stardawn.org> <825289A7-0CB6-4C91-AA14-CBE79CFD8166@media.mit.edu> <64181.84.157.24.199.1207226818.squirrel@mail.stardawn.org> Message-ID: <63682.88.73.197.54.1209974503.squirrel@mail.stardawn.org> I guess we don't need this patch anymore in unstable, so it's moot to apply it. If for any reason you'd like to do this nevertheless, don't use this patch; I have an updated version that works correctly. If not, just ignore this. Leslie From leslie.polzer at gmx.net Mon May 5 08:04:16 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 5 May 2008 10:04:16 +0200 (CEST) Subject: [elephant-devel] Re: Postmodern, Act II In-Reply-To: <6AAE571E5E624EB0856C563EF9067773@killer> References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org><64660.88.73.200.178.1209798228.squirrel@mail.stardawn.org><708B64FD70804C859F752F4C04FF8EAE@killer> <61788.88.73.200.178.1209804876.squirrel@mail.stardawn.org> <6AAE571E5E624EB0856C563EF9067773@killer> Message-ID: <63876.88.73.197.54.1209974656.squirrel@mail.stardawn.org> Alex, you explained that global-sync cache mode only works within a txn. I thought that per-transaction cache mode does this, and that global sync extends this behaviour so transactions/threads can share the cache? If not, what's the difference between them? Leslie From leslie.polzer at gmx.net Mon May 5 10:52:25 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 5 May 2008 12:52:25 +0200 (CEST) Subject: [elephant-devel] Bug in GET-INSTANCES-BY-RANGE Message-ID: <61804.88.73.197.54.1209984745.squirrel@mail.stardawn.org> Sorting on a slot field with NIL values and START, END = NIL doesn't work correctly: MYSTIC(23): (defpclass moo () ((slot1 :type list :initform nil :index t))) # MYSTIC(24): (make-instance 'moo) # MYSTIC(25): (get-instances-by-class 'moo) (#) MYSTIC(26): (get-instances-by-range 'moo 'slot1 nil nil) NIL Known behaviour? Leslie From leslie.polzer at gmx.net Mon May 5 12:07:25 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 5 May 2008 14:07:25 +0200 (CEST) Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE Message-ID: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> Now don't ask me how a NIL value gets mapped to an OID key in the class index. It happened, and I wasn't able to remove those things because of a bug in PM-INDEXED-BTREE. Diff: --- old-elephant/src/db-postmodern/pm-indexed-btree.lisp 2008-05-05 14:04:49.930108544 +0200 +++ new-elephant/src/db-postmodern/pm-indexed-btree.lisp 2008-05-05 14:04:49.930108544 +0200 @@ -74,8 +74,8 @@ (defmethod remove-kv (key (bt pm-indexed-btree)) "Remove a key / value pair, and update secondary indices." (with-trans-and-vars (bt) - (let ((value (get-value key bt))) - (when value + (multiple-value-bind (value found) (get-value key bt) + (when found (let ((indices (indices bt))) (loop for index being the hash-value of indices do (multiple-value-bind (index? secondary-key) Sorry it's not a Darcs patch. I find those patches pretty annoying to work with... Leslie From killerstorm at newmail.ru Mon May 5 13:15:43 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Mon, 5 May 2008 16:15:43 +0300 Subject: [elephant-devel] Re: Postmodern, Act II References: <62295.88.73.226.33.1209757291.squirrel@mail.stardawn.org><64660.88.73.200.178.1209798228.squirrel@mail.stardawn.org><708B64FD70804C859F752F4C04FF8EAE@killer><61788.88.73.200.178.1209804876.squirrel@mail.stardawn.org><6AAE571E5E624EB0856C563EF9067773@killer> <63876.88.73.197.54.1209974656.squirrel@mail.stardawn.org> Message-ID: LPP> you explained that global-sync cache mode only works within a txn. LPP> I thought that per-transaction cache mode does this, and that global LPP> sync extends this behaviour so transactions/threads can share the LPP> cache? LPP> If not, what's the difference between them? both modes rely on transactional semantics -- assumption that transactions are isolated and changes from one transaction do not (and should not) propagate into other one. and both use transaction boundaries -- begin/commit -- to their initialization/finalization. in per-transaction mode cache is created on beginning of transaction and is abandoned on commit. global-sync mode extends it reusing cache between transaction. when transactions starts, it checks if there are existing cache instances in a pool and takes one if it exists, or creates new one. if existing cache instance is used, it gets synchronized on first use -- and once within a single transaction. synchronization is actually reading what's changed and wiping those entries, however it is somewhat complicated to pull only changes _since_ last update. when transaction commits successfully, cache is returned into the pool and can be re-used later. sidenote: "global-sync" actually means "globally synchronized" that sort of means that all writes in all processes/threads are tracked and cache entries are invalidated (synchronized) accordingly, that allows us to re-use cache between transactions. so, both modes only makes sense when there is considerable amount of database reads within each transaction: per-transaction mode for obvious reasons, and sync cache because synchronization (that must be done inside each transaction) requires at least few database commands and it is not feasible to synchronize if it's only going to save single read command. interaction of global-sync cache and threads is another question -- currently it does not share data among concurrently running threads. if you have 10 threads running in parallel, you'll have 10 independent cache instances. cache pool is global though, so if you create new thread for each event, it won't be a problem -- it will take first cache in pool if available. it might seem like this behaviour is suboptimal -- it sharing cache among threads will save space. but it is a tradeoff -- shared cache will require some sort of locking, and that might reduce performance. so it actually depends on database usage patterns which scheme is prefered. as cache sharing is considerably more complex to implement, only isolated thing is implemented for now. From leslie.polzer at gmx.net Mon May 5 14:25:06 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 5 May 2008 16:25:06 +0200 (CEST) Subject: [elephant-devel] Postmodern: Removing the ABORT restart Message-ID: <63401.88.73.197.54.1209997506.squirrel@mail.stardawn.org> When an error occurs inside a txn, this restart will lose the controller. It should be replaced by the ABORT-TRANSACTION restart behaviour. Anyone against this? Leslie From klists at saphor.de Mon May 5 16:52:28 2008 From: klists at saphor.de (Marc) Date: Mon, 05 May 2008 18:52:28 +0200 Subject: [elephant-devel] Newbie question: Index on collections In-Reply-To: <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu> <47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> Message-ID: <481F3B4C.30004@saphor.de> Ian Eslick wrote: > Definitely work elephant-unstable if you want to test the new > functionality. Most of the activity in the main branch has been in > and around the postmodern interface. The limiting factor in merging > the branches is getting support for the new data store API (i.e. for > btrees w/ duplicate keys) in CL-SQL and Postmodern. Robert has made > some good progress on CL-SQL but we haven't had any work done on this > in Postmodern yet. > > I just merged out everything from the main elephant branch into > unstable, so you won't be missing anything; except of course it only > works for the BDB data store now. many thanks --- and BDB is the only datastore we use and can look into in any case! Best regards, Marc > > Ian > > On May 3, 2008, at 3:56 AM, Marc wrote: > >> Marc wrote: >>> Ian Eslick wrote: >>>> Great! Actually in writing the following e-mail, I realized that >>>> this had forced me to think through a bunch of things and so I >>>> wrote a version of the functionality in associations.lisp and just >>>> checked it in. >>>> >>>> Start playing with, reading and commenting on what's wrong with >>>> it. For example, I'd like a better API for adding/removing items >>>> from slots than overloading (setf (accessor instance) >>>> related-instance). I left a commented API proposal in the source >>>> code for you to look at. >>>> >>>> There are still bugs that I haven't thought through that will crop >>>> up when you change classes, redefine classes, etc. Let me know >>>> what you think. >>> Thanks for the file and the documentation in this mail. Starting >>> from this I'll now begin to dig into it and try to grope my way around. >>> >>> As to timing: this week is somewhat busy due to a conference, but I >>> still hope for some time in between sessions. >> I fear that the delay has been much longer than anticipated --- it >> has been very busy ever after the conference ---, but no w I hope >> finally to find some time --- provided that the issue is not long >> obsolete. >> >> Best regards, >> >> Marc >> >> P. S.: I've noticed that there have been few changes in the unstable >> branch lately, but many more in the stable one. Which one is the one >> to work on for this right now? >>> >>>> >>>> ======================== >>>> >>>> Download the elephant-unstable branch: >>>> >>>> darcs get >>>> http://www.common-lisp.net/project/elephant/darcs/elephant-unstable >>>> >>>> It's probably best to spend a little time reading and >>>> experimenting, then write up a proposal so we can discuss it a >>>> bit. The support for defining association slots (but not >>>> implementing them) is already implemented in metaclasses.lisp. >>>> >>>> The idea of an association is that instances of class A can find >>>> instances of class B that are 'associated' with it. Association >>>> relations can be undirected or directed, typically called >>>> many-to-many or 1-to-many. 1-to-many is a kind of inverse indexing >>>> and many-to-many requires a separate btree/class/table to maintain >>>> the undirected relation. Hopefully the following description will >>>> help clarify what I mean. >>>> >>>> >>>> One-to-Many associations: >>>> >>>> If class B has a slot that stores a single instance of A, then >>>> class A would like to have a method that returns the set of >>>> instances of B that refer to the instance of A. >>>> >>>> If A is person and B is job, then job->owner is a reference to A >>>> and A->jobs is a method (no slot storage) that refers to an index >>>> on job->owner. The index keys are A oids. >>>> >>>> If you look in indexed-slots.lisp, you'll get a sense of how >>>> indices work. set-valued-slots.lisp is also useful as it puts a >>>> btree in each instance's slot to store sets of objects using the >>>> pset abstraction in pset.lisp. Reading these three files should >>>> help alot to see how we use btrees and dup-btrees to maintain sets >>>> of objects and relations. >>>> >>>> >>>> One challenge is how to specify an association implicitly in slot >>>> definitions. Ideally you want it to be on the object that you are >>>> most often going to use to add and remove relations: >>>> >>>> (defpclass person () >>>> ((jobs :accessor jobs :association (job owner)))) >>>> >>>> (defpclass job () >>>> ((title :accessor title ...) >>>> (company :accessor company ...))) >>>> >>>> But this means that person has to change the definition of job and >>>> that can be problematic. So you could require agreement... >>>> >>>> (defpclass person () >>>> ((jobs :accessor jobs :association (job owner))) >>>> >>>> (defpclass job () >>>> ((title ...) >>>> (company ...) >>>> (owner :accessor owner :association person))) >>>> >>>> This means that job has a slot owner who's values are instances of >>>> the class person. This slot is also effectively indexed. A >>>> person has a method jobs which looks up the index in (job owner) >>>> and returns the list of instances of job that have the instance of >>>> A in the owner slot. >>>> >>>> >>>> Many-to-Many: >>>> >>>> To add many-to-many relations, any A->B relation also requires that >>>> we can find the relation from the other direction B->A. Typically >>>> this requires a special table to keep track of pairs of related >>>> objects. You can then query on either column to get the set of >>>> related objects. >>>> >>>> many-to-many association: A<->B >>>> (A1 B1) >>>> (A1 B2) >>>> (A2 B1) >>>> (A2 B3) >>>> >>>> (get-related assoc A1)->B1,B2 >>>> (get-related assoc B1)->A1,A2 >>>> >>>> We can simply stick with the index model and just ensure that there >>>> are indexes on both classes instead of one. When we add an element >>>> to one side, we also add it to the other. (This is what is >>>> implemented now) >>>> >>>> >>>> Optimizations/Alternatives: >>>> >>>> One optimization for one-to-many is to return a standard object >>>> implementing the pset protocol that just keeps a reference to the >>>> index and supports map operations over the appropriate subset of >>>> the objects in the index. >>>> >>>> Regards, >>>> Ian >>>> >>>> On Mar 19, 2008, at 2:42 AM, Marc wrote: >>>>> Hi Ian, >>>>>> >>>>>> My best guess is weeks, not months. We're hoping for April >>>>>> sometime, but it depends on a number of factors. Elephant is >>>>>> getting quite complex to test so the last few bugs can take some >>>>>> time. >>>>>> >>>>>> You can speed this process up if you want to help develop and >>>>>> test the association facility (most of the hooks are there so it >>>>>> shouldn't be too hard, even if you don't fully understand the >>>>>> elephant internals). If not, we have plenty of new features that >>>>>> need tests written against them! >>>>> Since we need this feature anyway for our application, I'm quite >>>>> prepared to join in the work on the association facility. Any >>>>> pointer on what would be the best entry point into that task? >>>>> >>>>> Best regards, >>>>> >>>>> Marc >>>>>> >>>>>> On Mar 18, 2008, at 12:50 AM, Marc wrote: >>>>>> >>>>>>> Ian Eslick wrote: >>>>>>>> The pset is probably your best bet. Other collection objects >>>>>>>> have more performance impact. With a pset you can ask: >>>>>>>> >>>>>>>> is X a friend of Y? (find-item X (friends Y)) => X | nil >>>>>>>> >>>>>>>> all people that X is friends of: >>>>>>>> >>>>>>>> (remove-nulls >>>>>>>> (map-class 'people (lambda (y) >>>>>>>> (when (find-item X (friends Y)) >>>>>>>> Y)) >>>>>>>> :collect t)) >>>>>>>> >>>>>>>> This requires walking all person objects. For larger DB's, you >>>>>>>> can build your own 'many-to-many' relation table or waiting for >>>>>>>> the next release which should do this for you. >>>>>>> Is there a time line (however preliminary) when this release >>>>>>> might be >>>>>>> available? >>>>>>>> >>>>>>>> (defpclass friend-relation () >>>>>>>> ((first :accessor first :initarg :first :index t) >>>>>>>> (second :accessor second :initarg :second :index t))) >>>>>>>> >>>>>>>> (defmethod add-friend-relation (x y) >>>>>>>> (make-instance 'friend-relation :first x :second y)) >>>>>>>> >>>>>>>> (defmethod friends-of (x) >>>>>>>> (union (mapcar #'second (get-instances-by-slot 'friend-relation >>>>>>>> 'first x)) >>>>>>>> (mapcar #'first (get-instances-by-slot 'friend-relation >>>>>>>> 'second x)))) >>>>>>>> >>>>>>>> Of course there are ways to do this more efficiently, but I >>>>>>>> think this is the idea. >>>>>>>> >>>>>>> Many thanks for this guidance. That does, indeed, give me an >>>>>>> idea of the >>>>>>> best way forward in our application. >>>>>>> >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> Marc >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mar 16, 2008, at 11:06 PM, Marc wrote: >>>>>>>> >>>>>>>>> Hello! >>>>>>>>> >>>>>>>>> I fear that this may be a rather stupid newbie question, but >>>>>>>>> neither from the documentation nor from the mailing list posts >>>>>>>>> I got an idea as to the best practice for the following type >>>>>>>>> of searches. Let's use an example akin to the one in the >>>>>>>>> tutorial: >>>>>>>>> >>>>>>>>> (defpclass person () >>>>>>>>> ((name :accessor name :index t) >>>>>>>>> (friends :accessor friends)) ;collection with names (say as >>>>>>>>> strings) of the friends of this person >>>>>>>>> ) >>>>>>>>> >>>>>>>>> Given a similar structure, what is the recommended way to >>>>>>>>> handle queries of the type "find all persons that X is a >>>>>>>>> friend of?" (i.e. queries on individual entries in slots that >>>>>>>>> contain collections)? What would be the best data structure >>>>>>>>> for the friends slot, assuming that the collection remains >>>>>>>>> fairly stable over time (psets, lists or other collections)? >>>>>>>>> >>>>>>>>> Thanks in advance for any hint! >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> >>>>>>>>> Marc >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> elephant-devel site list >>>>>>>>> elephant-devel at common-lisp.net >>>>>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> elephant-devel site list >>>>>>>> elephant-devel at common-lisp.net >>>>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> elephant-devel site list >>>>>>> elephant-devel at common-lisp.net >>>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>>> >>>>>> _______________________________________________ >>>>>> elephant-devel site list >>>>>> elephant-devel at common-lisp.net >>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> elephant-devel site list >>>>> elephant-devel at common-lisp.net >>>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>> >>>> _______________________________________________ >>>> elephant-devel site list >>>> elephant-devel at common-lisp.net >>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>> >>>> >>>> >>> >>> >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >>> >>> >>> >> >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > > From killerstorm at newmail.ru Mon May 5 17:50:56 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Mon, 5 May 2008 20:50:56 +0300 Subject: [elephant-devel] Re: Postmodern: Removing the ABORT restart References: <63401.88.73.197.54.1209997506.squirrel@mail.stardawn.org> Message-ID: LPP> When an error occurs inside a txn, this restart will lose the LPP> controller. It should be replaced by the ABORT-TRANSACTION restart LPP> behaviour. sorry, i didn't get -- what is "ABORT restart", how does it lose the controller and how are you going to remove it? From eslick at media.mit.edu Mon May 5 12:30:11 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 5 May 2008 08:30:11 -0400 Subject: [elephant-devel] Staus of the port of Ian's new functional toCL-SQL backend.... In-Reply-To: <1209955814.3971.1901.camel@penguin.yourdomain.com> References: <47DDE029.2090204@saphor.de> <16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu> <47DF49FF.8070206@saphor.de> <47E0B5E4.4000809@saphor.de> <86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu> <47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de> <02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu> <62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org> <1209852291.3971.1822.camel@penguin.yourdomain.com> <1209955814.3971.1901.camel@penguin.yourdomain.com> Message-ID: <628C46EB-87B2-41AA-B2E0-6FC7021361AD@media.mit.edu> Awesome! Thank you Robert. Henrik? Alex? Ian On May 4, 2008, at 10:50 PM, Robert L. Read wrote: > I have now committed an all-green solution for the CL-SQL backend to > elephant-unstable. > > I don't know that it will help the Postmodern work much --- perhaps > just > as a pointer for what needs to be done. > > Nonetheless, it is ready for someone who wants to work on making > Postmodern work with elephant-unstable to review if they like. > > > > > > On Sun, 2008-05-04 at 10:37 +0300, Alex Mizrahi wrote: >> RLR> implementing a new kind of btree that supports duplicates >> (hard). >> >> ain't btree with duplicates very similar to btree-index thing, >> because >> btree-index also supports duplicates? >> >> RLR> Since Postmodern uses the same kind of cursor control that the >> CL-SQL >> RLR> backend does, >> >> why do you think so? cursor are probably biggest difference between >> CL-SQL >> and postmodern -- instead of reading everything and sorting on lisp >> side, we >> use sorting done by postgresql, and pick stuff in portions, i.e. >> cursor-next >> is something like: >> >> SELECT k, v FROM btree11 WHERE ((k = $1) and (v > $2)) OR (k > $1) >> LIMIT 10 >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Mon May 5 14:17:18 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 5 May 2008 10:17:18 -0400 Subject: [elephant-devel] 1.0 Release Planning Message-ID: <3EB0F70A-56F6-4324-B9FA-D31B29E2B2D5@media.mit.edu> Robert and I have agreed about moving Elephant towards a 1.0 release. Elephant isn't perfect, but more and more people appear to be having success using it and we feel this kind of predictability is important for its ongoing adoption. The purpose of this 1.0 release is to provide a stable long-term platform for users with a commitment from us to back patch the release as necessary to fix any significant bugs. It also is a clear statement about our confidence in the platform. The major emphasis from this point forward is test coverage, stability and documentation. I'm particularly interested in documenting more use-cases and best-practices. Alex's discussion of using derived indices to make selecting messages by username and dates is one example. We would like to institute an effective feature freeze as of the upcoming beta release, so before we institute that freeze, I'd like invite everyone to weigh in on smaller features or annoyances that would make a big difference to the utility, usability or stability of 1.0. For example I'd like to implement/integrate a dump/restore capability to/from XML or s-expressions. You can see what we've got planned in the elephant trac roadmap: http://trac.common-lisp.net/elephant/roadmap Click on any of the milestone there to see the current tickets associated with it. Given that everyone is quite busy, it may take a few months to get to 1.0. The best way to contribute to this process is to start using elephant-unstable (which I will rename to something less scary) and make sure the test suite passes on as wide a variety of platforms, lisps and data stores as possible. In particular, testing the upgrade procedure on real applications from 0.9.1 to elephant-unstable would be very helpful. Thank you, Ian From read at robertlread.net Tue May 6 03:28:47 2008 From: read at robertlread.net (Robert L. Read) Date: Mon, 05 May 2008 22:28:47 -0500 Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE In-Reply-To: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> Message-ID: <1210044527.3971.1953.camel@penguin.yourdomain.com> OK, I have applied this patch. I didn't run the tests. Ordinarily I do before accepting a patch, but I assume if this causes a problem we will catch it quickly in our current state. Of course, please make sure you are all green before submitting a patch. On Mon, 2008-05-05 at 14:07 +0200, Leslie P. Polzer wrote: > Now don't ask me how a NIL value gets mapped to an OID key in the > class index. It happened, and I wasn't able to remove those things > because of a bug in PM-INDEXED-BTREE. Diff: > > --- old-elephant/src/db-postmodern/pm-indexed-btree.lisp 2008-05-05 > 14:04:49.930108544 +0200 > +++ new-elephant/src/db-postmodern/pm-indexed-btree.lisp 2008-05-05 > 14:04:49.930108544 +0200 > @@ -74,8 +74,8 @@ > (defmethod remove-kv (key (bt pm-indexed-btree)) > "Remove a key / value pair, and update secondary indices." > (with-trans-and-vars (bt) > - (let ((value (get-value key bt))) > - (when value > + (multiple-value-bind (value found) (get-value key bt) > + (when found > (let ((indices (indices bt))) > (loop for index being the hash-value of indices do > (multiple-value-bind (index? secondary-key) > > Sorry it's not a Darcs patch. I find those patches pretty annoying > to work with... > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Tue May 6 08:08:49 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Tue, 6 May 2008 10:08:49 +0200 (CEST) Subject: [elephant-devel] Re: Postmodern: Removing the ABORT restart In-Reply-To: References: <63401.88.73.197.54.1209997506.squirrel@mail.stardawn.org> Message-ID: <62688.88.73.198.168.1210061329.squirrel@mail.stardawn.org> > sorry, i didn't get -- what is "ABORT restart", http://www.lisp.org/HyperSpec/Body/res_abort.html > how does it lose the controller After invoking this restart, the controller is gone. I don't know exactly why, but I suppose the reason is that the current transaction isn't aborted. > how are you going to remove it? I don't want to remove it, I want to change its behaviour to do something more appropriate (i.e. do what the ABORT-TRANSACTION restart does). This can be achieved by providing the ABORT restart inside the transaction code via RESTART-CASE/RESTART-BIND. Leslie From leslie.polzer at gmx.net Tue May 6 08:15:27 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Tue, 6 May 2008 10:15:27 +0200 (CEST) Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE In-Reply-To: <1210044527.3971.1953.camel@penguin.yourdomain.com> References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> <1210044527.3971.1953.camel@penguin.yourdomain.com> Message-ID: <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> > OK, I have applied this patch. Thanks! I wonder how NIL values can be mapped to an OID in any case... > I didn't run the tests. Ordinarily I do before accepting a patch, but > I assume if this causes a problem we will catch it quickly in our > current state. It's a pretty trivial change. > Of course, please make sure you are all green before submitting a patch. Naturally. Leslie From eslick at media.mit.edu Tue May 6 09:05:23 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 6 May 2008 05:05:23 -0400 Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE In-Reply-To: <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> <1210044527.3971.1953.camel@penguin.yourdomain.com> <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> Message-ID: <2C6E0324-1641-488F-B6B9-C02B80C3B9B3@media.mit.edu> On May 6, 2008, at 4:15 AM, Leslie P. Polzer wrote: > >> OK, I have applied this patch. > Thanks! I wonder how NIL values can be mapped to an OID in any case... Do you mean in an index? A slot can have nil as a value and being nil is different than being unbound. Same thing is true with slot indices. An unbound slot produces no index key, a nil slot produces a nil index key. >> I didn't run the tests. Ordinarily I do before accepting a patch, >> but >> I assume if this causes a problem we will catch it quickly in our >> current state. > > It's a pretty trivial change. > >> Of course, please make sure you are all green before submitting a >> patch. > > Naturally. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From killerstorm at newmail.ru Tue May 6 10:59:49 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Tue, 6 May 2008 13:59:49 +0300 Subject: [elephant-devel] Re: Small bug in PM-INDEXED-BTREE References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org><1210044527.3971.1953.camel@penguin.yourdomain.com> <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> Message-ID: ??>> OK, I have applied this patch. LPP> Thanks! I wonder how NIL values can be mapped to an OID in any case... yep, that's pretty interesting. looks like a bug -- values in class index must be instances of that class -- but i do not see how it's possible at all.. but NIL are legal in indexed-btrees that are not class indices, so it seems we can't just ban them (to catch this bug) From leslie.polzer at gmx.net Tue May 6 11:18:22 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Tue, 6 May 2008 13:18:22 +0200 (CEST) Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE In-Reply-To: <2C6E0324-1641-488F-B6B9-C02B80C3B9B3@media.mit.edu> References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> <1210044527.3971.1953.camel@penguin.yourdomain.com> <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> <2C6E0324-1641-488F-B6B9-C02B80C3B9B3@media.mit.edu> Message-ID: <64564.88.73.198.168.1210072702.squirrel@mail.stardawn.org> > Do you mean in an index? A slot can have nil as a value and being nil > is different than being unbound. Same thing is true with slot > indices. An unbound slot produces no index key, a nil slot produces a > nil index key. It occured in the object index for a class; the OIDs of six objects were mapped to NIL. Leslie From eslick at media.mit.edu Tue May 6 14:34:31 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 6 May 2008 10:34:31 -0400 Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE In-Reply-To: <64564.88.73.198.168.1210072702.squirrel@mail.stardawn.org> References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> <1210044527.3971.1953.camel@penguin.yourdomain.com> <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> <2C6E0324-1641-488F-B6B9-C02B80C3B9B3@media.mit.edu> <64564.88.73.198.168.1210072702.squirrel@mail.stardawn.org> Message-ID: <89060EB4-2214-46F7-910A-283698AA7692@media.mit.edu> I assume this is not the new branch? Could it be a bug due to the behavior of delete-pobj? On May 6, 2008, at 7:18 AM, Leslie P. Polzer wrote: > >> Do you mean in an index? A slot can have nil as a value and being >> nil >> is different than being unbound. Same thing is true with slot >> indices. An unbound slot produces no index key, a nil slot >> produces a >> nil index key. > > It occured in the object index for a class; the OIDs of six objects > were mapped to NIL. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Tue May 6 15:16:47 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Tue, 6 May 2008 17:16:47 +0200 (CEST) Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE In-Reply-To: <89060EB4-2214-46F7-910A-283698AA7692@media.mit.edu> References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> <1210044527.3971.1953.camel@penguin.yourdomain.com> <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> <2C6E0324-1641-488F-B6B9-C02B80C3B9B3@media.mit.edu> <64564.88.73.198.168.1210072702.squirrel@mail.stardawn.org> <89060EB4-2214-46F7-910A-283698AA7692@media.mit.edu> Message-ID: <61511.88.73.198.168.1210087007.squirrel@mail.stardawn.org> > I assume this is not the new branch? Correct. > Could it be a bug due to the behavior of delete-pobj? Well possible. I used it (i.e. DROP-POBJECT or DROP-INSTANCES) in the case described. Leslie From killerstorm at newmail.ru Wed May 7 10:24:07 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Wed, 7 May 2008 13:24:07 +0300 Subject: [elephant-devel] Re: Postmodern: Removing the ABORT restart References: <63401.88.73.197.54.1209997506.squirrel@mail.stardawn.org> <62688.88.73.198.168.1210061329.squirrel@mail.stardawn.org> Message-ID: ??>> sorry, i didn't get -- what is "ABORT restart", LPP> http://www.lisp.org/HyperSpec/Body/res_abort.html "Typically, in an interactive listener, the invocation of abort returns to the Lisp reader phase of the Lisp read-eval-print loop, though in some batch or multi-processing situations there may be situations in which having it kill the running process is more appropriate." are you sure that it is a good idea to make it returning to a random point in program (after with-transaction) rather than to a command prompt (what people would expect)? and if people want to abort transaction only, can't they just call abort-transaction? ??>> how does it lose the controller LPP> After invoking this restart, the controller is gone. I don't know LPP> exactly why, but I suppose the reason is that the current transaction LPP> isn't aborted. maybe we'll better fix this bug? non-local control transfer should correctly abort transaction. i think "controller lost" is only possible when when something awful happens to the socket. i can't reproduce "controller lost" via abort restart. i.e. i run such code (ele:with-transaction () (make-instance 'something) (error "barf")) and when SLIME debugger opens, i choose ABORT restart, and it returns to REPL. and works quite fine afterwards.. if you have that "controller lost" thing reproducible, please share how to reproduce this From killerstorm at newmail.ru Wed May 7 11:04:20 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Wed, 7 May 2008 14:04:20 +0300 Subject: [elephant-devel] Staus of the port of Ian's new functionaltoCL-SQL backend.... References: <47DDE029.2090204@saphor.de><16965B4A-CAB4-40EE-9A85-D568C8A3BDEB@media.mit.edu><47DF49FF.8070206@saphor.de><47E0B5E4.4000809@saphor.de><86FF036F-E2E1-452E-B18C-787BF8205FFC@media.mit.edu><47E78550.4000704@saphor.de> <481C1A9F.8050100@saphor.de><02CEBFF4-BCA0-44A7-802D-19E1B7B13E67@media.mit.edu><62759.88.73.200.178.1209829252.squirrel@mail.stardawn.org><1209852291.3971.1822.camel@penguin.yourdomain.com><1209955814.3971.1901.camel@penguin.yourdomain.com> <628C46EB-87B2-41AA-B2E0-6FC7021361AD@media.mit.edu> Message-ID: IE> Awesome! Thank you Robert. IE> Henrik? Alex? yep, thanks Robert, i'm looking into this.. From killerstorm at newmail.ru Wed May 7 20:20:11 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Wed, 7 May 2008 23:20:11 +0300 Subject: [elephant-devel] Re: Derived Indicies References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> Message-ID: <4A9C590E61F0448B9801785B99D74D5F@killer> IE> Given that in Elephant, the cost of a btree instance is negligable, it postmodern backend each btree is backed by a database table, so i think it's not that negligable there.. IE> one way to do this is the just have a set of messages for each user in IE> a user slot (a pset or set-valued slot). but i think it's quite easy to make cheap psets -- just a dup-btree of pset-oid->item IE> messages. Of course to get the full efficiency you want, we'd have to IE> make a sorted-pset that sorts on some fn of the elements. sorted-pset? how can we sort something in database without having a key? i think that would be more like indexed-pset, with indices being btrees. as an interface, it could look like: (get-by-range (messages-of user) 'date yesterday nil) but, damn, it would be much harder to make cheap btrees with SQL.. IE> I'd like to think about different metaphors for solving these problems IE> so we don't end up turning Elephant's interface into a less efficient IE> variation of an ORM! i thought elephant is a BDB wrapper :) From leslie.polzer at gmx.net Thu May 8 06:53:26 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Thu, 8 May 2008 08:53:26 +0200 (CEST) Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: <4A9C590E61F0448B9801785B99D74D5F@killer> References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> <4A9C590E61F0448B9801785B99D74D5F@killer> Message-ID: <61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org> > as an interface, it could look like: > (get-by-range (messages-of user) 'date yesterday nil) >From my daily work I can say that this functionality would make Elephant way more useful. Leslie From leslie.polzer at gmx.net Thu May 8 10:14:21 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Thu, 8 May 2008 12:14:21 +0200 (CEST) Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: <61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org> References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> <4A9C590E61F0448B9801785B99D74D5F@killer> <61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org> Message-ID: <63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> Wouldn't it make sense to generalize the class index mechanism? At the moment we have one big global name space for indexed objects. Storing everything there is hellishly expensive for many purposes. Example: (intersection (get-instances-by-value 'message 'owner "Charlie") (get-instances-by-value 'message 'folder 'inbox) ...) has unholy complexity and is therefore unsuited to numbers as low as 1k objects. It would be nice to store Charlie's inbox right in Charlie's user object, making it a btree with indices defined by the user in DEFPCLASS. I wonder if it would be a lot of effort to generalize GET-INSTANCES-BY... so it takes a class root parameter, and to manage a multitude of indexed class btrees. Leslie From eslick at media.mit.edu Thu May 8 11:11:48 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Thu, 8 May 2008 07:11:48 -0400 Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: <4A9C590E61F0448B9801785B99D74D5F@killer> References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> <4A9C590E61F0448B9801785B99D74D5F@killer> Message-ID: <9D02DB92-0148-4065-9002-4DC1FB79B168@media.mit.edu> On May 7, 2008, at 4:20 PM, Alex Mizrahi wrote: > IE> Given that in Elephant, the cost of a btree instance is > negligable, > > it postmodern backend each btree is backed by a database table, > so i think it's not that negligable there.. Ah, I think the CL-SQL and BDB have different performance implications then. What is the purpose of doing it this way? Are you specializing the table on the type of the keys in some way? > IE> one way to do this is the just have a set of messages for each > user in > IE> a user slot (a pset or set-valued slot). > > but i think it's quite easy to make cheap psets -- just a dup-btree > of pset-oid->item That's nice; that was the intention of the default pset implementation. > IE> messages. Of course to get the full efficiency you want, we'd > have to > IE> make a sorted-pset that sorts on some fn of the elements. > > sorted-pset? how can we sort something in database without having a > key? BDB can sort on values as well. The pset abstraction is currently guaranteed to be an unordered collection or unique objects. > i think that would be more like indexed-pset, with indices being > btrees. > > as an interface, it could look like: > (get-by-range (messages-of user) 'date yesterday nil) > > but, damn, it would be much harder to make cheap btrees with SQL.. You could just do this directly using an indexed-btree if you really needed to. Probably better to work on a native-lisp solution than to try to contort SQL to do this! > IE> I'd like to think about different metaphors for solving these > problems > IE> so we don't end up turning Elephant's interface into a less > efficient > IE> variation of an ORM! > > i thought elephant is a BDB wrapper :) It was, once. It's slowly evolving into an OODB, but people are so used to SQL and ORM that I sense a strong leaning towards making the data representation and query access look like those you find in ORM systems. Especially since many users are using postmodern, and the effective way to make the Elephant model faster on top of Postmodern is to change the abstractions to look more like tables and SQL. Ian From henrik at evahjelte.com Thu May 8 12:23:34 2008 From: henrik at evahjelte.com (Henrik Hjelte) Date: Thu, 8 May 2008 14:23:34 +0200 Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: <9D02DB92-0148-4065-9002-4DC1FB79B168@media.mit.edu> References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu> <1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> <4A9C590E61F0448B9801785B99D74D5F@killer> <9D02DB92-0148-4065-9002-4DC1FB79B168@media.mit.edu> Message-ID: <50e8e4f60805080523v16d541a3h43e7fddabc94254a@mail.gmail.com> On Thu, May 8, 2008 at 1:11 PM, Ian Eslick wrote: > > On May 7, 2008, at 4:20 PM, Alex Mizrahi wrote: > >> IE> Given that in Elephant, the cost of a btree instance is negligable, >> >> it postmodern backend each btree is backed by a database table, >> so i think it's not that negligable there.. > > Ah, I think the CL-SQL and BDB have different performance implications then. > What is the purpose of doing it this way? Are you specializing the table > on the type of the keys in some way? The table is the closest thing to a btree structure in an sql database. For example cursor operations are done on table. That is the idea of using tables as btrees. /Henrik From eslick at media.mit.edu Thu May 8 12:55:37 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Thu, 8 May 2008 08:55:37 -0400 Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: <50e8e4f60805080523v16d541a3h43e7fddabc94254a@mail.gmail.com> References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu> <1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> <4A9C590E61F0448B9801785B99D74D5F@killer> <9D02DB92-0148-4065-9002-4DC1FB79B168@media.mit.edu> <50e8e4f60805080523v16d541a3h43e7fddabc94254a@mail.gmail.com> Message-ID: <85B20EA7-6B81-4091-BFD6-2E603EAC7C7A@media.mit.edu> > > The table is the closest thing to a btree structure in an sql > database. For example cursor operations are done on table. That is the > idea of using tables as btrees. BDB maps multiple btrees onto a single large btree. An Elephant btree key is a binary concatenation of btree-oid+key. This presumes that a custom sorting function is available. This is why Elephant btrees are 'cheap'. From eslick at media.mit.edu Thu May 8 14:28:45 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Thu, 8 May 2008 10:28:45 -0400 Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: <63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> <4A9C590E61F0448B9801785B99D74D5F@killer> <61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org> <63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> Message-ID: On May 8, 2008, at 6:14 AM, Leslie P. Polzer wrote: > > Wouldn't it make sense to generalize the class index mechanism? class indexing is basically a specialization ofbtrees. It's pretty easy to build your own indexes for special purposes and to store them wherever you want. Common idioms can be optimized via macros > At the moment we have one big global name space for indexed objects. > Storing everything there is hellishly expensive for many purposes. > > Example: > > (intersection > (get-instances-by-value 'message 'owner "Charlie") > (get-instances-by-value 'message 'folder 'inbox) > ...) > > has unholy complexity and is therefore unsuited to numbers > as low as 1k objects. You can always try Alex's approach, create a derived index on 'message which computes and sorts on: (format nil "~A::~A" owner folder). Postmodern doesn't sort on aggregates like BDB can or you could do (cons owner folder) instead. > It would be nice to store Charlie's inbox right in Charlie's > user object, making it a btree with indices defined by the > user in DEFPCLASS. You mean something like: (defpclass person () ((name ...) (inbox :accessor inbox :initform (make-indexed-btree message) :index-on (date sender)))) Which would create a indexed-btree that stored messages and auto- created indices on the date and sender slots? That is a bit more OODB/Lispy than the association mechanism I've implemented. However I don't think you need to overload defpclass to get this functionality. In fact I think it starts to get too ugly. inbox should be an object (class instance or btree) that has its own api. (defmethod initialize-instance :after ((p person) &rest args) (setf (inbox person) (make-inbox))) (defun make-inbox () (let ((btree (make-indexed-btree))) (add-index ...))) (defun get-messages-by-date (user start end) ...) (add-message (inbox charlie) message) --------------- The new association mechanism is basically a simple btree version of this, but the btree is implicit. (defpclass user () ((name :accessor name) (index :accessor index :associate (message user)))) (setf charlie (make-instance 'user)) (defpclass message () ((user :accessor user :associate user))) (make-instance 'message charlie) (make-instance 'message charlie) (inbox charlie) => returns two messages with charlie in the user slot However associations, like psets, are not sorted (dup-btree oid:instance-ref). The value of (user message) is a persistent object that is added to a dup-btree maintained by the metaclass protocol. It maps the oid of a user to the messages that store it. I couldn't find a clean way of making the indexed-slot mechanism work with associations, which was the reason for moving to dup-btrees. My thought was that the inbox function would be implemented either by a scan of the subset of messages indicated by the inbox association, or via a cheap join that intersects two arrays of oids, one from the association, one from the appropriate index so the result is a sorted list of oids that can be instantiated as needed. It would be reasonably cheap to do query caching of these sorted OIDs so that subsequent OFFSET & LIMIT style accesses over the same query set would be fast, just instantiating those messages that are needed. I haven't worked through the interface ideas, but this is the general idea that was, in part, driving the association mechanism. The derived index hack is still more efficient for large sets. Without changes to the data stores to create an efficient way of sorting concatenated values, I don't see a way to improve on it easily. > I wonder if it would be a lot of effort to generalize > GET-INSTANCES-BY... so it takes a class root parameter, and > to manage a multitude of indexed class btrees. A more general query language is probably the right solution for this interface. The query language would know about associations, derived indices, etc and perform query planning via introspection over the class objects. > Leslie > > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Thu May 8 15:32:35 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Thu, 8 May 2008 17:32:35 +0200 (CEST) Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> <4A9C590E61F0448B9801785B99D74D5F@killer> <61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org> <63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> Message-ID: <63364.88.73.251.95.1210260755.squirrel@mail.stardawn.org> > (defpclass person () > ((name ...) > (inbox :accessor inbox :initform (make-indexed-btree message) > :index-on (date sender)))) > > Which would create a indexed-btree that stored messages and auto- > created indices on the date and sender slots? > > That is a bit more OODB/Lispy than the association mechanism I've > implemented. However I don't think you need to overload defpclass to > get this functionality. In fact I think it starts to get too ugly. I was thinking of this, but the other way round; MESSAGE should contain the usual index declarations: (defpclass message () ((date :index t) ; create an index on DATE (sender :index t))) ; create an index on SENDER These indices would be created in the correct "indexing namespace" when the object is put into a container. This is easy to maintain with, say, an INDEXED-SET class. Whatever. This should probably also imply that objects of the MESSAGE class are not automatically registered in the store controller's class root (as it is now); this should only happen on (defpclass message () (...) (:index t)) > inbox should be an object (class instance or btree) that has its own > api. In fact that's my current solution but I don't like it much. It's not good for quick development. You need to think too much about the storage part. > However associations, like psets, are not sorted (dup-btree > oid:instance-ref). The value of (user message) is a persistent object > that is added to a dup-btree maintained by the metaclass protocol. It > maps the oid of a user to the messages that store it. All too complicated. IMHO a great feature of Elephant is that it let you work with your objects without worrying much about the storage backend. As of now, this feature still needs to get better. Sure, it works (at least for me), but it's not as nice as it could be. Elephant should take care of all the low-level sorting stuff (probably creating indices wherever needed or even sorting without indices for prototyping). > It would be reasonably cheap to do query caching of these sorted > OIDs so that subsequent OFFSET & LIMIT style accesses over the same > query set would be fast, just instantiating those messages that are needed. While I'm at it: OFFSET and LIMIT (a real limit which lets you specify an arbitrary Lisp expression) are things we definitely want to aim for in 1.0. They are not difficult to implement at all, but they don't work with GET-INSTANCES-BY-* and, worse, MAP-BTREE. This means everyone has to write their own version of these functions that take appropriate arguments and move the cursor around themselves instead of relying on a simple high-level API. I'd have implemented these extensions myself, but I thought it better to wait for the integration of the query language to add it. > The derived index hack is still more efficient for large sets. > Without changes to the data stores to create an efficient way of > sorting concatenated values, I don't see a way to improve on it easily. I'm not sure you actually need concatenated index values at all if you manage your objects correctly. I.e. putting them in appropriate containers (the natural OODB way) as opposed to throwing them all together in some indexing namespace and then tediously (for programmer and machine) selecting the stuff you need. Leslie From eslick at media.mit.edu Thu May 8 16:17:53 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Thu, 8 May 2008 12:17:53 -0400 Subject: [elephant-devel] Re: Derived Indicies In-Reply-To: <63364.88.73.251.95.1210260755.squirrel@mail.stardawn.org> References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu> <4A9C590E61F0448B9801785B99D74D5F@killer> <61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org> <63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> <63364.88.73.251.95.1210260755.squirrel@mail.stardawn.org> Message-ID: <11491124-8029-4762-BA6D-6479AE333CBB@media.mit.edu> On May 8, 2008, at 11:32 AM, Leslie P. Polzer wrote: > >> (defpclass person () >> ((name ...) >> (inbox :accessor inbox :initform (make-indexed-btree message) >> :index-on (date sender)))) >> >> Which would create a indexed-btree that stored messages and auto- >> created indices on the date and sender slots? >> >> That is a bit more OODB/Lispy than the association mechanism I've >> implemented. However I don't think you need to overload defpclass to >> get this functionality. In fact I think it starts to get too ugly. > > > I was thinking of this, but the other way round; MESSAGE should > contain > the usual index declarations: > > (defpclass message () > ((date :index t) ; create an index on DATE > (sender :index t))) ; create an index on SENDER > > These indices would be created in the correct "indexing namespace" > when the object is put into a container. This is easy to maintain > with, say, an INDEXED-SET class. Whatever. Can you be more explicit about what indexes you imagine being created here and where they are stored and how they are accessed? > This should probably also imply that objects of the MESSAGE class > are not automatically registered in the store controller's class > root (as it is now); this should only happen on > > (defpclass message () > (...) > (:index t)) How do you know what to sort the 'inbox' sorted-set by? Does it sort on message date or sender or both? FYI, the class indexing function is now implicit, since elephant maintains a master list of oid->class-schema to support the schema mechanism which is itself an indexed btree. I think I have an option to make this connection weak so the class index isn't updated automatically. > >> inbox should be an object (class instance or btree) that has its own >> api. > > In fact that's my current solution but I don't like it much. It's not > good for quick development. You need to think too much about the > storage part. I think we need to work through this model as an independent extension first. I'm leery at this point of premature optimization given the cost in complexity, testing, and API changes that these kinds of changes incur. (plus I'm not going to have time for a major upgrade for some time). > >> However associations, like psets, are not sorted (dup-btree >> oid:instance-ref). The value of (user message) is a persistent >> object >> that is added to a dup-btree maintained by the metaclass protocol. >> It >> maps the oid of a user to the messages that store it. > > All too complicated. IMHO a great feature of Elephant is that it let (setf (user message) charlie) and (inbox charlie) is too complicated? Associations implements and maintains your set-of-objects model without requiring you to explicitly add objects to the appropriate container. Seems like alot of gain for two lines of code! Moreover, it limits the proliferation of btrees which makes life easier for postmodern given the table-per-btree implementation restriction. > you work with your objects without worrying much about the storage > backend. As of now, this feature still needs to get better. > Sure, it works (at least for me), but it's not as nice as it could > be. Elephant should take care of all the low-level sorting stuff > (probably creating indices wherever needed or even sorting without > indices for prototyping). That would definitely be nice, but I'm not convinced the increased complexity is worth the benefit. These things need to work robustly in the presence of migration, schema changes, multiple stores, transactions, the existing MOP, disconnected operations and the limitations of all the different data stores, all without adversely impacting the base performance. One hard part in implementing something like this is telling the system how to hook into the slot-value and (setf slot-value) functions on individual class slots without incurring significant overhead. The query system should do the right thing without these connections via joins and you can add the association declarations when you need better performance. You could figure this out on the fly, of course, but building a large index can tie up the system for quite some time and you don't want that to happen randomly. Of course, all what you propose is doable. > >> It would be reasonably cheap to do query caching of these sorted >> OIDs so that subsequent OFFSET & LIMIT style accesses over the same >> query set would be fast, just instantiating those messages that are >> needed. > > While I'm at it: OFFSET and LIMIT (a real limit which lets you specify > an arbitrary Lisp expression) are things we definitely want to aim > for in 1.0. They are not difficult to implement at all, but they don't > work with GET-INSTANCES-BY-* and, worse, MAP-BTREE. This means > everyone has to write their own version of these functions that > take appropriate arguments and move the cursor around themselves > instead of relying on a simple high-level API. Can't you generalize this today as a higher order function that does this as a scan over an index, something like: (map-inverted-index class index (offset-limit-scanner offset limit- fn) :oids t) (defun offset-limit-scanner (offset limit-fn &optional (sc *store- controller*)) (let ((count 0)) (lambda (oid) (incf count) (when (> count offset) (let ((instance (controller-recreate-instance sc oid))) (when (limit-fn instance) (stop-mapping)))))) In general, I believe these are things to think about for as roadmap items for 1.1 and 1.2. They won't happen soon enough to justify delaying 1.0 for months. As I said above, I think this should be a contrib that implements this behind a macro that generates the appropriate methods for a set of generic functions. get-instances-by- class was always a convenience, not a catch-all. > I'd have implemented these extensions myself, but I thought it better > to wait for the integration of the query language to add it. Well, don't hold your breath. :) Unless someone other than me picks up the query system work, it could be months before I get around to it. > >> The derived index hack is still more efficient for large sets. >> Without changes to the data stores to create an efficient way of >> sorting concatenated values, I don't see a way to improve on it >> easily. > > I'm not sure you actually need concatenated index values at all > if you manage your objects correctly. I.e. putting them in appropriate > containers (the natural OODB way) as opposed to throwing them all > together in some indexing namespace and then tediously (for programmer > and machine) selecting the stuff you need. Hmmm...I'll have to think about that. Good ideas here, let's keep the ideas coming and even better, see some contributions/extensions that implement this without impacting the MOP and all the ultimate automation we might desire. > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From killerstorm at newmail.ru Thu May 8 16:45:56 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Thu, 8 May 2008 19:45:56 +0300 Subject: [elephant-devel] Re: Derived Indicies References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu><4A9C590E61F0448B9801785B99D74D5F@killer> <9D02DB92-0148-4065-9002-4DC1FB79B168@media.mit.edu> Message-ID: <3C8EE78BC8F344AA85B3F215807A4B04@killer> ??>> it postmodern backend each btree is backed by a database table, ??>> so i think it's not that negligable there.. IE> Ah, I think the CL-SQL and BDB have different performance implications IE> then. What is the purpose of doing it this way? Are you specializing IE> the table on the type of the keys in some way? yep, SQL is statically typed, so we only can have sorting if we specify actual type of the table. ??>> but i think it's quite easy to make cheap psets -- just a dup-btree ??>> of pset-oid->item IE> That's nice; that was the intention of the default pset implementation. really? default implementation allocates btree for each pset. but with dup btrees we can have one btree for many psets. IE>>> messages. Of course to get the full efficiency you want, we'd ??>> have to IE>>> make a sorted-pset that sorts on some fn of the elements. ??>> ??>> sorted-pset? how can we sort something in database without having a ??>> key? IE> BDB can sort on values as well. yes, I know, but how is it going to help us? we need a list of message objects sorted by their dates. apparently pset item here is a message object, but how will you tell BDB to sort it on a specific slot? ??>> but, damn, it would be much harder to make cheap btrees with SQL.. IE> Probably better to work on a native-lisp solution than to try to IE> contort SQL to do this! maybe. but it's not like that is completely undoable -- we can just have a table of tripples (btree_oid, key, value) sorted by all the three. the only tricky thing here is dealing with types, but that's solvable -- we can have table for each key and value type combination. ??>> i thought elephant is a BDB wrapper :) IE> It was, once. It's slowly evolving into an OODB, but people are so IE> used to SQL and ORM that I sense a strong leaning towards making the IE> data representation and query access look like those you find in ORM IE> systems. i think main problem is not how it looks, but that query language actually makes programming a lot easier. From killerstorm at newmail.ru Thu May 8 17:00:05 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Thu, 8 May 2008 20:00:05 +0300 Subject: [elephant-devel] Re: Derived Indicies References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu><4A9C590E61F0448B9801785B99D74D5F@killer><61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org><63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> Message-ID: <78DFB9857BF94EE3B7705237EF2091AE@killer> IE> (defpclass user () IE> ((name :accessor name) IE> (index :accessor index :associate (message user)))) IE> (setf charlie (make-instance 'user)) IE> (defpclass message () IE> ((user :accessor user :associate user))) IE> (make-instance 'message charlie) IE> (make-instance 'message charlie) IE> (inbox charlie) => returns two messages with charlie in the user slot ... IE> The derived index hack is still more efficient for large sets. IE> Without changes to the data stores to create an efficient way of IE> sorting concatenated values, I don't see a way to improve on it easily. but if we'll add that tuple sorting, will it be possible to make a nice front-end interface to it? maybe something like: (defpclass user () ((name :accessor name) (inbox :accessor index :associate (message user) :sort-by 'creation-time)) (get-by-range user 'inbox yesterday nil) ? From killerstorm at newmail.ru Thu May 8 17:00:05 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Thu, 8 May 2008 20:00:05 +0300 Subject: [elephant-devel] Re: Derived Indicies References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu><4A9C590E61F0448B9801785B99D74D5F@killer><61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org><63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> Message-ID: <78DFB9857BF94EE3B7705237EF2091AE@killer> IE> (defpclass user () IE> ((name :accessor name) IE> (index :accessor index :associate (message user)))) IE> (setf charlie (make-instance 'user)) IE> (defpclass message () IE> ((user :accessor user :associate user))) IE> (make-instance 'message charlie) IE> (make-instance 'message charlie) IE> (inbox charlie) => returns two messages with charlie in the user slot ... IE> The derived index hack is still more efficient for large sets. IE> Without changes to the data stores to create an efficient way of IE> sorting concatenated values, I don't see a way to improve on it easily. but if we'll add that tuple sorting, will it be possible to make a nice front-end interface to it? maybe something like: (defpclass user () ((name :accessor name) (inbox :accessor index :associate (message user) :sort-by 'creation-time)) (get-by-range user 'inbox yesterday nil) ? From killerstorm at newmail.ru Thu May 8 17:14:22 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Thu, 8 May 2008 20:14:22 +0300 Subject: [elephant-devel] Re: Derived Indicies References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu><4A9C590E61F0448B9801785B99D74D5F@killer><61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org><63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> <63364.88.73.251.95.1210260755.squirrel@mail.stardawn.org> Message-ID: ??>> The derived index hack is still more efficient for large sets. ??>> Without changes to the data stores to create an efficient way of ??>> sorting concatenated values, I don't see a way to improve on it ??>> easily. LPP> I'm not sure you actually need concatenated index values at all LPP> if you manage your objects correctly. I.e. putting them in appropriate LPP> containers (the natural OODB way) as opposed to throwing them all LPP> together in some indexing namespace and then tediously (for programmer LPP> and machine) selecting the stuff you need. actually tuple keys are very similar to multiple namespaces. with tuple keys we have essentially a table of triples (userid, time, message). for specific value of userid, you have a well-defined section of this table, that essentially represents inbox of that user. you can as well call userid "a separate namespace" and call that separate section "a separate btree", but that won't change much. From killerstorm at newmail.ru Thu May 8 17:14:22 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Thu, 8 May 2008 20:14:22 +0300 Subject: [elephant-devel] Re: Derived Indicies References: <0A2E5733-3BF1-4427-BBD5-6E6952B22AC2@media.mit.edu><1A304371-DBF0-489C-8893-064B2521B2A7@media.mit.edu><4A9C590E61F0448B9801785B99D74D5F@killer><61922.88.73.251.95.1210229606.squirrel@mail.stardawn.org><63987.88.73.251.95.1210241661.squirrel@mail.stardawn.org> <63364.88.73.251.95.1210260755.squirrel@mail.stardawn.org> Message-ID: ??>> The derived index hack is still more efficient for large sets. ??>> Without changes to the data stores to create an efficient way of ??>> sorting concatenated values, I don't see a way to improve on it ??>> easily. LPP> I'm not sure you actually need concatenated index values at all LPP> if you manage your objects correctly. I.e. putting them in appropriate LPP> containers (the natural OODB way) as opposed to throwing them all LPP> together in some indexing namespace and then tediously (for programmer LPP> and machine) selecting the stuff you need. actually tuple keys are very similar to multiple namespaces. with tuple keys we have essentially a table of triples (userid, time, message). for specific value of userid, you have a well-defined section of this table, that essentially represents inbox of that user. you can as well call userid "a separate namespace" and call that separate section "a separate btree", but that won't change much. From gtarcea at umich.edu Fri May 9 00:51:34 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Thu, 8 May 2008 20:51:34 -0400 Subject: [elephant-devel] How to get involved? Message-ID: Hi all, I'd like to help out in some way with Elephant, but I'm not quite sure how to participate. I'm using Elephant in a project while still climbing the lisp learning curve. That said, I've been doing software development long enough to be legal-to-drink only counting the years manning a keyboard :-) I figure I have 4 to 8 hours a week I can help out, maybe a bit more when my project (which I'm doing on my own time outside of my day job) and the needs for Elephant coincide. Since it looks like there is a push to get a 1.0 release ready I'm happy to help with tasks such as writing documentation, or running/writing tests. Any pointers would be much appreciated. Thanks, Glenn From eslick at media.mit.edu Fri May 9 02:26:44 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Thu, 8 May 2008 22:26:44 -0400 Subject: [elephant-devel] How to get involved? In-Reply-To: References: Message-ID: <11ACB1FC-8B1B-43A3-9C32-4051AC5BE3E2@media.mit.edu> Hi Glenn, It's great to have another pair of helping hands, welcome! I think the most important next steps, which will also help you get further up to speed on Elephant, is to write tests for the new features in the elephant-unstable branch. This includes the new slot types (set-valued, cached, associations), hierarchical indexing and schema evolution. My updates to the list may describe other changes that I'm not remembering off-hand. testassociations.lisp in the tests directory has a start on the associations and I think testindexing has some updates to handle the new indexing features. Please feel free to ask questions and/or submit documentation as you go. Thank you very much, Ian On May 8, 2008, at 8:51 PM, Glenn Tarcea wrote: > > Hi all, > > I'd like to help out in some way with Elephant, but I'm not quite > sure how to participate. I'm using Elephant in a project while still > climbing the lisp learning curve. That said, I've been doing > software development long enough to be legal-to-drink only counting > the years manning a keyboard :-) > > I figure I have 4 to 8 hours a week I can help out, maybe a bit more > when my project (which I'm doing on my own time outside of my day > job) and the needs for Elephant coincide. Since it looks like there > is a push to get a 1.0 release ready I'm happy to help with tasks > such as writing documentation, or running/writing tests. Any > pointers would be much appreciated. > > Thanks, > > Glenn > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From gtarcea at umich.edu Fri May 9 02:45:09 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Thu, 8 May 2008 22:45:09 -0400 Subject: [elephant-devel] How to get involved? In-Reply-To: <11ACB1FC-8B1B-43A3-9C32-4051AC5BE3E2@media.mit.edu> References: <11ACB1FC-8B1B-43A3-9C32-4051AC5BE3E2@media.mit.edu> Message-ID: <3EB34F54-754A-47B0-96FD-F0F7FB2ED924@umich.edu> Thanks Ian. I'll take a look at the tests and start there. I expect to devote a good part of Sunday on this (my wife is out of town, so now is a good time for me to start getting involved). I'll send out an update and questions. Thanks for the opportunity, I'm looking forward to getting to know and contribute to Elephant. Thanks, Glenn On May 8, 2008, at 10:26 PM, Ian Eslick wrote: > Hi Glenn, > > It's great to have another pair of helping hands, welcome! > > I think the most important next steps, which will also help you get > further up to speed on Elephant, is to write tests for the new > features in the elephant-unstable branch. This includes the new > slot types (set-valued, cached, associations), hierarchical > indexing and schema evolution. My updates to the list may describe > other changes that I'm not remembering off-hand. > > testassociations.lisp in the tests directory has a start on the > associations and I think testindexing has some updates to handle > the new indexing features. > > Please feel free to ask questions and/or submit documentation as > you go. > > Thank you very much, > Ian > > On May 8, 2008, at 8:51 PM, Glenn Tarcea wrote: > >> >> Hi all, >> >> I'd like to help out in some way with Elephant, but I'm not quite >> sure how to participate. I'm using Elephant in a project while >> still climbing the lisp learning curve. That said, I've been doing >> software development long enough to be legal-to-drink only >> counting the years manning a keyboard :-) >> >> I figure I have 4 to 8 hours a week I can help out, maybe a bit >> more when my project (which I'm doing on my own time outside of my >> day job) and the needs for Elephant coincide. Since it looks like >> there is a push to get a 1.0 release ready I'm happy to help with >> tasks such as writing documentation, or running/writing tests. Any >> pointers would be much appreciated. >> >> Thanks, >> >> Glenn >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > From lists at infoway.net Fri May 9 22:02:15 2008 From: lists at infoway.net (lists at infoway.net) Date: Fri, 9 May 2008 18:02:15 -0400 Subject: [elephant-devel] Query System Message-ID: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net> Hello everyone, I apologize for being disconnected for so long. I had volunteered to help in the query system and should have done more progress by now. Unfortunately, the same as some (most or all) of you, putting food on the table for my family has a higher priority and my current job has demanded 110% of my time lately. Enough excuses! I have been passively reading several of your email threads. I am convinced that a query system will bring a lot of value to Elephant. The question that still arises is whether or not people want a SQL-like syntax or a Lisp-like syntax. As Ian has suggested, publicly and/or privately, we should start designing the query system in a very basic form. The most critical part would be query optimization, which I'd rather work on after we have the basic query system in place. But there are a lot of decisions to make before we get there and coming to a consensus of how it should look and how it should work is of critical importance. From a simplistic point of view, a SQL-like syntax should allow for the execution of the basic relational algebraic operations (union, difference, cartesian product, projection, and selection). For the most part, these would not be difficult to implement. However, IMHO, there is an intrinsic "contradiction" in applying a SQL-like syntax on top of Elephant. Assume you have the following Tables (relations) in a SQL world: Books ( book_id, title, author ) Publishers ( publisher_id, name ) BooksPublishers ( book_id, publisher_id, year ) Suppose you wanted to get the cartesian product of all the books published in 2008, you could run a SQL query like: SELECT Books.*, Publishers.* FROM Books, Publishers, BooksPublishers WHERE Books.book_id = BooksPublishers.book_id AND Publishers.publisher_id = BooksPublishers.publisher_id AND BooksPublishers.year = 2008 The result will be a concatenation of all the columns from the Books and Publishers tables. In a SQL-world, you would access these results in a key-value pair type mode (e.g. Books.book_id = 1, Books.title = "1984", etc). However, when you think in terms of Elephant (at least my understanding of it), you're dealing with objects and not key-value pairs from multiple tables. So, instead of getting a concatenation of all the columns, you "should" be getting just a list of Book objects (or Publisher objects) that met your query criteria, such that when you iterate thru them, you could "query" their Publishers (or the Books). So, if we had something like (please keep in mind this is no suggestion to syntax or correctness but just for illustrative purposes): (defpclass book () ((title :accessor book-title :index t) (author :accessor book-author :index t) (published_copies :accessor book-copies :initform (make-pset)))) (defpclass publisher () ((name :accessor publisher-name :index t))) (defmethod add-published-copy ((bk book) (pb publisher) year) (insert-item '(pb year) (book-copies bk))) (defmethod map-published-copies (fn (bk book)) (map-pset fn (book-copies bk))) (setq objs (select book :where ((map-published-copies (lambda (item year) (= (second item) year)) $bk 2008))))) From then on, you could just iterate through the book objects in the result set for their respective published copies. The problem with this is that, ok, you get all the books that met your criteria but if you then wanted to get a list of all the published copies, you would need to apply the filter criteria again. The reason I think it "should behave" this way is because Elephant deals with sets of objects, and you use Lisp to navigate through the object space, whereas in a SQL- world you are not dealing with objects but with a result set that contains all the columns you asked for. If we were to emulate the same behavior in the query system, that would sort of defeat the purpose of Elephant. For that matter, you might as well use some of the other libraries (e.g. CL-SQL, cl-perec, cl-rdbms, etc). The above example is a very simple example. We haven't looked at SORTING, LIMIT, OFFSET, etc. Things which will simply make this whole dilemma more difficult. I haven't looked into Ian's association mechanism yet. Maybe the query system could/should be an extension to that with some specialized features to apply filter criteria instead (and possibly evolve into something similar to Ruby's ActiveRecord). I know the association mechanism is still being developed and I haven't really seen anyone comment much on it other than what Ian has mentioned. In one of Ian's comments, he said: "A more general query language is probably the right solution for this interface. The query language would know about associations, derived indices, etc and perform query planning via introspection over the class objects." At the same time, Robert said on another thread: "One might philosophically prefer SQL. I personally vaster prefer to work in a powerful programming language to accomplish these things. Obviously, whether two classes that refer to each other stand in a "parent-child" relationship or not depends entirely on the circumstances. I prefer to write simple functions such as "delete- order" below, which both utilize and (in a sense) expand the power of LISP applied to persistent objects." Leslie said on yet another thread: "While I'm at it: OFFSET and LIMIT (a real limit which lets you specify an arbitrary Lisp expression) are things we definitely want to aim for in 1.0. They are not difficult to implement at all, but they don't work with GET-INSTANCES-BY-* and, worse, MAP-BTREE. This means everyone has to write their own version of these functions that take appropriate arguments and move the cursor around themselves instead of relying on a simple high-level API. I'd have implemented these extensions myself, but I thought it better to wait for the integration of the query language to add it." And Alex said: "I think main problem is not how it looks, but that query language actually makes programming a lot easier." All those comments make sense. There seems to be a group agreement that something is needed, but everyone has their own ideas of how it should work. Both the query language and the associations are still being developed, so if we get consensus no how these should work, it may give a better direction to both feature sets. If anyone has any comments or suggestion as to whether a query system be of real interest/necessity and if so, which would be the preferred query syntax and expected behavior, that would really help. I'm willing to work on this in as much as possible with my limited knowledge of Lisp and Elephant. However, given a clear direction of where this should go, I will be able to focus better and learn faster what I haven't learned so far. Again, your feedback is much appreciated. I'm hopeful to be able to work more on this over the weekend, assuming I get some feedback from you guys. Thanks Daniel From eslick at media.mit.edu Sat May 10 01:54:50 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Fri, 9 May 2008 21:54:50 -0400 Subject: [elephant-devel] Query System In-Reply-To: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net> References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net> Message-ID: <90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu> Welcome back Daniel, we all know the work drill! Here are a few thoughts to throw into the mix... One advantage of the relational model is that you have implicit data structures (tables) that can be assembled from existing tables via the SQL query. This is nice because it means we don't have to explicitly create and maintain the structure for all these derived data structures. In a pure lisp model, you actually have to do all this maintenance yourself, especially the optimizations necessary for efficiency that add to complexity. I feel that Elephant should probably fall somewhere in-between. You maintain the data structures that you want to work with in your program logic, but the system can maintain pointers and indices and other relationships that make it easy and efficient to generate and work with subsets of objects efficiently (a user's inbox, for example). Some of the limitations/frustrations with the current system may be caused by people trying to do familiar relational tasks in the OODB framework. I also think that Robert's lisp-as-query-language works well for the prevalence model when all objects are in memory, but I think it's less practical in, say, BDB where you are going to disk alot. However, it's a good discipline to consider - when does it makes sense to add new syntax/apis and when does it make sense to use lisp directly. You mentioned associations. The best way to think about associations is that it is an easy way to maintain back pointers. For example, if a message object has a slot that contains a reference to a user, we may also want the user object to have an accessor that provides quick and efficient access to the collection of messages that point to it. That's what associations are for. You could do this by declaring after methods on (setf (user message) value) that add the message to a pset sitting in a user instance slot, but that gets tedious. As Leslie says, we're trying to make common cases simple and reasonably efficient. So the approach I'd like to see taken to designing the query framework is to capture the use cases and metaphors that people are really interested in and are encountering in real-world use and pick the largest subset that fits nicely into a clean, theoretical conceptual model. There are already a good number (Leslie, Alex, etc) on the list that we could start with. For example, I often find myself wanting to filter a set of objects by more than one parameter (messages from user U that are high priority between 4/1/08 and 5/1/08). What is the complexity of different approaches afforded by the existing Elephant implementation? In order of computational efficiency (I surmise): 1. scan all messages and collect/operate on only those matching all criteria 2. scan an index on messages instead of all messages; pick the one likely to yield the smallest subset 3. intersection: scan two or more indexes for subsets represented as sequences of oids, instantiate, filter and operate on the objects represented by the intersection. 4. create an index that orders objects by all three parameters and just walk the matching set. Trade off space for time. Any others? The other consideration is the conceptual framework we want to use to approach the problem. Procedural? Constraint satisfaction? Logical form? Graph matching? There are some good examples of existing OODB systems in lisp out there (PLOB, AllegroStore/AllegroCache, Statice, etc). If you search the list archives, I think I've forwarded references in the past. I tend to lean towards a constraint satisfaction approach, as my sketch demonstrations. "Operate on the set of objects that satisfy these constraints." There are a bunch of practical issues. Do we map query sets? Do we cache them? Do we represent them as lists? Are they lazily evaluated? If we don't have a DSL, but allow arbitrary lisp expressions, then there isn't enough information to automatically select indexes, perform intersections, etc. My other strong suggestion, besides starting by capturing the major use cases, is that we begin by implementing a procedural approach by implementing the building blocks for filter, sort, intersect, etc. If we take the list of four filtering approaches above, we can start writing code that do these things and use them to implement some of the use cases. The common building blocks and problems that we discover will inform the additions we'll want to the MOP, new implicit data structures like associations, the most convenient query syntax, etc. Plus it will be useful in the meantime. This fits into the classic lisp bottom-up DSL development model (well proselytized by Paul. Graham). Ian On May 9, 2008, at 6:02 PM, lists at infoway.net wrote: > Hello everyone, > > I apologize for being disconnected for so long. I had volunteered to > help in the query system and should have done more progress by now. > Unfortunately, the same as some (most or all) of you, putting food > on the table for my family has a higher priority and my current job > has demanded 110% of my time lately. > > Enough excuses! I have been passively reading several of your email > threads. I am convinced that a query system will bring a lot of > value to Elephant. The question that still arises is whether or not > people want a SQL-like syntax or a Lisp-like syntax. > > As Ian has suggested, publicly and/or privately, we should start > designing the query system in a very basic form. The most critical > part would be query optimization, which I'd rather work on after we > have the basic query system in place. But there are a lot of > decisions to make before we get there and coming to a consensus of > how it should look and how it should work is of critical importance. > > From a simplistic point of view, a SQL-like syntax should allow for > the execution of the basic relational algebraic operations (union, > difference, cartesian product, projection, and selection). For the > most part, these would not be difficult to implement. However, IMHO, > there is an intrinsic "contradiction" in applying a SQL-like syntax > on top of Elephant. > > Assume you have the following Tables (relations) in a SQL world: > > Books ( > book_id, > title, > author > ) > > Publishers ( > publisher_id, > name > ) > > BooksPublishers ( > book_id, > publisher_id, > year > ) > > Suppose you wanted to get the cartesian product of all the books > published in 2008, you could run a SQL query like: > > SELECT Books.*, Publishers.* FROM Books, Publishers, BooksPublishers > WHERE Books.book_id = BooksPublishers.book_id AND > Publishers.publisher_id = BooksPublishers.publisher_id AND > BooksPublishers.year = 2008 > > The result will be a concatenation of all the columns from the Books > and Publishers tables. In a SQL-world, you would access these > results in a key-value pair type mode (e.g. Books.book_id = 1, > Books.title = "1984", etc). However, when you think in terms of > Elephant (at least my understanding of it), you're dealing with > objects and not key-value pairs from multiple tables. So, instead of > getting a concatenation of all the columns, you "should" be getting > just a list of Book objects (or Publisher objects) that met your > query criteria, such that when you iterate thru them, you could > "query" their Publishers (or the Books). So, if we had something > like (please keep in mind this is no suggestion to syntax or > correctness but just for illustrative purposes): > > (defpclass book () > ((title :accessor book-title :index t) > (author :accessor book-author :index t) > (published_copies :accessor book-copies :initform (make-pset)))) > > (defpclass publisher () > ((name :accessor publisher-name :index t))) > > (defmethod add-published-copy ((bk book) (pb publisher) year) > (insert-item '(pb year) (book-copies bk))) > > (defmethod map-published-copies (fn (bk book)) > (map-pset fn (book-copies bk))) > > (setq objs (select book :where ((map-published-copies (lambda (item > year) (= (second item) year)) $bk 2008))))) > > From then on, you could just iterate through the book objects in the > result set for their respective published copies. The problem with > this is that, ok, you get all the books that met your criteria but > if you then wanted to get a list of all the published copies, you > would need to apply the filter criteria again. The reason I think it > "should behave" this way is because Elephant deals with sets of > objects, and you use Lisp to navigate through the object space, > whereas in a SQL-world you are not dealing with objects but with a > result set that contains all the columns you asked for. If we were > to emulate the same behavior in the query system, that would sort of > defeat the purpose of Elephant. For that matter, you might as well > use some of the other libraries (e.g. CL-SQL, cl-perec, cl-rdbms, > etc). > > The above example is a very simple example. We haven't looked at > SORTING, LIMIT, OFFSET, etc. Things which will simply make this > whole dilemma more difficult. > > I haven't looked into Ian's association mechanism yet. Maybe the > query system could/should be an extension to that with some > specialized features to apply filter criteria instead (and possibly > evolve into something similar to Ruby's ActiveRecord). I know the > association mechanism is still being developed and I haven't really > seen anyone comment much on it other than what Ian has mentioned. In > one of Ian's comments, he said: > > "A more general query language is probably the right solution > for this interface. The query language would know about > associations, derived indices, etc and perform query planning via > introspection over the class objects." > > At the same time, Robert said on another thread: > > "One might philosophically prefer SQL. I personally vaster > prefer to work in a powerful programming language to accomplish > these things. Obviously, whether two classes that refer to each > other stand in a "parent-child" relationship or not depends entirely > on the circumstances. I prefer to write simple functions such as > "delete-order" below, which both utilize and (in a sense) expand the > power of LISP applied to persistent objects." > > Leslie said on yet another thread: > > "While I'm at it: OFFSET and LIMIT (a real limit which lets you > specify an arbitrary Lisp expression) are things we definitely want > to aim for in 1.0. They are not difficult to implement at all, but > they don't work with GET-INSTANCES-BY-* and, worse, MAP-BTREE. This > means everyone has to write their own version of these functions > that take appropriate arguments and move the cursor around > themselves instead of relying on a simple high-level API. > > I'd have implemented these extensions myself, but I thought it > better to wait for the integration of the query language to add it." > > And Alex said: > > "I think main problem is not how it looks, but that query > language actually makes programming a lot easier." > > All those comments make sense. There seems to be a group agreement > that something is needed, but everyone has their own ideas of how it > should work. Both the query language and the associations are still > being developed, so if we get consensus no how these should work, it > may give a better direction to both feature sets. If anyone has any > comments or suggestion as to whether a query system be of real > interest/necessity and if so, which would be the preferred query > syntax and expected behavior, that would really help. > > I'm willing to work on this in as much as possible with my limited > knowledge of Lisp and Elephant. However, given a clear direction of > where this should go, I will be able to focus better and learn > faster what I haven't learned so far. > > Again, your feedback is much appreciated. I'm hopeful to be able to > work more on this over the weekend, assuming I get some feedback > from you guys. > > Thanks > Daniel > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From lists at infoway.net Sat May 10 04:26:21 2008 From: lists at infoway.net (lists at infoway.net) Date: Sat, 10 May 2008 00:26:21 -0400 Subject: [elephant-devel] Query System In-Reply-To: <90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu> References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net> <90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu> Message-ID: <9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net> Hi Ian, Thanks for your comments. They do make some points a bit clearer and bring others to the table. I'd like to see others comment as well before we continue moving forward. In summary, I agree to follow your suggestion. However, one thing that still remains unclear to me is the type of result expected from the query system. I was under the assumption that we want to continue handling objects of the persistent classes in Elephant. Thus, the results from the query should be a set (list, etc) of objects matching the criteria. However, reading your comments makes me think that what you're suggesting is that the result set be something closer to what a SQL query would return. So, going back to my example, instead of receiving a set of Books, you would get back something like: ((book_oid book_title book_author publisher_oid publisher_name publisher_year) (value11 value12 value 13, value14, value15 value16) (value21 value22 value23 value24 value25 value26) ... ) In other words, maybe something like a list that contains a list of the slot names returned and then the list of matching values, where the list of slots is either the concatenation of all the slots in the persistent classes (in the case of something like SELECT Books.*, Publishers.* ...) or the specific slots requested (e.g. SELECT Books.author, Publishers.name ...). And therefore, the result set would be treated as simply a set of values instead of a set of objects. Within the result set, you could include the OIDs so that you could eventually instantiate the particular object and work in via the object model, or you could simply just use the values returned, which is what you originally asked for in the SELECT statement. If my assumption is correct from what you're saying, that certainly clarifies a lot or my concerns and doubts. I'm sure more details will arise along the way, but that could be a starting point to sketch the system. Comments? Thanks, Daniel On May 9, 2008, at 9:54 PM, Ian Eslick wrote: > Welcome back Daniel, we all know the work drill! > > Here are a few thoughts to throw into the mix... > > One advantage of the relational model is that you have implicit data > structures (tables) that can be assembled from existing tables via > the SQL query. This is nice because it means we don't have to > explicitly create and maintain the structure for all these derived > data structures. In a pure lisp model, you actually have to do all > this maintenance yourself, especially the optimizations necessary > for efficiency that add to complexity. I feel that Elephant should > probably fall somewhere in-between. You maintain the data > structures that you want to work with in your program logic, but the > system can maintain pointers and indices and other relationships > that make it easy and efficient to generate and work with subsets of > objects efficiently (a user's inbox, for example). > > Some of the limitations/frustrations with the current system may be > caused by people trying to do familiar relational tasks in the OODB > framework. > > I also think that Robert's lisp-as-query-language works well for the > prevalence model when all objects are in memory, but I think it's > less practical in, say, BDB where you are going to disk alot. > However, it's a good discipline to consider - when does it makes > sense to add new syntax/apis and when does it make sense to use lisp > directly. > > You mentioned associations. The best way to think about > associations is that it is an easy way to maintain back pointers. > For example, if a message object has a slot that contains a > reference to a user, we may also want the user object to have an > accessor that provides quick and efficient access to the collection > of messages that point to it. That's what associations are for. > You could do this by declaring after methods on (setf (user message) > value) that add the message to a pset sitting in a user instance > slot, but that gets tedious. As Leslie says, we're trying to make > common cases simple and reasonably efficient. > > So the approach I'd like to see taken to designing the query > framework is to capture the use cases and metaphors that people are > really interested in and are encountering in real-world use and pick > the largest subset that fits nicely into a clean, theoretical > conceptual model. There are already a good number (Leslie, Alex, > etc) on the list that we could start with. > > For example, I often find myself wanting to filter a set of objects > by more than one parameter (messages from user U that are high > priority between 4/1/08 and 5/1/08). What is the complexity of > different approaches afforded by the existing Elephant implementation? > > In order of computational efficiency (I surmise): > 1. scan all messages and collect/operate on only those matching all > criteria > 2. scan an index on messages instead of all messages; pick the one > likely to yield the smallest subset > 3. intersection: scan two or more indexes for subsets represented as > sequences of oids, instantiate, filter and operate on the objects > represented by the intersection. > 4. create an index that orders objects by all three parameters and > just walk the matching set. Trade off space for time. > > Any others? > > The other consideration is the conceptual framework we want to use > to approach the problem. Procedural? Constraint satisfaction? > Logical form? Graph matching? There are some good examples of > existing OODB systems in lisp out there (PLOB, AllegroStore/ > AllegroCache, Statice, etc). If you search the list archives, I > think I've forwarded references in the past. > > I tend to lean towards a constraint satisfaction approach, as my > sketch demonstrations. "Operate on the set of objects that satisfy > these constraints." There are a bunch of practical issues. Do we > map query sets? Do we cache them? Do we represent them as lists? > Are they lazily evaluated? If we don't have a DSL, but allow > arbitrary lisp expressions, then there isn't enough information to > automatically select indexes, perform intersections, etc. > > > My other strong suggestion, besides starting by capturing the major > use cases, is that we begin by implementing a procedural approach by > implementing the building blocks for filter, sort, intersect, etc. > If we take the list of four filtering approaches above, we can start > writing code that do these things and use them to implement some of > the use cases. The common building blocks and problems that we > discover will inform the additions we'll want to the MOP, new > implicit data structures like associations, the most convenient > query syntax, etc. Plus it will be useful in the meantime. This > fits into the classic lisp bottom-up DSL development model (well > proselytized by Paul. Graham). > > Ian > > > > On May 9, 2008, at 6:02 PM, lists at infoway.net wrote: > >> Hello everyone, >> >> I apologize for being disconnected for so long. I had volunteered >> to help in the query system and should have done more progress by >> now. Unfortunately, the same as some (most or all) of you, putting >> food on the table for my family has a higher priority and my >> current job has demanded 110% of my time lately. >> >> Enough excuses! I have been passively reading several of your email >> threads. I am convinced that a query system will bring a lot of >> value to Elephant. The question that still arises is whether or not >> people want a SQL-like syntax or a Lisp-like syntax. >> >> As Ian has suggested, publicly and/or privately, we should start >> designing the query system in a very basic form. The most critical >> part would be query optimization, which I'd rather work on after we >> have the basic query system in place. But there are a lot of >> decisions to make before we get there and coming to a consensus of >> how it should look and how it should work is of critical importance. >> >> From a simplistic point of view, a SQL-like syntax should allow for >> the execution of the basic relational algebraic operations (union, >> difference, cartesian product, projection, and selection). For the >> most part, these would not be difficult to implement. However, >> IMHO, there is an intrinsic "contradiction" in applying a SQL-like >> syntax on top of Elephant. >> >> Assume you have the following Tables (relations) in a SQL world: >> >> Books ( >> book_id, >> title, >> author >> ) >> >> Publishers ( >> publisher_id, >> name >> ) >> >> BooksPublishers ( >> book_id, >> publisher_id, >> year >> ) >> >> Suppose you wanted to get the cartesian product of all the books >> published in 2008, you could run a SQL query like: >> >> SELECT Books.*, Publishers.* FROM Books, Publishers, >> BooksPublishers WHERE Books.book_id = BooksPublishers.book_id AND >> Publishers.publisher_id = BooksPublishers.publisher_id AND >> BooksPublishers.year = 2008 >> >> The result will be a concatenation of all the columns from the >> Books and Publishers tables. In a SQL-world, you would access these >> results in a key-value pair type mode (e.g. Books.book_id = 1, >> Books.title = "1984", etc). However, when you think in terms of >> Elephant (at least my understanding of it), you're dealing with >> objects and not key-value pairs from multiple tables. So, instead >> of getting a concatenation of all the columns, you "should" be >> getting just a list of Book objects (or Publisher objects) that met >> your query criteria, such that when you iterate thru them, you >> could "query" their Publishers (or the Books). So, if we had >> something like (please keep in mind this is no suggestion to syntax >> or correctness but just for illustrative purposes): >> >> (defpclass book () >> ((title :accessor book-title :index t) >> (author :accessor book-author :index t) >> (published_copies :accessor book-copies :initform (make-pset)))) >> >> (defpclass publisher () >> ((name :accessor publisher-name :index t))) >> >> (defmethod add-published-copy ((bk book) (pb publisher) year) >> (insert-item '(pb year) (book-copies bk))) >> >> (defmethod map-published-copies (fn (bk book)) >> (map-pset fn (book-copies bk))) >> >> (setq objs (select book :where ((map-published-copies (lambda (item >> year) (= (second item) year)) $bk 2008))))) >> >> From then on, you could just iterate through the book objects in >> the result set for their respective published copies. The problem >> with this is that, ok, you get all the books that met your criteria >> but if you then wanted to get a list of all the published copies, >> you would need to apply the filter criteria again. The reason I >> think it "should behave" this way is because Elephant deals with >> sets of objects, and you use Lisp to navigate through the object >> space, whereas in a SQL-world you are not dealing with objects but >> with a result set that contains all the columns you asked for. If >> we were to emulate the same behavior in the query system, that >> would sort of defeat the purpose of Elephant. For that matter, you >> might as well use some of the other libraries (e.g. CL-SQL, cl- >> perec, cl-rdbms, etc). >> >> The above example is a very simple example. We haven't looked at >> SORTING, LIMIT, OFFSET, etc. Things which will simply make this >> whole dilemma more difficult. >> >> I haven't looked into Ian's association mechanism yet. Maybe the >> query system could/should be an extension to that with some >> specialized features to apply filter criteria instead (and possibly >> evolve into something similar to Ruby's ActiveRecord). I know the >> association mechanism is still being developed and I haven't really >> seen anyone comment much on it other than what Ian has mentioned. >> In one of Ian's comments, he said: >> >> "A more general query language is probably the right solution >> for this interface. The query language would know about >> associations, derived indices, etc and perform query planning via >> introspection over the class objects." >> >> At the same time, Robert said on another thread: >> >> "One might philosophically prefer SQL. I personally vaster >> prefer to work in a powerful programming language to accomplish >> these things. Obviously, whether two classes that refer to each >> other stand in a "parent-child" relationship or not depends >> entirely on the circumstances. I prefer to write simple functions >> such as "delete-order" below, which both utilize and (in a sense) >> expand the power of LISP applied to persistent objects." >> >> Leslie said on yet another thread: >> >> "While I'm at it: OFFSET and LIMIT (a real limit which lets you >> specify an arbitrary Lisp expression) are things we definitely want >> to aim for in 1.0. They are not difficult to implement at all, but >> they don't work with GET-INSTANCES-BY-* and, worse, MAP-BTREE. This >> means everyone has to write their own version of these functions >> that take appropriate arguments and move the cursor around >> themselves instead of relying on a simple high-level API. >> >> I'd have implemented these extensions myself, but I thought it >> better to wait for the integration of the query language to add it." >> >> And Alex said: >> >> "I think main problem is not how it looks, but that query >> language actually makes programming a lot easier." >> >> All those comments make sense. There seems to be a group agreement >> that something is needed, but everyone has their own ideas of how >> it should work. Both the query language and the associations are >> still being developed, so if we get consensus no how these should >> work, it may give a better direction to both feature sets. If >> anyone has any comments or suggestion as to whether a query system >> be of real interest/necessity and if so, which would be the >> preferred query syntax and expected behavior, that would really help. >> >> I'm willing to work on this in as much as possible with my limited >> knowledge of Lisp and Elephant. However, given a clear direction of >> where this should go, I will be able to focus better and learn >> faster what I haven't learned so far. >> >> Again, your feedback is much appreciated. I'm hopeful to be able to >> work more on this over the weekend, assuming I get some feedback >> from you guys. >> >> Thanks >> Daniel >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From ryszard.szopa at gmail.com Sat May 10 13:35:59 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Sat, 10 May 2008 15:35:59 +0200 Subject: [elephant-devel] strange bug related to psets Message-ID: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> Hello, I have run into a very strange bug related to psets. I create some persistent objects with psets as slot-values, and at some point one of those psets that should be empty contains 5 integers, usually in descending order with a step equal 5. For example (values pset (pset-list pset)) => # (20 15 10 5 0) (the exact numbers may vary: most often it starts from 20). The bug appeared for the first time in a unit test. Before running that test, I open a new BDB store in an empty directory, so it is rather unlikely that my code at some point before puts the extra values into the pset. What is more, the bug does not appear the first time I run the test: only in the second and subsequent runs. In order to get rid (at lest temporarily) of this bug, it is enough to run (make-pset) somewhere in the code before creating my objects. It looks as if there was some pset floating in memory, which accidentally gets reused while making my new pset. Does anybody have any idea how to hunt this bug down? I am using elephant as in the darcs repository and the BerkeleyDB (4.6) backend, on SBCL 1.0.15, x86 Linux version. Oh, and I can actually reproduce this bug, but I wasn't able to factor it down not to contain major parts of the application I am writing. Thanks in advance, -- Richard PS. I'd like to apologize: I have asked a question ("question about persistent classes") on this group before, and got some interesting answers, but then failed to thank for them. I was planning to write an elaborate answer etc., but then there was ECLM, I had a lot of work to do... You probably know how it goes. So, let me fix my mistake now: thank you, Robert, Ian and Alex. -- http://szopa.tasak.gda.pl/ From eslick at media.mit.edu Sat May 10 14:12:48 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 10 May 2008 10:12:48 -0400 Subject: [elephant-devel] Query System In-Reply-To: <9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net> References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net> <90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu> <9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net> Message-ID: <0199E4E0-735C-49AF-8D76-2B16AFEC3F1F@media.mit.edu> On May 10, 2008, at 12:26 AM, lists at infoway.net wrote: > Hi Ian, > > Thanks for your comments. They do make some points a bit clearer and > bring others to the table. I'd like to see others comment as well > before we continue moving forward. > > In summary, I agree to follow your suggestion. However, one thing > that still remains unclear to me is the type of result expected from > the query system. See my comments below, but I think we want to avoid consing large sets where possible. So once we've defined a set of objects using a query expressions we may want to: 1) Perform a non-consing map operation over it 2) Return it in a list 3) intersect it with another query set 4) returning a list of slot values or mapping over the slot value #2 is easy to derive from #1 - that's what get-instances-by-value does today. #3, to be efficient, may require lazy query evaluation. #4 is what you discuss below. I think the general idea is that a query defines a set of objects, but how you use it defines the time/space cost of evaluating that set. I actually like the Django notion of lazy query sets - you can perform operations like intersecting two query sets, but the actual query isn't executed until you need to operate on a member of the set. This gives the query compiler the maximum information to optimize. > I was under the assumption that we want to continue handling objects > of the persistent classes in Elephant. Thus, the results from the > query should be a set (list, etc) of objects matching the criteria. > However, reading your comments makes me think that what you're > suggesting is that the result set be something closer to what a SQL > query would return. So, going back to my example, instead of > receiving a set of Books, you would get back something like: Wow, that was not the impression I intended to make! I definitely prefer an object-centric view by default. We may want to return pairs of objects, however. Returning values is easy to implement on top of a map, as the example below illustrates. > ((book_oid book_title book_author publisher_oid publisher_name > publisher_year) (value11 value12 value 13, value14, value15 value16) > (value21 value22 value23 value24 value25 value26) ... ) > > In other words, maybe something like a list that contains a list of > the slot names returned and then the list of matching values, where > the list of slots is either the concatenation of all the slots in > the persistent classes (in the case of something like SELECT > Books.*, Publishers.* ...) or the specific slots requested (e.g. > SELECT Books.author, Publishers.name ...). Of course it's easy to do the above with a map operation or wrapper with a closure that binds map-fn, the user's function, and slots, a list of slots to operate on. (lambda (obj) (apply #'map-fn (mapcar (lambda (slot) (funcall accessor obj)) slots))) So the user would see something like: (map-query (:slots name city state country) (lambda (name city state country) ...)) It would mostly be a convenience function built on top of the basic query mechanism, and not essential to start with... > And therefore, the result set would be treated as simply a set of > values instead of a set of objects. Within the result set, you could > include the OIDs so that you could eventually instantiate the > particular object and work in via the object model, or you could > simply just use the values returned, which is what you originally > asked for in the SELECT statement. I do think the semantics of a query should be that query results are a set of objects, rather than only persistent objects. It means the user can work with a result list, or map over the results; working with whatever my query string specifies. That said, we should start by just extracting sets of objects for users to map over that satisfy the constraints. We definitely should be able to specify that the system only return oids. > If my assumption is correct from what you're saying, that certainly > clarifies a lot or my concerns and doubts. I'm sure more details > will arise along the way, but that could be a starting point to > sketch the system. > > Comments? > > Thanks, > Daniel > > On May 9, 2008, at 9:54 PM, Ian Eslick wrote: > >> Welcome back Daniel, we all know the work drill! >> >> Here are a few thoughts to throw into the mix... >> >> One advantage of the relational model is that you have implicit >> data structures (tables) that can be assembled from existing tables >> via the SQL query. This is nice because it means we don't have to >> explicitly create and maintain the structure for all these derived >> data structures. In a pure lisp model, you actually have to do all >> this maintenance yourself, especially the optimizations necessary >> for efficiency that add to complexity. I feel that Elephant should >> probably fall somewhere in-between. You maintain the data >> structures that you want to work with in your program logic, but >> the system can maintain pointers and indices and other >> relationships that make it easy and efficient to generate and work >> with subsets of objects efficiently (a user's inbox, for example). >> >> Some of the limitations/frustrations with the current system may be >> caused by people trying to do familiar relational tasks in the OODB >> framework. >> >> I also think that Robert's lisp-as-query-language works well for >> the prevalence model when all objects are in memory, but I think >> it's less practical in, say, BDB where you are going to disk alot. >> However, it's a good discipline to consider - when does it makes >> sense to add new syntax/apis and when does it make sense to use >> lisp directly. >> >> You mentioned associations. The best way to think about >> associations is that it is an easy way to maintain back pointers. >> For example, if a message object has a slot that contains a >> reference to a user, we may also want the user object to have an >> accessor that provides quick and efficient access to the collection >> of messages that point to it. That's what associations are for. >> You could do this by declaring after methods on (setf (user >> message) value) that add the message to a pset sitting in a user >> instance slot, but that gets tedious. As Leslie says, we're trying >> to make common cases simple and reasonably efficient. >> >> So the approach I'd like to see taken to designing the query >> framework is to capture the use cases and metaphors that people are >> really interested in and are encountering in real-world use and >> pick the largest subset that fits nicely into a clean, theoretical >> conceptual model. There are already a good number (Leslie, Alex, >> etc) on the list that we could start with. >> >> For example, I often find myself wanting to filter a set of objects >> by more than one parameter (messages from user U that are high >> priority between 4/1/08 and 5/1/08). What is the complexity of >> different approaches afforded by the existing Elephant >> implementation? >> >> In order of computational efficiency (I surmise): >> 1. scan all messages and collect/operate on only those matching all >> criteria >> 2. scan an index on messages instead of all messages; pick the one >> likely to yield the smallest subset >> 3. intersection: scan two or more indexes for subsets represented >> as sequences of oids, instantiate, filter and operate on the >> objects represented by the intersection. >> 4. create an index that orders objects by all three parameters and >> just walk the matching set. Trade off space for time. >> >> Any others? >> >> The other consideration is the conceptual framework we want to use >> to approach the problem. Procedural? Constraint satisfaction? >> Logical form? Graph matching? There are some good examples of >> existing OODB systems in lisp out there (PLOB, AllegroStore/ >> AllegroCache, Statice, etc). If you search the list archives, I >> think I've forwarded references in the past. >> >> I tend to lean towards a constraint satisfaction approach, as my >> sketch demonstrations. "Operate on the set of objects that satisfy >> these constraints." There are a bunch of practical issues. Do we >> map query sets? Do we cache them? Do we represent them as lists? >> Are they lazily evaluated? If we don't have a DSL, but allow >> arbitrary lisp expressions, then there isn't enough information to >> automatically select indexes, perform intersections, etc. >> >> >> My other strong suggestion, besides starting by capturing the major >> use cases, is that we begin by implementing a procedural approach >> by implementing the building blocks for filter, sort, intersect, >> etc. If we take the list of four filtering approaches above, we >> can start writing code that do these things and use them to >> implement some of the use cases. The common building blocks and >> problems that we discover will inform the additions we'll want to >> the MOP, new implicit data structures like associations, the most >> convenient query syntax, etc. Plus it will be useful in the >> meantime. This fits into the classic lisp bottom-up DSL >> development model (well proselytized by Paul. Graham). >> >> Ian >> >> >> >> On May 9, 2008, at 6:02 PM, lists at infoway.net wrote: >> >>> Hello everyone, >>> >>> I apologize for being disconnected for so long. I had volunteered >>> to help in the query system and should have done more progress by >>> now. Unfortunately, the same as some (most or all) of you, putting >>> food on the table for my family has a higher priority and my >>> current job has demanded 110% of my time lately. >>> >>> Enough excuses! I have been passively reading several of your >>> email threads. I am convinced that a query system will bring a lot >>> of value to Elephant. The question that still arises is whether or >>> not people want a SQL-like syntax or a Lisp-like syntax. >>> >>> As Ian has suggested, publicly and/or privately, we should start >>> designing the query system in a very basic form. The most critical >>> part would be query optimization, which I'd rather work on after >>> we have the basic query system in place. But there are a lot of >>> decisions to make before we get there and coming to a consensus of >>> how it should look and how it should work is of critical importance. >>> >>> From a simplistic point of view, a SQL-like syntax should allow >>> for the execution of the basic relational algebraic operations >>> (union, difference, cartesian product, projection, and selection). >>> For the most part, these would not be difficult to implement. >>> However, IMHO, there is an intrinsic "contradiction" in applying a >>> SQL-like syntax on top of Elephant. >>> >>> Assume you have the following Tables (relations) in a SQL world: >>> >>> Books ( >>> book_id, >>> title, >>> author >>> ) >>> >>> Publishers ( >>> publisher_id, >>> name >>> ) >>> >>> BooksPublishers ( >>> book_id, >>> publisher_id, >>> year >>> ) >>> >>> Suppose you wanted to get the cartesian product of all the books >>> published in 2008, you could run a SQL query like: >>> >>> SELECT Books.*, Publishers.* FROM Books, Publishers, >>> BooksPublishers WHERE Books.book_id = BooksPublishers.book_id AND >>> Publishers.publisher_id = BooksPublishers.publisher_id AND >>> BooksPublishers.year = 2008 >>> >>> The result will be a concatenation of all the columns from the >>> Books and Publishers tables. In a SQL-world, you would access >>> these results in a key-value pair type mode (e.g. Books.book_id = >>> 1, Books.title = "1984", etc). However, when you think in terms of >>> Elephant (at least my understanding of it), you're dealing with >>> objects and not key-value pairs from multiple tables. So, instead >>> of getting a concatenation of all the columns, you "should" be >>> getting just a list of Book objects (or Publisher objects) that >>> met your query criteria, such that when you iterate thru them, you >>> could "query" their Publishers (or the Books). So, if we had >>> something like (please keep in mind this is no suggestion to >>> syntax or correctness but just for illustrative purposes): >>> >>> (defpclass book () >>> ((title :accessor book-title :index t) >>> (author :accessor book-author :index t) >>> (published_copies :accessor book-copies :initform (make-pset)))) >>> >>> (defpclass publisher () >>> ((name :accessor publisher-name :index t))) >>> >>> (defmethod add-published-copy ((bk book) (pb publisher) year) >>> (insert-item '(pb year) (book-copies bk))) >>> >>> (defmethod map-published-copies (fn (bk book)) >>> (map-pset fn (book-copies bk))) >>> >>> (setq objs (select book :where ((map-published-copies (lambda >>> (item year) (= (second item) year)) $bk 2008))))) >>> >>> From then on, you could just iterate through the book objects in >>> the result set for their respective published copies. The problem >>> with this is that, ok, you get all the books that met your >>> criteria but if you then wanted to get a list of all the published >>> copies, you would need to apply the filter criteria again. The >>> reason I think it "should behave" this way is because Elephant >>> deals with sets of objects, and you use Lisp to navigate through >>> the object space, whereas in a SQL-world you are not dealing with >>> objects but with a result set that contains all the columns you >>> asked for. If we were to emulate the same behavior in the query >>> system, that would sort of defeat the purpose of Elephant. For >>> that matter, you might as well use some of the other libraries >>> (e.g. CL-SQL, cl-perec, cl-rdbms, etc). >>> >>> The above example is a very simple example. We haven't looked at >>> SORTING, LIMIT, OFFSET, etc. Things which will simply make this >>> whole dilemma more difficult. >>> >>> I haven't looked into Ian's association mechanism yet. Maybe the >>> query system could/should be an extension to that with some >>> specialized features to apply filter criteria instead (and >>> possibly evolve into something similar to Ruby's ActiveRecord). I >>> know the association mechanism is still being developed and I >>> haven't really seen anyone comment much on it other than what Ian >>> has mentioned. In one of Ian's comments, he said: >>> >>> "A more general query language is probably the right solution >>> for this interface. The query language would know about >>> associations, derived indices, etc and perform query planning via >>> introspection over the class objects." >>> >>> At the same time, Robert said on another thread: >>> >>> "One might philosophically prefer SQL. I personally vaster >>> prefer to work in a powerful programming language to accomplish >>> these things. Obviously, whether two classes that refer to each >>> other stand in a "parent-child" relationship or not depends >>> entirely on the circumstances. I prefer to write simple functions >>> such as "delete-order" below, which both utilize and (in a sense) >>> expand the power of LISP applied to persistent objects." >>> >>> Leslie said on yet another thread: >>> >>> "While I'm at it: OFFSET and LIMIT (a real limit which lets you >>> specify an arbitrary Lisp expression) are things we definitely >>> want to aim for in 1.0. They are not difficult to implement at >>> all, but they don't work with GET-INSTANCES-BY-* and, worse, MAP- >>> BTREE. This means everyone has to write their own version of these >>> functions that take appropriate arguments and move the cursor >>> around themselves instead of relying on a simple high-level API. >>> >>> I'd have implemented these extensions myself, but I thought it >>> better to wait for the integration of the query language to add it." >>> >>> And Alex said: >>> >>> "I think main problem is not how it looks, but that query >>> language actually makes programming a lot easier." >>> >>> All those comments make sense. There seems to be a group agreement >>> that something is needed, but everyone has their own ideas of how >>> it should work. Both the query language and the associations are >>> still being developed, so if we get consensus no how these should >>> work, it may give a better direction to both feature sets. If >>> anyone has any comments or suggestion as to whether a query system >>> be of real interest/necessity and if so, which would be the >>> preferred query syntax and expected behavior, that would really >>> help. >>> >>> I'm willing to work on this in as much as possible with my limited >>> knowledge of Lisp and Elephant. However, given a clear direction >>> of where this should go, I will be able to focus better and learn >>> faster what I haven't learned so far. >>> >>> Again, your feedback is much appreciated. I'm hopeful to be able >>> to work more on this over the weekend, assuming I get some >>> feedback from you guys. >>> >>> Thanks >>> Daniel >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Sat May 10 14:25:34 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 10 May 2008 10:25:34 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> Message-ID: This happens in your application, or only in the tests? Can you give me the exact sequence and state you use to reproduce? Do you have a fresh lisp & fresh DB? Does it still show up if you close the store, open a fresh one, and try again? There is likely to be some state somewhere and identifying the environmental dependencies should help us localize where that is. Thank you, Ian On May 10, 2008, at 9:35 AM, Ryszard Szopa wrote: > Hello, > > I have run into a very strange bug related to psets. I create some > persistent objects with psets as slot-values, and at some point one of > those psets that should be empty contains 5 integers, usually in > descending order with a step equal 5. For example > > (values pset (pset-list pset)) => # (20 15 10 5 > 0) > > (the exact numbers may vary: most often it starts from 20). > > The bug appeared for the first time in a unit test. Before running > that test, I open a new BDB store in an empty directory, so it is > rather unlikely that my code at some point before puts the extra > values into the pset. What is more, the bug does not appear the first > time I run the test: only in the second and subsequent runs. In order > to get rid (at lest temporarily) of this bug, it is enough to run > (make-pset) somewhere in the code before creating my objects. > > It looks as if there was some pset floating in memory, which > accidentally gets reused while making my new pset. Does anybody have > any idea how to hunt this bug down? > > I am using elephant as in the darcs repository and the BerkeleyDB > (4.6) backend, on SBCL 1.0.15, x86 Linux version. > > Oh, and I can actually reproduce this bug, but I wasn't able to factor > it down not to contain major parts of the application I am writing. > > Thanks in advance, > > -- Richard > > PS. I'd like to apologize: I have asked a question ("question about > persistent classes") on this group before, and got some interesting > answers, but then failed to thank for them. I was planning to write an > elaborate answer etc., but then there was ECLM, I had a lot of work to > do... You probably know how it goes. So, let me fix my mistake now: > thank you, Robert, Ian and Alex. > > -- > http://szopa.tasak.gda.pl/ > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From ryszard.szopa at gmail.com Sat May 10 17:13:07 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Sat, 10 May 2008 19:13:07 +0200 Subject: [elephant-devel] strange bug related to psets In-Reply-To: References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> Message-ID: <19e19e410805101013t46083048r29a33a5932fa7307@mail.gmail.com> On Sat, May 10, 2008 at 4:25 PM, Ian Eslick wrote: > This happens in your application, or only in the tests? Until now I have spotted it only in my unit tests. > Can you give me the > exact sequence and state you use to reproduce? Do you have a fresh lisp & > fresh DB? Does it still show up if you close the store, open a fresh one, > and try again? There is likely to be some state somewhere and identifying > the environmental dependencies should help us localize where that is. My unit test works like this. 1. Remove all the files in the directory "test/store" 2. Open a bdb store located in "test/store" using `with-store' 3. Create a bunch of persistent objects (they have psets as slot-values). 4. Run some tests (this is the test for a query facility, so I query the db and see if I get the right objects back). 5. Close the store. The first time I run the test, everything is OK. The problem arises at point 4 during the second and subsequent runs (this may mean that it doesn't matter whether the store is fresh, but only whether the lisp image is fresh). Note that at the time the persistent classes are defined, there's a different store open (located in the directory "store"), and in point 2 I use with-store to open the test store. As I said, if between point 2 and 3 I call a few times (make-pset), the error doesn't occur. I put quite a lot of effort into isolating the exact code that causes the error, unsuccessfully. Cheers, -- Richard -- http://szopa.tasak.gda.pl/ From eslick at media.mit.edu Sat May 10 17:19:40 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 10 May 2008 13:19:40 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: <19e19e410805101013t46083048r29a33a5932fa7307@mail.gmail.com> References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> <19e19e410805101013t46083048r29a33a5932fa7307@mail.gmail.com> Message-ID: Ok, I'll look into this when I can. May be a few days or so. Pester me if you don't hear back from me next week sometime. Thanks, Ian On May 10, 2008, at 1:13 PM, Ryszard Szopa wrote: > On Sat, May 10, 2008 at 4:25 PM, Ian Eslick > wrote: >> This happens in your application, or only in the tests? > > Until now I have spotted it only in my unit tests. > >> Can you give me the >> exact sequence and state you use to reproduce? Do you have a fresh >> lisp & >> fresh DB? Does it still show up if you close the store, open a >> fresh one, >> and try again? There is likely to be some state somewhere and >> identifying >> the environmental dependencies should help us localize where that is. > > My unit test works like this. > > 1. Remove all the files in the directory "test/store" > 2. Open a bdb store located in "test/store" using `with-store' > 3. Create a bunch of persistent objects (they have psets as slot- > values). > 4. Run some tests (this is the test for a query facility, so I query > the db and see if I get the right objects back). > 5. Close the store. > > The first time I run the test, everything is OK. The problem arises at > point 4 during the second and subsequent runs (this may mean that it > doesn't matter whether the store is fresh, but only whether the lisp > image is fresh). > > Note that at the time the persistent classes are defined, there's a > different store open (located in the directory "store"), and in point > 2 I use with-store to open the test store. > > As I said, if between point 2 and 3 I call a few times (make-pset), > the error doesn't occur. > > I put quite a lot of effort into isolating the exact code that causes > the error, unsuccessfully. > > Cheers, > > -- Richard > > -- > http://szopa.tasak.gda.pl/ > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From clinton at unknownlamer.org Sat May 10 17:49:14 2008 From: clinton at unknownlamer.org (Clinton Ebadi) Date: Sat, 10 May 2008 13:49:14 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: (Ian Eslick's message of "Sat\, 10 May 2008 10\:25\:34 -0400") References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> Message-ID: <87mymy82xx.fsf@unknownlamer.org> Ian Eslick writes: > This happens in your application, or only in the tests? Can you give > me the exact sequence and state you use to reproduce? Do you have a > fresh lisp & fresh DB? Does it still show up if you close the store, > open a fresh one, and try again? There is likely to be some state > somewhere and identifying the environmental dependencies should help > us localize where that is. This happened to me too after closing and opening the database during testing in a program of mine. Empty psets created after some point would be filled with what looked like the oids of recently created objects, and they were the same on every run. My database deletion procedure was to close the store, delete the bdb files, and then connect again creating a fresh database. Alas, I removed the psets and used derived indices now so I can't post the code that triggered this. I do have the old version using psets in my darcs revision history so I might be able to set up a repo and revert things back there though. -- How can you accept social supression This weak state of mind in our time I demand release from hypocrisy I'd rather die than be held down, forced down From eslick at media.mit.edu Sat May 10 17:55:18 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 10 May 2008 13:55:18 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: <87mymy82xx.fsf@unknownlamer.org> References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> <87mymy82xx.fsf@unknownlamer.org> Message-ID: That's fascinating (and disturbing). What data-stores are you using? Ian On May 10, 2008, at 1:49 PM, Clinton Ebadi wrote: > Ian Eslick writes: > >> This happens in your application, or only in the tests? Can you give >> me the exact sequence and state you use to reproduce? Do you have a >> fresh lisp & fresh DB? Does it still show up if you close the store, >> open a fresh one, and try again? There is likely to be some state >> somewhere and identifying the environmental dependencies should help >> us localize where that is. > > This happened to me too after closing and opening the database during > testing in a program of mine. Empty psets created after some point > would be filled with what looked like the oids of recently created > objects, and they were the same on every run. > > My database deletion procedure was to close the store, delete the bdb > files, and then connect again creating a fresh database. > > Alas, I removed the psets and used derived indices now so I can't post > the code that triggered this. I do have the old version using psets in > my darcs revision history so I might be able to set up a repo and > revert things back there though. > > -- > How can you accept social supression > This weak state of mind in our time > I demand release from hypocrisy > I'd rather die than be held down, forced down > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Sat May 10 17:57:05 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 10 May 2008 13:57:05 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: <87mymy82xx.fsf@unknownlamer.org> References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> <87mymy82xx.fsf@unknownlamer.org> Message-ID: Also which version of the elephant tree? This may be fixed on unstable - there was quite a bit of general cleanup as I made the schema changes. Today, psets are just a wrapper around btrees - anyone tried to reproduce this using straight btrees instead of psets? Ian On May 10, 2008, at 1:49 PM, Clinton Ebadi wrote: > Ian Eslick writes: > >> This happens in your application, or only in the tests? Can you give >> me the exact sequence and state you use to reproduce? Do you have a >> fresh lisp & fresh DB? Does it still show up if you close the store, >> open a fresh one, and try again? There is likely to be some state >> somewhere and identifying the environmental dependencies should help >> us localize where that is. > > This happened to me too after closing and opening the database during > testing in a program of mine. Empty psets created after some point > would be filled with what looked like the oids of recently created > objects, and they were the same on every run. > > My database deletion procedure was to close the store, delete the bdb > files, and then connect again creating a fresh database. > > Alas, I removed the psets and used derived indices now so I can't post > the code that triggered this. I do have the old version using psets in > my darcs revision history so I might be able to set up a repo and > revert things back there though. > > -- > How can you accept social supression > This weak state of mind in our time > I demand release from hypocrisy > I'd rather die than be held down, forced down > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From clinton at unknownlamer.org Sat May 10 18:46:23 2008 From: clinton at unknownlamer.org (Clinton Ebadi) Date: Sat, 10 May 2008 14:46:23 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: (Ian Eslick's message of "Sat\, 10 May 2008 13\:57\:05 -0400") References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> <87mymy82xx.fsf@unknownlamer.org> Message-ID: <87hcd680ao.fsf@unknownlamer.org> Ian Eslick writes: > Also which version of the elephant tree? This may be fixed on > unstable - there was quite a bit of general cleanup as I made the > schema changes. Today, psets are just a wrapper around btrees - > anyone tried to reproduce this using straight btrees instead of psets? This is with stable. I tried to use unstable for the program at first, but ran into weird issues with objects of different types ending up in each others class indices. After switching to derived indices and some manual index generation the problems went away. I tried tracking down the problem for an hourish but gave up. I'm just using BDB. -- ntk is currently using "telnet fyodor 25" to send email From ryszard.szopa at gmail.com Sat May 10 20:35:53 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Sat, 10 May 2008 22:35:53 +0200 Subject: [elephant-devel] strange bug related to psets In-Reply-To: <87hcd680ao.fsf@unknownlamer.org> References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> <87mymy82xx.fsf@unknownlamer.org> <87hcd680ao.fsf@unknownlamer.org> Message-ID: <19e19e410805101335k3ccf4763qb027902bc27b2100@mail.gmail.com> On Sat, May 10, 2008 at 8:46 PM, Clinton Ebadi wrote: > Ian Eslick writes: >> Also which version of the elephant tree? This may be fixed on >> unstable - there was quite a bit of general cleanup as I made the >> schema changes. Today, psets are just a wrapper around btrees - >> anyone tried to reproduce this using straight btrees instead of psets? > > This is with stable. I tried to use unstable for the program at first, > but ran into weird issues with objects of different types ending up in > each others class indices. I ran into the whole thing using unstable, which means that the problem unfortunately hasn't disappeared. I also had the impression that these are OIDs that have somehow leaked in a place they didn't belong to. Anyway, it makes me feel better knowing that this problem isn't local for my application. Cheers, -- Richard -- http://szopa.tasak.gda.pl/ From leslie.polzer at gmx.net Sat May 10 21:19:48 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sat, 10 May 2008 23:19:48 +0200 (CEST) Subject: [elephant-devel] Query System In-Reply-To: <90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu> References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net> <90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu> Message-ID: <62876.88.73.195.19.1210454388.squirrel@mail.stardawn.org> > My other strong suggestion, besides starting by capturing the major > use cases, is that we begin by implementing a procedural approach by > implementing the building blocks for filter, sort, intersect, etc. If > we take the list of four filtering approaches above, we can start > writing code that do these things and use them to implement some of > the use cases. [...] This fits into the classic lisp bottom-up DSL > development model (well proselytized by Paul. Graham). I unanimously agree here. It's hard to come up with good use cases and strategies, so we should build it bit by bit and watch it grow. Coincidentally, this also fits in with my initial ramblings (cited by Daniel) about LIMIT and the likes. My first idea here was to add this functionality as keyword parameters to GET-INSTANCES-*. Leslie From eslick at media.mit.edu Sat May 10 21:56:46 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 10 May 2008 17:56:46 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: <87hcd680ao.fsf@unknownlamer.org> References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> <87mymy82xx.fsf@unknownlamer.org> <87hcd680ao.fsf@unknownlamer.org> Message-ID: <123F82EA-30E3-4EFB-AF8F-A76DE846EAC9@media.mit.edu> How long ago were you using unstable? Some of the class indexing problems should have been fixed by the time I did my last major update. If it was more recent, than at least for that problem I know where to look... Ian On May 10, 2008, at 2:46 PM, Clinton Ebadi wrote: > Ian Eslick writes: > >> Also which version of the elephant tree? This may be fixed on >> unstable - there was quite a bit of general cleanup as I made the >> schema changes. Today, psets are just a wrapper around btrees - >> anyone tried to reproduce this using straight btrees instead of >> psets? > > This is with stable. I tried to use unstable for the program at first, > but ran into weird issues with objects of different types ending up in > each others class indices. > > After switching to derived indices and some manual index generation > the problems went away. I tried tracking down the problem for an > hourish but gave up. > > I'm just using BDB. > > -- > ntk is currently using "telnet fyodor 25" to send > email > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Sat May 10 21:57:13 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 10 May 2008 17:57:13 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: <19e19e410805101335k3ccf4763qb027902bc27b2100@mail.gmail.com> References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> <87mymy82xx.fsf@unknownlamer.org> <87hcd680ao.fsf@unknownlamer.org> <19e19e410805101335k3ccf4763qb027902bc27b2100@mail.gmail.com> Message-ID: <77E5833B-4268-449A-87E3-907426F0CA1B@media.mit.edu> It's interesting to know that this problem appears to show up on both branches. That narrows down the possibilities a bit. Ian On May 10, 2008, at 4:35 PM, Ryszard Szopa wrote: > On Sat, May 10, 2008 at 8:46 PM, Clinton Ebadi > wrote: >> Ian Eslick writes: >>> Also which version of the elephant tree? This may be fixed on >>> unstable - there was quite a bit of general cleanup as I made the >>> schema changes. Today, psets are just a wrapper around btrees - >>> anyone tried to reproduce this using straight btrees instead of >>> psets? >> >> This is with stable. I tried to use unstable for the program at >> first, >> but ran into weird issues with objects of different types ending up >> in >> each others class indices. > > I ran into the whole thing using unstable, which means that the > problem unfortunately hasn't disappeared. I also had the impression > that these are OIDs that have somehow leaked in a place they didn't > belong to. > > Anyway, it makes me feel better knowing that this problem isn't local > for my application. > > Cheers, > > -- Richard > -- > http://szopa.tasak.gda.pl/ > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Sat May 10 22:41:02 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 10 May 2008 18:41:02 -0400 Subject: [elephant-devel] strange bug related to psets In-Reply-To: <87hcd680ao.fsf@unknownlamer.org> References: <19e19e410805100635h6fc2e9a0wdd27d47bf0aa05d1@mail.gmail.com> <87mymy82xx.fsf@unknownlamer.org> <87hcd680ao.fsf@unknownlamer.org> Message-ID: <4255FC27-4B3A-407E-B15F-D69A8038EB2E@media.mit.edu> On May 10, 2008, at 2:46 PM, Clinton Ebadi wrote: > Ian Eslick writes: > >> Also which version of the elephant tree? This may be fixed on >> unstable - there was quite a bit of general cleanup as I made the >> schema changes. Today, psets are just a wrapper around btrees - >> anyone tried to reproduce this using straight btrees instead of >> psets? > > This is with stable. I tried to use unstable for the program at first, > but ran into weird issues with objects of different types ending up in > each others class indices. Were you using inheritance? In unstable, a subclass can now be a member of it's parent's index. This is a different behavior than in the old branch. Actually I realized this isn't a good idea. I've added an :inherit keyword to association, index and derived-index slots that enables hierarchical indexing for all subclasses of that base class. You can override the default behavior by shadowing the slot in a subclass. This defaults to nil, so maintains the original behavior but allows this new feature. Default is that each named class has its own index. ELEPHANT-USER> (defpclass base () ((inherit :accessor inherit :index t :inherit t :initarg :inherit) (none :accessor none :index t :inherit nil :initarg :none) (num :accessor num :initarg :num))) # ELEPHANT-USER> (defpclass sub (base) ()) # Create some simple test cases ELEPHANT-USER> (make-instance 'base :inherit 1 :none 1 :num 1) # ELEPHANT-USER> (make-instance 'base :inherit 2 :none 2 :num 2) # ELEPHANT-USER> (make-instance 'sub :inherit 2 :none 2 :num 2) # ELEPHANT-USER> (make-instance 'sub :inherit 3 :none 3 :num 3) # Our base class index picks up the subclass instances as well. ELEPHANT-USER> (get-instances-by-value 'base 'inherit 1) (#) ELEPHANT-USER> (get-instances-by-value 'base 'inherit 2) (# #) ELEPHANT-USER> (get-instances-by-value 'base 'inherit 3) (#) Inherited indexes look the same from all subclasses, we could add an option to filter superclasses if you use a subclass classname to access an inherited index. ELEPHANT-USER> (get-instances-by-value 'sub 'inherit 2) (# #) ELEPHANT-USER> (get-instances-by-value 'sub 'inherit 3) (#) Non-inherited slots create a separate index for each class: ELEPHANT-USER> (get-instances-by-value 'base 'none 1) (#) ELEPHANT-USER> (get-instances-by-value 'base 'none 2) (#) ELEPHANT-USER> (get-instances-by-value 'base 'none 3) NIL ELEPHANT-USER> (get-instances-by-value 'sub 'none 1) NIL ELEPHANT-USER> (get-instances-by-value 'sub 'none 2) (#) ELEPHANT-USER> (get-instances-by-value 'sub 'none 3) (#) > After switching to derived indices and some manual index generation > the problems went away. I tried tracking down the problem for an > hourish but gave up. > > I'm just using BDB. > > -- > ntk is currently using "telnet fyodor 25" to send > email > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From bryan.emrys at gmail.com Sun May 11 21:22:06 2008 From: bryan.emrys at gmail.com (Bryan Emrys) Date: Sun, 11 May 2008 14:22:06 -0700 Subject: [elephant-devel] Clbuild In-Reply-To: <3EB34F54-754A-47B0-96FD-F0F7FB2ED924@umich.edu> References: <11ACB1FC-8B1B-43A3-9C32-4051AC5BE3E2@media.mit.edu> <3EB34F54-754A-47B0-96FD-F0F7FB2ED924@umich.edu> Message-ID: <200805111422.06254.bryan.emrys@gmail.com> At one point the clbuild maintainers looked at elephant but had some problems that prevented inclusion. Specifically: # - needs sb-posix, but doesn't declare that dependency, meaning that # it doesn't build even with a config file # - ele-clsql loads clsql from the .asd file, not using :depends-on, # which is completely unacceptable # (we blacklist ele-clsql in clbuild.lisp because of that) Did anyone ever respond? From gtarcea at umich.edu Sun May 11 23:50:19 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Sun, 11 May 2008 19:50:19 -0400 Subject: [elephant-devel] Some thoughts/Questions from testing Message-ID: <0B53E816-1703-4D6B-9E63-D030B6EA9E4B@umich.edu> What do you all think about allowing two different versions of Elephant to be installed at the same time? I tried to have both my original version of Elephant installed and also to install elephant- unstable. Currently there are a lot of dependencies on the system name sprinkled throughout the code that prevent this. My thought would be to allow someone to do a (require 'elephant-unstable) (or some other chosen system name) and have it load an alternative version of Elephant. All the package names would stay the same, just the systems would be different. The benefit to this is it would allow users to test out the new functionality before deciding to switch, and for testers, it would mean we could have both versions installed. It seems like this would be helpful right now when the BerkeleyDB interfaces have stabilized, but the postmodern interface hasn't. For me it would allow me to test and observe behavior in the old environment and then easily switch over to the new environment. If no one objects I can look into this further, at least drawing up a plan of attack and posting it for comment. Also, it looks like the testing framework is being migrated over to 5am. I assume this means that I should write any new tests using 5am? Thanks, Glenn From read at robertlread.net Mon May 12 00:01:14 2008 From: read at robertlread.net (Robert L. Read) Date: Sun, 11 May 2008 19:01:14 -0500 Subject: [elephant-devel] Some thoughts/Questions from testing In-Reply-To: <0B53E816-1703-4D6B-9E63-D030B6EA9E4B@umich.edu> References: <0B53E816-1703-4D6B-9E63-D030B6EA9E4B@umich.edu> Message-ID: <1210550474.18630.13.camel@penguin.yourdomain.com> On Sun, 2008-05-11 at 19:50 -0400, Glenn Tarcea wrote: > What do you all think about allowing two different versions of > Elephant to be installed at the same time? I tried to have both my > original version of Elephant installed and also to install elephant- > unstable. Currently there are a lot of dependencies on the system > name sprinkled throughout the code that prevent this. My thought > would be to allow someone to do a (require 'elephant-unstable) (or > some other chosen system name) and have it load an alternative > version of Elephant. All the package names would stay the same, just > the systems would be different. I think this could be very nice for the reasons you mention. I personally don't know how to do it. I suspect any workable solution would have to be an asdf-based solution, not just using 'requre. > > The benefit to this is it would allow users to test out the new > functionality before deciding to switch, and for testers, it would > mean we could have both versions installed. It seems like this would > be helpful right now when the BerkeleyDB interfaces have stabilized, > but the postmodern interface hasn't. For me it would allow me to test > and observe behavior in the old environment and then easily switch > over to the new environment. > > If no one objects I can look into this further, at least drawing up a > plan of attack and posting it for comment. I think it is definitely worth trying to figure out how to do it. However, an added complication is that any objects created are created under a certain version, which may not be readable by another version. Ian has objectified this with his "schema numbering" for classes in elephant-unstable. The idea situation is that Elephant version N+K can read any Elephant object written by version N. I think we are moving toward that capability. However, even if one version can't read another version, being able to have two distinct packages in one image could be useful---I encourage you to spend some time thinking through how it might work. > > Also, it looks like the testing framework is being migrated over to > 5am. I assume this means that I should write any new tests using 5am? Yes --- I believe we are already completely dependent on 5am for testing. > > Thanks, > > Glenn > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Mon May 12 00:12:16 2008 From: read at robertlread.net (Robert L. Read) Date: Sun, 11 May 2008 19:12:16 -0500 Subject: [elephant-devel] Clbuild In-Reply-To: <200805111422.06254.bryan.emrys@gmail.com> References: <11ACB1FC-8B1B-43A3-9C32-4051AC5BE3E2@media.mit.edu> <3EB34F54-754A-47B0-96FD-F0F7FB2ED924@umich.edu> <200805111422.06254.bryan.emrys@gmail.com> Message-ID: <1210551136.18630.24.camel@penguin.yourdomain.com> On Sun, 2008-05-11 at 14:22 -0700, Bryan Emrys wrote: > At one point the clbuild maintainers looked at elephant but had some problems that prevented inclusion. > > Specifically: > > # - needs sb-posix, but doesn't declare that dependency, meaning that > # it doesn't build even with a config file > # - ele-clsql loads clsql from the .asd file, not using :depends-on, > # which is completely unacceptable > # (we blacklist ele-clsql in clbuild.lisp because of that) > > Did anyone ever respond? Dear Bryan, Thanks for calling this to our attention. I doubt anyone ever responded to our being blacklisted, as I have no recollection of it. I have removed the explicit load, and my tests are red but I doubt that was the problem. I am not a master of ASDF --- can someone (Bryan?) tell me how we can integrate the fact that we depend on sb-posix into the ASDF (:depeonds-on ) expressions without dependence on SBCL? Can we just use a compiler pragma like: :depends-on (:uffi #+sbcl :sb-posix :cl-base64)) in our asdf files? If so we will have to do something similar for each supported system (see os.lisp): (defmacro in-directory ((dir) &body body) `(progn (#+sbcl sb-posix:chdir #+cmu unix:unix-chdir #+allegro excl:chdir #+lispworks hcl:change-directory #+openmcl ccl:cwd ,dir) , at body)) Finally, our documentation build system, make-ref.lisp, seems to be more deeply SBCL dependent than anything in Elephant; but that doesn't hurt much, as we normally build and publish the documentation with each release anyway. I wonder if we could just make the documentation build a separate package, in order to satisfy the clbuild requirements? > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From bryan.emrys at gmail.com Mon May 12 01:43:55 2008 From: bryan.emrys at gmail.com (Bryan Emrys) Date: Sun, 11 May 2008 18:43:55 -0700 Subject: [elephant-devel] Clbuild In-Reply-To: <1210551136.18630.24.camel@penguin.yourdomain.com> References: <200805111422.06254.bryan.emrys@gmail.com> <1210551136.18630.24.camel@penguin.yourdomain.com> Message-ID: <200805111843.55131.bryan.emrys@gmail.com> I'm not a master of anything, but pragmas seem to be the approach used in hunchentoot (see below). (asdf:defsystem :hunchentoot :serial t :version #.*hunchentoot-version* :depends-on (:chunga :cl-base64 :cl-fad :cl-ppcre #-(or :lispworks :hunchentoot-no-ssl) :cl+ssl :md5 :rfc2388 #+:sbcl :sb-bsd-sockets #+:sbcl :sb-posix :url-rewrite) :components ((:file "packages") (:file "conditions") #+:allegro (:file "port-acl") #+:clisp (:file "port-clisp") #+:cmu (:file "port-cmu") #+:lispworks (:file "port-lw") #+:openmcl (:file "port-mcl") #+:sbcl (:file "port-sbcl") (:file "specials") (:file "mime-types") (:file "util") (:file "log") (:file "cookie") (:file "reply") (:file "request") (:file "session") (:file "misc") (:file "easy-handlers") (:file "headers") #+(and :allegro :unix) (:file "unix-acl") #+(and :clisp :unix) (:file "unix-clisp") #+(and :cmu :unix) (:file "unix-cmu") #+(and :lispworks :unix) (:file "unix-lw") #+(and :openmcl :unix) (:file "unix-mcl") #+(and :sbcl :unix (not :win32)) (:file "unix-sbcl") (:file "server"))) Bryan On Sunday 11 May 2008 05:12:16 pm Robert L. Read wrote: > On Sun, 2008-05-11 at 14:22 -0700, Bryan Emrys wrote: > > At one point the clbuild maintainers looked at elephant but had some problems that prevented inclusion. > > > > Specifically: > > > > # - needs sb-posix, but doesn't declare that dependency, meaning that > > # it doesn't build even with a config file > > # - ele-clsql loads clsql from the .asd file, not using :depends-on, > > # which is completely unacceptable > > # (we blacklist ele-clsql in clbuild.lisp because of that) > > > > Did anyone ever respond? > > Dear Bryan, > Thanks for calling this to our attention. I doubt anyone ever > responded to our being blacklisted, as I have no recollection of it. > > I have removed the explicit load, and my tests are red but I doubt that > was the problem. > > I am not a master of ASDF --- can someone (Bryan?) tell me how we can > integrate the fact that we depend on sb-posix into the ASDF > (:depeonds-on ) expressions without dependence on SBCL? Can we just use > a compiler pragma like: > > :depends-on (:uffi > #+sbcl :sb-posix > :cl-base64)) > > > > in our asdf files? If so we will have to do something similar for > each supported system (see os.lisp): > > > (defmacro in-directory ((dir) &body body) > `(progn > (#+sbcl sb-posix:chdir > #+cmu unix:unix-chdir > #+allegro excl:chdir > #+lispworks hcl:change-directory > #+openmcl ccl:cwd > ,dir) > , at body)) > > Finally, our documentation build system, make-ref.lisp, seems to be more > deeply SBCL dependent than anything in Elephant; but that doesn't hurt > much, as we normally build and publish the documentation with each > release anyway. I wonder if we could just make the documentation build > a separate package, in order to satisfy the clbuild requirements? > > > > _______________________________________________ > > elephant-devel site list > > elephant-devel at common-lisp.net > > http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > From gtarcea at umich.edu Mon May 12 02:10:29 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Sun, 11 May 2008 22:10:29 -0400 Subject: [elephant-devel] Some thoughts/Questions from testing In-Reply-To: <1210550474.18630.13.camel@penguin.yourdomain.com> References: <0B53E816-1703-4D6B-9E63-D030B6EA9E4B@umich.edu> <1210550474.18630.13.camel@penguin.yourdomain.com> Message-ID: <7BC3CCDF-7ADD-4D19-8C26-A6424FE092AC@umich.edu> Thanks Robert. I'll look into it. I was thinking of something much simpler than you outlined. Rather than trying to support reading objects across versions, I was just looking to support two different versions of the elephant packages, and to tease out the load-op dependencies that refer to explicit names. Something like (thinking out loud here): (defconstant +elephant-system-name+ 'elephant) (defconstant +elephant-system-bdb+ 'ele-bdb) (defsystem #.+elephant-system-name+ .... And then in the code, rather than referring to :ele-bdb (for example) refer instead to +elephant-system-bdb+. Then, to support the same package names, but different systems, we just need to change the constants. So, elephant unstable would set the constants to, eg 'elephant-unstable, 'ele-bdb-unstable, 'ele-xxx- unstable etc... If we did something like this it would allow for both elephant and elephant-unstable on the system, and depending on which system the user loaded (require, or asdf:operation 'asdf:load-op) their code would either be referring to the released version, or the new "unstable" version. All package names would remain the same, so you couldn't easily switch without leaving and reentering lisp. As a first cut, though, this would be a low impact change with some immediate benefit. Since I'm new to the code I'm hesitant to propose any change that isn't simple and straightforward. At least not until I get a better feel for the code. I'll put some thought in to it and put out a proposal. Thanks, Glenn On May 11, 2008, at 8:01 PM, Robert L. Read wrote: > On Sun, 2008-05-11 at 19:50 -0400, Glenn Tarcea wrote: >> What do you all think about allowing two different versions of >> Elephant to be installed at the same time? I tried to have both my >> original version of Elephant installed and also to install elephant- >> unstable. Currently there are a lot of dependencies on the system >> name sprinkled throughout the code that prevent this. My thought >> would be to allow someone to do a (require 'elephant-unstable) (or >> some other chosen system name) and have it load an alternative >> version of Elephant. All the package names would stay the same, just >> the systems would be different. > > I think this could be very nice for the reasons you mention. I > personally don't know how to do it. I suspect any workable solution > would have to be an asdf-based solution, not just using 'requre. > >> >> The benefit to this is it would allow users to test out the new >> functionality before deciding to switch, and for testers, it would >> mean we could have both versions installed. It seems like this would >> be helpful right now when the BerkeleyDB interfaces have stabilized, >> but the postmodern interface hasn't. For me it would allow me to test >> and observe behavior in the old environment and then easily switch >> over to the new environment. >> >> If no one objects I can look into this further, at least drawing up a >> plan of attack and posting it for comment. > > I think it is definitely worth trying to figure out how to do it. > However, an added complication is that any objects created are created > under a certain version, which may not be readable by another version. > Ian has objectified this with his "schema numbering" for classes in > elephant-unstable. > > The idea situation is that Elephant version N+K can read any Elephant > object written by version N. I think we are moving toward that > capability. > > However, even if one version can't read another version, being able to > have two distinct packages in one image could be useful---I encourage > you to spend some time thinking through how it might work. > >> >> Also, it looks like the testing framework is being migrated over to >> 5am. I assume this means that I should write any new tests using 5am? > > Yes --- I believe we are already completely dependent on 5am for > testing. > >> >> Thanks, >> >> Glenn >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > From eslick at media.mit.edu Mon May 12 03:13:36 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sun, 11 May 2008 23:13:36 -0400 Subject: [elephant-devel] Some thoughts/Questions from testing In-Reply-To: <7BC3CCDF-7ADD-4D19-8C26-A6424FE092AC@umich.edu> References: <0B53E816-1703-4D6B-9E63-D030B6EA9E4B@umich.edu> <1210550474.18630.13.camel@penguin.yourdomain.com> <7BC3CCDF-7ADD-4D19-8C26-A6424FE092AC@umich.edu> Message-ID: Hmmmm... I think having to name all the systems differently in each branch is a poor methodology and likely to lead to confusion over time. Instead, I wrote a simple shell script to change .asd symbolic links in my system directory. We might bundle a shell script and batch file that one can run to manage multiple .asd files or source trees so that the right thing happens as a convenience for our users. It might be possible to integrate this into a special .asd file that manipulates links appropriately - but allowing users to adapt shell scripts is probably, ultimately, simpler. Ian PS - It might also be useful to have a clear message during the loading process about what version is actually being loaded into the image. At least that way you aren't confused about which version you are currently running! Perhaps something that prints a message when you open a store; or a big banner that makes it hard to miss what version is loading? ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;; Loading ELEPHANT 0.9.2 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; On May 11, 2008, at 10:10 PM, Glenn Tarcea wrote: > Thanks Robert. > > I'll look into it. I was thinking of something much simpler than you > outlined. Rather than trying to support reading objects across > versions, I was just looking to support two different versions of > the elephant packages, and to tease out the load-op dependencies > that refer to explicit names. > > Something like (thinking out loud here): > > (defconstant +elephant-system-name+ 'elephant) > (defconstant +elephant-system-bdb+ 'ele-bdb) > > (defsystem #.+elephant-system-name+ > .... > > And then in the code, rather than referring to :ele-bdb (for > example) refer instead to +elephant-system-bdb+. > > Then, to support the same package names, but different systems, we > just need to change the constants. So, elephant unstable would set > the constants to, eg 'elephant-unstable, 'ele-bdb-unstable, 'ele-xxx- > unstable etc... > > If we did something like this it would allow for both elephant and > elephant-unstable on the system, and depending on which system the > user loaded (require, or asdf:operation 'asdf:load-op) their code > would either be referring to the released version, or the new > "unstable" version. All package names would remain the same, so you > couldn't easily switch without leaving and reentering lisp. As a > first cut, though, this would be a low impact change with some > immediate benefit. > > Since I'm new to the code I'm hesitant to propose any change that > isn't simple and straightforward. At least not until I get a better > feel for the code. > > I'll put some thought in to it and put out a proposal. > > Thanks, > > Glenn > > On May 11, 2008, at 8:01 PM, Robert L. Read wrote: > >> On Sun, 2008-05-11 at 19:50 -0400, Glenn Tarcea wrote: >>> What do you all think about allowing two different versions of >>> Elephant to be installed at the same time? I tried to have both my >>> original version of Elephant installed and also to install elephant- >>> unstable. Currently there are a lot of dependencies on the system >>> name sprinkled throughout the code that prevent this. My thought >>> would be to allow someone to do a (require 'elephant-unstable) (or >>> some other chosen system name) and have it load an alternative >>> version of Elephant. All the package names would stay the same, just >>> the systems would be different. >> >> I think this could be very nice for the reasons you mention. I >> personally don't know how to do it. I suspect any workable solution >> would have to be an asdf-based solution, not just using 'requre. >> >>> >>> The benefit to this is it would allow users to test out the new >>> functionality before deciding to switch, and for testers, it would >>> mean we could have both versions installed. It seems like this would >>> be helpful right now when the BerkeleyDB interfaces have stabilized, >>> but the postmodern interface hasn't. For me it would allow me to >>> test >>> and observe behavior in the old environment and then easily switch >>> over to the new environment. >>> >>> If no one objects I can look into this further, at least drawing >>> up a >>> plan of attack and posting it for comment. >> >> I think it is definitely worth trying to figure out how to do it. >> However, an added complication is that any objects created are >> created >> under a certain version, which may not be readable by another >> version. >> Ian has objectified this with his "schema numbering" for classes in >> elephant-unstable. >> >> The idea situation is that Elephant version N+K can read any Elephant >> object written by version N. I think we are moving toward that >> capability. >> >> However, even if one version can't read another version, being able >> to >> have two distinct packages in one image could be useful---I encourage >> you to spend some time thinking through how it might work. >> >>> >>> Also, it looks like the testing framework is being migrated over to >>> 5am. I assume this means that I should write any new tests using >>> 5am? >> >> Yes --- I believe we are already completely dependent on 5am for >> testing. >> >>> >>> Thanks, >>> >>> Glenn >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From killerstorm at newmail.ru Mon May 12 07:55:28 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Mon, 12 May 2008 10:55:28 +0300 Subject: [elephant-devel] Some thoughts/Questions from testing References: <0B53E816-1703-4D6B-9E63-D030B6EA9E4B@umich.edu> Message-ID: <5DE025096F1D480D99637FC1FF9C84F9@killer> GT> What do you all think about allowing two different versions of GT> Elephant to be installed at the same time? that's pretty easy. let's suppose that "stable" elephant is somewhere on your asdf:*central-registry* path. but you want to load "unstable" version that resides in "~/elephant-unstable/" just to (pushnew #p"~/elephant-unstable/" asdf:*central-registry*) and load elephant as normal -- asdf will first look into unstable directory, and find packages there. if you're doing switching all the time, you can add a function into your rc file to do aforementioned trick, create a script or whatever. From killerstorm at newmail.ru Mon May 12 07:55:28 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Mon, 12 May 2008 10:55:28 +0300 Subject: [elephant-devel] Some thoughts/Questions from testing References: <0B53E816-1703-4D6B-9E63-D030B6EA9E4B@umich.edu> Message-ID: <5DE025096F1D480D99637FC1FF9C84F9@killer> GT> What do you all think about allowing two different versions of GT> Elephant to be installed at the same time? that's pretty easy. let's suppose that "stable" elephant is somewhere on your asdf:*central-registry* path. but you want to load "unstable" version that resides in "~/elephant-unstable/" just to (pushnew #p"~/elephant-unstable/" asdf:*central-registry*) and load elephant as normal -- asdf will first look into unstable directory, and find packages there. if you're doing switching all the time, you can add a function into your rc file to do aforementioned trick, create a script or whatever. From killerstorm at newmail.ru Mon May 12 08:03:33 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Mon, 12 May 2008 11:03:33 +0300 Subject: [elephant-devel] Clbuild References: <11ACB1FC-8B1B-43A3-9C32-4051AC5BE3E2@media.mit.edu><3EB34F54-754A-47B0-96FD-F0F7FB2ED924@umich.edu><200805111422.06254.bryan.emrys@gmail.com> <1210551136.18630.24.camel@penguin.yourdomain.com> Message-ID: RLR> (:depeonds-on ) expressions without dependence on SBCL? Can we just RLR> use a compiler pragma like: RLR> :depends-on (:uffi RLR> #+sbcl :sb-posix RLR> :cl-base64)) sure RLR> in our asdf files? If so we will have to do something similar for RLR> each supported system (see os.lisp): RLR> (defmacro in-directory ((dir) &body body) RLR> `(progn RLR> (#+sbcl sb-posix:chdir RLR> #+cmu unix:unix-chdir RLR> #+allegro excl:chdir RLR> #+lispworks hcl:change-directory RLR> #+openmcl ccl:cwd RLR> ,dir) RLR> , at body)) i think this stuff is built-in (loaded automatically) on everything but SBCL, so no intervention is required. (i've just checked this on everything but OpenMCL) From eslick at media.mit.edu Mon May 12 12:42:41 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 08:42:41 -0400 Subject: [elephant-devel] Clbuild In-Reply-To: <200805111843.55131.bryan.emrys@gmail.com> References: <200805111422.06254.bryan.emrys@gmail.com> <1210551136.18630.24.camel@penguin.yourdomain.com> <200805111843.55131.bryan.emrys@gmail.com> Message-ID: #+sbcl :sb-posix is the way to go. I hadn't realized that SBCL didn't include that by default like the rest of the lisps. Easy fix! Robert, any indication as to the source of your clsql errors? Removing the explicit load is good and looks easy. I'll let you check these fixes in. Also, I was thinking that we should keep the cl-sql database around for the sqlite3 store. Most system have it, it's easy to use, and can get people going without the install overhead of BDB and/or postmodern/ clsql+postgres or mysql. When we have a reasonable all-lisp backend then we can revisit the decision to support it. On May 11, 2008, at 9:43 PM, Bryan Emrys wrote: > I'm not a master of anything, but pragmas seem to be the approach > used in hunchentoot (see below). > > (asdf:defsystem :hunchentoot > :serial t > :version #.*hunchentoot-version* > :depends-on (:chunga > :cl-base64 > :cl-fad > :cl-ppcre > #-(or :lispworks :hunchentoot-no-ssl) :cl+ssl > :md5 > :rfc2388 > #+:sbcl :sb-bsd-sockets > #+:sbcl :sb-posix > :url-rewrite) > :components ((:file "packages") > (:file "conditions") > #+:allegro (:file "port-acl") > #+:clisp (:file "port-clisp") > #+:cmu (:file "port-cmu") > #+:lispworks (:file "port-lw") > #+:openmcl (:file "port-mcl") > #+:sbcl (:file "port-sbcl") > (:file "specials") > (:file "mime-types") > (:file "util") > (:file "log") > (:file "cookie") > (:file "reply") > (:file "request") > (:file "session") > (:file "misc") > (:file "easy-handlers") > (:file "headers") > #+(and :allegro :unix) (:file "unix-acl") > #+(and :clisp :unix) (:file "unix-clisp") > #+(and :cmu :unix) (:file "unix-cmu") > #+(and :lispworks :unix) (:file "unix-lw") > #+(and :openmcl :unix) (:file "unix-mcl") > #+(and :sbcl :unix (not :win32)) (:file "unix-sbcl") > (:file "server"))) > > > Bryan > > On Sunday 11 May 2008 05:12:16 pm Robert L. Read wrote: >> On Sun, 2008-05-11 at 14:22 -0700, Bryan Emrys wrote: >>> At one point the clbuild maintainers looked at elephant but had >>> some problems that prevented inclusion. >>> >>> Specifically: >>> >>> # - needs sb-posix, but doesn't declare that dependency, meaning >>> that >>> # it doesn't build even with a config file >>> # - ele-clsql loads clsql from the .asd file, not using :depends- >>> on, >>> # which is completely unacceptable >>> # (we blacklist ele-clsql in clbuild.lisp because of that) >>> >>> Did anyone ever respond? >> >> Dear Bryan, >> Thanks for calling this to our attention. I doubt anyone ever >> responded to our being blacklisted, as I have no recollection of it. >> >> I have removed the explicit load, and my tests are red but I doubt >> that >> was the problem. >> >> I am not a master of ASDF --- can someone (Bryan?) tell me how we >> can >> integrate the fact that we depend on sb-posix into the ASDF >> (:depeonds-on ) expressions without dependence on SBCL? Can we >> just use >> a compiler pragma like: >> >> :depends-on (:uffi >> #+sbcl :sb-posix >> :cl-base64)) >> >> >> >> in our asdf files? If so we will have to do something similar for >> each supported system (see os.lisp): >> >> >> (defmacro in-directory ((dir) &body body) >> `(progn >> (#+sbcl sb-posix:chdir >> #+cmu unix:unix-chdir >> #+allegro excl:chdir >> #+lispworks hcl:change-directory >> #+openmcl ccl:cwd >> ,dir) >> , at body)) >> >> Finally, our documentation build system, make-ref.lisp, seems to be >> more >> deeply SBCL dependent than anything in Elephant; but that doesn't >> hurt >> much, as we normally build and publish the documentation with each >> release anyway. I wonder if we could just make the documentation >> build >> a separate package, in order to satisfy the clbuild requirements? >> >> >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel >> > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Mon May 12 12:47:43 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 08:47:43 -0400 Subject: [elephant-devel] Re: Postmodern: Removing the ABORT restart In-Reply-To: References: <63401.88.73.197.54.1209997506.squirrel@mail.stardawn.org> <62688.88.73.198.168.1210061329.squirrel@mail.stardawn.org> Message-ID: <9D1BFC0E-3941-4701-90C4-DF4541610BAC@media.mit.edu> What is the status of this bug/issue? I'm losing track of all the bugs in the system. It would be very helpful (and your bug is less likely to be forgotten) if you could file Trac bugs against 1.0 for these reports? It's easy enough to get a common-lisp.net account to sign onto Trac. Thank you, Ian On May 7, 2008, at 6:24 AM, Alex Mizrahi wrote: > ??>> sorry, i didn't get -- what is "ABORT restart", > > LPP> http://www.lisp.org/HyperSpec/Body/res_abort.html > > "Typically, in an interactive listener, the invocation of abort > returns to > the Lisp reader phase of the Lisp read-eval-print loop, though in > some batch > or multi-processing situations there may be situations in which > having it > kill the running process is more appropriate." > > are you sure that it is a good idea to make it returning to a random > point > in program (after with-transaction) rather than to a command prompt > (what > people would expect)? > and if people want to abort transaction only, can't they just call > abort-transaction? > > ??>> how does it lose the controller > > LPP> After invoking this restart, the controller is gone. I don't know > LPP> exactly why, but I suppose the reason is that the current > transaction > LPP> isn't aborted. > > maybe we'll better fix this bug? non-local control transfer should > correctly > abort transaction. > i think "controller lost" is only possible when when something awful > happens > to the socket. > i can't reproduce "controller lost" via abort restart. > i.e. i run such code > > (ele:with-transaction () (make-instance 'something) (error "barf")) > > and when SLIME debugger opens, i choose ABORT restart, and it > returns to > REPL. > and works quite fine afterwards.. > > if you have that "controller lost" thing reproducible, please share > how to > reproduce this > > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Mon May 12 12:49:06 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 08:49:06 -0400 Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE In-Reply-To: <64564.88.73.198.168.1210072702.squirrel@mail.stardawn.org> References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> <1210044527.3971.1953.camel@penguin.yourdomain.com> <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> <2C6E0324-1641-488F-B6B9-C02B80C3B9B3@media.mit.edu> <64564.88.73.198.168.1210072702.squirrel@mail.stardawn.org> Message-ID: Does this bug still occur? On May 6, 2008, at 7:18 AM, Leslie P. Polzer wrote: > >> Do you mean in an index? A slot can have nil as a value and being >> nil >> is different than being unbound. Same thing is true with slot >> indices. An unbound slot produces no index key, a nil slot >> produces a >> nil index key. > > It occured in the object index for a class; the OIDs of six objects > were mapped to NIL. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Mon May 12 12:51:49 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 08:51:49 -0400 Subject: [elephant-devel] Cached slots and other tweaks for 1.0 Message-ID: <7E3A31D6-2992-4CC9-9053-FEC18209F84D@media.mit.edu> Cached slots: ------------- - Currently cached slots are not rolled back to their initial value if transaction aborts. I can add a facility to push the cached value on the initial read into a per-transaction list that can be used to reset the cached slots on an abort. Should this be by default or an exception? A global variable or a reader macro? - Currently there is no way to 'refresh' the instance to get data from the db or to force reads to go to the db instead of the cached value. Any suggestions on a model & interface for doing this? asdf-install: ------------- - What would be required to make elephant asdf-installable. There are several problems. 1) asdf-install's propensity to load all .asd files in the root directory. 2) the dependence on external libraries that are not asdf-installable. 3) the dependence on a C compiler for memutils. 4) the my-config.sexp file is not created by default, should we add an option to auto-create it either as a restart option or just by warning that one was created? Anything else? Would anyone like to volunteer to work on making elephant asdf- installable? Tracking bugs: --------------- As I said earlier, bug reports should be entered into Trac. I am going to be batching up my work on Elephant, so will not be able to get to bugs quickly and need help remembering the current status. As a community, this is a discipline I think we need to develop, particularly if we want to be back-patching 1.0 while continuing to work on 1.1 and beyond. Thank you, Ian From leslie.polzer at gmx.net Mon May 12 14:05:00 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 12 May 2008 16:05:00 +0200 (CEST) Subject: [elephant-devel] Small bug in PM-INDEXED-BTREE In-Reply-To: References: <62431.88.73.197.54.1209989245.squirrel@mail.stardawn.org> <1210044527.3971.1953.camel@penguin.yourdomain.com> <63658.88.73.198.168.1210061727.squirrel@mail.stardawn.org> <2C6E0324-1641-488F-B6B9-C02B80C3B9B3@media.mit.edu> <64564.88.73.198.168.1210072702.squirrel@mail.stardawn.org> Message-ID: <64770.88.73.214.122.1210601100.squirrel@mail.stardawn.org> > Does this bug still occur? Given that nothing changed in stable: I must suspect so, yes. I cannot reproduce it, though. Leslie From leslie.polzer at gmx.net Mon May 12 14:17:20 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 12 May 2008 16:17:20 +0200 (CEST) Subject: [elephant-devel] Re: Postmodern: Removing the ABORT restart In-Reply-To: <9D1BFC0E-3941-4701-90C4-DF4541610BAC@media.mit.edu> References: <63401.88.73.197.54.1209997506.squirrel@mail.stardawn.org> <62688.88.73.198.168.1210061329.squirrel@mail.stardawn.org> <9D1BFC0E-3941-4701-90C4-DF4541610BAC@media.mit.edu> Message-ID: <65061.88.73.214.122.1210601840.squirrel@mail.stardawn.org> > What is the status of this bug/issue? > > I'm losing track of all the bugs in the system. It would be very > helpful (and your bug is less likely to be forgotten) if you could > file Trac bugs against 1.0 for these reports? This thing was meant for discussion, not as a bug report. I haven't been able to reproduce this thing lately, either, so let's assume it has been solved. I'll be happy to file any reproducable bugs on Trac in the future in any case! Leslie From leslie.polzer at gmx.net Mon May 12 14:23:31 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 12 May 2008 16:23:31 +0200 (CEST) Subject: [elephant-devel] Cached slots and other tweaks for 1.0 In-Reply-To: <7E3A31D6-2992-4CC9-9053-FEC18209F84D@media.mit.edu> References: <7E3A31D6-2992-4CC9-9053-FEC18209F84D@media.mit.edu> Message-ID: <61109.88.73.214.122.1210602211.squirrel@mail.stardawn.org> > - Currently cached slots are not rolled back to their initial value if > transaction aborts. I can add a facility to push the cached value on > the initial read into a per-transaction list that can be used to reset > the cached slots on an abort. Would it be possible to automatically restore the cached slots on a failed transaction? This would be the most transparent behaviour IMHO. > - Currently there is no way to 'refresh' the instance to get data from > the db or to force reads to go to the db instead of the cached value. > Any suggestions on a model & interface for doing this? Hm, what's the use case for this, assuming that the cache works correctly? > 4) the my-config.sexp file is not created by default, should we add an > option to > auto-create it either as a restart option or just by warning that > one was created? It depends. IME the current config.sexp is mainly a thing for the BDB backend, so it's pretty useless for other backend users. I'm all for auto-creating and warning. Leslie From eslick at media.mit.edu Mon May 12 15:23:53 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 11:23:53 -0400 Subject: [elephant-devel] Cached slots and other tweaks for 1.0 In-Reply-To: <61109.88.73.214.122.1210602211.squirrel@mail.stardawn.org> References: <7E3A31D6-2992-4CC9-9053-FEC18209F84D@media.mit.edu> <61109.88.73.214.122.1210602211.squirrel@mail.stardawn.org> Message-ID: <496F79FB-568C-4B52-9D32-55E5616B7966@media.mit.edu> On May 12, 2008, at 10:23 AM, Leslie P. Polzer wrote: > >> - Currently cached slots are not rolled back to their initial value >> if >> transaction aborts. I can add a facility to push the cached value on >> the initial read into a per-transaction list that can be used to >> reset >> the cached slots on an abort. > > Would it be possible to automatically restore the cached slots > on a failed transaction? This would be the most transparent > behaviour IMHO. I think I've all but decided to take this approach. > >> - Currently there is no way to 'refresh' the instance to get data >> from >> the db or to force reads to go to the db instead of the cached value. >> Any suggestions on a model & interface for doing this? > > Hm, what's the use case for this, assuming that the cache works > correctly? A database shared among lisp images. The cached copy of the object may become stale. Also, cached slots are not currently thread safe. If you don't guarantee that only one thread at a time is working on a particular object, you can get problems. These slot values are not isolated by the transaction mechanism. I'm not sure we should solve that; the protection for cached slots should be at a higher level so it doesn't swamp the benefits. For example you can have a transient slot with a lock on it if you don't have too many of these objects, or a protected hash table... This might occur if there are two parallel sessions under the same user-id. Ian > >> 4) the my-config.sexp file is not created by default, should we add >> an >> option to >> auto-create it either as a restart option or just by warning that >> one was created? > > It depends. IME the current config.sexp is mainly a thing for the > BDB backend, so it's pretty useless for other backend users. True, unless you use it to add clsql library files or choose a compilation option for the memutil library. > I'm all for auto-creating and warning. Sounds good. > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From ryszard.szopa at gmail.com Mon May 12 18:08:18 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Mon, 12 May 2008 20:08:18 +0200 Subject: [elephant-devel] bug in map-inverted-index Message-ID: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> Hey, map-inverted-index shows some strange behavior. When you pass it `start' and `end' arguments and there are no objects that satisfy the range, it maps over the first object greater than end. I haven't have time to investigate it thoroughly, but I have written a 5am unit-test (you are using this test framework, aren't you?). (defpclass foo () ((str :type string :initarg :str :reader str-of :index t))) (test map-inverted-index-1 (with-open-store (`(:bdb "/tmp/store/")) (drop-instances (get-instances-by-class 'foo)) (loop :for st :in '("a" "b" "c" "d" "g" "ga" "gb" "gc" "z") :do (make-instance 'foo :str st)) (is (null (map-inverted-index (lambda (x y) (declare (ignore y)) x) 'foo 'str :start "f" :end "fz" :collect t))))) This was on Ubuntu Hardy Heron, with BerkeleyDB 4.5 from Ubuntu packages, SBCL 1.0.15 and the newest unstable branch of Elephant (I've just made darcs pull). Funny enough, a friend of mine who is using a Macintosh is not getting this bug (he also uses unstable but I'm not sure whether it is the newest version). -- http://szopa.tasak.gda.pl/ From eslick at media.mit.edu Mon May 12 18:15:27 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 14:15:27 -0400 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> Message-ID: I thought I had fixed all of these - I'll look into it... Ian On May 12, 2008, at 2:08 PM, Ryszard Szopa wrote: > Hey, > > map-inverted-index shows some strange behavior. When you pass it > `start' and `end' arguments and there are no objects that satisfy the > range, it maps over the first object greater than end. I haven't have > time to investigate it thoroughly, but I have written a 5am unit-test > (you are using this test framework, aren't you?). > > (defpclass foo () > ((str :type string > :initarg :str > :reader str-of > :index t))) > > (test map-inverted-index-1 > (with-open-store (`(:bdb "/tmp/store/")) > (drop-instances (get-instances-by-class 'foo)) > (loop :for st :in '("a" "b" "c" "d" "g" "ga" "gb" "gc" "z") > :do (make-instance 'foo :str st)) > (is (null (map-inverted-index (lambda (x y) > (declare (ignore y)) > x) 'foo 'str :start "f" :end "fz" :collect t))))) > > This was on Ubuntu Hardy Heron, with BerkeleyDB 4.5 from Ubuntu > packages, SBCL 1.0.15 and the newest unstable branch of Elephant (I've > just made darcs pull). Funny enough, a friend of mine who is using a > Macintosh is not getting this bug (he also uses unstable but I'm not > sure whether it is the newest version). > > -- > http://szopa.tasak.gda.pl/ > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Mon May 12 18:24:00 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 14:24:00 -0400 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> Message-ID: That test was great. I reproduced this on my Mac under Allegro with BDB 4.6. Will let you know what I find... I also added the test to testindexing.lisp. On May 12, 2008, at 2:08 PM, Ryszard Szopa wrote: > Hey, > > map-inverted-index shows some strange behavior. When you pass it > `start' and `end' arguments and there are no objects that satisfy the > range, it maps over the first object greater than end. I haven't have > time to investigate it thoroughly, but I have written a 5am unit-test > (you are using this test framework, aren't you?). > > (defpclass foo () > ((str :type string > :initarg :str > :reader str-of > :index t))) > > (test map-inverted-index-1 > (with-open-store (`(:bdb "/tmp/store/")) > (drop-instances (get-instances-by-class 'foo)) > (loop :for st :in '("a" "b" "c" "d" "g" "ga" "gb" "gc" "z") > :do (make-instance 'foo :str st)) > (is (null (map-inverted-index (lambda (x y) > (declare (ignore y)) > x) 'foo 'str :start "f" :end "fz" :collect t))))) > > This was on Ubuntu Hardy Heron, with BerkeleyDB 4.5 from Ubuntu > packages, SBCL 1.0.15 and the newest unstable branch of Elephant (I've > just made darcs pull). Funny enough, a friend of mine who is using a > Macintosh is not getting this bug (he also uses unstable but I'm not > sure whether it is the newest version). > > -- > http://szopa.tasak.gda.pl/ > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Mon May 12 18:34:52 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 14:34:52 -0400 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> Message-ID: <890CB4D4-435C-4EEE-9A00-4D3D44B12A18@media.mit.edu> map-index was missing a check for a valid key after the first value query. I've patched, verified and pushed a fix along with the test case to elephant-unstable. Thank you for finding this! Ian On May 12, 2008, at 2:24 PM, Ian Eslick wrote: > That test was great. I reproduced this on my Mac under Allegro with > BDB 4.6. Will let you know what I find... I also added the test to > testindexing.lisp. > > On May 12, 2008, at 2:08 PM, Ryszard Szopa wrote: > >> Hey, >> >> map-inverted-index shows some strange behavior. When you pass it >> `start' and `end' arguments and there are no objects that satisfy the >> range, it maps over the first object greater than end. I haven't have >> time to investigate it thoroughly, but I have written a 5am unit-test >> (you are using this test framework, aren't you?). >> >> (defpclass foo () >> ((str :type string >> :initarg :str >> :reader str-of >> :index t))) >> >> (test map-inverted-index-1 >> (with-open-store (`(:bdb "/tmp/store/")) >> (drop-instances (get-instances-by-class 'foo)) >> (loop :for st :in '("a" "b" "c" "d" "g" "ga" "gb" "gc" "z") >> :do (make-instance 'foo :str st)) >> (is (null (map-inverted-index (lambda (x y) >> (declare (ignore y)) >> x) 'foo 'str :start "f" :end "fz" :collect t))))) >> >> This was on Ubuntu Hardy Heron, with BerkeleyDB 4.5 from Ubuntu >> packages, SBCL 1.0.15 and the newest unstable branch of Elephant >> (I've >> just made darcs pull). Funny enough, a friend of mine who is using a >> Macintosh is not getting this bug (he also uses unstable but I'm not >> sure whether it is the newest version). >> >> -- >> http://szopa.tasak.gda.pl/ >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From ryszard.szopa at gmail.com Mon May 12 19:53:16 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Mon, 12 May 2008 21:53:16 +0200 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: <890CB4D4-435C-4EEE-9A00-4D3D44B12A18@media.mit.edu> References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> <890CB4D4-435C-4EEE-9A00-4D3D44B12A18@media.mit.edu> Message-ID: <19e19e410805121253g64976962hd6dbdc75845d9dc9@mail.gmail.com> On Mon, May 12, 2008 at 8:34 PM, Ian Eslick wrote: > map-index was missing a check for a valid key after the first value query. > I've patched, verified and pushed a fix along with the test case to > elephant-unstable. > > Thank you for finding this! My pleasure. Thank you for fixing it :-) Cheers, -- Richard -- http://szopa.tasak.gda.pl/ From ryszard.szopa at gmail.com Mon May 12 23:55:19 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Tue, 13 May 2008 01:55:19 +0200 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: <19e19e410805121253g64976962hd6dbdc75845d9dc9@mail.gmail.com> References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> <890CB4D4-435C-4EEE-9A00-4D3D44B12A18@media.mit.edu> <19e19e410805121253g64976962hd6dbdc75845d9dc9@mail.gmail.com> Message-ID: <19e19e410805121655p6146deedn6dfae26e60f092b8@mail.gmail.com> Huh... This is one of those things that are kind of hard to admit publicly... Apparently, I tricked myself into thinking that I was using elephant-unstable, while in fact I was using Elephant stable (please don't ask how this was possible). So, the bug I've mentioned in this thread and for which I've written the unit-test is also present in the stable version. BTW I get a missing slot error when I try to open a store using the unstable branch (this time it is elephant-unstable for real): (open-store `(:bdb ,(merge-pathnames #P "store" (asdf:component-pathname (asdf:find-system 'my-project)))))) => When attempting to set the slot's value to # (SETF of SLOT-VALUE), the slot ELEPHANT::CLASS-ROOT is missing from the object #. [Condition of type SIMPLE-ERROR] Cheers, -- Richard -- http://szopa.tasak.gda.pl/ From read at robertlread.net Tue May 13 00:00:35 2008 From: read at robertlread.net (Robert L. Read) Date: Mon, 12 May 2008 19:00:35 -0500 Subject: [elephant-devel] Clbuild In-Reply-To: References: <200805111422.06254.bryan.emrys@gmail.com> <1210551136.18630.24.camel@penguin.yourdomain.com> <200805111843.55131.bryan.emrys@gmail.com> Message-ID: <1210636835.18630.64.camel@penguin.yourdomain.com> On Mon, 2008-05-12 at 08:42 -0400, Ian Eslick wrote: > #+sbcl :sb-posix is the way to go. > > I hadn't realized that SBCL didn't include that by default like the > rest of the lisps. Easy fix! > > Robert, any indication as to the source of your clsql errors? I will look into tonight, when the kids are in bed. > > Removing the explicit load is good and looks easy. I'll let you check > these fixes in. Affirmative. > > Also, I was thinking that we should keep the cl-sql database around > for the sqlite3 store. Most system have it, it's easy to use, and can > get people going without the install overhead of BDB and/or postmodern/ > clsql+postgres or mysql. When we have a reasonable all-lisp backend > then we can revisit the decision to support it. Yes, that is a good point in favor of keeping cl-sql. > > > > On May 11, 2008, at 9:43 PM, Bryan Emrys wrote: > > > I'm not a master of anything, but pragmas seem to be the approach > > used in hunchentoot (see below). > > > > (asdf:defsystem :hunchentoot > > :serial t > > :version #.*hunchentoot-version* > > :depends-on (:chunga > > :cl-base64 > > :cl-fad > > :cl-ppcre > > #-(or :lispworks :hunchentoot-no-ssl) :cl+ssl > > :md5 > > :rfc2388 > > #+:sbcl :sb-bsd-sockets > > #+:sbcl :sb-posix > > :url-rewrite) > > :components ((:file "packages") > > (:file "conditions") > > #+:allegro (:file "port-acl") > > #+:clisp (:file "port-clisp") > > #+:cmu (:file "port-cmu") > > #+:lispworks (:file "port-lw") > > #+:openmcl (:file "port-mcl") > > #+:sbcl (:file "port-sbcl") > > (:file "specials") > > (:file "mime-types") > > (:file "util") > > (:file "log") > > (:file "cookie") > > (:file "reply") > > (:file "request") > > (:file "session") > > (:file "misc") > > (:file "easy-handlers") > > (:file "headers") > > #+(and :allegro :unix) (:file "unix-acl") > > #+(and :clisp :unix) (:file "unix-clisp") > > #+(and :cmu :unix) (:file "unix-cmu") > > #+(and :lispworks :unix) (:file "unix-lw") > > #+(and :openmcl :unix) (:file "unix-mcl") > > #+(and :sbcl :unix (not :win32)) (:file "unix-sbcl") > > (:file "server"))) > > > > > > Bryan > > > > On Sunday 11 May 2008 05:12:16 pm Robert L. Read wrote: > >> On Sun, 2008-05-11 at 14:22 -0700, Bryan Emrys wrote: > >>> At one point the clbuild maintainers looked at elephant but had > >>> some problems that prevented inclusion. > >>> > >>> Specifically: > >>> > >>> # - needs sb-posix, but doesn't declare that dependency, meaning > >>> that > >>> # it doesn't build even with a config file > >>> # - ele-clsql loads clsql from the .asd file, not using :depends- > >>> on, > >>> # which is completely unacceptable > >>> # (we blacklist ele-clsql in clbuild.lisp because of that) > >>> > >>> Did anyone ever respond? > >> > >> Dear Bryan, > >> Thanks for calling this to our attention. I doubt anyone ever > >> responded to our being blacklisted, as I have no recollection of it. > >> > >> I have removed the explicit load, and my tests are red but I doubt > >> that > >> was the problem. > >> > >> I am not a master of ASDF --- can someone (Bryan?) tell me how we > >> can > >> integrate the fact that we depend on sb-posix into the ASDF > >> (:depeonds-on ) expressions without dependence on SBCL? Can we > >> just use > >> a compiler pragma like: > >> > >> :depends-on (:uffi > >> #+sbcl :sb-posix > >> :cl-base64)) > >> > >> > >> > >> in our asdf files? If so we will have to do something similar for > >> each supported system (see os.lisp): > >> > >> > >> (defmacro in-directory ((dir) &body body) > >> `(progn > >> (#+sbcl sb-posix:chdir > >> #+cmu unix:unix-chdir > >> #+allegro excl:chdir > >> #+lispworks hcl:change-directory > >> #+openmcl ccl:cwd > >> ,dir) > >> , at body)) > >> > >> Finally, our documentation build system, make-ref.lisp, seems to be > >> more > >> deeply SBCL dependent than anything in Elephant; but that doesn't > >> hurt > >> much, as we normally build and publish the documentation with each > >> release anyway. I wonder if we could just make the documentation > >> build > >> a separate package, in order to satisfy the clbuild requirements? > >> > >> > >>> _______________________________________________ > >>> elephant-devel site list > >>> elephant-devel at common-lisp.net > >>> http://common-lisp.net/mailman/listinfo/elephant-devel > >> > >> _______________________________________________ > >> elephant-devel site list > >> elephant-devel at common-lisp.net > >> http://common-lisp.net/mailman/listinfo/elephant-devel > >> > > _______________________________________________ > > elephant-devel site list > > elephant-devel at common-lisp.net > > http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Tue May 13 00:02:08 2008 From: read at robertlread.net (Robert L. Read) Date: Mon, 12 May 2008 19:02:08 -0500 Subject: [elephant-devel] Some thoughts/Questions from testing In-Reply-To: <5DE025096F1D480D99637FC1FF9C84F9@killer> References: <0B53E816-1703-4D6B-9E63-D030B6EA9E4B@umich.edu> <5DE025096F1D480D99637FC1FF9C84F9@killer> Message-ID: <1210636928.18630.67.camel@penguin.yourdomain.com> On Mon, 2008-05-12 at 10:55 +0300, Alex Mizrahi wrote: > GT> What do you all think about allowing two different versions of > GT> Elephant to be installed at the same time? > > that's pretty easy. > > let's suppose that "stable" elephant is somewhere on your > asdf:*central-registry* path. > but you want to load "unstable" version that resides in > "~/elephant-unstable/" > > just to (pushnew #p"~/elephant-unstable/" asdf:*central-registry*) > and load elephant as normal -- asdf will first look into unstable directory, > and find packages there. This is a great little hack; I somehow new it but still waste time swiveling my asd symbolic links. > > if you're doing switching all the time, you can add a function into your rc > file to do aforementioned trick, create a script or whatever. > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Tue May 13 00:05:38 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 12 May 2008 20:05:38 -0400 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: <19e19e410805121655p6146deedn6dfae26e60f092b8@mail.gmail.com> References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> <890CB4D4-435C-4EEE-9A00-4D3D44B12A18@media.mit.edu> <19e19e410805121253g64976962hd6dbdc75845d9dc9@mail.gmail.com> <19e19e410805121655p6146deedn6dfae26e60f092b8@mail.gmail.com> Message-ID: <11EC3CD4-A7D3-46F9-9FB4-9412AC1A711B@media.mit.edu> The problem is on both branches then. I've fixed it on unstable. I don't intend to back patch, although the patch is trivial if someone wants to make it. Ian On May 12, 2008, at 7:55 PM, Ryszard Szopa wrote: > Huh... This is one of those things that are kind of hard to admit > publicly... Apparently, I tricked myself into thinking that I was > using elephant-unstable, while in fact I was using Elephant stable > (please don't ask how this was possible). So, the bug I've mentioned > in this thread and for which I've written the unit-test is also > present in the stable version. > > BTW I get a missing slot error when I try to open a store using the > unstable branch (this time it is elephant-unstable for real): > > (open-store `(:bdb ,(merge-pathnames #P "store" > (asdf:component-pathname (asdf:find-system 'my-project)))))) > => > When attempting to > set the slot's value to # (SETF of SLOT-VALUE), > the slot > ELEPHANT::CLASS-ROOT is missing from the object > #. > [Condition of type SIMPLE-ERROR] > > Cheers, > > -- Richard > -- > http://szopa.tasak.gda.pl/ > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From ryszard.szopa at gmail.com Tue May 13 13:41:45 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Tue, 13 May 2008 15:41:45 +0200 Subject: [elephant-devel] open-controller for bdb doesn't accept a pathname spec Message-ID: <19e19e410805130641j39e83052q3094f5d62db5a2a9@mail.gmail.com> Hey, As I mentioned before, I had trouble in opening a bdb store in elephant-unstable. I investigated the case, and it turned out that open-controller for bdb didn't accept a pathname as part of the store specification. A patch that fixes the problem is attached. I also wrote a unit-test for the thing, but I am not really sure how to incorporate it to elephant-tests, so I didn't include it in the patch. I would be very glad if someone helped me with this. (test open-store-bdb-w-pathname (finishes (open-store (:bdb #P"/tmp/")))) Cheers, -- Richard -- http://szopa.tasak.gda.pl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: open-controller-pathname-spec.patch Type: text/x-diff Size: 11508 bytes Desc: not available URL: From eslick at media.mit.edu Tue May 13 14:01:02 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 13 May 2008 10:01:02 -0400 Subject: [elephant-devel] open-controller for bdb doesn't accept a pathname spec In-Reply-To: <19e19e410805130641j39e83052q3094f5d62db5a2a9@mail.gmail.com> References: <19e19e410805130641j39e83052q3094f5d62db5a2a9@mail.gmail.com> Message-ID: <5126A351-3B94-48B3-B857-7CA8273B664B@media.mit.edu> I had just run into this myself too! Anyway, I made some tweaks to the patch and just pushed the fix. You could write a whitebox test for the auxiliary function I created called elephant-db-path which returns a pathname to probe-file regardless of the input spec. Ian On May 13, 2008, at 9:41 AM, Ryszard Szopa wrote: > Hey, > > As I mentioned before, I had trouble in opening a bdb store in > elephant-unstable. I investigated the case, and it turned out that > open-controller for bdb didn't accept a pathname as part of the store > specification. A patch that fixes the problem is attached. > > I also wrote a unit-test for the thing, but I am not really sure how > to incorporate it to elephant-tests, so I didn't include it in the > patch. I would be very glad if someone helped me with this. > > (test open-store-bdb-w-pathname > (finishes (open-store (:bdb #P"/tmp/")))) > > Cheers, > > -- Richard > > -- > http://szopa.tasak.gda.pl/ > spec.patch>_______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From ryszard.szopa at gmail.com Tue May 13 14:12:12 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Tue, 13 May 2008 16:12:12 +0200 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: <11EC3CD4-A7D3-46F9-9FB4-9412AC1A711B@media.mit.edu> References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> <890CB4D4-435C-4EEE-9A00-4D3D44B12A18@media.mit.edu> <19e19e410805121253g64976962hd6dbdc75845d9dc9@mail.gmail.com> <19e19e410805121655p6146deedn6dfae26e60f092b8@mail.gmail.com> <11EC3CD4-A7D3-46F9-9FB4-9412AC1A711B@media.mit.edu> Message-ID: <19e19e410805130712n3c074106k2b4367422642e61f@mail.gmail.com> On Tue, May 13, 2008 at 2:05 AM, Ian Eslick wrote: > The problem is on both branches then. I've fixed it on unstable. I don't > intend to back patch, although the patch is trivial if someone wants to make > it. I wanted to. The patch is attached. -- http://szopa.tasak.gda.pl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: map-inverted-index-range.patch Type: text/x-diff Size: 7437 bytes Desc: not available URL: From eslick at media.mit.edu Tue May 13 14:17:41 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 13 May 2008 10:17:41 -0400 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: <19e19e410805130712n3c074106k2b4367422642e61f@mail.gmail.com> References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> <890CB4D4-435C-4EEE-9A00-4D3D44B12A18@media.mit.edu> <19e19e410805121253g64976962hd6dbdc75845d9dc9@mail.gmail.com> <19e19e410805121655p6146deedn6dfae26e60f092b8@mail.gmail.com> <11EC3CD4-A7D3-46F9-9FB4-9412AC1A711B@media.mit.edu> <19e19e410805130712n3c074106k2b4367422642e61f@mail.gmail.com> Message-ID: <7AFD8356-8980-4DC2-BB6B-9B8730C5B36F@media.mit.edu> Robert or Alex, would you mind back patching on the main branch? We should deprecate the main branch and move everyone to the new branch as soon as possible. Alex, any update on when we'll get the postmodern backend to work on the new branch with dup-btrees, etc? Thank you! Ian On May 13, 2008, at 10:12 AM, Ryszard Szopa wrote: > On Tue, May 13, 2008 at 2:05 AM, Ian Eslick > wrote: >> The problem is on both branches then. I've fixed it on unstable. >> I don't >> intend to back patch, although the patch is trivial if someone >> wants to make >> it. > > I wanted to. The patch is attached. > > -- > http://szopa.tasak.gda.pl/ > range.patch>_______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Tue May 13 18:03:39 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Tue, 13 May 2008 20:03:39 +0200 (CEST) Subject: [elephant-devel] Lisp Btrees: design considerations Message-ID: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> I suppose the "binary paging" approach mentioned in the design considerations document means the problem of organizing the data efficiently on disk right from the start. Is this correct? Do you think it would make good sense to start working on the btree library without thinking much about on-disk efficiency, leaving this part for later? I'm not sure a btree where on-disk storage organization is separeted from the rest like that can achieve enough efficiency... Thanks, Leslie From eslick at media.mit.edu Tue May 13 19:46:10 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 13 May 2008 15:46:10 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> Message-ID: <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> I think they key decision was what serialization format we're going to use for btree nodes, log entries, etc and how that relates to caching data during transactions, maintaining empty lists, etc. The current serializer produces a byte sequence. If we continue with that model, how do we write/read this stuff from disk? How and where do we store it prior to committing a transaction? When we create a new key or value as a binary stream within a transaction, how is it stored in memory? If we want a multi-process, but non-socket based approach, we need to figure out how to store data in shared memory, etc. For example, in BDB, the primitive is the page. BTree nodes are layed out in one or more pages, each page has some binary metadata explaining it's type and organization (free list, etc). A short key value is written directly into the page, a long one is written into an overflow page, etc. Lots of details to deal with in managing variable sized data on disk. Pages that are dirty are kept in memory (which is why BDB can run out of transaction space; the pages overflow the max cache size when you are writing lots of data). However, to get started, the easiest thing is to reuse the existing memutils serializer, not worry about multi-process operation and not worry about fragmentation, sharing space and maintaining free lists (except perhaps for btree nodes). Something like: - btree nodes only keep pointers to variable sized keys stored elsewhere in the file - new keys and values of differing or less length are written in place, otherwise new space is allocated at the end of the file. - btree nodes are a fixed size page on-disk and keep some free-list information so we can reuse them. - transactions simply keep track of the primitive operations on the database and the associated data in a memory queue and write those ops to disk as part of the txn commit process. The pages and key/value pairs that will be touched in that operation are also stored in that txn log. - when a transaction commits, it replays the log to write everything to disk appropriately. The list of touched data is then passed up the commit chain to invalidate any pending transactions that have a conflict. Everything is speculative in this case, but we don't have to deal with locking. This is a nice balance between some lisp-sexp serialization format that performs poorly, and a highly-optimized low-level implementation which is blindingly fast. A big decision is: - Use cffi/uffi and do much of the serialization & btree implementation in C/static memory or do all of this in pools of static arrays and write a new serializer to operate on lisp data. I lean towards using cffi to manipulate static data, just because it's going to be easier to get performance via that method and it's also going to be much easier to do a multi-process implementation (by operating on static memory and primitive locks in a shared memory region). Predicated on that decision, getting started on the simplest possible btree/dup-btree implementation is the next, most valuable and educational step. The key pieces for a single-process lisp backend: - btrees and dup-btrees (indices can be built from these two easily enough) - the binary pages could be stored in static data and the primitives btree ops could directly manipulate data within the page? We pass a C function that directly sorts binary sequences rather than having to deserialize to sort. We'd need to write that in lisp to operate on static data or on lisp arrays. Deserializing on each key comparison is too expensive. - a set of transaction records (lisp structs and consts) - simply keeps tuples of (op {rd | wr} {btree+page-offset | value- offset} [values]) in a memory queue. Could use static memory for this to reduce load on GC - a blocking primitive library that serializes txn commits (i.e. write log to disk, write data to disk, write 'commit done' to log, invalidate pending/conflicting txns) A nice auxiliary hack would be: - rewrite memutils to entirely use uffi/cffi to manipulate static data rather than calling out to C to do it. Maintains efficiency but removes the compilation build step except for users of BDB So what do people think about the cffi->static-data vs. lisp->array- pool decision? Ian On May 13, 2008, at 2:03 PM, Leslie P. Polzer wrote: > > I suppose the "binary paging" approach mentioned in the design > considerations > document means the problem of organizing the data efficiently on disk > right from the start. Is this correct? > > Do you think it would make good sense to start working on the btree > library > without thinking much about on-disk efficiency, leaving this part > for later? > > I'm not sure a btree where on-disk storage organization is separeted > from the > rest like that can achieve enough efficiency... > > Thanks, > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From ryszard.szopa at gmail.com Tue May 13 21:36:59 2008 From: ryszard.szopa at gmail.com (Ryszard Szopa) Date: Tue, 13 May 2008 23:36:59 +0200 Subject: [elephant-devel] open-controller for bdb doesn't accept a pathname spec In-Reply-To: <5126A351-3B94-48B3-B857-7CA8273B664B@media.mit.edu> References: <19e19e410805130641j39e83052q3094f5d62db5a2a9@mail.gmail.com> <5126A351-3B94-48B3-B857-7CA8273B664B@media.mit.edu> Message-ID: <19e19e410805131436x74cec081mb332b2389b5efc51@mail.gmail.com> On Tue, May 13, 2008 at 4:01 PM, Ian Eslick wrote: > I had just run into this myself too! Anyway, I made some tweaks to the > patch and just pushed the fix. > > You could write a whitebox test for the auxiliary function I created called > elephant-db-path which returns a pathname to probe-file regardless of the > input spec. Here it is. There's one problem though: for POSIX-like systems, multiple subsequent slashes don't change the meaning of a pathname. This is not the case for CL pathnames. Should we care about it? BTW there are some aspects of the tests system I fail to understand. For example, elephant-tests fails to compile (tests/testconcurrency.lisp wants to use package BT, which cannot be found). -- Richard -- http://szopa.tasak.gda.pl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: elephant-db-path-test.patch Type: text/x-diff Size: 11988 bytes Desc: not available URL: From eslick at media.mit.edu Tue May 13 22:00:18 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 13 May 2008 18:00:18 -0400 Subject: [elephant-devel] open-controller for bdb doesn't accept a pathname spec In-Reply-To: <19e19e410805131436x74cec081mb332b2389b5efc51@mail.gmail.com> References: <19e19e410805130641j39e83052q3094f5d62db5a2a9@mail.gmail.com> <5126A351-3B94-48B3-B857-7CA8273B664B@media.mit.edu> <19e19e410805131436x74cec081mb332b2389b5efc51@mail.gmail.com> Message-ID: On May 13, 2008, at 5:36 PM, Ryszard Szopa wrote: > On Tue, May 13, 2008 at 4:01 PM, Ian Eslick > wrote: >> I had just run into this myself too! Anyway, I made some tweaks to >> the >> patch and just pushed the fix. >> >> You could write a whitebox test for the auxiliary function I >> created called >> elephant-db-path which returns a pathname to probe-file regardless >> of the >> input spec. > > Here it is. There's one problem though: for POSIX-like systems, > multiple subsequent slashes don't change the meaning of a pathname. > This is not the case for CL pathnames. Should we care about it? I don't think we should care. The CL standard is just fine unless we get complaints, which I don't think we have in the 4 years the project has been alive. > BTW there are some aspects of the tests system I fail to understand. > For example, elephant-tests fails to compile > (tests/testconcurrency.lisp wants to use package BT, which cannot be > found). testconcurrency hasn't been totally integrated yet. we need to add a test dependency on bordeaux threads (bt). elephant-tests.lisp works fine for me. What is the specific error you are seeing? Can you remind me what data store you are primarily using? > -- Richard > > -- > http://szopa.tasak.gda.pl/ > test.patch>_______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Wed May 14 02:02:54 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 13 May 2008 22:02:54 -0400 Subject: [elephant-devel] full text search solution? Message-ID: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu> Hello, Has anyone found a good solution for full text search in lisp? I'm interested in indexing website objects such as posts and perhaps external documents as well. BDB doesn't, to the best of my knowledge, have the appropriate building blocks for an efficient indexing system and you certainly don't want to do it on top of the current btree interface. I have an old full text index code base that supported wildcard and NEAR queries, all built on top of Elephant btrees. It was convenient but had a query time that slowed down linearly with the avg # of documents per word. I've decided that the best approach for me is to connect to a separate, probably external, system to which I can incrementally add content that will return something I can easily turn into an ordered list of OIDs. Most solutions I've run across require other languages, servers that add up to needless complexity for my modest application. In the lisp world I've only seen Montezuma, which isn't being developed or seriously maintained (unless it's just really stable I'd rather not fight with stale code). I am considering hacking something simple on top of postmodern that uses the new text indexing functions of Postgresql 8.3 and wondered if anyone here has insight into this application of postmodern or into the full-text indexing from lisp problem in general. Thank you, Ian From gtarcea at umich.edu Wed May 14 02:33:59 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Tue, 13 May 2008 22:33:59 -0400 Subject: [elephant-devel] full text search solution? In-Reply-To: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu> References: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu> Message-ID: <721FD699-D250-4864-9296-1735E034FA5B@umich.edu> Have you looked at Montezuma? http://projects.heavymeta.org/montezuma/ It's a port of Ferret (which is port of Lucene) to Lisp. It's also on the list for the google summer of code to add some additional functionality to it. BTW: If its alright I'm going to update the docs to add a section on having two versions of Elephant installed and using ASDF to manage them. Glenn On May 13, 2008, at 10:02 PM, Ian Eslick wrote: > Hello, > > Has anyone found a good solution for full text search in lisp? I'm > interested in indexing website objects such as posts and perhaps > external documents as well. BDB doesn't, to the best of my > knowledge, have the appropriate building blocks for an efficient > indexing system and you certainly don't want to do it on top of the > current btree interface. > > I have an old full text index code base that supported wildcard and > NEAR queries, all built on top of Elephant btrees. It was > convenient but had a query time that slowed down linearly with the > avg # of documents per word. > > I've decided that the best approach for me is to connect to a > separate, probably external, system to which I can incrementally > add content that will return something I can easily turn into an > ordered list of OIDs. > > Most solutions I've run across require other languages, servers > that add up to needless complexity for my modest application. In > the lisp world I've only seen Montezuma, which isn't being > developed or seriously maintained (unless it's just really stable > I'd rather not fight with stale code). > > I am considering hacking something simple on top of postmodern that > uses the new text indexing functions of Postgresql 8.3 and wondered > if anyone here has insight into this application of postmodern or > into the full-text indexing from lisp problem in general. > > Thank you, > Ian > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gtarcea at umich.edu Wed May 14 02:43:37 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Tue, 13 May 2008 22:43:37 -0400 Subject: [elephant-devel] full text search solution? In-Reply-To: <721FD699-D250-4864-9296-1735E034FA5B@umich.edu> References: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu> <721FD699-D250-4864-9296-1735E034FA5B@umich.edu> Message-ID: Never mind, I see from reading that you already found Monezuma.... But it does look like it might get some activity. Glenn On May 13, 2008, at 10:33 PM, Glenn Tarcea wrote: > Have you looked at Montezuma? http://projects.heavymeta.org/montezuma/ > > It's a port of Ferret (which is port of Lucene) to Lisp. It's also > on the list for the google summer of code to add some additional > functionality to it. > > BTW: If its alright I'm going to update the docs to add a section > on having two versions of Elephant installed and using ASDF to > manage them. > > Glenn > > > On May 13, 2008, at 10:02 PM, Ian Eslick wrote: > >> Hello, >> >> Has anyone found a good solution for full text search in lisp? >> I'm interested in indexing website objects such as posts and >> perhaps external documents as well. BDB doesn't, to the best of >> my knowledge, have the appropriate building blocks for an >> efficient indexing system and you certainly don't want to do it on >> top of the current btree interface. >> >> I have an old full text index code base that supported wildcard >> and NEAR queries, all built on top of Elephant btrees. It was >> convenient but had a query time that slowed down linearly with the >> avg # of documents per word. >> >> I've decided that the best approach for me is to connect to a >> separate, probably external, system to which I can incrementally >> add content that will return something I can easily turn into an >> ordered list of OIDs. >> >> Most solutions I've run across require other languages, servers >> that add up to needless complexity for my modest application. In >> the lisp world I've only seen Montezuma, which isn't being >> developed or seriously maintained (unless it's just really stable >> I'd rather not fight with stale code). >> >> I am considering hacking something simple on top of postmodern >> that uses the new text indexing functions of Postgresql 8.3 and >> wondered if anyone here has insight into this application of >> postmodern or into the full-text indexing from lisp problem in >> general. >> >> Thank you, >> Ian >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From eslick at media.mit.edu Wed May 14 03:09:52 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 13 May 2008 23:09:52 -0400 Subject: [elephant-devel] full text search solution? In-Reply-To: References: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu> <721FD699-D250-4864-9296-1735E034FA5B@umich.edu> Message-ID: <0F978366-F34F-4F91-8554-474DED798997@media.mit.edu> Thanks Glenn, Do you have experience with Montezuma? Ian On May 13, 2008, at 10:43 PM, Glenn Tarcea wrote: > Never mind, I see from reading that you already found Monezuma.... > But it does look like it might get some activity. > > Glenn > > On May 13, 2008, at 10:33 PM, Glenn Tarcea wrote: > >> Have you looked at Montezuma? http://projects.heavymeta.org/ >> montezuma/ >> >> It's a port of Ferret (which is port of Lucene) to Lisp. It's also >> on the list for the google summer of code to add some additional >> functionality to it. >> >> BTW: If its alright I'm going to update the docs to add a section >> on having two versions of Elephant installed and using ASDF to >> manage them. >> >> Glenn >> >> >> On May 13, 2008, at 10:02 PM, Ian Eslick wrote: >> >>> Hello, >>> >>> Has anyone found a good solution for full text search in lisp? >>> I'm interested in indexing website objects such as posts and >>> perhaps external documents as well. BDB doesn't, to the best of >>> my knowledge, have the appropriate building blocks for an >>> efficient indexing system and you certainly don't want to do it on >>> top of the current btree interface. >>> >>> I have an old full text index code base that supported wildcard >>> and NEAR queries, all built on top of Elephant btrees. It was >>> convenient but had a query time that slowed down linearly with the >>> avg # of documents per word. >>> >>> I've decided that the best approach for me is to connect to a >>> separate, probably external, system to which I can incrementally >>> add content that will return something I can easily turn into an >>> ordered list of OIDs. >>> >>> Most solutions I've run across require other languages, servers >>> that add up to needless complexity for my modest application. In >>> the lisp world I've only seen Montezuma, which isn't being >>> developed or seriously maintained (unless it's just really stable >>> I'd rather not fight with stale code). >>> >>> I am considering hacking something simple on top of postmodern >>> that uses the new text indexing functions of Postgresql 8.3 and >>> wondered if anyone here has insight into this application of >>> postmodern or into the full-text indexing from lisp problem in >>> general. >>> >>> Thank you, >>> Ian >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >>> >>> >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Wed May 14 03:16:58 2008 From: read at robertlread.net (Robert L. Read) Date: Tue, 13 May 2008 22:16:58 -0500 Subject: [elephant-devel] full text search solution? In-Reply-To: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu> References: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu> Message-ID: <1210735018.18630.136.camel@penguin.yourdomain.com> About two years ago I tried to use Montezuma and found it very buggy. It may have gotten better. I had better luck using "lucene-webeservice": http://lucene-ws.net/ Which is (obviously) a webservice interface to a normal Lucene installation. This worked pretty well for me. I used Closure-XML to construct and deconstruct the web-services. http://common-lisp.net/project/cxml/ CXML was far and away the best XML manipulator that I tried. One advantage of this solution is that any improvement to Lucene you get automatically, since you are really using Lucene, and not a re-implementation of it. Given the weight behind Lucene, that is going to be hard to beat. The webservice was not a problem for the application that I was working on. However, once could argue that you don't want the overhead of a webservice call. I would still be tempted to use or construct a C/file level interface to Lucene, for the reason that I don't think anyone can outdo Lucene. Clearly, Lucene-ws could be hacked into Lucene-LISPAPI, since it already knows how to deal with Lucene (although through Catalina/Tomcat and so on.). I suspect that Postgres, with its extensible types, would allow a very efficient use of full-text searching. I think this is major plus of postgres (and has been since 7.1, if I'm not mistaken --- though it may even better now.) I agree with you that a full-text searching system should NOT be built into Elephant, on the basis of separation of concerns. On Tue, 2008-05-13 at 22:02 -0400, Ian Eslick wrote: > Hello, > > Has anyone found a good solution for full text search in lisp? I'm > interested in indexing website objects such as posts and perhaps > external documents as well. BDB doesn't, to the best of my > knowledge, have the appropriate building blocks for an efficient > indexing system and you certainly don't want to do it on top of the > current btree interface. > > I have an old full text index code base that supported wildcard and > NEAR queries, all built on top of Elephant btrees. It was convenient > but had a query time that slowed down linearly with the avg # of > documents per word. > > I've decided that the best approach for me is to connect to a > separate, probably external, system to which I can incrementally add > content that will return something I can easily turn into an ordered > list of OIDs. > > Most solutions I've run across require other languages, servers that > add up to needless complexity for my modest application. In the lisp > world I've only seen Montezuma, which isn't being developed or > seriously maintained (unless it's just really stable I'd rather not > fight with stale code). > > I am considering hacking something simple on top of postmodern that > uses the new text indexing functions of Postgresql 8.3 and wondered if > anyone here has insight into this application of postmodern or into > the full-text indexing from lisp problem in general. > > Thank you, > Ian > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Wed May 14 03:18:22 2008 From: read at robertlread.net (Robert L. Read) Date: Tue, 13 May 2008 22:18:22 -0500 Subject: [elephant-devel] full text search solution? In-Reply-To: <721FD699-D250-4864-9296-1735E034FA5B@umich.edu> References: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu> <721FD699-D250-4864-9296-1735E034FA5B@umich.edu> Message-ID: <1210735102.18630.137.camel@penguin.yourdomain.com> On Tue, 2008-05-13 at 22:33 -0400, Glenn Tarcea wrote: > BTW: If its alright I'm going to update the docs to add a section on > having two versions of Elephant installed and using ASDF to manage > them. Please do. From read at robertlread.net Wed May 14 05:46:51 2008 From: read at robertlread.net (Robert L. Read) Date: Wed, 14 May 2008 00:46:51 -0500 Subject: [elephant-devel] Clbuild In-Reply-To: <200805111843.55131.bryan.emrys@gmail.com> References: <200805111422.06254.bryan.emrys@gmail.com> <1210551136.18630.24.camel@penguin.yourdomain.com> <200805111843.55131.bryan.emrys@gmail.com> Message-ID: <1210744011.23948.22.camel@penguin.yourdomain.com> Dear Bryan and Team, I have now removed the explicit load of cl-sql (I think) and put an sbcl pragma around an explicit depend in both the "elephant" darcs branch and the "elephant-unstable" darcs branch. I observe the following comments in the CLBUILD wnpp-projects file, which you mentioned: # endless problems: # - doesn't build at all without a user-written my-config.sexp # - needs sb-posix, but doesn't declare that dependency, meaning that # it doesn't build even with a config file # - ele-clsql loads clsql from the .asd file, not using :depends-on, # which is completely unacceptable # (we blacklist ele-clsql in clbuild.lisp because of that) elephant get_darcs http://www.common-lisp.net/project/elephant/darcs/elephant ... so it is possible that the second and third problem are solved from your point of view. However, the first problem may require some creativity. It seems to me the best we could hope for is to modify our build to use our standard template, "config.sexp", if my-config.sexp doesn't exist, and blare a warning that you probably should understand and set those defaults yourself. If we did that, perhaps on most systems (85%) it would build, but it will remain fragile (not least of all to new revisions of BDB). It might build more reliably for a CL-SQL backend or a postmodern backend, but of course you can't use those systems without knowing at least that your have to have postgres and run "createdb test", for example. Does this sound like a reasonable solution? From eslick at media.mit.edu Wed May 14 06:21:08 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 02:21:08 -0400 Subject: [elephant-devel] Clbuild In-Reply-To: <1210744011.23948.22.camel@penguin.yourdomain.com> References: <200805111422.06254.bryan.emrys@gmail.com> <1210551136.18630.24.camel@penguin.yourdomain.com> <200805111843.55131.bryan.emrys@gmail.com> <1210744011.23948.22.camel@penguin.yourdomain.com> Message-ID: <6403E946-3CF4-49DD-ACBA-2B1BB3C3AA29@media.mit.edu> > However, the first problem may require some creativity. > > It seems to me the best we could hope for is to modify our build to > use our standard template, > "config.sexp", if my-config.sexp doesn't exist, and blare a warning > that you probably should > understand and set those defaults yourself. If we did that, perhaps > on most systems (85%) it > would build, but it will remain fragile (not least of all to new > revisions of BDB). > It might build more reliably for a CL-SQL backend or a postmodern > backend, but of course you > can't use those systems without knowing at least that your have to > have postgres and run "createdb test", > for example. > My suggestions would that the process be: - load the core elephant code without committing to loading a particular data store - if my-config doesn't exist, create it with that big warning sign but continue (only BDB users depend on it, and BDB isn't being loaded as part of the core so they'll run into the errors when they try to open a specific data store) - Make sure the dependencies for elephant-tests is done right - Create an asdf:test-op for a basic set of elephant tests??? Perhaps it complains about my-config.sexp and forces you to set a store type to test with? Questions: - Is it OK with clbuild that we load :ele-bdb, :ele-clsql from within our open-store procedure? - How about the hack in elephant.asd to ensure that uffi is loaded when some of the code in that file is evaluated as part of the c library build procedure? Do we have a problem there? - In general, have you found in clbuild that CL-SQL is a good prototype for a project with multiple backends like ours. Cheers, Ian > Does this sound like a reasonable solution? > > > > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Wed May 14 07:17:01 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Wed, 14 May 2008 09:17:01 +0200 (CEST) Subject: [elephant-devel] Cached slots and other tweaks for 1.0 In-Reply-To: <496F79FB-568C-4B52-9D32-55E5616B7966@media.mit.edu> References: <7E3A31D6-2992-4CC9-9053-FEC18209F84D@media.mit.edu> <61109.88.73.214.122.1210602211.squirrel@mail.stardawn.org> <496F79FB-568C-4B52-9D32-55E5616B7966@media.mit.edu> Message-ID: <62870.88.73.240.7.1210749421.squirrel@mail.stardawn.org> > Also, cached slots are not currently thread safe. If you don't > guarantee that only one thread at a time is working on a particular > object, you can get problems. These slot values are not isolated by > the transaction mechanism. I'm not sure we should solve that; the > protection for cached slots should be at a higher level so it doesn't > swamp the benefits. I don't know what the best decision might be here. But I have a use case that might help; it has the following features: * I access the slots of two persistent objects. * The number of the slots and the times requested together produce very bad performance (think seconds) even with PM txn caching (for comparison, BDB is about three times faster) * The environment is multi-threaded (web server), but the slots won't be changed by any other process. * Ideally the slots would be cached only for this one function and the functions called by it (and only per-invocation, i.e. slot caches get refreshed right at the beginning of the function). * This is currently the only place in my app where I would need the performance advantages of slot caching. In all other places ACID is highly preferred and speed is sufficient. * The desired behaviour can be somewhat modelled by CLSQL's OO interface: - get the objects from the DB at the beginning - work with those in-memory objects - write back the values to the DB at the end of the process The difference is that I don't want the whole object (other slot values of it might be changed from outside!) but only a few selected slots. Leslie From leslie.polzer at gmx.net Wed May 14 07:28:35 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Wed, 14 May 2008 09:28:35 +0200 (CEST) Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> Message-ID: <63743.88.73.240.7.1210750115.squirrel@mail.stardawn.org> > For example, in BDB, the primitive is the page. BTree nodes are layed > out in one or more pages, each page has some binary metadata > explaining it's type and organization (free list, etc). A short key > value is written directly into the page, a long one is written into an > overflow page, etc. InnoDB also uses this approach. > - new keys and values of differing or less length are written in > place, otherwise new > space is allocated at the end of the file. Okay, but maybe let the user reserve extent space. > - transactions simply keep track of the primitive operations on the > database and the associated data in a memory queue and write those ops > to disk as part of the txn commit process. The pages and key/value > pairs that will be touched in that operation are also stored in that > txn log. > - when a transaction commits, it replays the log to write everything > to disk appropriately. The list of touched data is then passed up the > commit chain to invalidate any pending transactions that have a > conflict. Everything is speculative in this case, but we don't have > to deal with locking. I like this approach. > A big decision is: > - Use cffi/uffi and do much of the serialization & btree > implementation in C/static memory IME this FFI stuff can be quite hard to debug. > I lean towards using cffi to manipulate static data, just because it's > going to be easier to get performance via that method and it's also > going to be much easier to do a multi-process implementation (by > operating on static memory and primitive locks in a shared memory > region). I cannot estimate the performance trade-offs involved here, but in general I'm in favor of a Lisp-based approach... > - the binary pages could be stored in static data I don't think I understand this. What does "static" mean here (and above)? > and the primitives btree ops > could directly manipulate data within the page? We pass a C > function that > directly sorts binary sequences rather than having to deserialize > to sort. We'd > need to write that in lisp to operate on static data or on lisp > arrays. Deserializing > on each key comparison is too expensive. Yes. > A nice auxiliary hack would be: > - rewrite memutils to entirely use uffi/cffi to manipulate static data > rather > than calling out to C to do it. Maintains efficiency but removes > the compilation > build step except for users of BDB I have looked at memutils/the serializer, but it's very hard for me to replace it by something else, because I'm not sure what would be required of a replacement. The partial conclusion to which I came was that memutils just models a bivalent stream so the backend can communicate with the serializer. The data format is hard to figure out for me, however, because of all the UFFI stuff involved... Leslie From leslie.polzer at gmx.net Wed May 14 07:32:02 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Wed, 14 May 2008 09:32:02 +0200 (CEST) Subject: [elephant-devel] open-controller for bdb doesn't accept a pathname spec In-Reply-To: <19e19e410805131436x74cec081mb332b2389b5efc51@mail.gmail.com> References: <19e19e410805130641j39e83052q3094f5d62db5a2a9@mail.gmail.com> <5126A351-3B94-48B3-B857-7CA8273B664B@media.mit.edu> <19e19e410805131436x74cec081mb332b2389b5efc51@mail.gmail.com> Message-ID: <64043.88.73.240.7.1210750322.squirrel@mail.stardawn.org> > BTW there are some aspects of the tests system I fail to understand. > For example, elephant-tests fails to compile > (tests/testconcurrency.lisp wants to use package BT, which cannot be > found). BT is an alias for BORDEAUX-THREADS, which should be included by ASDF. If it isn't we got a bug. Leslie From eslick at media.mit.edu Wed May 14 13:07:13 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 09:07:13 -0400 Subject: [elephant-devel] open-controller for bdb doesn't accept a pathname spec In-Reply-To: <64043.88.73.240.7.1210750322.squirrel@mail.stardawn.org> References: <19e19e410805130641j39e83052q3094f5d62db5a2a9@mail.gmail.com> <5126A351-3B94-48B3-B857-7CA8273B664B@media.mit.edu> <19e19e410805131436x74cec081mb332b2389b5efc51@mail.gmail.com> <64043.88.73.240.7.1210750322.squirrel@mail.stardawn.org> Message-ID: <4C0E8BB3-4962-4608-B0F8-114300D09F9F@media.mit.edu> It used to be included in elephant.asd but should actually be in elephant-tests.asd since the core elephant doesn't depend on it. Ian On May 14, 2008, at 3:32 AM, Leslie P. Polzer wrote: > >> BTW there are some aspects of the tests system I fail to understand. >> For example, elephant-tests fails to compile >> (tests/testconcurrency.lisp wants to use package BT, which cannot be >> found). > > BT is an alias for BORDEAUX-THREADS, which should be included by ASDF. > If it isn't we got a bug. > > Leslie > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From gtarcea at umich.edu Wed May 14 13:11:05 2008 From: gtarcea at umich.edu (V. Glenn Tarcea) Date: Wed, 14 May 2008 09:11:05 -0400 Subject: [elephant-devel] full text search solution? In-Reply-To: <0F978366-F34F-4F91-8554-474DED798997@media.mit.edu> References: <313491AD-0655-4357-AD24-BA6249C6C427@media.mit.edu><721FD699-D250-4864-9296-1735E034FA5B@umich.edu> <0F978366-F34F-4F91-8554-474DED798997@media.mit.edu> Message-ID: <035f01c8b5c3$f8485800$4212a8c0@mimi2> Unfortunately I do not. It's been on my list, but I've also been concerned about project support. I was heartened to see it listed for the Google Summer of Code, and that write up was about features, rather than stabilization. That said, Lucene is under active development, where as Montezuma is tepid at best. Another approach is JFLI http://jfli.sourceforge.net/ which gives you Lisp bindings to Java. I *believe* though that it is LispWorks specific, so this may not work. It would be nice to see resurgence in Lisp. Glenn -----Original Message----- From: elephant-devel-bounces at common-lisp.net [mailto:elephant-devel-bounces at common-lisp.net] On Behalf Of Ian Eslick Sent: Tuesday, May 13, 2008 11:10 PM To: Elephant bugs and development Subject: Re: [elephant-devel] full text search solution? Thanks Glenn, Do you have experience with Montezuma? Ian On May 13, 2008, at 10:43 PM, Glenn Tarcea wrote: > Never mind, I see from reading that you already found Monezuma.... > But it does look like it might get some activity. > > Glenn > > On May 13, 2008, at 10:33 PM, Glenn Tarcea wrote: > >> Have you looked at Montezuma? http://projects.heavymeta.org/ >> montezuma/ >> >> It's a port of Ferret (which is port of Lucene) to Lisp. It's also >> on the list for the google summer of code to add some additional >> functionality to it. >> >> BTW: If its alright I'm going to update the docs to add a section >> on having two versions of Elephant installed and using ASDF to >> manage them. >> >> Glenn >> >> >> On May 13, 2008, at 10:02 PM, Ian Eslick wrote: >> >>> Hello, >>> >>> Has anyone found a good solution for full text search in lisp? >>> I'm interested in indexing website objects such as posts and >>> perhaps external documents as well. BDB doesn't, to the best of >>> my knowledge, have the appropriate building blocks for an >>> efficient indexing system and you certainly don't want to do it on >>> top of the current btree interface. >>> >>> I have an old full text index code base that supported wildcard >>> and NEAR queries, all built on top of Elephant btrees. It was >>> convenient but had a query time that slowed down linearly with the >>> avg # of documents per word. >>> >>> I've decided that the best approach for me is to connect to a >>> separate, probably external, system to which I can incrementally >>> add content that will return something I can easily turn into an >>> ordered list of OIDs. >>> >>> Most solutions I've run across require other languages, servers >>> that add up to needless complexity for my modest application. In >>> the lisp world I've only seen Montezuma, which isn't being >>> developed or seriously maintained (unless it's just really stable >>> I'd rather not fight with stale code). >>> >>> I am considering hacking something simple on top of postmodern >>> that uses the new text indexing functions of Postgresql 8.3 and >>> wondered if anyone here has insight into this application of >>> postmodern or into the full-text indexing from lisp problem in >>> general. >>> >>> Thank you, >>> Ian >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >>> >>> >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel _______________________________________________ elephant-devel site list elephant-devel at common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Wed May 14 13:40:40 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 09:40:40 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <63743.88.73.240.7.1210750115.squirrel@mail.stardawn.org> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <63743.88.73.240.7.1210750115.squirrel@mail.stardawn.org> Message-ID: <733ED1ED-D102-4283-85F7-B946A5F62097@media.mit.edu> On May 14, 2008, at 3:28 AM, Leslie P. Polzer wrote: > >> For example, in BDB, the primitive is the page. BTree nodes are >> layed >> out in one or more pages, each page has some binary metadata >> explaining it's type and organization (free list, etc). A short key >> value is written directly into the page, a long one is written into >> an >> overflow page, etc. > > InnoDB also uses this approach. There is a massive body of work and many variations on ways to lay out indexing structures and data on disk. The tradeoffs depend on your specific needs. My recommendation is we don't get ambitious and stick with a simple BTree or perhaps B+Tree for now. If we abstract properly, we should be able to replace the underlying page storage mechanism later. > >> - new keys and values of differing or less length are written in >> place, otherwise new >> space is allocated at the end of the file. > > Okay, but maybe let the user reserve extent space. > >> - transactions simply keep track of the primitive operations on the >> database and the associated data in a memory queue and write those >> ops >> to disk as part of the txn commit process. The pages and key/value >> pairs that will be touched in that operation are also stored in that >> txn log. >> - when a transaction commits, it replays the log to write everything >> to disk appropriately. The list of touched data is then passed up >> the >> commit chain to invalidate any pending transactions that have a >> conflict. Everything is speculative in this case, but we don't have >> to deal with locking. > > I like this approach. > > >> A big decision is: >> - Use cffi/uffi and do much of the serialization & btree >> implementation in C/static memory > > IME this FFI stuff can be quite hard to debug. That's true, but the CFFI operations would basically be the primitive set that are already used in memutils to implement buffer streams. > >> I lean towards using cffi to manipulate static data, just because >> it's >> going to be easier to get performance via that method and it's also >> going to be much easier to do a multi-process implementation (by >> operating on static memory and primitive locks in a shared memory >> region). > > I cannot estimate the performance trade-offs involved here, > but in general I'm in favor of a Lisp-based approach... I think if we abstract correctly with a more direct approach in mind, we could always go back and do it the other way later. Probably best to start with lisp, it just may mean more work in the meantime to rewrite the functionality in memutils and serializer2. > >> - the binary pages could be stored in static data > > I don't think I understand this. What does "static" mean here > (and above)? Static data just means that allocated from the C heap, not the lisp garbage collector. Lisp data structures > >> and the primitives btree ops >> could directly manipulate data within the page? We pass a C >> function that >> directly sorts binary sequences rather than having to deserialize >> to sort. We'd >> need to write that in lisp to operate on static data or on lisp >> arrays. Deserializing >> on each key comparison is too expensive. > > Yes. > > >> A nice auxiliary hack would be: >> - rewrite memutils to entirely use uffi/cffi to manipulate static >> data >> rather >> than calling out to C to do it. Maintains efficiency but removes >> the compilation >> build step except for users of BDB > > I have looked at memutils/the serializer, but it's very hard for me > to replace it by something else, because I'm not sure what would be > required of a replacement. > > The partial conclusion to which I came was that memutils just models > a bivalent stream so the backend can communicate with the serializer. That's essentially correct, although memutils was written to implement buffer streams which are then used by the serializer - in effect it's the serializer that uses memutils to send serialized buffers to the data stores. It was originally designed for BDB and has the benefit of avoiding an extra copy step when talking to the BDB API. If you are serializing structures into a lisp array, to pass them to C you need to copy them to a C array, and BDB then copies that array into the appropriate cached page and ultimately writes that page to disk. Using memutils as-is is actually more expensive for a lisp-only solution if we're storing pages in lisp arrays (which hopefully will quickly become tenured and stop being copied around by the collector). We would then write into a C array with memutils, copy into a lisp array and write that to disk rather than writing directly to the lisp array. > The data format is hard to figure out for me, however, because of > all the UFFI stuff involved... The serializer is probably fine as it is from a functional standpoint, but to do it in lisp we'll need to change out all the primitives it uses (buffer-write-int32, buffer-write-byte, etc). buffers streams are wrappers around C arrays (defstruct buffer-stream "A stream-like interface to foreign (alien) char buffers." (buffer (allocate-foreign-object :unsigned-char 10) :type array-or- pointer-char) (size 0 :type fixnum) (position 0 :type fixnum) (length 10 :type fixnum)) We allocate that object directly from the C heap (just like malloc) using the allocate-foreign-object call, this is currently in uffi and using the uffi compat layer from cffi leads to errors. Size is the current size of valid data in the array (a write pointer). The position is the current read pointer and the length is the total size of the allocated region. If I write two int32's into the stream the values above are: size 8, position 0, length 10 If the BTree node binary data is stored in a lisp octet array, and we don't want to deserialize to compare keys, then we need to write the procedure that you can find in libberkeley-db.c in the src/db-bdb file. It performs a content-sensitive comparison on a byte-by-byte basis. This has huge benefits of allowing us to do key comparisons 'in-place' without creating lisp objects. We pass this C function to BDB which uses it to compare keys and values directly in the binary file pages. I tell you what, if we wrap an abstraction around a lisp paged btree approach, we can cheat with memutils for the time being and replace it later. Every field written to a binary page should have an extent associated with it. To compare keys, we copy that extent to a buffer stream, run the comparison operation in the existing memutils, etc. To serialize/deserialize, we just copy to/from the lisp binary page and the buffer stream abstraction. To lookup data in a page you would do something like: serialize key to a buffer stream (bs1) database file => read fixed sized page into an array page => copy serialized key to buffer stream (bs2) compare bs1 and bs2 look at btree page to decide what next key to compare or, copy the value to a buffer stream bs3 and run the deserializer. Later this should be something like: serialize key to a lisp array database file => read fixed sized page into an array byte-by-byte comparison of key in lisp array to a field at offset + length in the binary page on success, deserialize from the offset + length field in the binary page. I'm sorry if this is confusing. I have some code dealing with fixed pools of binary pages and some attempts at a btree and log implementation in src/contrib/db-lisp. Rucksack is also an excellent source of ideas. the Rucksack persistence model is too different from elephant for me to be comfortable adapting it, but it has a much more elegant serializer and an all-in-lisp implementation of btrees (as I recall). That is another great place to start getting ideas. Ian > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Wed May 14 13:53:54 2008 From: read at robertlread.net (Robert L. Read) Date: Wed, 14 May 2008 08:53:54 -0500 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> Message-ID: <1210773234.23948.47.camel@penguin.yourdomain.com> I am certainly in favor of using fixed-size pages as a basic model. I think this is very common. It has the advantage of allowing page-based caching more easily. One can even contemplate things like a RAID-style striping across machines on a LAN or WAN. I think we should create a really pure-lisp solution, that doesn't on C in any way. Certainly we should do that first. However, I have personal reason for preferring a completely LISP-based serializer: I think there is a lot of opportunity for major performance enhancements by making the serializer more efficient---by which I mean producing very compressed representations of the data serialized. For example, if we could produce a 10-fold reduction in the size of the compressed data, we would have a game-changing improvement, which would probably make us much faster than BDB or any relational system. If that is true, then almost any amount of CPU time spent in the codec is warranted. A 10-fold improvement may seem unlikely until one stops to consider the applicability of Huffman coding of English strings, prefix-forward-index encoding of ordered strings, Huffman encoding on our little type-designators, the possibility of keeping a dictionary in a cached page for LZW-style compression, the possibility of detecting a Poisson distribution of integer values by declaration or statistics, etc. However, I think this is a very "researchy" line of thought. I would like to have the ability to explore these ideas by writing them up in LISP code, even though they definitely would represent premature optimization until we had a basic B-Tree working. It's not 100% clear to me that C would offer a more performant solution in any case. It is probably true that a high-performance serializer would have to be tuned to various LISP implementations. On Tue, 2008-05-13 at 15:46 -0400, Ian Eslick wrote: > I think they key decision was what serialization format we're going to > use for btree nodes, log entries, etc and how that relates to caching > data during transactions, maintaining empty lists, etc. > > The current serializer produces a byte sequence. If we continue with > that model, how do we write/read this stuff from disk? How and where > do we store it prior to committing a transaction? > > When we create a new key or value as a binary stream within a > transaction, how is it stored in memory? If we want a multi-process, > but non-socket based approach, we need to figure out how to store data > in shared memory, etc. > > For example, in BDB, the primitive is the page. BTree nodes are layed > out in one or more pages, each page has some binary metadata > explaining it's type and organization (free list, etc). A short key > value is written directly into the page, a long one is written into an > overflow page, etc. Lots of details to deal with in managing variable > sized data on disk. Pages that are dirty are kept in memory (which is > why BDB can run out of transaction space; the pages overflow the max > cache size when you are writing lots of data). > > > However, to get started, the easiest thing is to reuse the existing > memutils serializer, not worry about multi-process operation and not > worry about fragmentation, sharing space and maintaining free lists > (except perhaps for btree nodes). > > Something like: > - btree nodes only keep pointers to variable sized keys stored > elsewhere in the file > - new keys and values of differing or less length are written in > place, otherwise new > space is allocated at the end of the file. > - btree nodes are a fixed size page on-disk and keep some free-list > information so we can reuse them. > - transactions simply keep track of the primitive operations on the > database and the associated data in a memory queue and write those ops > to disk as part of the txn commit process. The pages and key/value > pairs that will be touched in that operation are also stored in that > txn log. > - when a transaction commits, it replays the log to write everything > to disk appropriately. The list of touched data is then passed up the > commit chain to invalidate any pending transactions that have a > conflict. Everything is speculative in this case, but we don't have > to deal with locking. > > This is a nice balance between some lisp-sexp serialization format > that performs poorly, and a highly-optimized low-level implementation > which is blindingly fast. > > A big decision is: > - Use cffi/uffi and do much of the serialization & btree > implementation in C/static memory > or do all of this in pools of static arrays and write a new > serializer to operate on lisp data. > > I lean towards using cffi to manipulate static data, just because it's > going to be easier to get performance via that method and it's also > going to be much easier to do a multi-process implementation (by > operating on static memory and primitive locks in a shared memory > region). > > Predicated on that decision, getting started on the simplest possible > btree/dup-btree implementation is the next, most valuable and > educational step. > > The key pieces for a single-process lisp backend: > - btrees and dup-btrees (indices can be built from these two easily > enough) > - the binary pages could be stored in static data and the > primitives btree ops > could directly manipulate data within the page? We pass a C > function that > directly sorts binary sequences rather than having to deserialize > to sort. We'd > need to write that in lisp to operate on static data or on lisp > arrays. Deserializing > on each key comparison is too expensive. > - a set of transaction records (lisp structs and consts) > - simply keeps tuples of (op {rd | wr} {btree+page-offset | value- > offset} [values]) > in a memory queue. Could use static memory for this to reduce > load on GC > - a blocking primitive library that serializes txn commits > (i.e. write log to disk, write data to disk, write 'commit done' to > log, > invalidate pending/conflicting txns) > > A nice auxiliary hack would be: > - rewrite memutils to entirely use uffi/cffi to manipulate static data > rather > than calling out to C to do it. Maintains efficiency but removes > the compilation > build step except for users of BDB > > > So what do people think about the cffi->static-data vs. lisp->array- > pool decision? > > > Ian > > On May 13, 2008, at 2:03 PM, Leslie P. Polzer wrote: > > > > > I suppose the "binary paging" approach mentioned in the design > > considerations > > document means the problem of organizing the data efficiently on disk > > right from the start. Is this correct? > > > > Do you think it would make good sense to start working on the btree > > library > > without thinking much about on-disk efficiency, leaving this part > > for later? > > > > I'm not sure a btree where on-disk storage organization is separeted > > from the > > rest like that can achieve enough efficiency... > > > > Thanks, > > > > Leslie > > > > _______________________________________________ > > elephant-devel site list > > elephant-devel at common-lisp.net > > http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Wed May 14 13:55:45 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 09:55:45 -0400 Subject: [elephant-devel] Cached slots and other tweaks for 1.0 In-Reply-To: <62870.88.73.240.7.1210749421.squirrel@mail.stardawn.org> References: <7E3A31D6-2992-4CC9-9053-FEC18209F84D@media.mit.edu> <61109.88.73.214.122.1210602211.squirrel@mail.stardawn.org> <496F79FB-568C-4B52-9D32-55E5616B7966@media.mit.edu> <62870.88.73.240.7.1210749421.squirrel@mail.stardawn.org> Message-ID: <013B9843-1215-4C96-A8A6-3716EEEECFCF@media.mit.edu> That might be a better approach than what I have now. I'll think about this. Thank you for the suggestion! Ian On May 14, 2008, at 3:17 AM, Leslie P. Polzer wrote: > >> Also, cached slots are not currently thread safe. If you don't >> guarantee that only one thread at a time is working on a particular >> object, you can get problems. These slot values are not isolated by >> the transaction mechanism. I'm not sure we should solve that; the >> protection for cached slots should be at a higher level so it doesn't >> swamp the benefits. > > I don't know what the best decision might be here. > But I have a use case that might help; it has the following > features: > > * I access the slots of two persistent objects. > > * The number of the slots and the times requested > together produce very bad performance (think seconds) > even with PM txn caching (for comparison, BDB is about > three times faster) > > * The environment is multi-threaded (web server), but the > slots won't be changed by any other process. > > * Ideally the slots would be cached only for this one > function and the functions called by it (and only > per-invocation, i.e. slot caches get refreshed right at > the beginning of the function). > > * This is currently the only place in my app where I would > need the performance advantages of slot caching. In all > other places ACID is highly preferred and speed is sufficient. > > * The desired behaviour can be somewhat modelled by CLSQL's > OO interface: > > - get the objects from the DB at the beginning > > - work with those in-memory objects > > - write back the values to the DB at the end of the process > > The difference is that I don't want the whole object (other slot > values of it might be changed from outside!) but only a few > selected slots. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Wed May 14 14:02:05 2008 From: read at robertlread.net (Robert L. Read) Date: Wed, 14 May 2008 09:02:05 -0500 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <733ED1ED-D102-4283-85F7-B946A5F62097@media.mit.edu> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <63743.88.73.240.7.1210750115.squirrel@mail.stardawn.org> <733ED1ED-D102-4283-85F7-B946A5F62097@media.mit.edu> Message-ID: <1210773725.23948.55.camel@penguin.yourdomain.com> On Wed, 2008-05-14 at 09:40 -0400, Ian Eslick wrote: > If the BTree node binary data is stored in a lisp octet array, and > we > don't want to deserialize to compare keys, then we need to write the > procedure that you can find in libberkeley-db.c in the src/db-bdb > file. It performs a content-sensitive comparison on a byte-by-byte > basis. This has huge benefits of allowing us to do key comparisons > 'in-place' without creating lisp objects. We pass this C function > to > BDB which uses it to compare keys and values directly in the binary > file pages. I understand the value of this, but is it not possible to write the comparator on serialized values in LISP? This would then be passed down to the lowest level IO operation, which is still in LISP. In general I'm imagining a system that does just that as its "query language", --- take a very lisp-like specification of a predicate and pass it down to the lowest level I/O operation, or as low as you can get, and apply it there. This allows us to retain the elegant "mapcar" and "map-reduce" strategies as basic querying mechanisms. If we make a modular serializer, that serializes on the basis of data type, then it is indeed a challenge for the implementor to implement a function that performs comparisons on the serialized representation of that type...but a very worthwhile challenge, I think. From eslick at media.mit.edu Wed May 14 14:05:35 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 10:05:35 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <1210773234.23948.47.camel@penguin.yourdomain.com> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <1210773234.23948.47.camel@penguin.yourdomain.com> Message-ID: <50E56DF4-358C-4B02-9545-7DF8B9D221C0@media.mit.edu> Here is an interesting discussion of using lisp for a large, high- performance system and the reasons to use static/C data. I wasn't advocating using C at all, and in fact removing all dependency on it for the lisp-only backend. With cffi or uffi you can get direct access to memory primitives like allocating data and reading/writing pointers without needing a C compiler or library. The only difference is that you aren't using lisp heap data structures. I'm all for performance improvements and agree that there is reasonable compressibility in much of the data that we generate, but given that disk space is virtually free these days and that you are increasing time, perhaps significantly, to save space (i.e. you now have to decompress each key prior to doing an ordering comparison unless you are being very format sensitive like having a global tokenization of language strings). I'm supportive, but skeptical. :) Perhaps there would be a way to hook into the comparison function and provide hints to the serializer so that it generates a tagged, compressed format that a custom comparison operator can dispatch on and the user/system hacker could write these domain or format specific extensions of the serializer. One big point from all this is that we can't do this on other backends because BDB is doing the comparisons internally and it needs a C function which performs ordering comparisons. We don't want to artificially instill this restriction in an all-lisp backend. Ian On May 14, 2008, at 9:53 AM, Robert L. Read wrote: > I am certainly in favor of using fixed-size pages as a basic model. I > think this is very common. It has the advantage of allowing page- > based > caching more easily. One can even contemplate things like a RAID- > style > striping across machines on a LAN or WAN. > > I think we should create a really pure-lisp solution, that doesn't > on C > in any way. Certainly we should do that first. > > However, I have personal reason for preferring a completely LISP-based > serializer: I think there is a lot of opportunity for major > performance > enhancements by making the serializer more efficient---by which I mean > producing very compressed representations of the data serialized. For > example, if we could produce a 10-fold reduction in the size of the > compressed data, we would have a game-changing improvement, which > would > probably make us much faster than BDB or any relational system. If > that > is true, then almost any amount of CPU time spent in the codec is > warranted. > > A 10-fold improvement may seem unlikely until one stops to consider > the > applicability of Huffman coding of English strings, prefix-forward- > index > encoding of ordered strings, Huffman encoding on our little > type-designators, the possibility of keeping a dictionary in a cached > page for LZW-style compression, the possibility of detecting a Poisson > distribution of integer values by declaration or statistics, etc. > However, I think this is a very "researchy" line of thought. I would > like to have the ability to explore these ideas by writing them up in > LISP code, even though they definitely would represent premature > optimization until we had a basic B-Tree working. > > It's not 100% clear to me that C would offer a more performant > solution > in any case. It is probably true that a high-performance serializer > would have to be tuned to various LISP implementations. > > > > On Tue, 2008-05-13 at 15:46 -0400, Ian Eslick wrote: >> I think they key decision was what serialization format we're going >> to >> use for btree nodes, log entries, etc and how that relates to caching >> data during transactions, maintaining empty lists, etc. >> >> The current serializer produces a byte sequence. If we continue with >> that model, how do we write/read this stuff from disk? How and where >> do we store it prior to committing a transaction? >> >> When we create a new key or value as a binary stream within a >> transaction, how is it stored in memory? If we want a multi-process, >> but non-socket based approach, we need to figure out how to store >> data >> in shared memory, etc. >> >> For example, in BDB, the primitive is the page. BTree nodes are >> layed >> out in one or more pages, each page has some binary metadata >> explaining it's type and organization (free list, etc). A short key >> value is written directly into the page, a long one is written into >> an >> overflow page, etc. Lots of details to deal with in managing >> variable >> sized data on disk. Pages that are dirty are kept in memory (which >> is >> why BDB can run out of transaction space; the pages overflow the max >> cache size when you are writing lots of data). >> >> >> However, to get started, the easiest thing is to reuse the existing >> memutils serializer, not worry about multi-process operation and not >> worry about fragmentation, sharing space and maintaining free lists >> (except perhaps for btree nodes). >> >> Something like: >> - btree nodes only keep pointers to variable sized keys stored >> elsewhere in the file >> - new keys and values of differing or less length are written in >> place, otherwise new >> space is allocated at the end of the file. >> - btree nodes are a fixed size page on-disk and keep some free-list >> information so we can reuse them. >> - transactions simply keep track of the primitive operations on the >> database and the associated data in a memory queue and write those >> ops >> to disk as part of the txn commit process. The pages and key/value >> pairs that will be touched in that operation are also stored in that >> txn log. >> - when a transaction commits, it replays the log to write everything >> to disk appropriately. The list of touched data is then passed up >> the >> commit chain to invalidate any pending transactions that have a >> conflict. Everything is speculative in this case, but we don't have >> to deal with locking. >> >> This is a nice balance between some lisp-sexp serialization format >> that performs poorly, and a highly-optimized low-level implementation >> which is blindingly fast. >> >> A big decision is: >> - Use cffi/uffi and do much of the serialization & btree >> implementation in C/static memory >> or do all of this in pools of static arrays and write a new >> serializer to operate on lisp data. >> >> I lean towards using cffi to manipulate static data, just because >> it's >> going to be easier to get performance via that method and it's also >> going to be much easier to do a multi-process implementation (by >> operating on static memory and primitive locks in a shared memory >> region). >> >> Predicated on that decision, getting started on the simplest possible >> btree/dup-btree implementation is the next, most valuable and >> educational step. >> >> The key pieces for a single-process lisp backend: >> - btrees and dup-btrees (indices can be built from these two easily >> enough) >> - the binary pages could be stored in static data and the >> primitives btree ops >> could directly manipulate data within the page? We pass a C >> function that >> directly sorts binary sequences rather than having to deserialize >> to sort. We'd >> need to write that in lisp to operate on static data or on lisp >> arrays. Deserializing >> on each key comparison is too expensive. >> - a set of transaction records (lisp structs and consts) >> - simply keeps tuples of (op {rd | wr} {btree+page-offset | value- >> offset} [values]) >> in a memory queue. Could use static memory for this to reduce >> load on GC >> - a blocking primitive library that serializes txn commits >> (i.e. write log to disk, write data to disk, write 'commit done' to >> log, >> invalidate pending/conflicting txns) >> >> A nice auxiliary hack would be: >> - rewrite memutils to entirely use uffi/cffi to manipulate static >> data >> rather >> than calling out to C to do it. Maintains efficiency but removes >> the compilation >> build step except for users of BDB >> >> >> So what do people think about the cffi->static-data vs. lisp->array- >> pool decision? >> >> >> Ian >> >> On May 13, 2008, at 2:03 PM, Leslie P. Polzer wrote: >> >>> >>> I suppose the "binary paging" approach mentioned in the design >>> considerations >>> document means the problem of organizing the data efficiently on >>> disk >>> right from the start. Is this correct? >>> >>> Do you think it would make good sense to start working on the btree >>> library >>> without thinking much about on-disk efficiency, leaving this part >>> for later? >>> >>> I'm not sure a btree where on-disk storage organization is separeted >>> from the >>> rest like that can achieve enough efficiency... >>> >>> Thanks, >>> >>> Leslie >>> >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Wed May 14 14:13:14 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 10:13:14 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <1210773234.23948.47.camel@penguin.yourdomain.com> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <1210773234.23948.47.camel@penguin.yourdomain.com> Message-ID: Here is the reference: http://www.paulgraham.com/carl.html I was quite wrong-headed on the performance point. The biggest time cost in database operations is time spent moving data to/from disk, in comparison CPU time is quite small. I was thinking about the internal copying overhead, which is still quite small. Oracle and others have had good success with compression. To get the juices flowing, this is a very clear (although a bit dated) articulation of the ideas: http://web.cecs.pdx.edu/~len/compression.pdf Ian On May 14, 2008, at 9:53 AM, Robert L. Read wrote: > I am certainly in favor of using fixed-size pages as a basic model. I > think this is very common. It has the advantage of allowing page- > based > caching more easily. One can even contemplate things like a RAID- > style > striping across machines on a LAN or WAN. > > I think we should create a really pure-lisp solution, that doesn't > on C > in any way. Certainly we should do that first. > > However, I have personal reason for preferring a completely LISP-based > serializer: I think there is a lot of opportunity for major > performance > enhancements by making the serializer more efficient---by which I mean > producing very compressed representations of the data serialized. For > example, if we could produce a 10-fold reduction in the size of the > compressed data, we would have a game-changing improvement, which > would > probably make us much faster than BDB or any relational system. If > that > is true, then almost any amount of CPU time spent in the codec is > warranted. > > A 10-fold improvement may seem unlikely until one stops to consider > the > applicability of Huffman coding of English strings, prefix-forward- > index > encoding of ordered strings, Huffman encoding on our little > type-designators, the possibility of keeping a dictionary in a cached > page for LZW-style compression, the possibility of detecting a Poisson > distribution of integer values by declaration or statistics, etc. > However, I think this is a very "researchy" line of thought. I would > like to have the ability to explore these ideas by writing them up in > LISP code, even though they definitely would represent premature > optimization until we had a basic B-Tree working. > > It's not 100% clear to me that C would offer a more performant > solution > in any case. It is probably true that a high-performance serializer > would have to be tuned to various LISP implementations. > > > > On Tue, 2008-05-13 at 15:46 -0400, Ian Eslick wrote: >> I think they key decision was what serialization format we're going >> to >> use for btree nodes, log entries, etc and how that relates to caching >> data during transactions, maintaining empty lists, etc. >> >> The current serializer produces a byte sequence. If we continue with >> that model, how do we write/read this stuff from disk? How and where >> do we store it prior to committing a transaction? >> >> When we create a new key or value as a binary stream within a >> transaction, how is it stored in memory? If we want a multi-process, >> but non-socket based approach, we need to figure out how to store >> data >> in shared memory, etc. >> >> For example, in BDB, the primitive is the page. BTree nodes are >> layed >> out in one or more pages, each page has some binary metadata >> explaining it's type and organization (free list, etc). A short key >> value is written directly into the page, a long one is written into >> an >> overflow page, etc. Lots of details to deal with in managing >> variable >> sized data on disk. Pages that are dirty are kept in memory (which >> is >> why BDB can run out of transaction space; the pages overflow the max >> cache size when you are writing lots of data). >> >> >> However, to get started, the easiest thing is to reuse the existing >> memutils serializer, not worry about multi-process operation and not >> worry about fragmentation, sharing space and maintaining free lists >> (except perhaps for btree nodes). >> >> Something like: >> - btree nodes only keep pointers to variable sized keys stored >> elsewhere in the file >> - new keys and values of differing or less length are written in >> place, otherwise new >> space is allocated at the end of the file. >> - btree nodes are a fixed size page on-disk and keep some free-list >> information so we can reuse them. >> - transactions simply keep track of the primitive operations on the >> database and the associated data in a memory queue and write those >> ops >> to disk as part of the txn commit process. The pages and key/value >> pairs that will be touched in that operation are also stored in that >> txn log. >> - when a transaction commits, it replays the log to write everything >> to disk appropriately. The list of touched data is then passed up >> the >> commit chain to invalidate any pending transactions that have a >> conflict. Everything is speculative in this case, but we don't have >> to deal with locking. >> >> This is a nice balance between some lisp-sexp serialization format >> that performs poorly, and a highly-optimized low-level implementation >> which is blindingly fast. >> >> A big decision is: >> - Use cffi/uffi and do much of the serialization & btree >> implementation in C/static memory >> or do all of this in pools of static arrays and write a new >> serializer to operate on lisp data. >> >> I lean towards using cffi to manipulate static data, just because >> it's >> going to be easier to get performance via that method and it's also >> going to be much easier to do a multi-process implementation (by >> operating on static memory and primitive locks in a shared memory >> region). >> >> Predicated on that decision, getting started on the simplest possible >> btree/dup-btree implementation is the next, most valuable and >> educational step. >> >> The key pieces for a single-process lisp backend: >> - btrees and dup-btrees (indices can be built from these two easily >> enough) >> - the binary pages could be stored in static data and the >> primitives btree ops >> could directly manipulate data within the page? We pass a C >> function that >> directly sorts binary sequences rather than having to deserialize >> to sort. We'd >> need to write that in lisp to operate on static data or on lisp >> arrays. Deserializing >> on each key comparison is too expensive. >> - a set of transaction records (lisp structs and consts) >> - simply keeps tuples of (op {rd | wr} {btree+page-offset | value- >> offset} [values]) >> in a memory queue. Could use static memory for this to reduce >> load on GC >> - a blocking primitive library that serializes txn commits >> (i.e. write log to disk, write data to disk, write 'commit done' to >> log, >> invalidate pending/conflicting txns) >> >> A nice auxiliary hack would be: >> - rewrite memutils to entirely use uffi/cffi to manipulate static >> data >> rather >> than calling out to C to do it. Maintains efficiency but removes >> the compilation >> build step except for users of BDB >> >> >> So what do people think about the cffi->static-data vs. lisp->array- >> pool decision? >> >> >> Ian >> >> On May 13, 2008, at 2:03 PM, Leslie P. Polzer wrote: >> >>> >>> I suppose the "binary paging" approach mentioned in the design >>> considerations >>> document means the problem of organizing the data efficiently on >>> disk >>> right from the start. Is this correct? >>> >>> Do you think it would make good sense to start working on the btree >>> library >>> without thinking much about on-disk efficiency, leaving this part >>> for later? >>> >>> I'm not sure a btree where on-disk storage organization is separeted >>> from the >>> rest like that can achieve enough efficiency... >>> >>> Thanks, >>> >>> Leslie >>> >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Wed May 14 14:20:57 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 10:20:57 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <1210773725.23948.55.camel@penguin.yourdomain.com> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <63743.88.73.240.7.1210750115.squirrel@mail.stardawn.org> <733ED1ED-D102-4283-85F7-B946A5F62097@media.mit.edu> <1210773725.23948.55.camel@penguin.yourdomain.com> Message-ID: <3ACDDE79-A26D-47B5-8E85-9D6A7CE4E24D@media.mit.edu> On May 14, 2008, at 10:02 AM, Robert L. Read wrote: > On Wed, 2008-05-14 at 09:40 -0400, Ian Eslick wrote: >> If the BTree node binary data is stored in a lisp octet array, and >> we >> don't want to deserialize to compare keys, then we need to write the >> procedure that you can find in libberkeley-db.c in the src/db-bdb >> file. It performs a content-sensitive comparison on a byte-by-byte >> basis. This has huge benefits of allowing us to do key comparisons >> 'in-place' without creating lisp objects. We pass this C function >> to >> BDB which uses it to compare keys and values directly in the binary >> file pages. > > I understand the value of this, but is it not possible to write the > comparator on serialized values in LISP? This would then be passed > down > to the lowest level IO operation, which is still in LISP. I think we can do a much more elegant job, and I like your vision here. > In general I'm imagining a system that does just that as its "query > language", --- take a very lisp-like specification of a predicate and > pass it down to the lowest level I/O operation, or as low as you can > get, and apply it there. This allows us to retain the elegant > "mapcar" > and "map-reduce" strategies as basic querying mechanisms. It would be nice to unify the comparisons in the map operators with those in the database!! > If we make a modular serializer, that serializes on the basis of data > type, then it is indeed a challenge for the implementor to implement a > function that performs comparisons on the serialized representation of > that type...but a very worthwhile challenge, I think. > It should be since we always know the type before we consume: (defmethod compare-serialized-< ((typetag (eql +string+)) stream1 stream2) Use ops like read-octet and read-int32 to read data from the stream and compare... Recurse on substructure as necessary (let ((type1 (read-octet stream1)) (type2 (read-octet stream2))) (if (eq type1 type2) (compare-serialized type1 stream1 stream2) (< type1 type2)))) Which of course is a common idiom that you could compress into: (dispatch-on-subtype stream1 stream2) Which returns an ordering on types or an ordering on the values of the same types. Makes it easy to implement equal and equalp sorting directly on top of the dbs! This code isn't like to be terribly pretty given the preponderance of low-level ops, but if you had a compressed data format you could create a tag for it and dispatch your own comparison function when that tag was reached... Ian > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From killerstorm at newmail.ru Wed May 14 15:36:16 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Wed, 14 May 2008 18:36:16 +0300 Subject: [elephant-devel] Lisp Btrees: design considerations References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com> Message-ID: IE> I was quite wrong-headed on the performance point. The biggest time IE> cost in database operations is time spent moving data to/from disk, in IE> comparison CPU time is quite small. this depends on size of a database -- relatively small ones can be cached in RAM, and with asynchronous writes disk won't be a bottleneck. also note that RAM gets larger and cheaper each year, so current "relatively small ones", on scale of several gigabytes, were considered big some years ago. also database are often optimized to minimize HDD seek times. but for now we have quite affordable Solid State Drives which do not have any seek time overhead at all and are pretty fast, so such optimization do not have any sense anymore. also elephant's workload is likely to be very different from relational database's one -- with RDMBS people often try to do as much work as possible with single optimized query, while Elephant users tend to do lots of small queries -- because it's, um, easy. also Elephant lacks optimization tricks that RDBMS can do, so it has to rely on crude processing power. so i believe that Elephant will be CPU-bound for most of uses/users, and any complications like compression will make it slower. people who will use Elephant for storage and rapid retrieval of some terabytes of text will find compression a cool feature, but i seriously doubt that there are many people who consider elephant a tool for such stuff. if you want to optimize something, better make sure that small objects, like numbers, object pointers, small strings etc. can be stored/retrieved with as little overhead as possible -- i think most values in database are such. From killerstorm at newmail.ru Wed May 14 15:36:16 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Wed, 14 May 2008 18:36:16 +0300 Subject: [elephant-devel] Lisp Btrees: design considerations References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com> Message-ID: IE> I was quite wrong-headed on the performance point. The biggest time IE> cost in database operations is time spent moving data to/from disk, in IE> comparison CPU time is quite small. this depends on size of a database -- relatively small ones can be cached in RAM, and with asynchronous writes disk won't be a bottleneck. also note that RAM gets larger and cheaper each year, so current "relatively small ones", on scale of several gigabytes, were considered big some years ago. also database are often optimized to minimize HDD seek times. but for now we have quite affordable Solid State Drives which do not have any seek time overhead at all and are pretty fast, so such optimization do not have any sense anymore. also elephant's workload is likely to be very different from relational database's one -- with RDMBS people often try to do as much work as possible with single optimized query, while Elephant users tend to do lots of small queries -- because it's, um, easy. also Elephant lacks optimization tricks that RDBMS can do, so it has to rely on crude processing power. so i believe that Elephant will be CPU-bound for most of uses/users, and any complications like compression will make it slower. people who will use Elephant for storage and rapid retrieval of some terabytes of text will find compression a cool feature, but i seriously doubt that there are many people who consider elephant a tool for such stuff. if you want to optimize something, better make sure that small objects, like numbers, object pointers, small strings etc. can be stored/retrieved with as little overhead as possible -- i think most values in database are such. From eslick at media.mit.edu Wed May 14 16:41:42 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 12:41:42 -0400 Subject: [elephant-devel] Quick hack for lisp backend. Message-ID: I just spent some time, with the context of this discussion, review the very excellent rucksack code base. I think that most of the facilities needed by our datastore are quite congruent with what Arthur did to implement his system. If we really want to get going fast, we should start by building a datastore out of the rucksack btree - but not necessarily buy into his full object model (although he has schemas, a similar MOP implementation, indexing, etc). The system has an all-lisp serializer, an object cache, a persistent heap, etc that should all work pretty well with the Elephant MOP and object model. You might even be able to use the transaction subsystem - it is based on a similar versioning scheme to the one I provided, but I don't know enough about it to know for sure how congruent it is. The extensions we'd need to make to this, at least, are multi-process support and perhaps changes to the versioning model to handle the slot- level persistence notion in elephant? Let me know if you look into this, I'd be happy to review the code and make some suggestions for what can be reused from that code base and how it would play into the rest of the elephant system. Ian From lists at infoway.net Wed May 14 17:45:26 2008 From: lists at infoway.net (lists at infoway.net) Date: Wed, 14 May 2008 13:45:26 -0400 Subject: [elephant-devel] Query System In-Reply-To: <0199E4E0-735C-49AF-8D76-2B16AFEC3F1F@media.mit.edu> References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net> <90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu> <9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net> <0199E4E0-735C-49AF-8D76-2B16AFEC3F1F@media.mit.edu> Message-ID: I've digested Ian and Leslie's comments on this thread and think we're all on the same page now. I've been reading different papers and other OODBMs in trying to propose a querying syntax and am pretty close to completing this now. However, I have some doubts for which I'll need someone's help with, which will allow me to continue drafting my proposal. I believe Ian (if it was someone else, I apologize for not quoting you) mentioned that we could, as a later phase, introspect the class and ask for the indices available in order to do some query optimizations. So my question goes somewhere along those lines and I'm wondering (please excuse my limited knowledge of Lisp and thank you for helping my learning curve) if it's possible to obtain the list of slots and/or slot-accessor functions for a given class. Thanks, Daniel From eslick at media.mit.edu Wed May 14 18:15:44 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 14:15:44 -0400 Subject: [elephant-devel] Query System In-Reply-To: References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net> <90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu> <9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net> <0199E4E0-735C-49AF-8D76-2B16AFEC3F1F@media.mit.edu> Message-ID: Hi Daniel, Welcome to the Metaobject Protocol, aka the MOP! There is a rich embedded language for introspecting over and manipulating the class system in lisp that Elephant uses heavily. You could have a procedure extract the and caches the essential information from the class slot definitions into a little structure. You can do this the first time the query system encounters a class, and add a hook into the class instantiation code that invalidates the cache on class redefinition. It's worth diving into it, but it can take awhile to wrap your head around the concepts, not to mention the idiosyncrasies of the MOP. I'm happy to review your proposal offline or online as you prefer. If you look at src/elephant/package.lisp we import symbols selectively from the metaobject protocol package specific to each lisp. The functions you are looking for are: class-slots - return a list of slot-definition-objects Each of the special elephant slots returned by this function is a subtype of the class standard-effective-slot-definition and persistent- effective-slot-definition. Direct slots are representations of the arguments to the defclass form. Effective slots are computed when the class is 'finalized' (i.e. the first instance is instantiated and it's inheritance hierarchy is fully defined). A generic function called slot-definition-using-class dispatches on the type of the effective slot to implement the common-lisp primitive (slot-value obj slotname) The special types you'll care about are: - persistent-effective-slot-definition persistent-p is a predicate on the slot definition that tells you if it is an instance of or subclass of a persistent slot. - indexed-effective-slot-definition (has special slots named: indices and base-class; predicate is indexed-p) - derived-effective-slot-definition (derived from the index slot also contains a slot: fn for determining the value to index on) - set-valued-effective-slot-definition (slot stores a reference to a pset; set-valued-p) - association-effective-slot-definition (association-p; this is complicated so we should handle this as a special case in the query system later) I have accessors to get these slots from the class object such as association-slot-defs, indexed-slot-defs, and persistent-slot-defs. Read metaclasses.lisp for more information on these definitions. If you know an index exists on a class slot, you can simply call (find- inverted-index class-object slotname) to have the system fetch it for you. This should be enough for you to define a structure like: (defstruct class-info classname slotypes) Where slottypes is an alist: - ((slotname . :indexed) (slotname . :persistent) (slotname . :transient)) or something like that. Ian On May 14, 2008, at 1:45 PM, lists at infoway.net wrote: > I've digested Ian and Leslie's comments on this thread and think > we're all on the same page now. I've been reading different papers > and other OODBMs in trying to propose a querying syntax and am > pretty close to completing this now. However, I have some doubts for > which I'll need someone's help with, which will allow me to continue > drafting my proposal. > > I believe Ian (if it was someone else, I apologize for not quoting > you) mentioned that we could, as a later phase, introspect the class > and ask for the indices available in order to do some query > optimizations. So my question goes somewhere along those lines and > I'm wondering (please excuse my limited knowledge of Lisp and thank > you for helping my learning curve) if it's possible to obtain the > list of slots and/or slot-accessor functions for a given class. > > Thanks, > Daniel > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From gtarcea at umich.edu Wed May 14 18:25:50 2008 From: gtarcea at umich.edu (V. Glenn Tarcea) Date: Wed, 14 May 2008 14:25:50 -0400 Subject: [elephant-devel] Query System In-Reply-To: References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net><90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu><9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net><0199E4E0-735C-49AF-8D76-2B16AFEC3F1F@media.mit.edu> Message-ID: <03c101c8b5ef$f0001e90$4212a8c0@mimi2> Some great books and PDFs: The Art of the Meta Object Protocol (available on Amazon) Object-Oriented Programming in COMMON LISP (Keene, also on Amazon) For Advance mind twisting Lisp, get Paul Graham's OnLisp (available as a downloadable PDF). My brain hurts just trying to figure out some of the macros (and even more advance features) he rolls out: http://www.paulgraham.com/onlisptext.html I would have to say that "The Art of the Meta Object Protocol" is one of the best comp sci books in my collection. Glenn -----Original Message----- From: elephant-devel-bounces at common-lisp.net [mailto:elephant-devel-bounces at common-lisp.net] On Behalf Of Ian Eslick Sent: Wednesday, May 14, 2008 2:16 PM To: Elephant bugs and development Subject: Re: [elephant-devel] Query System Hi Daniel, Welcome to the Metaobject Protocol, aka the MOP! There is a rich embedded language for introspecting over and manipulating the class system in lisp that Elephant uses heavily. You could have a procedure extract the and caches the essential information from the class slot definitions into a little structure. You can do this the first time the query system encounters a class, and add a hook into the class instantiation code that invalidates the cache on class redefinition. It's worth diving into it, but it can take awhile to wrap your head around the concepts, not to mention the idiosyncrasies of the MOP. I'm happy to review your proposal offline or online as you prefer. If you look at src/elephant/package.lisp we import symbols selectively from the metaobject protocol package specific to each lisp. The functions you are looking for are: class-slots - return a list of slot-definition-objects Each of the special elephant slots returned by this function is a subtype of the class standard-effective-slot-definition and persistent- effective-slot-definition. Direct slots are representations of the arguments to the defclass form. Effective slots are computed when the class is 'finalized' (i.e. the first instance is instantiated and it's inheritance hierarchy is fully defined). A generic function called slot-definition-using-class dispatches on the type of the effective slot to implement the common-lisp primitive (slot-value obj slotname) The special types you'll care about are: - persistent-effective-slot-definition persistent-p is a predicate on the slot definition that tells you if it is an instance of or subclass of a persistent slot. - indexed-effective-slot-definition (has special slots named: indices and base-class; predicate is indexed-p) - derived-effective-slot-definition (derived from the index slot also contains a slot: fn for determining the value to index on) - set-valued-effective-slot-definition (slot stores a reference to a pset; set-valued-p) - association-effective-slot-definition (association-p; this is complicated so we should handle this as a special case in the query system later) I have accessors to get these slots from the class object such as association-slot-defs, indexed-slot-defs, and persistent-slot-defs. Read metaclasses.lisp for more information on these definitions. If you know an index exists on a class slot, you can simply call (find- inverted-index class-object slotname) to have the system fetch it for you. This should be enough for you to define a structure like: (defstruct class-info classname slotypes) Where slottypes is an alist: - ((slotname . :indexed) (slotname . :persistent) (slotname . :transient)) or something like that. Ian On May 14, 2008, at 1:45 PM, lists at infoway.net wrote: > I've digested Ian and Leslie's comments on this thread and think > we're all on the same page now. I've been reading different papers > and other OODBMs in trying to propose a querying syntax and am > pretty close to completing this now. However, I have some doubts for > which I'll need someone's help with, which will allow me to continue > drafting my proposal. > > I believe Ian (if it was someone else, I apologize for not quoting > you) mentioned that we could, as a later phase, introspect the class > and ask for the indices available in order to do some query > optimizations. So my question goes somewhere along those lines and > I'm wondering (please excuse my limited knowledge of Lisp and thank > you for helping my learning curve) if it's possible to obtain the > list of slots and/or slot-accessor functions for a given class. > > Thanks, > Daniel > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel _______________________________________________ elephant-devel site list elephant-devel at common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel From lists at infoway.net Wed May 14 19:06:43 2008 From: lists at infoway.net (lists at infoway.net) Date: Wed, 14 May 2008 15:06:43 -0400 Subject: [elephant-devel] Query System In-Reply-To: <03c101c8b5ef$f0001e90$4212a8c0@mimi2> References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net><90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu><9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net><0199E4E0-735C-49AF-8D76-2B16AFEC3F1F@media.mit.edu> <03c101c8b5ef$f0001e90$4212a8c0@mimi2> Message-ID: <1A481811-B8DA-406C-822A-11A012B61397@infoway.net> Ian/Glenn, Thanks for the feedback. I knew it should be somewhere buried within the MOP and last week I ordered The Art of the MOP but am waiting for it to arrive. I also know it will take me some time to grasp it. I just wanted to get some familiarity for this particular need (more specifically, I was thinking of allowing the query to ask for things like Person.name or something like that, but wanted to know if it was possible to "pre-compile" the query and make sure that the name slot exists on the Person class; BTW, sorry for my non-Lisp syntax). Anyway, I am reading OnLisp. Definitely impressed and perplexed as to what can be done with macros, but I'm getting it slowly :) I hope to be able to submit a draft proposal by tomorrow. Maybe I'll send to you privately for review and comment before posting it to the list - and possibly ridicule myself :) Anyway, will look into your suggestions right away. Thanks, Daniel On May 14, 2008, at 2:25 PM, V. Glenn Tarcea wrote: > > Some great books and PDFs: > > The Art of the Meta Object Protocol (available on Amazon) > Object-Oriented Programming in COMMON LISP (Keene, also on Amazon) > > For Advance mind twisting Lisp, get Paul Graham's OnLisp (available > as a > downloadable PDF). My brain hurts just trying to figure out some of > the > macros (and even more advance features) he rolls out: > http://www.paulgraham.com/onlisptext.html > > I would have to say that "The Art of the Meta Object Protocol" is > one of the > best comp sci books in my collection. > > Glenn > > -----Original Message----- > From: elephant-devel-bounces at common-lisp.net > [mailto:elephant-devel-bounces at common-lisp.net] On Behalf Of Ian > Eslick > Sent: Wednesday, May 14, 2008 2:16 PM > To: Elephant bugs and development > Subject: Re: [elephant-devel] Query System > > Hi Daniel, > > Welcome to the Metaobject Protocol, aka the MOP! There is a rich > embedded language for introspecting over and manipulating the class > system in lisp that Elephant uses heavily. You could have a procedure > extract the and caches the essential information from the class slot > definitions into a little structure. You can do this the first time > the query system encounters a class, and add a hook into the class > instantiation code that invalidates the cache on class redefinition. > It's worth diving into it, but it can take awhile to wrap your head > around the concepts, not to mention the idiosyncrasies of the MOP. > > I'm happy to review your proposal offline or online as you prefer. > > If you look at src/elephant/package.lisp we import symbols selectively > from the metaobject protocol package specific to each lisp. The > functions you are looking for are: > > class-slots - return a list of slot-definition-objects > > Each of the special elephant slots returned by this function is a > subtype of the class standard-effective-slot-definition and > persistent- > effective-slot-definition. Direct slots are representations of the > arguments to the defclass form. Effective slots are computed when the > class is 'finalized' (i.e. the first instance is instantiated and it's > inheritance hierarchy is fully defined). A generic function called > slot-definition-using-class dispatches on the type of the effective > slot to implement the common-lisp primitive (slot-value obj slotname) > > The special types you'll care about are: > - persistent-effective-slot-definition > persistent-p is a predicate on the slot definition that tells you > if it is an instance of or subclass of a persistent slot. > - indexed-effective-slot-definition > (has special slots named: indices and base-class; predicate is > indexed-p) > - derived-effective-slot-definition > (derived from the index slot also contains a slot: fn for > determining the value to index on) > - set-valued-effective-slot-definition > (slot stores a reference to a pset; set-valued-p) > - association-effective-slot-definition > (association-p; this is complicated so we should handle this as a > special case in the query system later) > > I have accessors to get these slots from the class object such as > association-slot-defs, indexed-slot-defs, and persistent-slot-defs. > > Read metaclasses.lisp for more information on these definitions. > > If you know an index exists on a class slot, you can simply call > (find- > inverted-index class-object slotname) to have the system fetch it for > you. > > This should be enough for you to define a structure like: > > (defstruct class-info classname slotypes) > > Where slottypes is an alist: > - ((slotname . :indexed) (slotname . :persistent) > (slotname . :transient)) > > or something like that. > > Ian > > On May 14, 2008, at 1:45 PM, lists at infoway.net wrote: > >> I've digested Ian and Leslie's comments on this thread and think >> we're all on the same page now. I've been reading different papers >> and other OODBMs in trying to propose a querying syntax and am >> pretty close to completing this now. However, I have some doubts for >> which I'll need someone's help with, which will allow me to continue >> drafting my proposal. >> >> I believe Ian (if it was someone else, I apologize for not quoting >> you) mentioned that we could, as a later phase, introspect the class >> and ask for the indices available in order to do some query >> optimizations. So my question goes somewhere along those lines and >> I'm wondering (please excuse my limited knowledge of Lisp and thank >> you for helping my learning curve) if it's possible to obtain the >> list of slots and/or slot-accessor functions for a given class. >> >> Thanks, >> Daniel >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Wed May 14 19:15:46 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 15:15:46 -0400 Subject: [elephant-devel] Query System In-Reply-To: <1A481811-B8DA-406C-822A-11A012B61397@infoway.net> References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net><90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu><9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net><0199E4E0-735C-49AF-8D76-2B16AFEC3F1F@media.mit.edu> <03c101c8b5ef$f0001e90$4212a8c0@mimi2> <1A481811-B8DA-406C-822A-11A012B61397@infoway.net> Message-ID: On May 14, 2008, at 3:06 PM, lists at infoway.net wrote: > Ian/Glenn, > > Thanks for the feedback. I knew it should be somewhere buried within > the MOP and last week I ordered The Art of the MOP but am waiting > for it to arrive. I also know it will take me some time to grasp it. > I just wanted to get some familiarity for this particular need (more > specifically, I was thinking of allowing the query to ask for things > like Person.name or something like that, but wanted to know if it > was possible to "pre-compile" the query and make sure that the name > slot exists on the Person class; BTW, sorry for my non-Lisp syntax). If you download cl-weblocks and look in src/stores/elephant/ elephant.lisp you'll see a simple query implementation supporting the weblocks query semantics for paged table views that uses indices to perform ordering when available and also does reasonably efficient range queries (you walk the whole index, but only create the instances you need and perform early termination. If you want to get into query compilation, you can generate closures that are highly efficient, but check their preconditions prior to execution, recompiling on violations. I'd wait on this performance step and get an interpreter to work first. Don't sweat syntax too much, I'm sure it will evolve as we play with it so better to get something working and evolve and extend it and learn as we go. It's great to see so much energy on the list these days! > Anyway, I am reading OnLisp. Definitely impressed and perplexed as > to what can be done with macros, but I'm getting it slowly :) > > I hope to be able to submit a draft proposal by tomorrow. Maybe I'll > send to you privately for review and comment before posting it to > the list - and possibly ridicule myself :) > > Anyway, will look into your suggestions right away. > > Thanks, > Daniel > > On May 14, 2008, at 2:25 PM, V. Glenn Tarcea wrote: > >> >> Some great books and PDFs: >> >> The Art of the Meta Object Protocol (available on Amazon) >> Object-Oriented Programming in COMMON LISP (Keene, also on Amazon) >> >> For Advance mind twisting Lisp, get Paul Graham's OnLisp (available >> as a >> downloadable PDF). My brain hurts just trying to figure out some of >> the >> macros (and even more advance features) he rolls out: >> http://www.paulgraham.com/onlisptext.html >> >> I would have to say that "The Art of the Meta Object Protocol" is >> one of the >> best comp sci books in my collection. >> >> Glenn >> >> -----Original Message----- >> From: elephant-devel-bounces at common-lisp.net >> [mailto:elephant-devel-bounces at common-lisp.net] On Behalf Of Ian >> Eslick >> Sent: Wednesday, May 14, 2008 2:16 PM >> To: Elephant bugs and development >> Subject: Re: [elephant-devel] Query System >> >> Hi Daniel, >> >> Welcome to the Metaobject Protocol, aka the MOP! There is a rich >> embedded language for introspecting over and manipulating the class >> system in lisp that Elephant uses heavily. You could have a >> procedure >> extract the and caches the essential information from the class slot >> definitions into a little structure. You can do this the first time >> the query system encounters a class, and add a hook into the class >> instantiation code that invalidates the cache on class redefinition. >> It's worth diving into it, but it can take awhile to wrap your head >> around the concepts, not to mention the idiosyncrasies of the MOP. >> >> I'm happy to review your proposal offline or online as you prefer. >> >> If you look at src/elephant/package.lisp we import symbols >> selectively >> from the metaobject protocol package specific to each lisp. The >> functions you are looking for are: >> >> class-slots - return a list of slot-definition-objects >> >> Each of the special elephant slots returned by this function is a >> subtype of the class standard-effective-slot-definition and >> persistent- >> effective-slot-definition. Direct slots are representations of the >> arguments to the defclass form. Effective slots are computed when >> the >> class is 'finalized' (i.e. the first instance is instantiated and >> it's >> inheritance hierarchy is fully defined). A generic function called >> slot-definition-using-class dispatches on the type of the effective >> slot to implement the common-lisp primitive (slot-value obj slotname) >> >> The special types you'll care about are: >> - persistent-effective-slot-definition >> persistent-p is a predicate on the slot definition that tells you >> if it is an instance of or subclass of a persistent slot. >> - indexed-effective-slot-definition >> (has special slots named: indices and base-class; predicate is >> indexed-p) >> - derived-effective-slot-definition >> (derived from the index slot also contains a slot: fn for >> determining the value to index on) >> - set-valued-effective-slot-definition >> (slot stores a reference to a pset; set-valued-p) >> - association-effective-slot-definition >> (association-p; this is complicated so we should handle this as a >> special case in the query system later) >> >> I have accessors to get these slots from the class object such as >> association-slot-defs, indexed-slot-defs, and persistent-slot-defs. >> >> Read metaclasses.lisp for more information on these definitions. >> >> If you know an index exists on a class slot, you can simply call >> (find- >> inverted-index class-object slotname) to have the system fetch it for >> you. >> >> This should be enough for you to define a structure like: >> >> (defstruct class-info classname slotypes) >> >> Where slottypes is an alist: >> - ((slotname . :indexed) (slotname . :persistent) >> (slotname . :transient)) >> >> or something like that. >> >> Ian >> >> On May 14, 2008, at 1:45 PM, lists at infoway.net wrote: >> >>> I've digested Ian and Leslie's comments on this thread and think >>> we're all on the same page now. I've been reading different papers >>> and other OODBMs in trying to propose a querying syntax and am >>> pretty close to completing this now. However, I have some doubts for >>> which I'll need someone's help with, which will allow me to continue >>> drafting my proposal. >>> >>> I believe Ian (if it was someone else, I apologize for not quoting >>> you) mentioned that we could, as a later phase, introspect the class >>> and ask for the indices available in order to do some query >>> optimizations. So my question goes somewhere along those lines and >>> I'm wondering (please excuse my limited knowledge of Lisp and thank >>> you for helping my learning curve) if it's possible to obtain the >>> list of slots and/or slot-accessor functions for a given class. >>> >>> Thanks, >>> Daniel >>> _______________________________________________ >>> elephant-devel site list >>> elephant-devel at common-lisp.net >>> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel >> >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From gtarcea at umich.edu Wed May 14 19:20:47 2008 From: gtarcea at umich.edu (V. Glenn Tarcea) Date: Wed, 14 May 2008 15:20:47 -0400 Subject: [elephant-devel] Query System In-Reply-To: <1A481811-B8DA-406C-822A-11A012B61397@infoway.net> References: <3265B9F6-0748-4436-A444-24C5864E9F14@infoway.net><90CE91E8-966F-4639-98D7-50B9B7B11EA3@media.mit.edu><9C269981-D71E-4C3E-A7FD-5A7AC26024EC@infoway.net><0199E4E0-735C-49AF-8D76-2B16AFEC3F1F@media.mit.edu><03c101c8b5ef$f0001e90$4212a8c0@mimi2> <1A481811-B8DA-406C-822A-11A012B61397@infoway.net> Message-ID: <03cd01c8b5f7$9d519c20$4212a8c0@mimi2> Daniel, I'm just a hanger on who is trying to figure out this wonderful new world of Lisp and Elephant. I love the discussions on this list. Elephant does things I've always been interested in but never knew anything about. The discussions in this group are very helpful. Lisp makes me feel mentally challenged (well... more so than my usually state :-) ). Glenn -----Original Message----- From: elephant-devel-bounces at common-lisp.net [mailto:elephant-devel-bounces at common-lisp.net] On Behalf Of lists at infoway.net Sent: Wednesday, May 14, 2008 3:07 PM To: Elephant bugs and development Subject: Re: [elephant-devel] Query System Ian/Glenn, Thanks for the feedback. I knew it should be somewhere buried within the MOP and last week I ordered The Art of the MOP but am waiting for it to arrive. I also know it will take me some time to grasp it. I just wanted to get some familiarity for this particular need (more specifically, I was thinking of allowing the query to ask for things like Person.name or something like that, but wanted to know if it was possible to "pre-compile" the query and make sure that the name slot exists on the Person class; BTW, sorry for my non-Lisp syntax). Anyway, I am reading OnLisp. Definitely impressed and perplexed as to what can be done with macros, but I'm getting it slowly :) I hope to be able to submit a draft proposal by tomorrow. Maybe I'll send to you privately for review and comment before posting it to the list - and possibly ridicule myself :) Anyway, will look into your suggestions right away. Thanks, Daniel On May 14, 2008, at 2:25 PM, V. Glenn Tarcea wrote: > > Some great books and PDFs: > > The Art of the Meta Object Protocol (available on Amazon) > Object-Oriented Programming in COMMON LISP (Keene, also on Amazon) > > For Advance mind twisting Lisp, get Paul Graham's OnLisp (available > as a > downloadable PDF). My brain hurts just trying to figure out some of > the > macros (and even more advance features) he rolls out: > http://www.paulgraham.com/onlisptext.html > > I would have to say that "The Art of the Meta Object Protocol" is > one of the > best comp sci books in my collection. > > Glenn > > -----Original Message----- > From: elephant-devel-bounces at common-lisp.net > [mailto:elephant-devel-bounces at common-lisp.net] On Behalf Of Ian > Eslick > Sent: Wednesday, May 14, 2008 2:16 PM > To: Elephant bugs and development > Subject: Re: [elephant-devel] Query System > > Hi Daniel, > > Welcome to the Metaobject Protocol, aka the MOP! There is a rich > embedded language for introspecting over and manipulating the class > system in lisp that Elephant uses heavily. You could have a procedure > extract the and caches the essential information from the class slot > definitions into a little structure. You can do this the first time > the query system encounters a class, and add a hook into the class > instantiation code that invalidates the cache on class redefinition. > It's worth diving into it, but it can take awhile to wrap your head > around the concepts, not to mention the idiosyncrasies of the MOP. > > I'm happy to review your proposal offline or online as you prefer. > > If you look at src/elephant/package.lisp we import symbols selectively > from the metaobject protocol package specific to each lisp. The > functions you are looking for are: > > class-slots - return a list of slot-definition-objects > > Each of the special elephant slots returned by this function is a > subtype of the class standard-effective-slot-definition and > persistent- > effective-slot-definition. Direct slots are representations of the > arguments to the defclass form. Effective slots are computed when the > class is 'finalized' (i.e. the first instance is instantiated and it's > inheritance hierarchy is fully defined). A generic function called > slot-definition-using-class dispatches on the type of the effective > slot to implement the common-lisp primitive (slot-value obj slotname) > > The special types you'll care about are: > - persistent-effective-slot-definition > persistent-p is a predicate on the slot definition that tells you > if it is an instance of or subclass of a persistent slot. > - indexed-effective-slot-definition > (has special slots named: indices and base-class; predicate is > indexed-p) > - derived-effective-slot-definition > (derived from the index slot also contains a slot: fn for > determining the value to index on) > - set-valued-effective-slot-definition > (slot stores a reference to a pset; set-valued-p) > - association-effective-slot-definition > (association-p; this is complicated so we should handle this as a > special case in the query system later) > > I have accessors to get these slots from the class object such as > association-slot-defs, indexed-slot-defs, and persistent-slot-defs. > > Read metaclasses.lisp for more information on these definitions. > > If you know an index exists on a class slot, you can simply call > (find- > inverted-index class-object slotname) to have the system fetch it for > you. > > This should be enough for you to define a structure like: > > (defstruct class-info classname slotypes) > > Where slottypes is an alist: > - ((slotname . :indexed) (slotname . :persistent) > (slotname . :transient)) > > or something like that. > > Ian > > On May 14, 2008, at 1:45 PM, lists at infoway.net wrote: > >> I've digested Ian and Leslie's comments on this thread and think >> we're all on the same page now. I've been reading different papers >> and other OODBMs in trying to propose a querying syntax and am >> pretty close to completing this now. However, I have some doubts for >> which I'll need someone's help with, which will allow me to continue >> drafting my proposal. >> >> I believe Ian (if it was someone else, I apologize for not quoting >> you) mentioned that we could, as a later phase, introspect the class >> and ask for the indices available in order to do some query >> optimizations. So my question goes somewhere along those lines and >> I'm wondering (please excuse my limited knowledge of Lisp and thank >> you for helping my learning curve) if it's possible to obtain the >> list of slots and/or slot-accessor functions for a given class. >> >> Thanks, >> Daniel >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel _______________________________________________ elephant-devel site list elephant-devel at common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Wed May 14 19:20:06 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 15:20:06 -0400 Subject: [elephant-devel] 1.0 Release Message-ID: If the current flurry of interest and discussion around the query system and an all-lisp backend starts to become actual coding, I would support a delay in 1.0 and keep going on our current efforts to produce an interim release that stabilizes the core content. The nice thing is that both of these projects are very modular and will have only a minimal impact on the core code base. We'll keep on track with our release process once the postmodern implementation on unstable is in and keep an eye on these two projects as to timing and release naming. In the meantime, let's keep these projects out of the standard build process (elephant.asd) and put them in contrib or at least under optional elephant-query.asd or ele-lisp.asd system definitions. Thank you, Ian From read at robertlread.net Thu May 15 03:15:32 2008 From: read at robertlread.net (Robert L. Read) Date: Wed, 14 May 2008 22:15:32 -0500 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <1210773234.23948.47.camel@penguin.yourdomain.com> Message-ID: <1210821332.23948.110.camel@penguin.yourdomain.com> I feel compelled to respectfully disagree with Alex's views stated below. The purpose of compression is of course NOT to make the store smaller. It is to decrease the amount of I/O that is required to access the store. I think this is where most of the time is spent. I suspect Alex is correct that even if Elephant were coded perfectly (which it is not), there would be some databases for which it might be CPU-bound. It is hard for me to imagine this is true of most databases. I would like to make three points. Firstly, irrespective of the ever-changing ratio of I/O costs within the storage hierarchy, smaller will always be faster. Solid state drives and gigantic RAMs don't change this; they merely move the benefit by allowing one to store more data inside the the cache. As technology improves, this cache can be thought of as RAM caching disk, or fast ram caching slow ram, or register sets caching fast ram. The fact that locality of reference is a positive good for performance remains an iron-clad law as long as you actually have a storage hierarchy with different levels of smaller and faster vs. bigger and slower storage. All of my professional and academic experience leads me to believe that any database application is I/O bound and finding a way to trade a constant amount of CPU time to decrease I/O load is a good idea. Secondly the compression that we're talking about must not be "optimal" compression that seeks maximum compression. It is instead something that can be efficiently computed, yet saves space. Finally, the examples that you name, small objects, integers, small strings, and object pointers, are precisely those objects for which it is efficient to compute compact representations. For example, simple frequency-based Huffman encoding based on character frequency will give about a factor of 3 compression on strings. If one imagines an integer-valued slot with a Poisson distribution of values, for example having an average value of less than a thousand but still occasionally containing a much large number, then one can expect a simple codec that employs this fact to use about 10 bits for each value, and 30 bits for the outliers. The result is again a 3-fold reduction in I/O. Object pointers may of course cross the entire space in theory, but OIDs that we store in the database may indeed come from a set of quite knowable size, allowing us to use less than 64 bits or 32 bits to represent them. If we know the type of a slot to be only OIDs of a particular class, we may very well know that that class has only 2**20 members, and therefore demands only 20 bits, as opposed to 64, to represent the entire space of possible OIDs. If one imagines a page-based system that brings in a group of objects in one page, then getting 3 times as many objects for the same I/O is a huge overall win. However, my point here was that I wanted to stay in "pure" LISP so that I/we would have the power to do this. (I understand now that Ian was not proposing otherwise.) Whether it will work is somewhat academic, because it should certainly be deployed as a "strategy" -- that is, the system either decides based on statistics to employ compression for a given slot, or you declare a slot to be :poisson 1000 or :english-text or something to suggest the most efficient codec. As you point out, developing a type-specific codec, that might do a very good job compressing and improving performance based upon specialized knowledge of a particular application-dependent datatype may be quite rare, but I think it is an important ability. Therefore, since we should undoubtedly just develop a plain B+Tree first, and any implementation of compression should be a strategy-based approach, this argument is somewhat academic. However, in theory someone could write a compressing serializer right now, independently of implementing a pure-lisp Btree-based backend. For example, the serializer that Rucksack has written could be inserted in place of our existing serializer (or even set a configurable option) if we found it to be a better codec. On Wed, 2008-05-14 at 18:36 +0300, Alex Mizrahi wrote: > IE> I was quite wrong-headed on the performance point. The biggest > time > IE> cost in database operations is time spent moving data to/from > disk, in > IE> comparison CPU time is quite small. > > this depends on size of a database -- relatively small ones can be > cached in > RAM, and with asynchronous writes disk won't be a bottleneck. > also note that RAM gets larger and cheaper each year, so current > "relatively > small ones", on scale of several gigabytes, were considered big some > years > ago. > > also database are often optimized to minimize HDD seek times. but for > now we > have quite affordable Solid State Drives which do not have any seek > time > overhead at all and are pretty fast, so such optimization do not have > any > sense anymore. > > also elephant's workload is likely to be very different from > relational > database's one -- with RDMBS people often try to do as much work as > possible > with single optimized query, while Elephant users tend to do lots of > small > queries -- because it's, um, easy. also Elephant lacks optimization > tricks > that RDBMS can do, so it has to rely on crude processing power. > > so i believe that Elephant will be CPU-bound for most of uses/users, > and any > complications like compression will make it slower. > > people who will use Elephant for storage and rapid retrieval of some > terabytes of text will find compression a cool feature, but i > seriously > doubt that there are many people who consider elephant a tool for > such > stuff. > > if you want to optimize something, better make sure that small > objects, like > numbers, object pointers, small strings etc. can be stored/retrieved > with as > little overhead as possible -- i think most values in database are > such. From eslick at media.mit.edu Thu May 15 03:27:16 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 14 May 2008 23:27:16 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <1210821332.23948.110.camel@penguin.yourdomain.com> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <1210773234.23948.47.camel@penguin.yourdomain.com> <1210821332.23948.110.camel@penguin.yourdomain.com> Message-ID: <4FCA9D44-E1B7-4A03-88C9-C5272A79C3C6@media.mit.edu> > However, in theory someone could write a compressing serializer right > now, independently of implementing a pure-lisp Btree-based backend. Definitely true, and quite welcome. > For example, the serializer that Rucksack has written could be > inserted > in place of our existing serializer (or even set a configurable > option) > if we found it to be a better codec. Actually I wouldn't do that quite yet. If we're going to be putting things into the C world via BDB, then we should keep the serializer that is efficient at doing that! However, I suspect that postmodern, CL-SQL and the lisp backend would all benefit from a lisp serializer that serialized lisp objects into lisp byte arrays. Today we serialize a lisp object into a buffer- stream, then copy the buffer stream back into a lisp array, then write it out via postmodern or CL-SQL's interface which, for CL-SQL, then copies it into a base 64 text string... Ian From read at robertlread.net Thu May 15 03:43:13 2008 From: read at robertlread.net (Robert L. Read) Date: Wed, 14 May 2008 22:43:13 -0500 Subject: [elephant-devel] bug in map-inverted-index In-Reply-To: <7AFD8356-8980-4DC2-BB6B-9B8730C5B36F@media.mit.edu> References: <19e19e410805121108w63f32b56tad863a168f9f5a29@mail.gmail.com> <890CB4D4-435C-4EEE-9A00-4D3D44B12A18@media.mit.edu> <19e19e410805121253g64976962hd6dbdc75845d9dc9@mail.gmail.com> <19e19e410805121655p6146deedn6dfae26e60f092b8@mail.gmail.com> <11EC3CD4-A7D3-46F9-9FB4-9412AC1A711B@media.mit.edu> <19e19e410805130712n3c074106k2b4367422642e61f@mail.gmail.com> <7AFD8356-8980-4DC2-BB6B-9B8730C5B36F@media.mit.edu> Message-ID: <1210822993.23948.116.camel@penguin.yourdomain.com> On Tue, 2008-05-13 at 10:17 -0400, Ian Eslick wrote: > Robert or Alex, would you mind back patching on the main branch? I have now applied Richards patch to the main branch -- thanks! (I didn't run the tests, which is a bit naughty, but time is short.) > > We should deprecate the main branch and move everyone to the new > branch as soon as possible. Alex, any update on when we'll get the > postmodern backend to work on the new branch with dup-btrees, etc? > > Thank you! > Ian > > On May 13, 2008, at 10:12 AM, Ryszard Szopa wrote: > > > On Tue, May 13, 2008 at 2:05 AM, Ian Eslick > > wrote: > >> The problem is on both branches then. I've fixed it on unstable. > >> I don't > >> intend to back patch, although the patch is trivial if someone > >> wants to make > >> it. > > > > I wanted to. The patch is attached. > > > > -- > > http://szopa.tasak.gda.pl/ > > > range.patch>_______________________________________________ > > elephant-devel site list > > elephant-devel at common-lisp.net > > http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Thu May 15 03:48:15 2008 From: read at robertlread.net (Robert L. Read) Date: Wed, 14 May 2008 22:48:15 -0500 Subject: [elephant-devel] open-controller for bdb doesn't accept a pathname spec In-Reply-To: <19e19e410805131436x74cec081mb332b2389b5efc51@mail.gmail.com> References: <19e19e410805130641j39e83052q3094f5d62db5a2a9@mail.gmail.com> <5126A351-3B94-48B3-B857-7CA8273B664B@media.mit.edu> <19e19e410805131436x74cec081mb332b2389b5efc51@mail.gmail.com> Message-ID: <1210823295.23948.120.camel@penguin.yourdomain.com> On Tue, 2008-05-13 at 23:36 +0200, Ryszard Szopa wrote: > On Tue, May 13, 2008 at 4:01 PM, Ian Eslick wrote: > > I had just run into this myself too! Anyway, I made some tweaks to the > > patch and just pushed the fix. > > > > You could write a whitebox test for the auxiliary function I created called > > elephant-db-path which returns a pathname to probe-file regardless of the > > input spec. > > Here it is. There's one problem though: for POSIX-like systems, > multiple subsequent slashes don't change the meaning of a pathname. > This is not the case for CL pathnames. Should we care about it? > > BTW there are some aspects of the tests system I fail to understand. > For example, elephant-tests fails to compile > (tests/testconcurrency.lisp wants to use package BT, which cannot be > found). I applied this extra test to the elephant-unstable branch. > > -- Richard > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Thu May 15 03:53:00 2008 From: read at robertlread.net (Robert L. Read) Date: Wed, 14 May 2008 22:53:00 -0500 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <4FCA9D44-E1B7-4A03-88C9-C5272A79C3C6@media.mit.edu> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <1210773234.23948.47.camel@penguin.yourdomain.com> <1210821332.23948.110.camel@penguin.yourdomain.com> <4FCA9D44-E1B7-4A03-88C9-C5272A79C3C6@media.mit.edu> Message-ID: <1210823580.23948.126.camel@penguin.yourdomain.com> On Wed, 2008-05-14 at 23:27 -0400, Ian Eslick wrote: > Actually I wouldn't do that quite yet. If we're going to be putting > things into the C world via BDB, then we should keep the serializer > that is efficient at doing that! > Well, if we're willing to have type-specific serializers, I don't know why we couldn't use back-end specific serializers as well, and leave what we have in place for BDB. >However, I suspect that postmodern, CL-SQL and the lisp backend would >all benefit from a lisp serializer that serialized lisp objects into >lisp byte arrays. Today we serialize a lisp object into a buffer- >stream, then copy the buffer stream back into a lisp array, then write >it out via postmodern or CL-SQL's interface which, for CL-SQL, then >copies it into a base 64 text string... Right. And the only good reason to retain the Cl-SQL backend, given the superiority of the postmodern one, and the fact that postmodern doesn't base64 encode things, is to support SQlite3. But of course, if we had a pure-lisp backend, one of the incentives to support SQLite3 would go away. From mmstud at gmail.com Thu May 15 07:08:50 2008 From: mmstud at gmail.com (Marko Tapio Manninen) Date: Thu, 15 May 2008 10:08:50 +0300 Subject: [elephant-devel] 64bit BDB & CCL load problem Message-ID: <9920053B-DECE-4C85-9FAD-E21C3DAC0524@gmail.com> Hi, i have a slight problem using elephant berkeley store on my web app made with openmcl and hunchentoot. I have compiled 64 bit version of BDB (mac intel). I thought if you could help me forward with this. Following is the error: . . . ; registering # as CHUNGA Attempting to load libmemutil.dylib... Loaded /usr/local/asdf-install/site/elephant/src/memutil/ libmemutil.dylib ; loading system definition from /usr/local/asdf-install/site/elephant/ ele-bdb.asd into # ; registering # as ELE-BDB Attempting to load libmemutil.dylib... Loaded /usr/local/asdf-install/site/elephant/src/memutil/ libmemutil.dylib Loading /usr/local/BerkeleyDB.4.5/lib/libdb-4.5.dylib Attempting to load libberkeley-db.dylib... Loaded /usr/local/asdf-install/site/elephant/src/db-bdb/libberkeley- db.dylib --- short hung here.... and then debug trace Berkeley DB error: Resource temporarily unavailable [Condition of type DB-BDB::DB-ERROR] --- Backtrace: 0: (DB-BDB::DB-ENV-OPEN # "/Users/mmstud/ tmp/" ':AUTO-COMMIT 'NIL ':INIT-CDB 'NIL ':INIT-LOCK 'T ':INIT-LOG 'T ':INIT-MPOOL 'T ':INIT-REP 'NIL ':INIT-TXN 'T ':RECOVER 'NIL ':RECOVER- FATAL 'NIL ':CREATE 'T ':LOCKDOWN 'NIL ':PRIVATE 'NIL ':SYSTEM-MEM 'NIL ':THREAD 'T ':MODE 416) 1: (# # ':RECOVER 'NIL ':RECOVER-FATAL 'NIL ':THREAD 'T ':DEADLOCK-DETECT 'NIL) 2: (ELEPHANT:OPEN-STORE '(:BDB "/Users/mmstud/tmp/")) 3: (#) 4: (# #) 5: (CCL::%FASLOAD "/Users/mmstud/lisp/myapps/qa/qa.dx64fsl" #(# # CCL::%BAD-FASL # # # # # # # ...)) 6: (CCL::%LOAD #P"/Users/mmstud/lisp/myapps/qa/qa.dx64fsl" 'NIL 'NIL ':ERROR ':DEFAULT) 7: (LOAD #P"/Users/mmstud/lisp/myapps/qa/qa.dx64fsl" ':VERBOSE 'NIL ':PRINT 'NIL ':IF-DOES-NOT-EXIST ':ERROR ':EXTERNAL-FORMAT ':DEFAULT) 8: (# # #) 9: (CCL::%%BEFORE-AND-AFTER-COMBINED-METHOD-DCODE '(NIL # . 4265217)) 10: (CCL::%%STANDARD-COMBINED-METHOD-DCODE '(NIL (#) #) 4265217) 11: (ASDF:OPERATE 'ASDF:LOAD-OP ':QA) 12: (CCL::CALL-CHECK-REGS 'ASDF:OOS 'ASDF:LOAD-OP ':QA) Regards, Marko From killerstorm at newmail.ru Thu May 15 07:39:14 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Thu, 15 May 2008 10:39:14 +0300 Subject: [elephant-devel] Lisp Btrees: design considerations References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com> <1210821332.23948.110.camel@penguin.yourdomain.com> Message-ID: <8A9B375E3FCA40B1A3887976B80A9399@killer> RLR> The purpose of compression is of course NOT to make the store smaller. RLR> It is to decrease the amount of I/O that is required to access the RLR> store. time to do I/O does not linearly depend on I/O amount. for example, HDD reads data in blocks, with each block definitely not less than 512 bytes (most likely larger due to HDD doing read-aheads). if you compress your data into 50 bytes, 10 times, it won't make I/O anyhow faster. moreover, random reads for HDD are orders (!) of magnitude slower than linear reads -- time to reposition device head is larger than time reading consecutive blocks. i can show this in numbers, just checked on iometer, doing random reads: 512 byte blocks: 106 operations per second (~10 ms seek time), 0.05 MiB/sec 64 KiB blocks: 95 operations per second (~10 ms seek time), 5.95 MiB/sec you see, we have 120 times (!) more I/O, but very low difference in number of operations per second. so compression only will matter to people who store at least hundred KiB as their values. or it will matter if people are doing lots of linear reads -- but it's hard for me to imagine such database access patterns. RLR> there would be some databases for which it might be CPU-bound. It is RLR> hard for me to imagine this is true of most databases. then it's worth defining what is "most databases" in case of elephant -- optimization without having a concrete target absolutely has no sense. IMHO most people will be working with databases smaller than few gigabytes -- so read I/O has no sense for them (DDR reads are on scale of gigabytes per second), write I/O might matter, though. RLR> If one imagines an integer-valued slot with a Poisson distribution of RLR> values, for example having an average value of less than a thousand RLR> but still occasionally containing a much large number, then one can RLR> expect a simple codec that employs this fact to use about 10 bits for RLR> each value, and 30 bits for the outliers. The result is again a RLR> 3-fold reduction in I/O. bits, huh? what storage nowadays works on bit level? even cache lines are 32 bytes wide, so make no difference to read 1 byte or 32 of them. RLR> Object pointers may of course cross the entire space in theory, but RLR> OIDs that we store in the database may indeed come from a set of quite RLR> knowable size, allowing us to use less than 64 bits or 32 bits to RLR> represent them. If we know the type of a slot to be only OIDs of a RLR> particular class, we may very well know that that class has only 2**20 RLR> members, and therefore demands only 20 bits, as opposed to 64, to RLR> represent the entire space of possible OIDs. if you expect database being large enough for I/O to matter, why do you think then that there would be that few objects? From gtarcea at umich.edu Thu May 15 22:50:00 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Thu, 15 May 2008 18:50:00 -0400 Subject: [elephant-devel] Quick hack for lisp backend. In-Reply-To: References: Message-ID: I had never looked at Rucksack before. The code is very well structured indeed! A couple of thoughts: 1. Could the Rucksack B-Tree code be separated out, abstracted a little and put in a separate project (eg, a cl-btree, or ldb for example)? This would allow other lisp projects that are also looking for a native B-Tree backend to make use of it. And all those projects would benefit from the work done. 2. It appears that Rucksack is still be developed. Could the author (s) be contacted and see if they are interested in creating a universal lisp backend that could be shared between Elephant and Rucksack (more rows in the water so to speak)? 3. I think that if the Rucksack B-Tree can be used in some way its the fastest way to get the project off the ground and get some momentum behind it. Glenn On May 14, 2008, at 12:41 PM, Ian Eslick wrote: > I just spent some time, with the context of this discussion, review > the very excellent rucksack code base. I think that most of the > facilities needed by our datastore are quite congruent with what > Arthur did to implement his system. > > If we really want to get going fast, we should start by building a > datastore out of the rucksack btree - but not necessarily buy into > his full object model (although he has schemas, a similar MOP > implementation, indexing, etc). > > The system has an all-lisp serializer, an object cache, a > persistent heap, etc that should all work pretty well with the > Elephant MOP and object model. You might even be able to use the > transaction subsystem - it is based on a similar versioning scheme > to the one I provided, but I don't know enough about it to know for > sure how congruent it is. > > The extensions we'd need to make to this, at least, are multi- > process support and perhaps changes to the versioning model to > handle the slot-level persistence notion in elephant? > > Let me know if you look into this, I'd be happy to review the code > and make some suggestions for what can be reused from that code > base and how it would play into the rest of the elephant system. > > Ian > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > From read at robertlread.net Fri May 16 03:06:17 2008 From: read at robertlread.net (Robert L. Read) Date: Thu, 15 May 2008 22:06:17 -0500 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <8A9B375E3FCA40B1A3887976B80A9399@killer> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <1210773234.23948.47.camel@penguin.yourdomain.com> <1210821332.23948.110.camel@penguin.yourdomain.com> <8A9B375E3FCA40B1A3887976B80A9399@killer> Message-ID: <1210907177.29423.45.camel@penguin.yourdomain.com> On Thu, 2008-05-15 at 10:39 +0300, Alex Mizrahi wrote: > RLR> The purpose of compression is of course NOT to make the store smaller. > RLR> It is to decrease the amount of I/O that is required to access the > RLR> store. > > time to do I/O does not linearly depend on I/O amount. > > for example, HDD reads data in blocks, with each block definitely not less > than 512 bytes (most likely larger due to HDD doing read-aheads). > if you compress your data into 50 bytes, 10 times, it won't make I/O anyhow > faster. You are correct, if you mean the time to perform a single "point query" --- that is, a to look up a value by a single key. Of course, compressing it won't hurt. > > moreover, random reads for HDD are orders (!) of magnitude slower than > linear reads -- time to reposition device head is larger than time reading > consecutive blocks. > > i can show this in numbers, just checked on iometer, doing random reads: > 512 byte blocks: 106 operations per second (~10 ms seek time), 0.05 MiB/sec > 64 KiB blocks: 95 operations per second (~10 ms seek time), 5.95 MiB/sec > > you see, we have 120 times (!) more I/O, but very low difference in number > of operations per second. > so compression only will matter to people who store at least hundred KiB as > their values. > > or it will matter if people are doing lots of linear reads -- but it's hard > for me to imagine such database access patterns. I think linear reads are quite common. They are, after all, why we have implemented range queries. Moreover, the act of reading all the data into memory is quite common if you are using a "Object Prevalance" model. In a relational database a query optimizer has a very accurate model of the number of pages that will have to be read by any given query, and attempts to search the space of all possible plans in order to find the best one. In many cases, this is indeed a linear scan; for example many join operations can best be performed by performing a linear scan of at least one of the tables in question. The aggregate functions (sum, min, max, avg, count) are another example. Another typical example is "access all page views of my website from Thursday to Friday". > > RLR> there would be some databases for which it might be CPU-bound. It is > RLR> hard for me to imagine this is true of most databases. > > then it's worth defining what is "most databases" in case of elephant -- > optimization without having a concrete target absolutely has no sense. Yes, I agree. I'm familiar with what you might call "enterprise" applications --- large database-backed websites, with maybe a few million rows, of a total size of only tens of gigabytes. For such applications, I think my assumptions hold; and these are the same assumptions that I dealt with in my dissertation. In fact my entire professional career could be summarize as applying caching to more and more objects within an enterprise to avoid I/O costs. Perhaps you can post a description of your database usage pattern, to explain why it is dominated by point queries. > > IMHO most people will be working with databases smaller than few > gigabytes -- > so read I/O has no sense for them (DDR reads are on scale of gigabytes per > second), > write I/O might matter, though. Yes, write I/O does matter. I am assuming this will be done in fairly large pages---8K at one time was the standard; I'm not sure what is favored now. However, even in a 4 gigabyte or smaller database the read time really matters. > > RLR> If one imagines an integer-valued slot with a Poisson distribution of > RLR> values, for example having an average value of less than a thousand > RLR> but still occasionally containing a much large number, then one can > RLR> expect a simple codec that employs this fact to use about 10 bits for > RLR> each value, and 30 bits for the outliers. The result is again a > RLR> 3-fold reduction in I/O. > > bits, huh? what storage nowadays works on bit level? You are quite correct that most usage of storage is done at the byte and word level---I am arguing that this is an opportunity for us. Of course, one must understand that the page will be read and written as an 8K chunk of bytes, which will contain carefully packed bit sets that do not respect byte boundaries. The result will be that more objects can fit into an 8K page, which will tend to decrease I/O by that amount. The byte is the smallest object that can be written to a file, but that is irrelevant to how data is laid out within a block of bytes. > even cache lines are 32 bytes wide, so make no difference to read 1 byte or > 32 of them. Correct -- you read 8K at a time, and hope that because you care getting 300 objects in one page instead of 100, you don't have to read the next two pages, and that because you have fewer pages to cache, the chance that any page you need to look up will reside in your cache is much higher. > > RLR> Object pointers may of course cross the entire space in theory, but > RLR> OIDs that we store in the database may indeed come from a set of quite > RLR> knowable size, allowing us to use less than 64 bits or 32 bits to > RLR> represent them. If we know the type of a slot to be only OIDs of a > RLR> particular class, we may very well know that that class has only 2**20 > RLR> members, and therefore demands only 20 bits, as opposed to 64, to > RLR> represent the entire space of possible OIDs. > > if you expect database being large enough for I/O to matter, why do you > think then that there would be that few objects? A database can't be so small that I/O doesn't matter. I suppose a database can be so small that performance doesn't matter---but if performance matters, then I/O matters. I think one of our differences is that you are looking at Elephant as it is, and I imagining Elephant with a caching system, and perhaps a file system that has a cache, and I'm not neglecting the relatively long time it takes for a CPU to retrieve a byte from memory which is not already in the CPU cache. > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Fri May 16 08:57:53 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Fri, 16 May 2008 10:57:53 +0200 (CEST) Subject: [elephant-devel] Quick hack for lisp backend. In-Reply-To: References: Message-ID: <63080.88.73.218.126.1210928273.squirrel@mail.stardawn.org> > I had never looked at Rucksack before. The code is very well > structured indeed! Yes, we're lucky. > A couple of thoughts: > > 1. Could the Rucksack B-Tree code be separated out, abstracted a > little and put in a separate project (eg, a cl-btree, or ldb for > example)? This would allow other lisp projects that are also looking > for a native B-Tree backend to make use of it. And all those projects > would benefit from the work done. I have stated my desire for this in the past. Would you like to work on it? > 2. It appears that Rucksack is still be developed. Could the author > (s) be contacted and see if they are interested in creating a > universal lisp backend that could be shared between Elephant and > Rucksack (more rows in the water so to speak)? Why not, give it a try. :) > 3. I think that if the Rucksack B-Tree can be used in some way its > the fastest way to get the project off the ground and get some > momentum behind it. Yes. I have feared that the dream of a Lisp btree backend will take a long time to become true, but with the already existing code we might get started really fast. Leslie From linus at nordberg.se Fri May 16 10:26:47 2008 From: linus at nordberg.se (Linus Nordberg) Date: Fri, 16 May 2008 12:26:47 +0200 Subject: [elephant-devel] Trouble loading libdb-4.5.so (SBCL, FreeBSD @ amd64) Message-ID: Hi, I'm trying to get Elephant running with SBCL-1.0.16 built with threading support, FreeBSD-6.2 on amd64, db-4.5 from ports. --8<---------------cut here---------------start------------->8--- * (asdf-install:install 'elephant) [...] ; $ gcc -L/usr/local/lib/db45/ -I/usr/local/include/db45/ -shared -fPIC -Wall -O2 -g /usr/local/lib/sbcl/site/elephant/src/db-bdb/libberkeley-db.c -o /usr/local/lib/sbcl/site/elephant/src/db-bdb/libberkeley-db.so -lm -ldb-4.5 Loading /usr/local/lib/db45/libdb-4.5.so fatal error encountered in SBCL pid 27021(tid 5424640): GC invariant lost, file "gencgc.c", line 829 Welcome to LDB, a low-level debugger for the Lisp runtime environment. ldb> backtrace Backtrace: 0: Foreign function lose, fp = 0x800c8ec30, ra = 0x40d5d6 1: Foreign function gc_alloc_update_page_tables, fp = 0x800c8ec60, ra = 0x4153ba 2: Foreign function gc_alloc_with_region, fp = 0x800c8ec90, ra = 0x416161 3: Foreign function alloc, fp = 0x800c8ecb0, ra = 0x418def 4: Foreign function _start, fp = 0x800c8ecd0, ra = 0x40930d 5: Foreign function alloc_unboxed, fp = 0x800c8ecf0, ra = 0x4094ac 6: Foreign function alloc_sap, fp = 0x800c8ed10, ra = 0x40960b 7: Foreign function interrupt_handle_now, fp = 0x800c8ed70, ra = 0x40ddd4 ldb> --8<---------------cut here---------------end--------------->8--- I'm really too clueless to even know where to start with this. I suppose it might well be an SBCL or libdb issue but I thought I'd rather check with you first. The same problem is easily reproduced with (require 'ele-bdb) which BTW shows that site/elephant/src/memutil/libmemutil.so is successfully loaded (uffi-1.6.0). --8<---------------cut here---------------start------------->8--- Attempting to load libmemutil.so... Loaded /usr/local/lib/sbcl/site/elephant/src/memutil/libmemutil.so Loading /usr/local/lib/db45/libdb-4.5.so fatal error encountered in SBCL pid 29178(tid 5423616): GC invariant lost, file "gencgc.c", line 829 --8<---------------cut here---------------end--------------->8--- The executive summary of SBCL's run-tests.sh goes like this: --8<---------------cut here---------------start------------->8--- Expected failure: callback.impure.lisp / UNDERFLOW-DETECTION Invalid exit status: clos-cache.impure.lisp Unexpected success: debug.impure.lisp / (UNDEFINED-FUNCTION BUG-346) Expected failure: debug.impure.lisp / (UNDEFINED-FUNCTION BUG-353) Expected failure: external-format.impure.lisp / (CHARACTER-DECODE-LARGE FORCE-END-OF-FILE) Invalid exit status: threads.impure.lisp Invalid exit status: timer.impure.lisp test failed, expected 104 return code, got 1 --8<---------------cut here---------------end--------------->8--- FWIW, the threads failure looks like this: --8<---------------cut here---------------start------------->8--- // Running /usr/local/ports/basejail/usr/ports/lang/sbcl/work/sbcl-1.0.16/tests/threads.test.sh /waiting Killed --8<---------------cut here---------------end--------------->8--- Please let me know if there's any more information that I should supply. Thanks. -- Linus From henrik at evahjelte.com Fri May 16 10:54:06 2008 From: henrik at evahjelte.com (Henrik Hjelte) Date: Fri, 16 May 2008 12:54:06 +0200 Subject: [elephant-devel] Quick hack for lisp backend. In-Reply-To: References: Message-ID: <50e8e4f60805160354v2db28084nf88dfa354b0104fb@mail.gmail.com> On Fri, May 16, 2008 at 12:50 AM, Glenn Tarcea wrote: > > I had never looked at Rucksack before. The code is very well structured > indeed! > > A couple of thoughts: > > 1. Could the Rucksack B-Tree code be separated out, abstracted a little and > put in a separate project (eg, a cl-btree, or ldb for example)? This would > allow other lisp projects that are also looking for a native B-Tree backend > to make use of it. And all those projects would benefit from the work done. I have a lot of experience with the rucksack b-tree code. It is great code, but once you want to start using it you will want full control of the file. I don't think a separate project will be fruitful. Copy/Paste or Kill/Yank is probably better IMO. > > 2. It appears that Rucksack is still be developed. Could the author(s) be > contacted and see if they are interested in creating a universal lisp > backend that could be shared between Elephant and Rucksack (more rows in the > water so to speak)? Previously Arthur Lemmens said he wanted to minimize dependencies. But you never know. > > 3. I think that if the Rucksack B-Tree can be used in some way its the > fastest way to get the project off the ground and get some momentum behind > it. Yes, It will be a fast and good start. There are testcases to copy as well. /Henrik Hjelte From leslie.polzer at gmx.net Fri May 16 11:19:07 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Fri, 16 May 2008 13:19:07 +0200 (CEST) Subject: [elephant-devel] Trouble loading libdb-4.5.so (SBCL, FreeBSD @ amd64) In-Reply-To: References: Message-ID: <62817.88.73.218.126.1210936747.squirrel@mail.stardawn.org> Elephant is borked in SBCL 1.0.16 (due to its heavy UFFI magic lifting), and it hasn't been fixed in the SBCL development repository yet, either. Use an earlier version of SBCL. Leslie From linus at nordberg.se Fri May 16 13:01:54 2008 From: linus at nordberg.se (Linus Nordberg) Date: Fri, 16 May 2008 15:01:54 +0200 Subject: [elephant-devel] Re: Trouble loading libdb-4.5.so (SBCL, FreeBSD @ amd64) References: <62817.88.73.218.126.1210936747.squirrel@mail.stardawn.org> Message-ID: "Leslie P. Polzer" wrote Fri, 16 May 2008 13:19:07 +0200 (CEST): | Elephant is borked in SBCL 1.0.16 (due to its heavy UFFI magic lifting), | and it hasn't been fixed in the SBCL development repository yet, either. Thanks. 1.0.10 shows the same behaviour while 1.0.6 seem to do better wrt to SBCL and FFI: Loading /usr/local/lib/db45/libdb-4.5.so debugger invoked on a SIMPLE-ERROR: Error opening shared object "/usr/local/lib/db45/libdb-4.5.so": /lib/libpthread.so.2: Undefined symbol "__usleep". This seem more related to libdb and threading. -- Linus From leslie.polzer at gmx.net Fri May 16 13:26:38 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Fri, 16 May 2008 15:26:38 +0200 (CEST) Subject: [elephant-devel] Re: Trouble loading libdb-4.5.so (SBCL, FreeBSD @ amd64) In-Reply-To: References: <62817.88.73.218.126.1210936747.squirrel@mail.stardawn.org> Message-ID: <61076.88.73.218.126.1210944398.squirrel@mail.stardawn.org> > Thanks. 1.0.10 shows the same behaviour Okay, this might or might not be the bug in .16 that I was referring to then. Leslie From killerstorm at newmail.ru Fri May 16 14:14:18 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Fri, 16 May 2008 17:14:18 +0300 Subject: [elephant-devel] Lisp Btrees: design considerations References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com><1210821332.23948.110.camel@penguin.yourdomain.com><8A9B375E3FCA40B1A3887976B80A9399@killer> <1210907177.29423.45.camel@penguin.yourdomain.com> Message-ID: <3A11F6AA4E744DD783A8307AE9732C2A@killer> ??>> or it will matter if people are doing lots of linear reads -- but it's ??>> hard for me to imagine such database access patterns. RLR> I think linear reads are quite common. They are, after all, why we RLR> have implemented range queries. Moreover, the act of reading all the RLR> data into memory is quite common if you are using a "Object RLR> Prevalance" model. exactly. if we are speaking of a server, it reads all the data into memory and then works for monthes for it, so I/O performance is near to irrelevant for it. RLR> find the best one. In many cases, this is indeed a linear scan; RLR> for example many join operations can best be performed by performing RLR> a linear scan of at least one of the tables in question. The aggregate RLR> functions (sum, min, max, avg, count) are another example. Another RLR> typical example is "access all page views of my website from Thursday RLR> to Friday". if we've agreed that I/O performance matters only for large databases, linear scans will take very significant amount of time in any case. it could be two hours without optimization and a hour with, but for me that's not that large difference: if one can wait a hour to get results, can't he wait two hours then? RLR> more objects within an enterprise to avoid I/O costs. Perhaps you can RLR> post a description of your database usage pattern, to explain why it RLR> is dominated by point queries. simply because it's interactive -- reading full table just to do a join and get few values out of there is simply non-acceptable for interactive applications, because as table grows query will take prohibitive amount of time. to avoid full scans i've added derived indices that can be used to pick only stuff that is displayed, and utility function that allows iterating across multiple indices at once. ??>> bits, huh? what storage nowadays works on bit level? RLR> You are quite correct that most usage of storage is done at the byte RLR> and word level---I am arguing that this is an opportunity for us. Of RLR> course, one must understand that the page will be read and written as RLR> an 8K chunk of bytes, which will contain carefully packed bit sets RLR> that do not respect byte boundaries. doesn't this add node addressing problems? how do you point on a specific node from other pages with such approach? RLR> file system that has a cache, and I'm not neglecting the relatively RLR> long time it takes for a CPU to retrieve a byte from memory which is RLR> not already in the CPU cache. if we were using carefully optimized assembly language, then probably indeed memory latency/bandwidth would be a limiting factor (however, if gigabytes per second bandwidth is not enough, most likely that means that algorithms -- like using each-to-each comparisons -- suck). but if we are using high-level language like Lisp, it's much more likely that CPU overhead will be large comparing to memory latency. and as compression, even simpiest/fastest one will only add to CPU overhead, i'm very skeptical about it's benefits. by the way, interesting fact: some time ago i've found that even with cache enabled in elehant/postmodern slots reads are relatively slow -- on 10 thousands per second scale or so. it was quite surprising that abusing function is form-slot-key (that is used in CL-SQL too): (defun form-slot-key (oid name) (format nil "~A ~A" oid name)) and even more surprising was to find out that there is no portable and fast way to convert interger to string. so i had to use SBCL's internal function called sb-impl::quick-integer-to-string -- apparently SBCL developers knew about this problem so they made this function for themselves. so i think that subtle effects like I/O bandwidth will be only seen in we'll hardcore optimize elephant and applications themselves. but as it is now, storage backend won't be of a big influence of overal performance and just needs to be moderately good From bryan.emrys at gmail.com Fri May 16 14:31:27 2008 From: bryan.emrys at gmail.com (Bryan Emrys) Date: Fri, 16 May 2008 07:31:27 -0700 Subject: [elephant-devel] Trouble loading libdb-4.5.so (SBCL, FreeBSD @ amd64) In-Reply-To: <62817.88.73.218.126.1210936747.squirrel@mail.stardawn.org> References: <62817.88.73.218.126.1210936747.squirrel@mail.stardawn.org> Message-ID: <200805160731.27270.bryan.emrys@gmail.com> Anyone contacted the SBCL team on this? Bryan On Friday 16 May 2008 04:19:07 am Leslie P. Polzer wrote: > > Elephant is borked in SBCL 1.0.16 (due to its heavy UFFI magic lifting), > and it hasn't been fixed in the SBCL development repository yet, either. > > Use an earlier version of SBCL. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > From eslick at media.mit.edu Fri May 16 15:26:56 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Fri, 16 May 2008 11:26:56 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <3A11F6AA4E744DD783A8307AE9732C2A@killer> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com><1210821332.23948.110.camel@penguin.yourdomain.com><8A9B375E3FCA40B1A3887976B80A9399@killer> <1210907177.29423.45.camel@penguin.yourdomain.com> <3A11F6AA4E744DD783A8307AE9732C2A@killer> Message-ID: <60D907E5-DC83-48C5-BF05-DA9A2109FAC4@media.mit.edu> > I just wanted to clarify a few points in this interesting discussion. > but if we are using high-level language like Lisp, it's much more > likely that CPU overhead will be large comparing to memory latency. > and as compression, even simpiest/fastest one will only add to CPU > overhead, i'm very skeptical about it's benefits. I doubt that using a Lisp with a modern compiler affects CPU time in any interesting way; I've observed that algorithm implementation with reasonable data structure choices is within 10's of % points of the same algorithm in C. I don't care how fast your CPU is, it's the data access patterns that dominate time unless you're working on small or highly local datasets, then the inner loop speed _might_ matter. > by the way, interesting fact: some time ago i've found that even > with cache enabled in elehant/postmodern > slots reads are relatively slow -- on 10 thousands per second scale > or so. it was quite surprising > that abusing function is form-slot-key (that is used in CL-SQL too): At least in Allegro, the profiler only counts thread time and that much of the delay shown by 'time' vs. the profiler is taken up waiting on cache-miss IO. It may be that thread time is only a handful of percentage points of the total time? > (defun form-slot-key (oid name) > (format nil "~A ~A" oid name)) Format is notoriously slow. Still, I find it hard to believe that it matters that much. (See above) > and even more surprising was to find out that there is no portable > and fast way to convert interger to string. > so i had to use SBCL's internal function called sb-impl::quick- > integer-to-string -- apparently SBCL > developers knew about this problem so they made this function for > themselves. For fun, I did a test with princ and for integers only it's 2x faster, for the above idiom it's only about 50% faster, but some of that is the with-string-stream idea. If you do it manually as they did and write it to an array one character at a time it can get pretty fast, but probably only 3x-4x. > so i think that subtle effects like I/O bandwidth will be only seen > in we'll hardcore optimize elephant and applications themselves. > but as it is now, storage backend won't be of a big influence of > overal performance and just needs to be moderately good I agree with this in general. However I feel like this whole discussion is premature optimization being done while blindfolded. I do think that over time it would be nice to give people tools like compression to enable/disable for their particular applications, but it's probably not worth all the debate until we solve some of the bigger problems in our copious free time. By the way, if you are caching your data in memory and using the most conservative transaction model of BDB (blocking commits) you are write bandwidth limited. At the end of each little transaction, you have to write a log entry to disk, then flush a 16kb page to disk, then wait for the controller to say OK, then write the log again, then return to your regularly scheduled programming. In prevalence, you only write the log and occasionally dump to disk. I suspect that the SQL back ends function similarly. When you finish a transaction you have to wait for this to all happen over a socket... Prevalence is much, much faster because you don't have to flush data structures on each commit, so cl-prevalence performance with Elephant's data and transaction abstractions would be a really nice design point. I wonder if we would get some of this benefit from a Rucksack adaptation? Ian From eslick at media.mit.edu Fri May 16 15:33:31 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Fri, 16 May 2008 11:33:31 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <60D907E5-DC83-48C5-BF05-DA9A2109FAC4@media.mit.edu> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com><1210821332.23948.110.camel@penguin.yourdomain.com><8A9B375E3FCA40B1A3887976B80A9399@killer> <1210907177.29423.45.camel@penguin.yourdomain.com> <3A11F6AA4E744DD783A8307AE9732C2A@killer> <60D907E5-DC83-48C5-BF05-DA9A2109FAC4@media.mit.edu> Message-ID: I checked, Rucksack writes dirty objects to disk on each commit - so should have the same performance profile as our existing backends, of course with different constants... Ian On May 16, 2008, at 11:26 AM, Ian Eslick wrote: >> > > I just wanted to clarify a few points in this interesting discussion. > >> but if we are using high-level language like Lisp, it's much more >> likely that CPU overhead will be large comparing to memory latency. >> and as compression, even simpiest/fastest one will only add to CPU >> overhead, i'm very skeptical about it's benefits. > > I doubt that using a Lisp with a modern compiler affects CPU time in > any interesting way; I've observed that algorithm implementation > with reasonable data structure choices is within 10's of % points of > the same algorithm in C. I don't care how fast your CPU is, it's > the data access patterns that dominate time unless you're working on > small or highly local datasets, then the inner loop speed _might_ > matter. > >> by the way, interesting fact: some time ago i've found that even >> with cache enabled in elehant/postmodern >> slots reads are relatively slow -- on 10 thousands per second scale >> or so. it was quite surprising >> that abusing function is form-slot-key (that is used in CL-SQL too): > > At least in Allegro, the profiler only counts thread time and that > much of the delay shown by 'time' vs. the profiler is taken up > waiting on cache-miss IO. It may be that thread time is only a > handful of percentage points of the total time? > >> (defun form-slot-key (oid name) >> (format nil "~A ~A" oid name)) > > Format is notoriously slow. Still, I find it hard to believe that > it matters that much. (See above) > >> and even more surprising was to find out that there is no portable >> and fast way to convert interger to string. >> so i had to use SBCL's internal function called sb-impl::quick- >> integer-to-string -- apparently SBCL >> developers knew about this problem so they made this function for >> themselves. > > For fun, I did a test with princ and for integers only it's 2x > faster, for the above idiom it's only about 50% faster, but some of > that is the with-string-stream idea. If you do it manually as they > did and write it to an array one character at a time it can get > pretty fast, but probably only 3x-4x. > >> so i think that subtle effects like I/O bandwidth will be only seen >> in we'll hardcore optimize elephant and applications themselves. >> but as it is now, storage backend won't be of a big influence of >> overal performance and just needs to be moderately good > > I agree with this in general. However I feel like this whole > discussion is premature optimization being done while blindfolded. > I do think that over time it would be nice to give people tools like > compression to enable/disable for their particular applications, but > it's probably not worth all the debate until we solve some of the > bigger problems in our copious free time. > > By the way, if you are caching your data in memory and using the > most conservative transaction model of BDB (blocking commits) you > are write bandwidth limited. At the end of each little transaction, > you have to write a log entry to disk, then flush a 16kb page to > disk, then wait for the controller to say OK, then write the log > again, then return to your regularly scheduled programming. In > prevalence, you only write the log and occasionally dump to disk. I > suspect that the SQL back ends function similarly. When you finish > a transaction you have to wait for this to all happen over a socket... > > Prevalence is much, much faster because you don't have to flush data > structures on each commit, so cl-prevalence performance with > Elephant's data and transaction abstractions would be a really nice > design point. I wonder if we would get some of this benefit from a > Rucksack adaptation? > > Ian > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Fri May 16 16:43:34 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Fri, 16 May 2008 18:43:34 +0200 (CEST) Subject: [elephant-devel] Trouble loading libdb-4.5.so (SBCL, FreeBSD @ amd64) In-Reply-To: <200805160731.27270.bryan.emrys@gmail.com> References: <62817.88.73.218.126.1210936747.squirrel@mail.stardawn.org> <200805160731.27270.bryan.emrys@gmail.com> Message-ID: <64004.88.73.218.126.1210956214.squirrel@mail.stardawn.org> > Anyone contacted the SBCL team on this? This is an issue that has been well-known on sbcl-devel for the last few weeks. Leslie From linus at nordberg.se Fri May 16 17:39:31 2008 From: linus at nordberg.se (Linus Nordberg) Date: Fri, 16 May 2008 19:39:31 +0200 Subject: [elephant-devel] Re: Trouble loading libdb-4.5.so (SBCL, FreeBSD @ amd64) References: <62817.88.73.218.126.1210936747.squirrel@mail.stardawn.org> Message-ID: Linus Nordberg wrote Fri, 16 May 2008 15:01:54 +0200: | Thanks. 1.0.10 shows the same behaviour while 1.0.6 seem to do better | wrt to SBCL and FFI: I spoke too early. This was on SBCL 1.0.6 without threading support. Building 1.0.6 with threading results in The CC environment variable has not been set in SB-GROVEL. Since this variable should always be set during the SBCL build process, this might indicate an SBCL with a broken contrib installation. and similar. This is clearly not for me. I'd better find a plain old Debian box on x86 or something. Thanks anyway. -- Linus From killerstorm at newmail.ru Fri May 16 19:50:39 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Fri, 16 May 2008 22:50:39 +0300 Subject: [elephant-devel] Lisp Btrees: design considerations References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com><1210821332.23948.110.camel@penguin.yourdomain.com><8A9B375E3FCA40B1A3887976B80A9399@killer><1210907177.29423.45.camel@penguin.yourdomain.com><3A11F6AA4E744DD783A8307AE9732C2A@killer> <60D907E5-DC83-48C5-BF05-DA9A2109FAC4@media.mit.edu> Message-ID: <20E15061E450411CAEA0AF8A58AFCC4A@killer> IE> I doubt that using a Lisp with a modern compiler affects CPU time in IE> any interesting way; IE> I've observed that algorithm implementation with reasonable data IE> structure choices is within 10's of % points of the same algorithm in IE> C. it's possible to optimize single function, but typical (idiomatic) Lisp programs: * generate megatons of garbage * use dynamic type dispatch (lacking type declarations) and often have signficantly inferior performance. IE> I don't care how fast your CPU is, it's the data access patterns that IE> dominate time unless you're working on small or highly local datasets, IE> then the inner loop speed _might_ matter. i'm assuming you're speaking about memory bandwidth, because i think disk I/O will matter only on relatively large databases. memory bandwidth is also eaten in large quantities by garbage generator. let's take a concrete piece of software: Elephant's serializer. i'm assuming it's important piece of code and was optimized a lot? i'm taking test case from code Robert once have posted, little bit modifed: (defparameter *teststring* "supercalifragiliciousexpialidocious") (defun dotest (n s) (dotimes (x n) (in-out-value s))) ELE-TESTS> (time (dotest (truncate (* 1000 1000) (length *teststring*)) *teststring*)) Evaluation took: 0.249 seconds of real time 0.248016 seconds of user run time 0.0 seconds of system run time 0 calls to %EVAL 0 page faults and 6,398,328 bytes consed. ;; testing on core2 2.1 GHz, SBCL 1.0.13 not that great, ain't it? it processed roughly 10 megabytes of raw string data per second (5 MB serialized, 5 MB deserialized) -- that's 10 times slower than linear read spead of modern HDDs. and for 1 MB of data it produced 2 MB of garbage. or there is something wrong with my test? ??>> that abusing function is form-slot-key (that is used in CL-SQL too): IE> At least in Allegro, the profiler only counts thread time and that IE> much of the delay shown by 'time' vs. the profiler is taken up waiting IE> on cache-miss IO. It may be that thread time is only a handful of IE> percentage points of the total time? i'm pretty sure there was very little I/O in my tests. ??>> and even more surprising was to find out that there is no portable ??>> and fast way to convert interger to string. ??>> so i had to use SBCL's internal function called sb-impl::quick- ??>> integer-to-string -- apparently SBCL ??>> developers knew about this problem so they made this function for ??>> themselves. IE> For fun, I did a test with princ and for integers only it's 2x faster, IE> for the above idiom it's only about 50% faster, but some of that is IE> the with-string-stream idea. If you do it manually as they did and IE> write it to an array one character at a time it can get pretty fast, IE> but probably only 3x-4x. maybe they fixed something in SBCL? i'm using version 1.0.13 and get very different results, with functions that are actually in elephant (db-postmodern's function is optimized, db-clsql's is not): CL-USER> (defun dotest-pm (n o) (loop for i from 0 below n summing (length (db-postmodern::form-slot-key i o)))) DOTEST-PM CL-USER> (defun dotest-clsql (n o) (loop for i from 0 below n summing (length (db-clsql::form-slot-key i o)))) DOTEST-CLSQL CL-USER> (time (dotest-pm 100000 'o)) Evaluation took: 0.05 seconds of real time 0.052003 seconds of user run time 0.0 seconds of system run time [Run times include 0.024 seconds GC run time.] 0 calls to %EVAL 0 page faults and 5,592,976 bytes consed. 688890 CL-USER> (time (dotest-clsql 100000 'o)) Evaluation took: 1.502 seconds of real time 1.416088 seconds of user run time 0.084005 seconds of system run time [Run times include 0.7 seconds GC run time.] 0 calls to %EVAL 0 page faults and 404,779,792 bytes consed. 688890 optimized version seems to be 30 times faster. IE> I agree with this in general. However I feel like this whole IE> discussion is premature optimization being done while blindfolded. well, you guys started it discussing compression and other aspects :) IE> Prevalence is much, much faster because you don't have to flush data IE> structures on each commit, so cl-prevalence performance with IE> Elephant's data and transaction abstractions would be a really nice IE> design point. yep, that would be very interesting IE> I wonder if we would get some of this benefit from a Rucksack IE> adaptation? perhaps there is a way to hack it one way or another. one (high-level) way is to serialize dirty objects into different location (log) in some customized format that will allow us bring them back to normal data store later. another (low-level) way is to hijack heap (as far as i understood from a brief look rucksack does all reads/writes via heap object) to log changes instead of writing them to disk. also it might be good to use mmaped files -- OS can decide when to write changes by itself, while we could write log for safety. but mmaped files are definitely not pure Lisp :( From eslick at media.mit.edu Fri May 16 22:54:35 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Fri, 16 May 2008 18:54:35 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <20E15061E450411CAEA0AF8A58AFCC4A@killer> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com><1210821332.23948.110.camel@penguin.yourdomain.com><8A9B375E3FCA40B1A3887976B80A9399@killer><1210907177.29423.45.camel@penguin.yourdomain.com><3A11F6AA4E744DD783A8307AE9732C2A@killer> <60D907E5-DC83-48C5-BF05-DA9A2109FAC4@media.mit.edu> <20E15061E450411CAEA0AF8A58AFCC4A@killer> Message-ID: <4ED67803-2D37-484D-8579-F59C831BF298@media.mit.edu> I don't disagree with you that it's easy to create expensive/slow programs. My only point was that when it matters, it doesn't necessarily have to be a determinative issue. The big overheads in Elephant are due to Lisp's data model and the use of the MOP. The serializer is currently designed for safety and generality across lisps with reasonable, but not completely optimal performance (since the whole point is dealing with the implementation of the lisp data model, doing well would require specializing on each lisp's internal data representation so I can just do byte copies rather than using the general indirection apis). The code is there; perhaps I should selective re-enable the optimizations for SBCL... Perhaps these points underscore your argument about the overhead of Lisp. It is more work to write fast programs in lisp because performance is at odds from the common idioms. Other languages, like C, make it easy to write efficient programs, but much harder to write programs quickly as well as to maintain large programs. There is no free lunch, as my physicists friends are fond of saying. I was curious where the time was going in the example. The deserializer creates fresh strings of type 'character. Aren't characters 4 bytes in unicode SBCL? That would mean you're handling 8M bytes and for 4M bytes of data it produced 2M bytes of garbage (probably all consing in the test framework). Allegro uses UTF-16 so the same test is only 2MBytes of total string input. I'm not sure where the 25% overhead came from though. The time cost is still surprising though. ELE-USER> (time (dotest (truncate (* 1000 1000) (length *teststring*)) *teststring*)) ; cpu time (non-gc) 140 msec user, 0 msec system ; cpu time (gc) 0 msec user, 0 msec system ; cpu time (total) 140 msec user, 0 msec system ; real time 151 msec ; space allocation: ; 228,879 cons cells, 2,516,384 other bytes, 0 static bytes NIL ELE-USER> About half of the memory overhead is consing in dotimes, etc. However, notice that it didn't trigger a GC. Even when the GC runs, it isn't a significant cause of time overhead. Calling free to malloc a bunch is also a little expensive... Cheers, Ian PS - There was also an ill-conceived safety check that crept into the serializer recently which accounts for about half of the time in the example. I'll have to think about why it is there. On May 16, 2008, at 3:50 PM, Alex Mizrahi wrote: > IE> I doubt that using a Lisp with a modern compiler affects CPU > time in > IE> any interesting way; > IE> I've observed that algorithm implementation with reasonable data > IE> structure choices is within 10's of % points of the same > algorithm in > IE> C. > > it's possible to optimize single function, but typical (idiomatic) > Lisp programs: > * generate megatons of garbage > * use dynamic type dispatch (lacking type declarations) > and often have signficantly inferior performance. > > IE> I don't care how fast your CPU is, it's the data access > patterns that > IE> dominate time unless you're working on small or highly local > datasets, > IE> then the inner loop speed _might_ matter. > > i'm assuming you're speaking about memory bandwidth, because i think > disk I/O will matter only on relatively large databases. > > memory bandwidth is also eaten in large quantities by garbage > generator. > > let's take a concrete piece of software: Elephant's serializer. i'm > assuming it's important piece of code and was optimized a lot? > > i'm taking test case from code Robert once have posted, little bit > modifed: > > (defparameter *teststring* "supercalifragiliciousexpialidocious") > (defun dotest (n s) > (dotimes (x n) > (in-out-value s))) > > ELE-TESTS> (time (dotest (truncate (* 1000 1000) (length > *teststring*)) *teststring*)) > Evaluation took: > 0.249 seconds of real time > 0.248016 seconds of user run time > 0.0 seconds of system run time > 0 calls to %EVAL > 0 page faults and > 6,398,328 bytes consed. > > > ;; testing on core2 2.1 GHz, SBCL 1.0.13 > > not that great, ain't it? > it processed roughly 10 megabytes of raw string data per second (5 > MB serialized, 5 MB deserialized) -- that's 10 times slower than > linear read spead of modern HDDs. > and for 1 MB of data it produced 2 MB of garbage. > > or there is something wrong with my test? > > > ??>> that abusing function is form-slot-key (that is used in CL-SQL > too): > > IE> At least in Allegro, the profiler only counts thread time and that > IE> much of the delay shown by 'time' vs. the profiler is taken up > waiting > IE> on cache-miss IO. It may be that thread time is only a handful of > IE> percentage points of the total time? > > i'm pretty sure there was very little I/O in my tests. > > ??>> and even more surprising was to find out that there is no > portable > ??>> and fast way to convert interger to string. > ??>> so i had to use SBCL's internal function called sb-impl::quick- > ??>> integer-to-string -- apparently SBCL > ??>> developers knew about this problem so they made this function for > ??>> themselves. > > IE> For fun, I did a test with princ and for integers only it's 2x > faster, > IE> for the above idiom it's only about 50% faster, but some of that > is > IE> the with-string-stream idea. If you do it manually as they did > and > IE> write it to an array one character at a time it can get pretty > fast, > IE> but probably only 3x-4x. > > maybe they fixed something in SBCL? i'm using version 1.0.13 and get > very different results, with functions that are actually in elephant > (db-postmodern's function is optimized, db-clsql's is not): > > CL-USER> (defun dotest-pm (n o) > (loop for i from 0 below n > summing (length > (db-postmodern::form-slot-key i o)))) > DOTEST-PM > > CL-USER> (defun dotest-clsql (n o) > (loop for i from 0 below n > summing (length > (db-clsql::form-slot-key i o)))) > DOTEST-CLSQL > > > CL-USER> (time (dotest-pm 100000 'o)) > Evaluation took: > 0.05 seconds of real time > 0.052003 seconds of user run time > 0.0 seconds of system run time > [Run times include 0.024 seconds GC run time.] > 0 calls to %EVAL > 0 page faults and > 5,592,976 bytes consed. > 688890 > CL-USER> (time (dotest-clsql 100000 'o)) > Evaluation took: > 1.502 seconds of real time > 1.416088 seconds of user run time > 0.084005 seconds of system run time > [Run times include 0.7 seconds GC run time.] > 0 calls to %EVAL > 0 page faults and > 404,779,792 bytes consed. > 688890 > > > optimized version seems to be 30 times faster. > > > IE> I agree with this in general. However I feel like this whole > IE> discussion is premature optimization being done while blindfolded. > > well, you guys started it discussing compression and other aspects :) > > IE> Prevalence is much, much faster because you don't have to flush > data > IE> structures on each commit, so cl-prevalence performance with > IE> Elephant's data and transaction abstractions would be a really > nice > IE> design point. > > yep, that would be very interesting > > IE> I wonder if we would get some of this benefit from a Rucksack > IE> adaptation? > > perhaps there is a way to hack it one way or another. > one (high-level) way is to serialize dirty objects into different > location (log) in some customized format that will allow us bring > them back to normal data store later. > another (low-level) way is to hijack heap (as far as i understood > from a brief look rucksack does all reads/writes via heap object) to > log changes instead of writing them to disk. > > also it might be good to use mmaped files -- OS can decide when to > write changes by itself, while we could write log for safety. > but mmaped files are definitely not pure Lisp :( > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Fri May 16 23:02:39 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Fri, 16 May 2008 19:02:39 -0400 Subject: [elephant-devel] Lisp Btrees: design considerations In-Reply-To: <4ED67803-2D37-484D-8579-F59C831BF298@media.mit.edu> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org><13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu><1210773234.23948.47.camel@penguin.yourdomain.com><1210821332.23948.110.camel@penguin.yourdomain.com><8A9B375E3FCA40B1A3887976B80A9399@killer><1210907177.29423.45.camel@penguin.yourdomain.com><3A11F6AA4E744DD783A8307AE9732C2A@killer> <60D907E5-DC83-48C5-BF05-DA9A2109FAC4@media.mit.edu> <20E15061E450411CAEA0AF8A58AFCC4A@killer> <4ED67803-2D37-484D-8579-F59C831BF298@media.mit.edu> Message-ID: <7A6D4EFA-C68B-4755-9CEB-3065CF761A67@media.mit.edu> I should also say that the serializer has to be thread safe, so there is some non-trivial time overhead in terms of locking on each serializer call. I wish I could do primitive atomic ops or at least something like a futex in common lisp, then I could create deadlock free data structures and dispense with this overhead.... Ian On May 16, 2008, at 6:54 PM, Ian Eslick wrote: > I don't disagree with you that it's easy to create expensive/slow > programs. My only point was that when it matters, it doesn't > necessarily have to be a determinative issue. > > The big overheads in Elephant are due to Lisp's data model and the > use of the MOP. The serializer is currently designed for safety and > generality across lisps with reasonable, but not completely optimal > performance (since the whole point is dealing with the > implementation of the lisp data model, doing well would require > specializing on each lisp's internal data representation so I can > just do byte copies rather than using the general indirection > apis). The code is there; perhaps I should selective re-enable the > optimizations for SBCL... > > Perhaps these points underscore your argument about the overhead of > Lisp. It is more work to write fast programs in lisp because > performance is at odds from the common idioms. Other languages, > like C, make it easy to write efficient programs, but much harder to > write programs quickly as well as to maintain large programs. There > is no free lunch, as my physicists friends are fond of saying. > > > I was curious where the time was going in the example. The > deserializer creates fresh strings of type 'character. Aren't > characters 4 bytes in unicode SBCL? That would mean you're handling > 8M bytes and for 4M bytes of data it produced 2M bytes of garbage > (probably all consing in the test framework). Allegro uses UTF-16 > so the same test is only 2MBytes of total string input. I'm not > sure where the 25% overhead came from though. > > The time cost is still surprising though. > > ELE-USER> (time (dotest (truncate (* 1000 1000) (length > *teststring*)) *teststring*)) > ; cpu time (non-gc) 140 msec user, 0 msec system > ; cpu time (gc) 0 msec user, 0 msec system > ; cpu time (total) 140 msec user, 0 msec system > ; real time 151 msec > ; space allocation: > ; 228,879 cons cells, 2,516,384 other bytes, 0 static bytes > NIL > ELE-USER> > > About half of the memory overhead is consing in dotimes, etc. > However, notice that it didn't trigger a GC. Even when the GC runs, > it isn't a significant cause of time overhead. Calling free to > malloc a bunch is also a little expensive... > > Cheers, > Ian > > PS - There was also an ill-conceived safety check that crept into > the serializer recently which accounts for about half of the time in > the example. I'll have to think about why it is there. > > > > > On May 16, 2008, at 3:50 PM, Alex Mizrahi wrote: > >> IE> I doubt that using a Lisp with a modern compiler affects CPU >> time in >> IE> any interesting way; >> IE> I've observed that algorithm implementation with reasonable data >> IE> structure choices is within 10's of % points of the same >> algorithm in >> IE> C. >> >> it's possible to optimize single function, but typical (idiomatic) >> Lisp programs: >> * generate megatons of garbage >> * use dynamic type dispatch (lacking type declarations) >> and often have signficantly inferior performance. >> >> IE> I don't care how fast your CPU is, it's the data access >> patterns that >> IE> dominate time unless you're working on small or highly local >> datasets, >> IE> then the inner loop speed _might_ matter. >> >> i'm assuming you're speaking about memory bandwidth, because i think >> disk I/O will matter only on relatively large databases. >> >> memory bandwidth is also eaten in large quantities by garbage >> generator. >> >> let's take a concrete piece of software: Elephant's serializer. i'm >> assuming it's important piece of code and was optimized a lot? >> >> i'm taking test case from code Robert once have posted, little bit >> modifed: >> >> (defparameter *teststring* "supercalifragiliciousexpialidocious") >> (defun dotest (n s) >> (dotimes (x n) >> (in-out-value s))) >> >> ELE-TESTS> (time (dotest (truncate (* 1000 1000) (length >> *teststring*)) *teststring*)) >> Evaluation took: >> 0.249 seconds of real time >> 0.248016 seconds of user run time >> 0.0 seconds of system run time >> 0 calls to %EVAL >> 0 page faults and >> 6,398,328 bytes consed. >> >> >> ;; testing on core2 2.1 GHz, SBCL 1.0.13 >> >> not that great, ain't it? >> it processed roughly 10 megabytes of raw string data per second (5 >> MB serialized, 5 MB deserialized) -- that's 10 times slower than >> linear read spead of modern HDDs. >> and for 1 MB of data it produced 2 MB of garbage. >> >> or there is something wrong with my test? >> >> >> ??>> that abusing function is form-slot-key (that is used in CL-SQL >> too): >> >> IE> At least in Allegro, the profiler only counts thread time and >> that >> IE> much of the delay shown by 'time' vs. the profiler is taken up >> waiting >> IE> on cache-miss IO. It may be that thread time is only a handful >> of >> IE> percentage points of the total time? >> >> i'm pretty sure there was very little I/O in my tests. >> >> ??>> and even more surprising was to find out that there is no >> portable >> ??>> and fast way to convert interger to string. >> ??>> so i had to use SBCL's internal function called sb-impl::quick- >> ??>> integer-to-string -- apparently SBCL >> ??>> developers knew about this problem so they made this function >> for >> ??>> themselves. >> >> IE> For fun, I did a test with princ and for integers only it's 2x >> faster, >> IE> for the above idiom it's only about 50% faster, but some of >> that is >> IE> the with-string-stream idea. If you do it manually as they >> did and >> IE> write it to an array one character at a time it can get pretty >> fast, >> IE> but probably only 3x-4x. >> >> maybe they fixed something in SBCL? i'm using version 1.0.13 and >> get very different results, with functions that are actually in >> elephant >> (db-postmodern's function is optimized, db-clsql's is not): >> >> CL-USER> (defun dotest-pm (n o) >> (loop for i from 0 below n >> summing (length >> (db-postmodern::form-slot-key i o)))) >> DOTEST-PM >> >> CL-USER> (defun dotest-clsql (n o) >> (loop for i from 0 below n >> summing (length >> (db-clsql::form-slot-key i o)))) >> DOTEST-CLSQL >> >> >> CL-USER> (time (dotest-pm 100000 'o)) >> Evaluation took: >> 0.05 seconds of real time >> 0.052003 seconds of user run time >> 0.0 seconds of system run time >> [Run times include 0.024 seconds GC run time.] >> 0 calls to %EVAL >> 0 page faults and >> 5,592,976 bytes consed. >> 688890 >> CL-USER> (time (dotest-clsql 100000 'o)) >> Evaluation took: >> 1.502 seconds of real time >> 1.416088 seconds of user run time >> 0.084005 seconds of system run time >> [Run times include 0.7 seconds GC run time.] >> 0 calls to %EVAL >> 0 page faults and >> 404,779,792 bytes consed. >> 688890 >> >> >> optimized version seems to be 30 times faster. >> >> >> IE> I agree with this in general. However I feel like this whole >> IE> discussion is premature optimization being done while >> blindfolded. >> >> well, you guys started it discussing compression and other aspects :) >> >> IE> Prevalence is much, much faster because you don't have to flush >> data >> IE> structures on each commit, so cl-prevalence performance with >> IE> Elephant's data and transaction abstractions would be a really >> nice >> IE> design point. >> >> yep, that would be very interesting >> >> IE> I wonder if we would get some of this benefit from a Rucksack >> IE> adaptation? >> >> perhaps there is a way to hack it one way or another. >> one (high-level) way is to serialize dirty objects into different >> location (log) in some customized format that will allow us bring >> them back to normal data store later. >> another (low-level) way is to hijack heap (as far as i understood >> from a brief look rucksack does all reads/writes via heap object) to >> log changes instead of writing them to disk. >> >> also it might be good to use mmaped files -- OS can decide when to >> write changes by itself, while we could write log for safety. >> but mmaped files are definitely not pure Lisp :( >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From linus at nordberg.se Fri May 16 23:19:54 2008 From: linus at nordberg.se (Linus Nordberg) Date: Sat, 17 May 2008 01:19:54 +0200 Subject: [elephant-devel] Re: Trouble loading libdb-4.5.so (SBCL, FreeBSD @ amd64) References: <62817.88.73.218.126.1210936747.squirrel@mail.stardawn.org> Message-ID: Linus Nordberg wrote Fri, 16 May 2008 15:01:54 +0200: | | Elephant is borked in SBCL 1.0.16 (due to its heavy UFFI magic lifting), | | and it hasn't been fixed in the SBCL development repository yet, either. | | Thanks. 1.0.10 shows the same behaviour while 1.0.6 seem to do better | wrt to SBCL and FFI: | | Loading /usr/local/lib/db45/libdb-4.5.so | | debugger invoked on a SIMPLE-ERROR: | Error opening shared object "/usr/local/lib/db45/libdb-4.5.so": | /lib/libpthread.so.2: Undefined symbol "__usleep". | | This seem more related to libdb and threading. Sorry for keeping on following up on my own postings, but threaded SBCL-1.0.16, elephant-0.9.0 with bdb, uffi-1.6.0 play together in a jail on FreeBSD-6.2 @ amd64. At least the BDB tests doesn't complain. I had to rebuild db45 without the `--enable-pthread_api' configure option to get rid of the "undefined symbol __usleep" thing while loading libdb-4.5.so. Will that break other things wrt Elephant use of BDB? -- Linus From eslick at media.mit.edu Sat May 3 14:33:06 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 3 May 2008 10:33:06 -0400 Subject: [elephant-devel] 13 Warnings when loading elephant (SBCL, Mac OS X) In-Reply-To: <0B3AA84C-1964-4824-812E-F96C3DD84E31@jensteich.de> References: <0B3AA84C-1964-4824-812E-F96C3DD84E31@jensteich.de> Message-ID: <9F3CEF52-C645-4595-A69B-3B980A083D3D@media.mit.edu> You shouldn't need to worry. Does it pass all the regression tests? On Apr 7, 2008, at 1:50 AM, Jens Teich wrote: > SBCL 1.0.15 > Mac OS X 10.5 > > Do I have to worry about any of these warnings? Jens > > ; SLIME 2006-04-20 > CL-USER> (asdf :elephant) > ; loading system definition from > ; /Users/jensteich/Lisp/elephant/elephant.asd into # > ; registering # as ELEPHANT > ; loading system definition from /Users/jensteich/Lisp/cl-base64.asd > into > ; # > ; registering # as CL-BASE64 > ; registering # as CL-BASE64-TESTS > ; loading system definition from /Users/jensteich/Lisp/uffi.asd into > ; # > ; registering # as UFFI > > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > package.lisp" (written 10 MAY 2006 02:42:59 AM): > ; compiling (IN-PACKAGE #:CL-USER) > ; compiling (DEFPACKAGE #:UFFI ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/package.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > primitives.lisp" (written 18 APR 2006 02:07:09 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFMACRO DEF-CONSTANT ...) > ; compiling (DEFMACRO DEF-TYPE ...) > ; compiling (DEFMACRO NULL-CHAR-P ...) > ; compiling (DEFMACRO DEF-FOREIGN-TYPE ...) > ; compiling (DEFVAR +TYPE-CONVERSION-HASH+ ...) > ; compiling (DEFVAR *CMU-DEF-TYPE-HASH* ...) > ; compiling (DEFVAR *CMU-SBCL-DEF-TYPE-LIST* ...) > ; compiling (DEFVAR *TYPE-CONVERSION-LIST* ...) > ; compiling (SETQ *TYPE-CONVERSION-LIST* ...) > ; compiling (DOLIST (TYPE *TYPE-CONVERSION-LIST*) ...) > ; compiling (DOLIST (TYPE *CMU-SBCL-DEF-TYPE-LIST*) ...) > ; compiling (DEFUN BASIC-CONVERT-FROM-UFFI-TYPE ...) > ; compiling (DEFUN %CONVERT-FROM-UFFI-TYPE ...) > ; compiling (DEFUN CONVERT-FROM-UFFI-TYPE ...) > ; compiling (WHEN (CHAR= #\a ...) ...) > ; compiling (WHEN (NOT #) ...) > ; compiling (DEFUN MAKE-LISP-NAME ...) > ; compiling (SETQ *FEATURES* ...) > ; compiling (SETQ *FEATURES* ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/primitives.fasl written > ; compilation finished in 0:00:01 > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > objects.lisp" (written 14 AUG 2006 06:26:22 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFUN SIZE-OF-FOREIGN-TYPE ...) > ; compiling (DEFMACRO ALLOCATE-FOREIGN-OBJECT ...) > ; compiling (DEFMACRO FREE-FOREIGN-OBJECT ...) > ; compiling (DEFMACRO NULL-POINTER-P ...) > ; compiling (DEFMACRO MAKE-NULL-POINTER ...) > ; compiling (DEFMACRO MAKE-POINTER ...) > ; compiling (DEFMACRO CHAR-ARRAY-TO-POINTER ...) > ; compiling (DEFMACRO DEREF-POINTER ...) > ; compiling (DEFMACRO ENSURE-CHAR-CHARACTER ...) > ; compiling (DEFMACRO ENSURE-CHAR-INTEGER ...) > ; compiling (DEFMACRO ENSURE-CHAR-STORABLE ...) > ; compiling (DEFMACRO POINTER-ADDRESS ...) > ; compiling (DEFMACRO WITH-FOREIGN-OBJECT ...) > ; compiling (DEFMACRO WITH-FOREIGN-OBJECTS ...) > ; compiling (DEFMACRO WITH-CAST-POINTER ...) > ; compiling (DEFMACRO DEF-FOREIGN-VAR ...) > ; compiling (DEFMACRO DEF-POINTER-VAR ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/objects.fasl written > ; compilation finished in 0:00:01 > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > aggregates.lisp" (written 18 APR 2006 02:07:09 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFMACRO DEF-ENUM ...) > ; compiling (DEFMACRO DEF-ARRAY-POINTER ...) > ; compiling (DEFUN PROCESS-STRUCT-FIELDS ...) > ; compiling (DEFMACRO DEF-STRUCT ...) > ; compiling (DEFMACRO GET-SLOT-VALUE ...) > ; compiling (DEFMACRO GET-SLOT-POINTER ...) > ; compiling (DEFMACRO DEREF-ARRAY ...) > ; compiling (DEFMACRO DEF-UNION ...) > ; compiling (SB-EXT:WITHOUT-PACKAGE-LOCKS (DEFVAR *SYSTEM-COPY- > FN* ...) ...) > ; compiling (DEFUN CONVERT-FROM-FOREIGN-USB8 ...) > > ; file: /Users/jensteich/Lisp/uffi-1.6.0/src/aggregates.lisp > ; in: DEFUN CONVERT-FROM-FOREIGN-USB8 > ; (MAKE-ARRAY UFFI::LEN :ELEMENT-TYPE '(UNSIGNED-BYTE 8)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MOD 536870911) CONS NULL), not a > INTEGER. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MOD 536870911) CONS NULL), not a > LIST. > > ; (FUNCALL UFFI::*SYSTEM-COPY-FN* UFFI::SAP 0 UFFI::RESULT > ; UFFI::+SYSTEM-COPY-OFFSET+ > ; (* UFFI::LEN UFFI::+SYSTEM-COPY-MULTIPLIER+)) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/aggregates.fasl written > ; compilation finished in 0:00:01 > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > functions.lisp" (written 13 APR 2007 07:49:01 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFUN PROCESS-FUNCTION-ARGS ...) > ; compiling (DEFUN PROCESS-ONE-FUNCTION-ARG ...) > ; compiling (DEFUN ALLEGRO-CONVERT-RETURN-TYPE ...) > ; compiling (DEFUN FUNCALLABLE-LAMBDA-LIST ...) > ; compiling (DEFUN CONVERT-LISPWORKS-ARGS ...) > ; compiling (DEFUN PREPROCESS-NAMES ...) > ; compiling (DEFUN PREPROCESS-ARGS ...) > ; compiling (DEFMACRO DEF-FUNCTION ...) > ; compiling (DEFMACRO %DEF-FUNCTION ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/functions.fasl written > ; compilation finished in 0:00:00 > STYLE-WARNING: redefining SIZE-OF-FOREIGN-TYPE in DEFUN > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > strings.lisp" (written 14 AUG 2006 08:25:09 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEF-POINTER-VAR +NULL-CSTRING-POINTER+ ...) > ; compiling (DEFMACRO CONVERT-FROM-CSTRING ...) > ; compiling (DEFMACRO CONVERT-TO-CSTRING ...) > ; compiling (DEFMACRO FREE-CSTRING ...) > ; compiling (DEFMACRO WITH-CSTRING ...) > ; compiling (DEFMACRO WITH-CSTRINGS ...) > ; compiling (DEFMACRO CONVERT-TO-FOREIGN-STRING ...) > ; compiling (DEFMACRO CONVERT-FROM-FOREIGN-STRING ...); in: LAMBDA NIL > ; (LET* ((UFFI::OBJ (CAR (CDR #:WHOLE48))) > ; (LENGTH > ; (IF (SB-KERNEL::KEYWORD-SUPPLIED-P :LENGTH #:KEYWORDS-51) > ; (SB-KERNEL::LOOKUP-KEYWORD :LENGTH #:KEYWORDS-51) > NIL)) > ; (UFFI::LOCALE > ; (IF #:SUPPLIEDP-52 (SB-KERNEL::LOOKUP-KEYWORD ':LOCALE > #:KEYWORDS-51) > ; :DEFAULT)) > ; (UFFI::NULL-TERMINATED-P > ; (IF #:SUPPLIEDP-53 > ; (SB-KERNEL::LOOKUP-KEYWORD ':NULL-TERMINATED-P > #:KEYWORDS-51) T))) > ; (BLOCK UFFI:CONVERT-FROM-FOREIGN-STRING > ; (LET ((UFFI::STORED-OBJ #)) > ; `(LET (#) > ; (IF # ,NIL #))))) > ; > ; caught STYLE-WARNING: > ; The variable LOCALE is defined but never used. > > ; file: /Users/jensteich/Lisp/uffi-1.6.0/src/strings.lisp > ; in: DEFMACRO CONVERT-FROM-FOREIGN-STRING > ; (DEFMACRO UFFI:CONVERT-FROM-FOREIGN-STRING > ; (UFFI::OBJ > ; &KEY LENGTH (UFFI::LOCALE :DEFAULT) (UFFI::NULL- > TERMINATED-P T)) > ; (LET ((UFFI::STORED-OBJ (GENSYM))) > ; `(LET ((,UFFI::STORED-OBJ ,UFFI::OBJ)) > ; (IF (UFFI:NULL-POINTER-P ,UFFI::STORED-OBJ) ,NIL > ; (UFFI::SBCL-NATURALIZE-CSTRING # :LENGTH ,LENGTH > ; :NULL-TERMINATED-P ,UFFI::NULL-TERMINATED-P))))) > ; --> EVAL-WHEN SB-C::%DEFMACRO LET* > ; ==> > ; (LET* ((UFFI::OBJ (CAR (CDR #:WHOLE48))) > ; (LENGTH > ; (IF (SB-KERNEL::KEYWORD-SUPPLIED-P :LENGTH #:KEYWORDS-51) > ; (SB-KERNEL::LOOKUP-KEYWORD :LENGTH #:KEYWORDS-51) > NIL)) > ; (UFFI::LOCALE > ; (IF #:SUPPLIEDP-52 (SB-KERNEL::LOOKUP-KEYWORD ':LOCALE > #:KEYWORDS-51) > ; :DEFAULT)) > ; (UFFI::NULL-TERMINATED-P > ; (IF #:SUPPLIEDP-53 > ; (SB-KERNEL::LOOKUP-KEYWORD ':NULL-TERMINATED-P > #:KEYWORDS-51) T))) > ; (BLOCK UFFI:CONVERT-FROM-FOREIGN-STRING > ; (LET ((UFFI::STORED-OBJ #)) > ; `(LET (#) > ; (IF # ,NIL #))))) > ; > ; caught STYLE-WARNING: > ; The variable LOCALE is defined but never used. > > ; compiling (DEFMACRO ALLOCATE-FOREIGN-STRING ...) > ; compiling (DEFUN FOREIGN-STRING-LENGTH ...) > ; compiling (DEFMACRO WITH-FOREIGN-STRING ...) > ; compiling (DEFMACRO WITH-FOREIGN-STRINGS ...) > ; compiling (DEFUN SBCL-NATURALIZE-CSTRING ...) > > ; file: /Users/jensteich/Lisp/uffi-1.6.0/src/strings.lisp > ; in: DEFUN SBCL-NATURALIZE-CSTRING > ; (SUBSEQ UFFI::CASTED 0 LENGTH) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR NULL (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) > ; (*))), not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR NULL (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) > (*))), not a LIST. > > ; (COPY-SEQ (SUBSEQ UFFI::CASTED 0 LENGTH)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (SIMPLE-ARRAY * (*)) CONS NULL > ; SB-KERNEL:EXTENDED-SEQUENCE), not a > VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (SIMPLE-ARRAY * (*)) CONS NULL > ; SB-KERNEL:EXTENDED-SEQUENCE), not a > LIST. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (SIMPLE-ARRAY * (*)) CONS NULL > ; SB-KERNEL:EXTENDED-SEQUENCE), not a > SB-KERNEL:EXTENDED-SEQUENCE. > > ; (COPY-SEQ UFFI::CASTED) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR NULL (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) > ; (*))), not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR NULL (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) > (*))), not a LIST. > > ; (FUNCALL UFFI::*SYSTEM-COPY-FN* UFFI::SAP 0 UFFI::RESULT > ; UFFI::+SYSTEM-COPY-OFFSET+ (* LENGTH UFFI::+SYSTEM-COPY- > MULTIPLIER+)) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; compiling (DEF-FUNCTION "strlen" ...) > ; compiling (DEF-TYPE CHAR-PTR-DEF ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/strings.fasl written > ; compilation finished in 0:00:01 > WARNING: > COMPILE-FILE warned while performing # on > #. > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > libraries.lisp" (written 22 JUL 2007 09:09:39 PM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFVAR *LOADED-LIBRARIES* ...) > ; compiling (DEFUN DEFAULT-FOREIGN-LIBRARY-TYPE ...) > ; compiling (DEFUN FOREIGN-LIBRARY-TYPES ...) > ; compiling (DEFUN FIND-FOREIGN-LIBRARY ...) > ; compiling (DEFUN LOAD-FOREIGN-LIBRARY ...) > > ; file: /Users/jensteich/Lisp/uffi-1.6.0/src/libraries.lisp > ; in: DEFUN LOAD-FOREIGN-LIBRARY > ; (FLET ((UFFI::LOAD-FAILURE () > ; (ERROR "Unable to load foreign library \"~A\"." > UFFI::FILENAME))) > ; (WHEN (AND UFFI::FILENAME (OR (NULL #) (PROBE-FILE > UFFI::FILENAME))) > ; (IF (PATHNAMEP UFFI::FILENAME) (SETQ UFFI::FILENAME > (NAMESTRING #))) > ; (IF > ; (AND (NOT UFFI::FORCE-LOAD) > ; (FIND UFFI::FILENAME UFFI::*LOADED-LIBRARIES* :TEST > ; #'STRING-EQUAL)) > ; T > ; (PROGN > ; (HANDLER-CASE # #) > ; (PUSH UFFI::FILENAME UFFI::*LOADED-LIBRARIES*) > ; T)))) > ; > ; note: deleting unused function > ; (FLET LOAD-FAILURE) > > ; compiling (DEFUN CONVERT-SUPPORTING-LIBRARIES-TO-STRING ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/libraries.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > os.lisp" (written 18 APR 2006 02:07:09 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFUN GETENV ...) > ; compiling (DEFUN RUN-SHELL-COMMAND ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/os.fasl written > ; compilation finished in 0:00:00 > STYLE-WARNING: redefining STRLEN in DEFUN > ; > ; compilation unit finished > ; caught 2 STYLE-WARNING conditions > ; printed 12 notes > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > package.lisp" (written 10 MAY 2006 02:42:59 AM): > ; compiling (IN-PACKAGE #:CL-USER) > ; compiling (DEFPACKAGE #:UFFI ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/package.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > primitives.lisp" (written 18 APR 2006 02:07:09 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFMACRO DEF-CONSTANT ...) > ; compiling (DEFMACRO DEF-TYPE ...) > ; compiling (DEFMACRO NULL-CHAR-P ...) > ; compiling (DEFMACRO DEF-FOREIGN-TYPE ...) > ; compiling (DEFVAR +TYPE-CONVERSION-HASH+ ...) > ; compiling (DEFVAR *CMU-DEF-TYPE-HASH* ...) > ; compiling (DEFVAR *CMU-SBCL-DEF-TYPE-LIST* ...) > ; compiling (DEFVAR *TYPE-CONVERSION-LIST* ...) > ; compiling (SETQ *TYPE-CONVERSION-LIST* ...) > ; compiling (DOLIST (TYPE *TYPE-CONVERSION-LIST*) ...) > ; compiling (DOLIST (TYPE *CMU-SBCL-DEF-TYPE-LIST*) ...) > ; compiling (DEFUN BASIC-CONVERT-FROM-UFFI-TYPE ...) > ; compiling (DEFUN %CONVERT-FROM-UFFI-TYPE ...) > ; compiling (DEFUN CONVERT-FROM-UFFI-TYPE ...) > ; compiling (WHEN (CHAR= #\a ...) ...) > ; compiling (WHEN (NOT #) ...) > ; compiling (DEFUN MAKE-LISP-NAME ...) > ; compiling (SETQ *FEATURES* ...) > ; compiling (SETQ *FEATURES* ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/primitives.fasl written > ; compilation finished in 0:00:00 > STYLE-WARNING: redefining BASIC-CONVERT-FROM-UFFI-TYPE in DEFUN > STYLE-WARNING: redefining %CONVERT-FROM-UFFI-TYPE in DEFUN > STYLE-WARNING: redefining CONVERT-FROM-UFFI-TYPE in DEFUN > STYLE-WARNING: redefining MAKE-LISP-NAME in DEFUN > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > objects.lisp" (written 14 AUG 2006 06:26:22 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFUN SIZE-OF-FOREIGN-TYPE ...) > ; > ; caught STYLE-WARNING: > ; redefining SIZE-OF-FOREIGN-TYPE in DEFUN > > ; compiling (DEFMACRO ALLOCATE-FOREIGN-OBJECT ...) > ; compiling (DEFMACRO FREE-FOREIGN-OBJECT ...) > ; compiling (DEFMACRO NULL-POINTER-P ...) > ; compiling (DEFMACRO MAKE-NULL-POINTER ...) > ; compiling (DEFMACRO MAKE-POINTER ...) > ; compiling (DEFMACRO CHAR-ARRAY-TO-POINTER ...) > ; compiling (DEFMACRO DEREF-POINTER ...) > ; compiling (DEFMACRO ENSURE-CHAR-CHARACTER ...) > ; compiling (DEFMACRO ENSURE-CHAR-INTEGER ...) > ; compiling (DEFMACRO ENSURE-CHAR-STORABLE ...) > ; compiling (DEFMACRO POINTER-ADDRESS ...) > ; compiling (DEFMACRO WITH-FOREIGN-OBJECT ...) > ; compiling (DEFMACRO WITH-FOREIGN-OBJECTS ...) > ; compiling (DEFMACRO WITH-CAST-POINTER ...) > ; compiling (DEFMACRO DEF-FOREIGN-VAR ...) > ; compiling (DEFMACRO DEF-POINTER-VAR ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/objects.fasl written > ; compilation finished in 0:00:00 > WARNING: > COMPILE-FILE warned while performing # on > #. > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > aggregates.lisp" (written 18 APR 2006 02:07:09 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFMACRO DEF-ENUM ...) > ; compiling (DEFMACRO DEF-ARRAY-POINTER ...) > ; compiling (DEFUN PROCESS-STRUCT-FIELDS ...) > ; compiling (DEFMACRO DEF-STRUCT ...) > ; compiling (DEFMACRO GET-SLOT-VALUE ...) > ; compiling (DEFMACRO GET-SLOT-POINTER ...) > ; compiling (DEFMACRO DEREF-ARRAY ...) > ; compiling (DEFMACRO DEF-UNION ...) > ; compiling (SB-EXT:WITHOUT-PACKAGE-LOCKS (DEFVAR *SYSTEM-COPY- > FN* ...) ...) > ; compiling (DEFUN CONVERT-FROM-FOREIGN-USB8 ...) > > ; file: /Users/jensteich/Lisp/uffi-1.6.0/src/aggregates.lisp > ; in: DEFUN CONVERT-FROM-FOREIGN-USB8 > ; (MAKE-ARRAY UFFI::LEN :ELEMENT-TYPE '(UNSIGNED-BYTE 8)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MOD 536870911) CONS NULL), not a > INTEGER. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MOD 536870911) CONS NULL), not a > LIST. > > ; (FUNCALL UFFI::*SYSTEM-COPY-FN* UFFI::SAP 0 UFFI::RESULT > ; UFFI::+SYSTEM-COPY-OFFSET+ > ; (* UFFI::LEN UFFI::+SYSTEM-COPY-MULTIPLIER+)) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/aggregates.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > functions.lisp" (written 13 APR 2007 07:49:01 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFUN PROCESS-FUNCTION-ARGS ...) > ; compiling (DEFUN PROCESS-ONE-FUNCTION-ARG ...) > ; compiling (DEFUN ALLEGRO-CONVERT-RETURN-TYPE ...) > ; compiling (DEFUN FUNCALLABLE-LAMBDA-LIST ...) > ; compiling (DEFUN CONVERT-LISPWORKS-ARGS ...) > ; compiling (DEFUN PREPROCESS-NAMES ...) > ; compiling (DEFUN PREPROCESS-ARGS ...) > ; compiling (DEFMACRO DEF-FUNCTION ...) > ; compiling (DEFMACRO %DEF-FUNCTION ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/functions.fasl written > ; compilation finished in 0:00:01 > STYLE-WARNING: redefining PROCESS-FUNCTION-ARGS in DEFUN > STYLE-WARNING: redefining PROCESS-ONE-FUNCTION-ARG in DEFUN > STYLE-WARNING: redefining ALLEGRO-CONVERT-RETURN-TYPE in DEFUN > STYLE-WARNING: redefining FUNCALLABLE-LAMBDA-LIST in DEFUN > STYLE-WARNING: redefining CONVERT-LISPWORKS-ARGS in DEFUN > STYLE-WARNING: redefining PREPROCESS-NAMES in DEFUN > STYLE-WARNING: redefining PREPROCESS-ARGS in DEFUN > STYLE-WARNING: redefining PROCESS-STRUCT-FIELDS in DEFUN > STYLE-WARNING: redefining CONVERT-FROM-FOREIGN-USB8 in DEFUN > STYLE-WARNING: redefining SIZE-OF-FOREIGN-TYPE in DEFUN > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > strings.lisp" (written 14 AUG 2006 08:25:09 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEF-POINTER-VAR +NULL-CSTRING-POINTER+ ...) > ; compiling (DEFMACRO CONVERT-FROM-CSTRING ...) > ; compiling (DEFMACRO CONVERT-TO-CSTRING ...) > ; compiling (DEFMACRO FREE-CSTRING ...) > ; compiling (DEFMACRO WITH-CSTRING ...) > ; compiling (DEFMACRO WITH-CSTRINGS ...) > ; compiling (DEFMACRO CONVERT-TO-FOREIGN-STRING ...) > ; compiling (DEFMACRO CONVERT-FROM-FOREIGN-STRING ...); in: LAMBDA NIL > ; (LET* ((UFFI::OBJ (CAR (CDR #:WHOLE48))) > ; (LENGTH > ; (IF (SB-KERNEL::KEYWORD-SUPPLIED-P :LENGTH #:KEYWORDS-51) > ; (SB-KERNEL::LOOKUP-KEYWORD :LENGTH #:KEYWORDS-51) > NIL)) > ; (UFFI::LOCALE > ; (IF #:SUPPLIEDP-52 (SB-KERNEL::LOOKUP-KEYWORD ':LOCALE > #:KEYWORDS-51) > ; :DEFAULT)) > ; (UFFI::NULL-TERMINATED-P > ; (IF #:SUPPLIEDP-53 > ; (SB-KERNEL::LOOKUP-KEYWORD ':NULL-TERMINATED-P > #:KEYWORDS-51) T))) > ; (BLOCK UFFI:CONVERT-FROM-FOREIGN-STRING > ; (LET ((UFFI::STORED-OBJ #)) > ; `(LET (#) > ; (IF # ,NIL #))))) > ; > ; caught STYLE-WARNING: > ; The variable LOCALE is defined but never used. > > ; file: /Users/jensteich/Lisp/uffi-1.6.0/src/strings.lisp > ; in: DEFMACRO CONVERT-FROM-FOREIGN-STRING > ; (DEFMACRO UFFI:CONVERT-FROM-FOREIGN-STRING > ; (UFFI::OBJ > ; &KEY LENGTH (UFFI::LOCALE :DEFAULT) (UFFI::NULL- > TERMINATED-P T)) > ; (LET ((UFFI::STORED-OBJ (GENSYM))) > ; `(LET ((,UFFI::STORED-OBJ ,UFFI::OBJ)) > ; (IF (UFFI:NULL-POINTER-P ,UFFI::STORED-OBJ) ,NIL > ; (UFFI::SBCL-NATURALIZE-CSTRING # :LENGTH ,LENGTH > ; :NULL-TERMINATED-P > ; ,UFFI::NULL-TERMINATED- > P))))) > ; --> EVAL-WHEN SB-C::%DEFMACRO LET* > ; ==> > ; (LET* ((UFFI::OBJ (CAR (CDR #:WHOLE48))) > ; (LENGTH > ; (IF (SB-KERNEL::KEYWORD-SUPPLIED-P :LENGTH #:KEYWORDS-51) > ; (SB-KERNEL::LOOKUP-KEYWORD :LENGTH #:KEYWORDS-51) > NIL)) > ; (UFFI::LOCALE > ; (IF #:SUPPLIEDP-52 (SB-KERNEL::LOOKUP-KEYWORD ':LOCALE > #:KEYWORDS-51) > ; :DEFAULT)) > ; (UFFI::NULL-TERMINATED-P > ; (IF #:SUPPLIEDP-53 > ; (SB-KERNEL::LOOKUP-KEYWORD ':NULL-TERMINATED-P > #:KEYWORDS-51) T))) > ; (BLOCK UFFI:CONVERT-FROM-FOREIGN-STRING > ; (LET ((UFFI::STORED-OBJ #)) > ; `(LET (#) > ; (IF # ,NIL #))))) > ; > ; caught STYLE-WARNING: > ; The variable LOCALE is defined but never used. > > ; compiling (DEFMACRO ALLOCATE-FOREIGN-STRING ...) > ; compiling (DEFUN FOREIGN-STRING-LENGTH ...) > ; compiling (DEFMACRO WITH-FOREIGN-STRING ...) > ; compiling (DEFMACRO WITH-FOREIGN-STRINGS ...) > ; compiling (DEFUN SBCL-NATURALIZE-CSTRING ...) > > ; file: /Users/jensteich/Lisp/uffi-1.6.0/src/strings.lisp > ; in: DEFUN SBCL-NATURALIZE-CSTRING > ; (SUBSEQ UFFI::CASTED 0 LENGTH) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR NULL (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) > ; (*))), not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR NULL (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) > (*))), not a LIST. > > ; (COPY-SEQ (SUBSEQ UFFI::CASTED 0 LENGTH)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (SIMPLE-ARRAY * (*)) CONS NULL > ; SB-KERNEL:EXTENDED-SEQUENCE), not a > VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (SIMPLE-ARRAY * (*)) CONS NULL > ; SB-KERNEL:EXTENDED-SEQUENCE), not a > LIST. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (SIMPLE-ARRAY * (*)) CONS NULL > ; SB-KERNEL:EXTENDED-SEQUENCE), not a > SB-KERNEL:EXTENDED-SEQUENCE. > > ; (COPY-SEQ UFFI::CASTED) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR NULL (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) > ; (*))), not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR NULL (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) > (*))), not a LIST. > > ; (FUNCALL UFFI::*SYSTEM-COPY-FN* UFFI::SAP 0 UFFI::RESULT > ; UFFI::+SYSTEM-COPY-OFFSET+ (* LENGTH UFFI::+SYSTEM-COPY- > MULTIPLIER+)) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; compiling (DEF-FUNCTION "strlen" ...) > ; > ; caught STYLE-WARNING: > ; redefining STRLEN in DEFUN > > ; compiling (DEF-TYPE CHAR-PTR-DEF ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/strings.fasl written > ; compilation finished in 0:00:00 > WARNING: > COMPILE-FILE warned while performing # on > #. > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > libraries.lisp" (written 22 JUL 2007 09:09:39 PM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFVAR *LOADED-LIBRARIES* ...) > ; compiling (DEFUN DEFAULT-FOREIGN-LIBRARY-TYPE ...) > ; compiling (DEFUN FOREIGN-LIBRARY-TYPES ...) > ; compiling (DEFUN FIND-FOREIGN-LIBRARY ...) > ; compiling (DEFUN LOAD-FOREIGN-LIBRARY ...) > > ; file: /Users/jensteich/Lisp/uffi-1.6.0/src/libraries.lisp > ; in: DEFUN LOAD-FOREIGN-LIBRARY > ; (FLET ((UFFI::LOAD-FAILURE () > ; (ERROR "Unable to load foreign library \"~A\"." > UFFI::FILENAME))) > ; (WHEN (AND UFFI::FILENAME (OR (NULL #) (PROBE-FILE > UFFI::FILENAME))) > ; (IF (PATHNAMEP UFFI::FILENAME) (SETQ UFFI::FILENAME > (NAMESTRING #))) > ; (IF > ; (AND (NOT UFFI::FORCE-LOAD) > ; (FIND UFFI::FILENAME UFFI::*LOADED-LIBRARIES* :TEST > ; #'STRING-EQUAL)) > ; T > ; (PROGN > ; (HANDLER-CASE # #) > ; (PUSH UFFI::FILENAME UFFI::*LOADED-LIBRARIES*) > ; T)))) > ; > ; note: deleting unused function > ; (FLET LOAD-FAILURE) > > ; compiling (DEFUN CONVERT-SUPPORTING-LIBRARIES-TO-STRING ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/libraries.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/uffi-1.6.0/src/ > os.lisp" (written 18 APR 2006 02:07:09 AM): > ; compiling (IN-PACKAGE #:UFFI) > ; compiling (DEFUN GETENV ...) > ; compiling (DEFUN RUN-SHELL-COMMAND ...) > > ; /Users/jensteich/Lisp/uffi-1.6.0/src/os.fasl written > ; compilation finished in 0:00:00 > STYLE-WARNING: redefining FOREIGN-STRING-LENGTH in DEFUN > STYLE-WARNING: redefining SBCL-NATURALIZE-CSTRING in DEFUN > STYLE-WARNING: redefining STRLEN in DEFUN > STYLE-WARNING: redefining DEFAULT-FOREIGN-LIBRARY-TYPE in DEFUN > STYLE-WARNING: redefining FOREIGN-LIBRARY-TYPES in DEFUN > STYLE-WARNING: redefining FIND-FOREIGN-LIBRARY in DEFUN > STYLE-WARNING: redefining LOAD-FOREIGN-LIBRARY in DEFUN > STYLE-WARNING: redefining CONVERT-SUPPORTING-LIBRARIES-TO-STRING in > DEFUN > STYLE-WARNING: redefining GETENV in DEFUN > STYLE-WARNING: redefining RUN-SHELL-COMMAND in DEFUN > ; compiling file "/Users/jensteich/Lisp/elephant/src/utils/ > package.lisp" (written 12 APR 2007 04:47:51 AM): > ; compiling (IN-PACKAGE :CL-USER) > ; compiling (DEFPACKAGE #:ELEPHANT-UTILS ...) > > ; /Users/jensteich/Lisp/elephant/src/utils/package.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/utils/ > convenience.lisp" (written 12 APR 2007 04:47:51 AM): > ; compiling (IN-PACKAGE :ELEPHANT-UTILS) > ; compiling (DEFMACRO WITH-GENSYMS ...) > ; compiling (DEFMACRO DO-SUBSETS ...) > ; compiling (DEFUN SUBSETS ...) > ; compiling (DEFUN REMOVE-KEYWORDS ...) > ; compiling (DEFUN CONCAT-SEPARATED-STRINGS ...) > ; compiling (DEFUN APPEND-SUBLISTS ...) > ; compiling (DEFMACRO IFRET ...) > ; compiling (DEFMACRO AIF ...) > ; compiling (DEFMACRO AWHEN ...) > ; compiling (DEFUN MKLIST ...) > ; compiling (DEFUN REMOVE-INDEXED-ELEMENT-AND-ADJUST ...) > > ; /Users/jensteich/Lisp/elephant/src/utils/convenience.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/utils/ > locks.lisp" (written 12 APR 2007 04:47:51 AM): > ; compiling (IN-PACKAGE :ELEPHANT-UTILS) > ; compiling (DEFUN ELE-MAKE-LOCK ...) > ; compiling (DEFMACRO ELE-WITH-LOCK ...) > ; compiling (DEFUN ELE-MAKE-FAST-LOCK ...) > ; compiling (DEFMACRO ELE-WITH-FAST-LOCK ...) > ; compiling (DEFUN ELE-THREAD-HASH-KEY ...) > > ; /Users/jensteich/Lisp/elephant/src/utils/locks.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/utils/ > os.lisp" (written 29 APR 2007 11:54:51 PM): > ; compiling (IN-PACKAGE :ELEPHANT-UTILS) > ; compiling (DEFMACRO IN-DIRECTORY ...) > ; compiling (DEFUN LAUNCH-BACKGROUND-PROGRAM ...) > ; compiling (DEFUN KILL-BACKGROUND-PROGRAM ...) > > ; /Users/jensteich/Lisp/elephant/src/utils/os.fasl written > ; compilation finished in 0:00:01 > Attempting to load libmemutil.dylib... > Loaded /Users/jensteich/Lisp/elephant/src/memutil/libmemutil.dylib > ; compiling file "/Users/jensteich/Lisp/elephant/src/memutil/ > memutil.lisp" (written 29 APR 2007 04:10:55 AM): > ; compiling (DEFPACKAGE ELEPHANT-MEMUTIL ...) > ; compiling (IN-PACKAGE "ELEPHANT-MEMUTIL") > ; compiling (DEF-TYPE POINTER-INT ...) > ; compiling (DEF-TYPE POINTER-VOID ...) > ; compiling (DEF-FOREIGN-TYPE ARRAY-OR-POINTER-CHAR ...) > ; compiling (DEF-TYPE ARRAY-OR-POINTER-CHAR ...) > ; compiling (DEF-FUNCTION ("copy_buf" COPY-BUFS) ...) > ; compiling (DEFVAR +NULL-VOID+ ...) > ; compiling (DEFVAR +NULL-CHAR+ ...) > ; compiling (DEFVAR *BUFFER-STREAMS* ...) > ; compiling (DEFVAR *BUFFER-STREAMS-LOCK* ...) > ; compiling (DEFSTRUCT BUFFER-STREAM ...) > ; compiling (DEFUN GRAB-BUFFER-STREAM ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN GRAB-BUFFER-STREAM > ; (LENGTH ELEPHANT-MEMUTIL::*BUFFER-STREAMS*) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; compiling (DEFUN RETURN-BUFFER-STREAM ...) > ; compiling (DEFMACRO WITH-BUFFER-STREAMS ...) > ; compiling (DEFUN READ-INT32 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN READ-INT32 > ; (DEFUN ELEPHANT-MEMUTIL::READ-INT32 > ; (ELEPHANT-MEMUTIL::BUF ELEPHANT-MEMUTIL::OFFSET) > ; "Read a 32-bit signed integer from a foreign char buffer." > ; (DECLARE (TYPE (ALIEN (* UNSIGNED-CHAR)) ELEPHANT-MEMUTIL::BUF) > ; (TYPE FIXNUM ELEPHANT-MEMUTIL::OFFSET)) > ; (THE (SIGNED-BYTE 32) (DEREF (CAST (SAP-ALIEN # #) (* #))))) > ; --> PROGN EVAL-WHEN > ; ==> > ; (SB-IMPL::%DEFUN 'ELEPHANT-MEMUTIL::READ-INT32 > ; (SB-INT:NAMED-LAMBDA ELEPHANT-MEMUTIL::READ-INT32 > ; (ELEPHANT-MEMUTIL::BUF > ; ELEPHANT-MEMUTIL::OFFSET) > ; (DECLARE > ; (TYPE (ALIEN #) ELEPHANT- > MEMUTIL::BUF) > ; (TYPE FIXNUM > ; ELEPHANT-MEMUTIL::OFFSET)) > ; (BLOCK ELEPHANT- > MEMUTIL::READ-INT32 > ; (THE (SIGNED-BYTE 32) > (DEREF #)))) > ; "Read a 32-bit signed integer from a foreign > char buffer." > ; 'NIL (SB-C:SOURCE-LOCATION)) > ; > ; note: doing signed word to integer coercion (cost 20) to " value>" > > ; compiling (DEFUN READ-INT64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN READ-INT64 > ; (DEREF > ; (CAST > ; (SAP-ALIEN > ; (SB-SYS:SAP+ (ALIEN-SAP ELEPHANT-MEMUTIL::BUF) ELEPHANT- > MEMUTIL::OFFSET) > ; (* UNSIGNED-CHAR)) > ; (* (SIGNED 64)))) > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; --> SB-SYS:SIGNED-SAP-REF-64 LOGIOR > ; ==> > ; (ASH (SB-SYS:SIGNED-SAP-REF-32 SB-VM::SAP (+ SB-VM::OFFSET 4)) 32) > ; > ; note: forced to do full call > ; unable to do inline ASH (cost 2) because: > ; The first argument is a (SIGNED-BYTE 32), not a FIXNUM. > ; The result is a (VALUES > ; (INTEGER -9223372036854775808 > 9223372032559808512) > ; &OPTIONAL), not a (VALUES FIXNUM &REST T). > ; unable to do inline ASH (cost 3) because: > ; The first argument is a (SIGNED-BYTE 32), not a (UNSIGNED- > BYTE 32). > ; The result is a (VALUES > ; (INTEGER -9223372036854775808 > 9223372032559808512) > ; &OPTIONAL), not a (VALUES (UNSIGNED-BYTE > 32) &REST T). > ; etc. > > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; --> SB-SYS:SIGNED-SAP-REF-64 > ; ==> > ; (LOGIOR (SB-SYS:SAP-REF-32 SB-VM::SAP SB-VM::OFFSET) > ; (ASH (SB-SYS:SIGNED-SAP-REF-32 SB-VM::SAP (+ SB- > VM::OFFSET 4)) 32)) > ; > ; note: forced to do static-fun Two-arg-ior (cost 53) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (UNSIGNED-BYTE 32), not a FIXNUM. > ; The second argument is a (INTEGER -9223372036854775808 > ; 9223372032559808512), not a FIXNUM. > ; The result is a (VALUES (SIGNED-BYTE 64) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > ; T > ). > ; unable to do inline (signed-byte 32) arithmetic (cost 3) > because: > ; The first argument is a (UNSIGNED-BYTE 32), not a (SIGNED- > BYTE 32). > ; The second argument is a (INTEGER -9223372036854775808 > ; 9223372032559808512), not a > (SIGNED-BYTE 32). > ; The result is a (VALUES (SIGNED-BYTE 64) &OPTIONAL), not a > (VALUES > ; > (SIGNED-BYTE > ; > 32) > ; > &REST T). > ; etc. > > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; --> SB-SYS:SIGNED-SAP-REF-64 LOGIOR > ; ==> > ; (ASH (SB-SYS:SIGNED-SAP-REF-32 SB-VM::SAP (+ SB-VM::OFFSET 4)) 32) > ; > ; note: doing signed word to integer coercion (cost 20) > > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; --> SB-SYS:SIGNED-SAP-REF-64 > ; ==> > ; (LOGIOR (SB-SYS:SAP-REF-32 SB-VM::SAP SB-VM::OFFSET) > ; (ASH (SB-SYS:SIGNED-SAP-REF-32 SB-VM::SAP (+ SB- > VM::OFFSET 4)) 32)) > ; > ; note: doing unsigned word to integer coercion (cost 20), for: > ; the first argument of static-fun Two-arg-ior > > ; compiling (DEFUN READ-UINT32 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN READ-UINT32 > ; (DEFUN ELEPHANT-MEMUTIL::READ-UINT32 > ; (ELEPHANT-MEMUTIL::BUF ELEPHANT-MEMUTIL::OFFSET) > ; "Read a 32-bit unsigned integer from a foreign char buffer." > ; (DECLARE (TYPE (ALIEN (* UNSIGNED-CHAR)) ELEPHANT-MEMUTIL::BUF) > ; (TYPE FIXNUM ELEPHANT-MEMUTIL::OFFSET)) > ; (THE (UNSIGNED-BYTE 32) (DEREF (CAST (SAP-ALIEN # #) (* #))))) > ; --> PROGN EVAL-WHEN > ; ==> > ; (SB-IMPL::%DEFUN 'ELEPHANT-MEMUTIL::READ-UINT32 > ; (SB-INT:NAMED-LAMBDA ELEPHANT-MEMUTIL::READ- > UINT32 > ; (ELEPHANT-MEMUTIL::BUF > ; ELEPHANT-MEMUTIL::OFFSET) > ; (DECLARE > ; (TYPE (ALIEN #) ELEPHANT- > MEMUTIL::BUF) > ; (TYPE FIXNUM > ; ELEPHANT-MEMUTIL::OFFSET)) > ; (BLOCK ELEPHANT- > MEMUTIL::READ-UINT32 > ; (THE (UNSIGNED-BYTE 32) > (DEREF #)))) > ; "Read a 32-bit unsigned integer from a foreign > char buffer." > ; 'NIL (SB-C:SOURCE-LOCATION)) > ; > ; note: doing unsigned word to integer coercion (cost 20) to > "" > > ; compiling (DEFUN READ-UINT64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN READ-UINT64 > ; (DEREF > ; (CAST > ; (SAP-ALIEN > ; (SB-SYS:SAP+ (ALIEN-SAP ELEPHANT-MEMUTIL::BUF) ELEPHANT- > MEMUTIL::OFFSET) > ; (* UNSIGNED-CHAR)) > ; (* (SIGNED 64)))) > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; --> SB-SYS:SIGNED-SAP-REF-64 LOGIOR > ; ==> > ; (ASH (SB-SYS:SIGNED-SAP-REF-32 SB-VM::SAP (+ SB-VM::OFFSET 4)) 32) > ; > ; note: forced to do full call > ; unable to do inline ASH (cost 2) because: > ; The first argument is a (SIGNED-BYTE 32), not a FIXNUM. > ; The result is a (VALUES > ; (INTEGER -9223372036854775808 > 9223372032559808512) > ; &OPTIONAL), not a (VALUES FIXNUM &REST T). > ; unable to do inline ASH (cost 3) because: > ; The first argument is a (SIGNED-BYTE 32), not a (UNSIGNED- > BYTE 32). > ; The result is a (VALUES > ; (INTEGER -9223372036854775808 > 9223372032559808512) > ; &OPTIONAL), not a (VALUES (UNSIGNED-BYTE > 32) &REST T). > ; etc. > > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; --> SB-SYS:SIGNED-SAP-REF-64 > ; ==> > ; (LOGIOR (SB-SYS:SAP-REF-32 SB-VM::SAP SB-VM::OFFSET) > ; (ASH (SB-SYS:SIGNED-SAP-REF-32 SB-VM::SAP (+ SB- > VM::OFFSET 4)) 32)) > ; > ; note: forced to do static-fun Two-arg-ior (cost 53) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (UNSIGNED-BYTE 32), not a FIXNUM. > ; The second argument is a (INTEGER -9223372036854775808 > ; 9223372032559808512), not a FIXNUM. > ; The result is a (VALUES (SIGNED-BYTE 64) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > ; T > ). > ; unable to do inline (signed-byte 32) arithmetic (cost 3) > because: > ; The first argument is a (UNSIGNED-BYTE 32), not a (SIGNED- > BYTE 32). > ; The second argument is a (INTEGER -9223372036854775808 > ; 9223372032559808512), not a > (SIGNED-BYTE 32). > ; The result is a (VALUES (SIGNED-BYTE 64) &OPTIONAL), not a > (VALUES > ; > (SIGNED-BYTE > ; > 32) > ; > &REST T). > ; etc. > > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; --> SB-SYS:SIGNED-SAP-REF-64 LOGIOR > ; ==> > ; (ASH (SB-SYS:SIGNED-SAP-REF-32 SB-VM::SAP (+ SB-VM::OFFSET 4)) 32) > ; > ; note: doing signed word to integer coercion (cost 20) > > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; --> SB-SYS:SIGNED-SAP-REF-64 > ; ==> > ; (LOGIOR (SB-SYS:SAP-REF-32 SB-VM::SAP SB-VM::OFFSET) > ; (ASH (SB-SYS:SIGNED-SAP-REF-32 SB-VM::SAP (+ SB- > VM::OFFSET 4)) 32)) > ; > ; note: doing unsigned word to integer coercion (cost 20), for: > ; the first argument of static-fun Two-arg-ior > > ; compiling (DEFUN READ-FLOAT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN READ-FLOAT > ; (DEFUN ELEPHANT-MEMUTIL::READ-FLOAT > ; (ELEPHANT-MEMUTIL::BUF ELEPHANT-MEMUTIL::OFFSET) > ; "Read a single-float from a foreign char buffer." > ; (DECLARE (TYPE (ALIEN (* UNSIGNED-CHAR)) ELEPHANT-MEMUTIL::BUF) > ; (TYPE FIXNUM ELEPHANT-MEMUTIL::OFFSET)) > ; (THE SINGLE-FLOAT (DEREF (CAST (SAP-ALIEN # #) (* SINGLE- > FLOAT))))) > ; --> PROGN EVAL-WHEN > ; ==> > ; (SB-IMPL::%DEFUN 'ELEPHANT-MEMUTIL::READ-FLOAT > ; (SB-INT:NAMED-LAMBDA ELEPHANT-MEMUTIL::READ-FLOAT > ; (ELEPHANT-MEMUTIL::BUF > ; ELEPHANT-MEMUTIL::OFFSET) > ; (DECLARE > ; (TYPE (ALIEN #) ELEPHANT- > MEMUTIL::BUF) > ; (TYPE FIXNUM > ; ELEPHANT-MEMUTIL::OFFSET)) > ; (BLOCK ELEPHANT- > MEMUTIL::READ-FLOAT > ; (THE SINGLE-FLOAT (DEREF > #)))) > ; "Read a single-float from a foreign char buffer." > ; '(SB-C:LAMBDA-WITH-LEXENV NIL NIL NIL > ; (ELEPHANT-MEMUTIL::BUF ELEPHANT- > MEMUTIL::OFFSET) > ; (DECLARE (TYPE (ALIEN #) ELEPHANT-MEMUTIL::BUF) > ; (TYPE FIXNUM ELEPHANT-MEMUTIL::OFFSET)) > ; (BLOCK ELEPHANT-MEMUTIL::READ-FLOAT > ; (THE SINGLE-FLOAT (DEREF #)))) > ; (SB-C:SOURCE-LOCATION)) > ; > ; note: doing float to pointer coercion (cost 13) to "" > > ; compiling (DEFUN READ-DOUBLE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN READ-DOUBLE > ; (DEFUN ELEPHANT-MEMUTIL::READ-DOUBLE > ; (ELEPHANT-MEMUTIL::BUF ELEPHANT-MEMUTIL::OFFSET) > ; "Read a double-float from a foreign char buffer." > ; (DECLARE (TYPE (ALIEN (* UNSIGNED-CHAR)) ELEPHANT-MEMUTIL::BUF) > ; (TYPE FIXNUM ELEPHANT-MEMUTIL::OFFSET)) > ; (THE DOUBLE-FLOAT (DEREF (CAST (SAP-ALIEN # #) (* DOUBLE- > FLOAT))))) > ; --> PROGN EVAL-WHEN > ; ==> > ; (SB-IMPL::%DEFUN 'ELEPHANT-MEMUTIL::READ-DOUBLE > ; (SB-INT:NAMED-LAMBDA ELEPHANT-MEMUTIL::READ- > DOUBLE > ; (ELEPHANT-MEMUTIL::BUF > ; ELEPHANT-MEMUTIL::OFFSET) > ; (DECLARE > ; (TYPE (ALIEN #) ELEPHANT- > MEMUTIL::BUF) > ; (TYPE FIXNUM > ; ELEPHANT-MEMUTIL::OFFSET)) > ; (BLOCK ELEPHANT- > MEMUTIL::READ-DOUBLE > ; (THE DOUBLE-FLOAT (DEREF > #)))) > ; "Read a double-float from a foreign char buffer." > ; '(SB-C:LAMBDA-WITH-LEXENV NIL NIL NIL > ; (ELEPHANT-MEMUTIL::BUF ELEPHANT- > MEMUTIL::OFFSET) > ; (DECLARE (TYPE (ALIEN #) ELEPHANT-MEMUTIL::BUF) > ; (TYPE FIXNUM ELEPHANT-MEMUTIL::OFFSET)) > ; (BLOCK ELEPHANT-MEMUTIL::READ-DOUBLE > ; (THE DOUBLE-FLOAT (DEREF #)))) > ; (SB-C:SOURCE-LOCATION)) > ; > ; note: doing float to pointer coercion (cost 13) to "" > > ; compiling (DEFUN WRITE-INT32 ...) > ; compiling (DEFUN WRITE-INT64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN WRITE-INT64 > ; (SETF (DEREF > ; (CAST > ; (SAP-ALIEN (SB-SYS:SAP+ # ELEPHANT-MEMUTIL::OFFSET) > ; (* UNSIGNED-CHAR)) > ; (* (SIGNED 64)))) > ; ELEPHANT-MEMUTIL::NUM) > ; --> LET* MULTIPLE-VALUE-BIND LET SB-ALIEN-INTERNALS:%SET-DEREF > ; --> SB-ALIEN-INTERNALS:DEPOSIT-ALIEN-VALUE LET > ; --> SB-ALIEN-INTERNALS:MAYBE-WITH-PINNED-OBJECTS PROGN LET SETF > ; --> SB-KERNEL:%SET-SIGNED-SAP-REF-64 PROGN SB-KERNEL:%SET-SAP-REF-32 > ; ==> > ; (LOGAND SB-VM::VALUE 4294967295) > ; > ; note: forced to do static-fun Two-arg-and (cost 53) > ; unable to do inline (unsigned-byte 32) arithmetic (cost 2) > because: > ; The first argument is a (SIGNED-BYTE 64), not a (UNSIGNED- > BYTE 32). > ; unable to do inline (unsigned-byte 32) arithmetic (cost 2) > because: > ; The first argument is a (SIGNED-BYTE 64), not a (SIGNED-BYTE > 32). > ; etc. > > ; --> LET* MULTIPLE-VALUE-BIND LET SB-ALIEN-INTERNALS:%SET-DEREF > ; --> SB-ALIEN-INTERNALS:DEPOSIT-ALIEN-VALUE LET > ; --> SB-ALIEN-INTERNALS:MAYBE-WITH-PINNED-OBJECTS PROGN LET SETF > ; --> SB-KERNEL:%SET-SIGNED-SAP-REF-64 PROGN > ; --> SB-KERNEL:%SET-SIGNED-SAP-REF-32 > ; ==> > ; (ASH SB-VM::VALUE -32) > ; > ; note: forced to do full call > ; unable to do inline ASH (cost 2) because: > ; The first argument is a (SIGNED-BYTE 64), not a FIXNUM. > ; The result is a (VALUES (SIGNED-BYTE 32) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > ; T > ). > ; unable to do inline ASH (cost 3) because: > ; The first argument is a (SIGNED-BYTE 64), not a (UNSIGNED- > BYTE 32). > ; The result is a (VALUES (SIGNED-BYTE 32) &OPTIONAL), not a > (VALUES > ; > (UNSIGNED-BYTE > ; > 32) > ; > &REST T). > ; etc. > > ; compiling (DEFUN WRITE-UINT32 ...) > ; compiling (DEFUN WRITE-UINT64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN WRITE-UINT64 > ; (SETF (DEREF > ; (CAST > ; (SAP-ALIEN (SB-SYS:SAP+ # ELEPHANT-MEMUTIL::OFFSET) > ; (* UNSIGNED-CHAR)) > ; (* (UNSIGNED 64)))) > ; ELEPHANT-MEMUTIL::NUM) > ; --> LET* MULTIPLE-VALUE-BIND LET SB-ALIEN-INTERNALS:%SET-DEREF > ; --> SB-ALIEN-INTERNALS:DEPOSIT-ALIEN-VALUE LET > ; --> SB-ALIEN-INTERNALS:MAYBE-WITH-PINNED-OBJECTS PROGN LET SETF > ; --> SB-KERNEL:%SET-SAP-REF-64 PROGN SB-KERNEL:%SET-SAP-REF-32 > ; ==> > ; (LOGAND SB-VM::VALUE 4294967295) > ; > ; note: forced to do static-fun Two-arg-and (cost 53) > ; unable to do inline (unsigned-byte 32) arithmetic (cost 2) > because: > ; The first argument is a (UNSIGNED-BYTE 64), not a (UNSIGNED- > BYTE 32). > ; unable to do inline (unsigned-byte 32) arithmetic (cost 2) > because: > ; The first argument is a (UNSIGNED-BYTE 64), not a (SIGNED- > BYTE 32). > ; etc. > > ; ==> > ; (ASH SB-VM::VALUE -32) > ; > ; note: forced to do full call > ; unable to do inline ASH (cost 2) because: > ; The first argument is a (UNSIGNED-BYTE 64), not a FIXNUM. > ; The result is a (VALUES (UNSIGNED-BYTE 32) &OPTIONAL), not a > (VALUES > ; FIXNUM > ; &REST > T). > ; unable to do inline ASH (cost 3) because: > ; The first argument is a (UNSIGNED-BYTE 64), not a (UNSIGNED- > BYTE 32). > ; etc. > > ; compiling (DEFUN WRITE-FLOAT ...) > ; compiling (DEFUN WRITE-DOUBLE ...) > ; compiling (DEFUN OFFSET-CHAR-POINTER ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN OFFSET-CHAR-POINTER > ; (SAP-ALIEN > ; (SB-SYS:SAP+ (ALIEN-SAP ELEPHANT-MEMUTIL::P) ELEPHANT- > MEMUTIL::OFFSET) > ; (* UNSIGNED-CHAR)) > ; ==> > ; (SB-ALIEN-INTERNALS:%SAP-ALIEN > ; (SB-SYS:SAP+ (ALIEN-SAP ELEPHANT-MEMUTIL::P) ELEPHANT- > MEMUTIL::OFFSET) > ; '#) > ; > ; note: unable to > ; optimize > ; because: > ; could not optimize away %SAP-ALIEN: forced to do runtime > ; allocation of alien-value structure > ; > ; note: doing SAP to pointer coercion (cost 20) > > ; compiling (DEFMACRO BYTE-LENGTH ...) > ; compiling (DEF-FUNCTION ("copy_buf" %COPY-STR-TO-BUF) ...) > ; compiling (DEFUN COPY-STR-TO-BUF ...) > ; compiling (DEFUN PROCESS-STRUCT-SLOT-DEFS ...) > ; compiling (DEFMACRO WITH-STRUCT-SLOTS ...) > ; compiling (DEFUN RESIZE-BUFFER-STREAM ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN RESIZE-BUFFER-STREAM > ; (LET ((ELEPHANT-MEMUTIL::NEWLEN (MAX LENGTH (* ELEPHANT- > MEMUTIL::LEN 2)))) > ; (DECLARE (TYPE FIXNUM ELEPHANT-MEMUTIL::NEWLEN)) > ; (LET ((ELEPHANT-MEMUTIL::NEWBUF > ; (UFFI:ALLOCATE-FOREIGN-OBJECT :UNSIGNED-CHAR > ; ELEPHANT-MEMUTIL::NEWLEN))) > ; (WHEN (UFFI:NULL-POINTER-P ELEPHANT-MEMUTIL::NEWBUF) > ; (ERROR > ; "Failed to allocate buffer stream of length ~A. allocate- > foreign-object returned a null pointer" > ; ELEPHANT-MEMUTIL::NEWLEN)) > ; (ELEPHANT-MEMUTIL::COPY-BUFS ELEPHANT-MEMUTIL::NEWBUF 0 > ; ELEPHANT-MEMUTIL::BUF 0 > ; ELEPHANT-MEMUTIL::SIZE) > ; (UFFI:FREE-FOREIGN-OBJECT ELEPHANT-MEMUTIL::BUF) > ; (SETF ELEPHANT-MEMUTIL::BUF ELEPHANT-MEMUTIL::NEWBUF) > ; (SETF ELEPHANT-MEMUTIL::LEN ELEPHANT-MEMUTIL::NEWLEN) > ; NIL)) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN RESIZE-BUFFER-STREAM-NO-COPY ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN RESIZE-BUFFER-STREAM-NO-COPY > ; (LET ((ELEPHANT-MEMUTIL::NEWLEN (MAX LENGTH (* ELEPHANT- > MEMUTIL::LEN 2)))) > ; (DECLARE (TYPE FIXNUM ELEPHANT-MEMUTIL::NEWLEN)) > ; (LET ((ELEPHANT-MEMUTIL::NEWBUF > ; (UFFI:ALLOCATE-FOREIGN-OBJECT :UNSIGNED-CHAR > ; ELEPHANT-MEMUTIL::NEWLEN))) > ; (WHEN (UFFI:NULL-POINTER-P ELEPHANT-MEMUTIL::NEWBUF) > ; (ERROR > ; "Failed to allocate buffer stream of length ~A. allocate- > foreign-object returned a null pointer" > ; ELEPHANT-MEMUTIL::NEWLEN)) > ; (UFFI:FREE-FOREIGN-OBJECT ELEPHANT-MEMUTIL::BUF) > ; (SETF ELEPHANT-MEMUTIL::BUF ELEPHANT-MEMUTIL::NEWBUF) > ; (SETF ELEPHANT-MEMUTIL::LEN ELEPHANT-MEMUTIL::NEWLEN) > ; NIL)) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN RESET-BUFFER-STREAM ...) > ; compiling (DEFUN BUFFER-WRITE-BYTE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-BYTE > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {1296D5D9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-376) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-WRITE-INT32 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-INT32 > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {129D9F51}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-386) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-WRITE-UINT32 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-UINT32 > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12A18581}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-395) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-WRITE-INT64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-INT64 > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12A63551}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-404) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-WRITE-UINT64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-UINT64 > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {1177F999}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-413) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-WRITE-FLOAT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-FLOAT > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {117BB351}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-424) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-WRITE-DOUBLE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-DOUBLE > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {11803F91}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-436) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-WRITE-STRING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-STRING > ; (LET* ((ELEPHANT-MEMUTIL::STR-BYTES > ; (ELEPHANT-MEMUTIL:BYTE-LENGTH ELEPHANT-MEMUTIL::S)) > ; (ELEPHANT-MEMUTIL::NEEDED > ; (+ ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::STR-BYTES))) > ; (DECLARE (TYPE FIXNUM ELEPHANT-MEMUTIL::STR-BYTES ELEPHANT- > MEMUTIL::NEEDED) > ; (DYNAMIC-EXTENT ELEPHANT-MEMUTIL::STR-BYTES ELEPHANT- > MEMUTIL::NEEDED)) > ; (WHEN (> ELEPHANT-MEMUTIL::NEEDED ELEPHANT-MEMUTIL::LEN) > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED)) > ; (ELEPHANT-MEMUTIL::COPY-STR-TO-BUF ELEPHANT-MEMUTIL::BUF > ; ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::S 0 ELEPHANT- > MEMUTIL::STR-BYTES) > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; NIL) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-READ-BYTE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-BYTE > ; (INCF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS)) > ; --> LET* FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {1195F5C9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-470) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-READ-BYTE-VECTOR ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-BYTE > ; (INCF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS)) > ; --> LET* FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {119B5659}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-483) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-WRITE-BYTE-VECTOR ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-BYTE-VECTOR > ; (LENGTH ELEPHANT-MEMUTIL::BV) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (AREF ELEPHANT-MEMUTIL::BV ELEPHANT-MEMUTIL::I) > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-REF/CHECK-BOUNDS ARRAY SB-INT:INDEX) > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > > ; in: DEFUN BUFFER-WRITE-BYTE > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {11AF7BA1}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-526) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-READ-TO-ARRAY-OFFSET ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-TO-ARRAY-OFFSET > ; (LENGTH ELEPHANT-MEMUTIL::ARRY) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (SETF (AREF ELEPHANT-MEMUTIL::ARRY > ; (+ ELEPHANT-MEMUTIL::I ELEPHANT-MEMUTIL::OFFSET)) > ; (ELEPHANT-MEMUTIL:BUFFER-READ-BYTE ELEPHANT-MEMUTIL::BS)) > ; --> SB-KERNEL:%ASET > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-SET/CHECK-BOUNDS ARRAY SB-INT:INDEX > ; SB-C::NEW-VALUE) > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > > ; in: DEFUN BUFFER-READ-BYTE > ; (INCF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS)) > ; --> LET* FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {11BB6309}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-552) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; in: DEFUN BUFFER-READ-TO-ARRAY-OFFSET > ; (SETF (AREF ELEPHANT-MEMUTIL::ARRY > ; (+ ELEPHANT-MEMUTIL::I ELEPHANT-MEMUTIL::OFFSET)) > ; (ELEPHANT-MEMUTIL:BUFFER-READ-BYTE ELEPHANT-MEMUTIL::BS)) > ; --> SB-KERNEL:%ASET > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-SET/CHECK-BOUNDS ARRAY SB-INT:INDEX > ; SB-C::NEW-VALUE) > ; > ; note: doing signed word to integer coercion (cost 20) > > ; compiling (DEFUN BUFFER-WRITE-FROM-ARRAY-OFFSET ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-FROM-ARRAY-OFFSET > ; (AREF ELEPHANT-MEMUTIL::ARRY (+ ELEPHANT-MEMUTIL::I ELEPHANT- > MEMUTIL::OFFSET)) > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-REF/CHECK-BOUNDS ARRAY SB-INT:INDEX) > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (DOTIMES (ELEPHANT-MEMUTIL::I LENGTH ELEPHANT-MEMUTIL::ARRY) > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE > ; (AREF ELEPHANT-MEMUTIL::ARRY > ; (+ ELEPHANT-MEMUTIL::I ELEPHANT-MEMUTIL::OFFSET)) > ; ELEPHANT-MEMUTIL::BS)) > ; --> DO BLOCK LET TAGBODY UNLESS IF >= IF > ; ==> > ; (< SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The second argument is a INTEGER, not a FIXNUM. > > ; (+ ELEPHANT-MEMUTIL::I ELEPHANT-MEMUTIL::OFFSET) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912) &OPTIONAL), not > a (VALUES > ; FIXNUM > ; &REST > T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The first argument is a UNSIGNED-BYTE, not a (SIGNED-BYTE 32). > ; The result is a (VALUES (INTEGER -536870912) &OPTIONAL), not > a (VALUES > ; (SIGNED > -BYTE > ; 32 > ) > ; &REST > T). > ; etc. > > ; (DOTIMES (ELEPHANT-MEMUTIL::I LENGTH ELEPHANT-MEMUTIL::ARRY) > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE > ; (AREF ELEPHANT-MEMUTIL::ARRY > ; (+ ELEPHANT-MEMUTIL::I ELEPHANT-MEMUTIL::OFFSET)) > ; ELEPHANT-MEMUTIL::BS)) > ; --> DO BLOCK LET TAGBODY PSETQ PSETF LET* MULTIPLE-VALUE-BIND LET 1+ > ; ==> > ; (+ ELEPHANT-MEMUTIL::I 1) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; etc. > > ; in: DEFUN BUFFER-WRITE-BYTE > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {11C703C9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-570) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-WRITE-OID ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-INT32 > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {11D42E29}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-589) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-READ-OID ...) > ; compiling (DEFUN BUFFER-READ-INT ...) > ; compiling (DEFUN BUFFER-READ-FIXNUM ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-FIXNUM > ; (THE FIXNUM (ELEPHANT-MEMUTIL:BUFFER-READ-FIXNUM32 ELEPHANT- > MEMUTIL::BS)) > ; > ; note: type assertion too complex to check: > ; (VALUES FIXNUM &REST T). > > ; compiling (DEFUN BUFFER-WRITE-INT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-INT32 > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12409EE9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-604) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFUN BUFFER-READ-UINT ...) > ; compiling (DEFUN BUFFER-WRITE-UINT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-WRITE-UINT32 > ; (SETF ELEPHANT-MEMUTIL::SIZE ELEPHANT-MEMUTIL::NEEDED) > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {124536C9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-615) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; compiling (DEFCONSTANT +2^32+ ...) > ; compiling (DEFCONSTANT +2^64+ ...) > ; compiling (DEFUN BUFFER-READ-FIXNUM32 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-FIXNUM32 > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION 4)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {124B5609}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-624) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-READ-INT32 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-INT32 > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION 4)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {124E4189}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-632) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-READ-UINT32 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-UINT32 > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION 4)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12518D11}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-638) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-READ-FIXNUM64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-FIXNUM64 > ; (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT-MEMUTIL::BS) > ; --> TRULY-THE SB-KERNEL:%INSTANCE-REF THE > ; ==> > ; ELEPHANT-MEMUTIL::BS > ; > ; note: deleting unreachable code > > ; (ASH SECOND 32) > ; > ; note: forced to do full call > ; unable to do inline ASH (cost 2) because: > ; The first argument is a (SIGNED-BYTE 32), not a FIXNUM. > ; The result is a (VALUES > ; (INTEGER -9223372036854775808 > 9223372032559808512) > ; &OPTIONAL), not a (VALUES FIXNUM &REST T). > ; unable to do inline ASH (cost 3) because: > ; The first argument is a (SIGNED-BYTE 32), not a (UNSIGNED- > BYTE 32). > ; The result is a (VALUES > ; (INTEGER -9223372036854775808 > 9223372032559808512) > ; &OPTIONAL), not a (VALUES (UNSIGNED-BYTE > 32) &REST T). > ; etc. > > ; (+ FIRST (ASH SECOND 32)) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (SIGNED-BYTE 32), not a FIXNUM. > ; The second argument is a (INTEGER -9223372036854775808 > ; 9223372032559808512), not a FIXNUM. > ; The result is a (VALUES > ; (INTEGER -9223372039002259456 > 9223372034707292159) > ; &OPTIONAL), not a (VALUES FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The second argument is a (INTEGER -9223372036854775808 > ; 9223372032559808512), not a > (SIGNED-BYTE 32). > ; The result is a (VALUES > ; (INTEGER -9223372039002259456 > 9223372034707292159) > ; &OPTIONAL), not a (VALUES (SIGNED-BYTE 32) > &REST T). > ; etc. > > ; (ASH FIRST 32) > ; > ; note: forced to do full call > ; unable to do inline ASH (cost 2) because: > ; The first argument is a (SIGNED-BYTE 32), not a FIXNUM. > ; The result is a (VALUES > ; (INTEGER -9223372036854775808 > 9223372032559808512) > ; &OPTIONAL), not a (VALUES FIXNUM &REST T). > ; unable to do inline ASH (cost 3) because: > ; The first argument is a (SIGNED-BYTE 32), not a (UNSIGNED- > BYTE 32). > ; The result is a (VALUES > ; (INTEGER -9223372036854775808 > 9223372032559808512) > ; &OPTIONAL), not a (VALUES (UNSIGNED-BYTE > 32) &REST T). > ; etc. > > ; (+ SECOND (ASH FIRST 32)) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (SIGNED-BYTE 32), not a FIXNUM. > ; The second argument is a (INTEGER -9223372036854775808 > ; 9223372032559808512), not a FIXNUM. > ; The result is a (VALUES > ; (INTEGER -9223372039002259456 > 9223372034707292159) > ; &OPTIONAL), not a (VALUES FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The second argument is a (INTEGER -9223372036854775808 > ; 9223372032559808512), not a > (SIGNED-BYTE 32). > ; The result is a (VALUES > ; (INTEGER -9223372039002259456 > 9223372034707292159) > ; &OPTIONAL), not a (VALUES (SIGNED-BYTE 32) > &REST T). > ; etc. > > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION 8)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12553EE9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-644) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL::READ-INT32 > ; (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT-MEMUTIL::BS) (+ > POSITION 4)) > ; > ; note: doing signed word to integer coercion (cost 20) > > ; compiling (DEFUN BUFFER-READ-INT64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-INT64 > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION 8)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12623C09}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-650) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-READ-UINT64 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-UINT64 > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION 8)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {126510D9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-656) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-READ-FLOAT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-FLOAT > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION 4)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12677DE9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-662) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL::READ-FLOAT > ; (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT-MEMUTIL::BS) > POSITION) > ; > ; note: doing float to pointer coercion (cost 13) to "" > > ; compiling (DEFUN BUFFER-READ-DOUBLE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-DOUBLE > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION 8)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {126CB249}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-668) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL::READ-DOUBLE > ; (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT-MEMUTIL::BS) > POSITION) > ; > ; note: doing float to pointer coercion (cost 13) to "" > > ; compiling (DEFUN BUFFER-READ-UCS1-STRING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-UCS1-STRING > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION ELEPHANT-MEMUTIL:BYTE-LENGTH)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {1271E5C1}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-674) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN BUFFER-READ-UCS4-STRING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/memutil/memutil.lisp > ; in: DEFUN BUFFER-READ-UCS4-STRING > ; (SETF (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > MEMUTIL::BS) > ; (+ POSITION ELEPHANT-MEMUTIL:BYTE-LENGTH)) > ; --> LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {127AAE09}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-721) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFPARAMETER +LITTLE-ENDIAN+ ...) > ; compiling (DEFUN LITTLE-ENDIAN-P ...) > > ; /Users/jensteich/Lisp/elephant/src/memutil/memutil.fasl written > ; compilation finished in 0:00:01 > STYLE-WARNING: redefining COPY-BUFS in DEFUN > STYLE-WARNING: redefining PROCESS-STRUCT-SLOT-DEFS in DEFUN > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > package.lisp" (written 28 APR 2007 04:31:16 AM): > ; compiling (IN-PACKAGE :CL-USER) > ; compiling (DEFPACKAGE ELEPHANT ...) > ; compiling (IN-PACKAGE "ELE") > ; compiling (DEFPACKAGE :ELEPHANT-USER ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/package.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > variables.lisp" (written 07 MAY 2007 05:04:18 AM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DEFVAR *ELEPHANT-CODE-VERSION* ...) > ; compiling (DEFVAR *ELEPHANT-UNMARKED-CODE-VERSION* ...) > ; compiling (DEFVAR *ELEPHANT-PROPERTIES-LABEL* ...) > ; compiling (DEFVAR *USER-CONFIGURABLE-PARAMETERS* ...) > ; compiling (DEFVAR *CIRCULARITY-INITIAL-HASH-SIZE* ...) > ; compiling (DEFPARAMETER *MAP-USING-DEGREE2* ...) > ; compiling (DEFVAR *BERKELEY-DB-CACHESIZE* ...) > ; compiling (DEFVAR *RESOURCED-BYTE-SPEC* ...) > ; compiling (DEFVAR *STORE-CONTROLLER* ...) > ; compiling (DEFVAR *CURRENT-TRANSACTION* ...) > ; compiling (DEFVAR *WARN-ON-MANUAL-CLASS-FINALIZATION* ...) > ; compiling (MAPCAR (LAMBDA # ...) ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/variables.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > transactions.lisp" (written 28 APR 2007 04:31:16 AM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DEFGENERIC EXECUTE-TRANSACTION ...) > ; compiling (DEFUN MAKE-TRANSACTION-RECORD ...) > ; compiling (DEFUN TRANSACTION-STORE ...) > ; compiling (DEFUN TRANSACTION-OBJECT ...) > ; compiling (DEFUN TRANSACTION-OBJECT-P ...) > ; compiling (DEFUN OWNED-TXN-P ...) > ; compiling (DEFINE-CONDITION TRANSACTION-RETRY-COUNT-EXCEEDED ...) > ; compiling (DEFMACRO WITH-TRANSACTION ...) > ; compiling (DEFMACRO ENSURE-TRANSACTION ...) > ; compiling (DEFMACRO WITH-BATCH-TRANSACTION ...) > ; compiling (DEFGENERIC CONTROLLER-START-TRANSACTION ...) > ; compiling (DEFGENERIC CONTROLLER-COMMIT-TRANSACTION ...) > ; compiling (DEFGENERIC CONTROLLER-ABORT-TRANSACTION ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/transactions.fasl > written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > metaclasses.lisp" (written 23 APR 2007 04:26:53 AM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DECLAIM (OPTIMIZE # ...)) > ; compiling (DEFCLASS PERSISTENT ...) > ; compiling (DEFMETHOD PRINT-OBJECT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/metaclasses.lisp > ; in: DEFMETHOD PRINT-OBJECT (PERSISTENT T) > ; (FORMAT STREAM "#<~A oid:~A>" (TYPE-OF ELEPHANT::OBJ) > ; (ELEPHANT::OID ELEPHANT::OBJ)) > ; ==> > ; (FORMAT SB-C::DEST (FORMATTER "#<~A oid:~A>") #:G11 #:G12) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MEMBER NIL T) STREAM (VECTOR > CHARACTER) > ; (VECTOR NIL) BASE-STRING), not a > STREAM. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MEMBER NIL T) STREAM (VECTOR > CHARACTER) > ; (VECTOR NIL) BASE-STRING), not a > (MEMBER T). > > ; compiling (DEFCLASS PERSISTENT-METACLASS ...) > ; compiling (DEFMACRO DEFPCLASS ...) > ; compiling (DEFUN ADD-PERSISTENT-METACLASS-ARGUMENT ...) > ; compiling (DEFMETHOD PERSISTENT-SLOTS ...) > ; compiling (DEFMETHOD PERSISTENT-SLOTS ...) > ; compiling (DEFMETHOD OLD-PERSISTENT-SLOTS ...) > ; compiling (DEFMETHOD UPDATE-PERSISTENT-SLOTS ...) > ; compiling (DEFCLASS PERSISTENT-SLOT-DEFINITION ...) > ; compiling (DEFCLASS PERSISTENT-DIRECT-SLOT-DEFINITION ...) > ; compiling (DEFCLASS PERSISTENT-EFFECTIVE-SLOT-DEFINITION ...) > ; compiling (DEFCLASS TRANSIENT-SLOT-DEFINITION ...) > ; compiling (DEFCLASS TRANSIENT-DIRECT-SLOT-DEFINITION ...) > ; compiling (DEFCLASS TRANSIENT-EFFECTIVE-SLOT-DEFINITION ...) > ; compiling (DEFGENERIC TRANSIENT ...) > ; compiling (DEFCLASS INDEXING-RECORD ...) > ; compiling (DEFMETHOD PRINT-OBJECT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/metaclasses.lisp > ; in: DEFMETHOD PRINT-OBJECT (INDEXING-RECORD T) > ; (LENGTH (ELEPHANT::INDEXING-RECORD-SLOTS ELEPHANT::OBJ)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (LENGTH (ELEPHANT::INDEXING-RECORD-DERIVED ELEPHANT::OBJ)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (FORMAT STREAM "#INDEXING-RECORD" > ; (LENGTH (ELEPHANT::INDEXING-RECORD-SLOTS ELEPHANT::OBJ)) > ; (LENGTH (ELEPHANT::INDEXING-RECORD-DERIVED > ELEPHANT::OBJ))) > ; ==> > ; (FORMAT SB-C::DEST (FORMATTER "#INDEXING-RECORD ~A>") #:G128 > ; #:G129) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MEMBER NIL T) STREAM (VECTOR > CHARACTER) > ; (VECTOR NIL) BASE-STRING), not a > STREAM. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MEMBER NIL T) STREAM (VECTOR > CHARACTER) > ; (VECTOR NIL) BASE-STRING), not a > (MEMBER T). > > ; compiling (DEFMETHOD INDEXED-RECORD ...) > ; compiling (DEFMETHOD INDEXED-RECORD ...) > ; compiling (DEFMETHOD OLD-INDEXED-RECORD ...) > ; compiling (DEFMETHOD UPDATE-INDEXED-RECORD ...) > ; compiling (DEFMETHOD MAKE-NEW-INDEXED-RECORD ...) > ; compiling (DEFMETHOD REMOVED-INDEXING? ...) > ; compiling (DEFUN INDEXED-SLOT-NAMES-FROM-DEFS ...) > ; compiling (DEFMETHOD REGISTER-INDEXED-SLOT ...) > ; compiling (DEFMETHOD UNREGISTER-INDEXED-SLOT ...) > ; compiling (DEFMETHOD REGISTER-DERIVED-INDEX ...) > ; compiling (DEFMETHOD UNREGISTER-DERIVED-INDEX ...) > ; compiling (DEFMETHOD INDEXED ...) > ; compiling (DEFMETHOD PREVIOUSLY-INDEXED ...) > ; compiling (DEFMETHOD INDEXED ...) > ; compiling (DEFMETHOD INDEXED ...) > ; compiling (DEFVAR *INHIBIT-INDEXING-LIST* ...) > ; compiling (DEFUN INHIBIT-INDEXING ...) > ; compiling (DEFUN UNINHIBIT-INDEXING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/metaclasses.lisp > ; in: DEFUN UNINHIBIT-INDEXING > ; (DELETE ELEPHANT::UID ELEPHANT::*INHIBIT-INDEXING-LIST*) > ; > ; note: unable to > ; convert to EQ test > ; due to type uncertainty: > ; The second argument is a SEQUENCE, not a LIST. > > ; compiling (DEFMETHOD SLOT-DEFINITION-ALLOCATION ...) > ; compiling (DEFMETHOD DIRECT-SLOT-DEFINITION-CLASS ...) > ; compiling (DEFMETHOD VALIDATE-SUPERCLASS ...) > ; compiling (DEFMETHOD VALIDATE-SUPERCLASS ...) > ; compiling (DEFGENERIC PERSISTENT-P ...) > ; compiling (DEFMETHOD EFFECTIVE-SLOT-DEFINITION-CLASS ...) > ; compiling (DEFUN ENSURE-TRANSIENT-CHAIN ...) > ; compiling (DEFMETHOD COMPUTE-EFFECTIVE-SLOT-DEFINITION-INITARGS ...) > ; compiling (DEFUN FIND-SLOT-DEF-BY-NAME ...) > ; compiling (DEFUN PERSISTENT-SLOT-DEFS ...) > ; compiling (DEFUN TRANSIENT-SLOT-DEFS ...) > ; compiling (DEFUN PERSISTENT-SLOT-NAMES ...) > ; compiling (DEFUN TRANSIENT-SLOT-NAMES ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/metaclasses.fasl written > ; compilation finished in 0:00:00 > STYLE-WARNING: Implicitly creating new generic function PERSISTENT- > SLOTS. > STYLE-WARNING: > Implicitly creating new generic function OLD-PERSISTENT-SLOTS. > STYLE-WARNING: > Implicitly creating new generic function UPDATE-PERSISTENT-SLOTS. > STYLE-WARNING: Implicitly creating new generic function INDEXED- > RECORD. > STYLE-WARNING: Implicitly creating new generic function OLD-INDEXED- > RECORD. > STYLE-WARNING: > Implicitly creating new generic function UPDATE-INDEXED-RECORD. > STYLE-WARNING: > Implicitly creating new generic function MAKE-NEW-INDEXED-RECORD. > STYLE-WARNING: Implicitly creating new generic function REMOVED- > INDEXING?. > STYLE-WARNING: > Implicitly creating new generic function REGISTER-INDEXED-SLOT. > STYLE-WARNING: > Implicitly creating new generic function UNREGISTER-INDEXED-SLOT. > STYLE-WARNING: > Implicitly creating new generic function REGISTER-DERIVED-INDEX. > STYLE-WARNING: > Implicitly creating new generic function UNREGISTER-DERIVED-INDEX. > STYLE-WARNING: Implicitly creating new generic function PREVIOUSLY- > INDEXED. > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > classes.lisp" (written 29 APR 2007 04:10:54 AM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DEFVAR *DEBUG-SI* ...) > ; compiling (DECLAIM (OPTIMIZE #)) > ; compiling (DEFMETHOD INITIALIZE-INSTANCE ...) > ; compiling (DEFCLASS PERSISTENT-OBJECT ...) > ; compiling (DEFMETHOD SHARED-INITIALIZE ...) > ; compiling (DEFMETHOD FINALIZE-INHERITANCE ...) > ; compiling (DEFMETHOD REINITIALIZE-INSTANCE ...) > ; compiling (DEFMETHOD SHARED-INITIALIZE ...) > ; compiling (DEFUN INITIALIZE-PERSISTENT-SLOTS ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classes.lisp > ; in: DEFUN INITIALIZE-PERSISTENT-SLOTS > ; (FUNCALL ELEPHANT::INITFUN) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; compiling (DEFMETHOD UPDATE-INSTANCE-FOR-REDEFINED-CLASS ...) > ; compiling (DEFMETHOD CHANGE-CLASS ...) > ; compiling (DEFMETHOD CHANGE-CLASS ...) > ; compiling (DEFMETHOD UPDATE-INSTANCE-FOR-DIFFERENT-CLASS ...) > ; compiling (DEFMETHOD SLOT-VALUE-USING-CLASS ...) > ; compiling (DEFMETHOD (SETF SLOT-VALUE-USING-CLASS) ...) > ; compiling (DEFMETHOD SLOT-BOUNDP-USING-CLASS ...) > ; compiling (DEFMETHOD SLOT-BOUNDP-USING-CLASS ...) > ; compiling (DEFMETHOD SLOT-MAKUNBOUND-USING-CLASS ...) > ; compiling (DEFUN VALID-PERSISTENT-REFERENCE-P ...) > ; compiling (DEFINE-CONDITION CROSS-REFERENCE-ERROR ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classes.lisp > ; in: DEFINE-CONDITION CROSS-REFERENCE-ERROR > ; (FORMAT STREAM > ; "Attempted to write object ~A with home store ~A into > store ~A" > ; (ELEPHANT::CROSS-REFERENCE-ERROR-OBJECT CONDITION) > ; (ELEPHANT::CROSS-REFERENCE-ERROR-HOME-CONTROLLER > CONDITION) > ; (ELEPHANT::CROSS-REFERENCE-ERROR-FOREIGN-CONTROLLER > CONDITION)) > ; ==> > ; (FORMAT SB-C::DEST > ; (FORMATTER > ; "Attempted to write object ~A with home store ~A into > store ~A") > ; #:G607 #:G608 #:G609) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MEMBER NIL T) STREAM (VECTOR > CHARACTER) > ; (VECTOR NIL) BASE-STRING), not a > STREAM. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MEMBER NIL T) STREAM (VECTOR > CHARACTER) > ; (VECTOR NIL) BASE-STRING), not a > (MEMBER T). > > ; compiling (DEFUN SIGNAL-CROSS-REFERENCE-ERROR ...) > ; compiling (DEFUN MAKE-PERSISTENT-READER ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classes.lisp > ; in: DEFUN MAKE-PERSISTENT-READER > ; (LAMBDA (ELEPHANT::INSTANCE) > ; (DECLARE (TYPE ELEPHANT:PERSISTENT-OBJECT ELEPHANT::INSTANCE)) > ; (ELEPHANT::PERSISTENT-SLOT-READER (ELEPHANT::GET-CON > ELEPHANT::INSTANCE) > ; ELEPHANT::INSTANCE ELEPHANT::NAME)) > ; --> FUNCTION LOCALLY SB-C::%FUNCALL MULTIPLE-VALUE-BIND LET UNLESS > IF > ; ==> > ; (TYPEP #:G660 'ELEPHANT:PERSISTENT-OBJECT) > ; > ; note: can't open-code test of unknown type PERSISTENT-OBJECT > > ; compiling (DEFUN MAKE-PERSISTENT-WRITER ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classes.lisp > ; in: DEFUN MAKE-PERSISTENT-WRITER > ; (LAMBDA (ELEPHANT::NEW-VALUE ELEPHANT::INSTANCE) > ; (DECLARE (OPTIMIZE (SPEED 3)) > ; (TYPE ELEPHANT:PERSISTENT-OBJECT ELEPHANT::INSTANCE)) > ; (ELEPHANT::PERSISTENT-SLOT-WRITER (ELEPHANT::GET-CON > ELEPHANT::INSTANCE) > ; ELEPHANT::NEW-VALUE ELEPHANT::INSTANCE ELEPHANT::NAME)) > ; --> FUNCTION LOCALLY SB-C::%FUNCALL MULTIPLE-VALUE-BIND LET UNLESS > IF > ; ==> > ; (TYPEP #:G667 'ELEPHANT:PERSISTENT-OBJECT) > ; > ; note: can't open-code test of unknown type PERSISTENT-OBJECT > > ; compiling (DEFUN MAKE-PERSISTENT-SLOT-BOUNDP ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classes.lisp > ; in: DEFUN MAKE-PERSISTENT-SLOT-BOUNDP > ; (LAMBDA (ELEPHANT::INSTANCE) > ; (DECLARE (TYPE ELEPHANT:PERSISTENT-OBJECT ELEPHANT::INSTANCE)) > ; (ELEPHANT::PERSISTENT-SLOT-BOUNDP (ELEPHANT::GET-CON > ELEPHANT::INSTANCE) > ; ELEPHANT::INSTANCE ELEPHANT::NAME)) > ; --> FUNCTION LOCALLY SB-C::%FUNCALL MULTIPLE-VALUE-BIND LET UNLESS > IF > ; ==> > ; (TYPEP #:G673 'ELEPHANT:PERSISTENT-OBJECT) > ; > ; note: can't open-code test of unknown type PERSISTENT-OBJECT > > ; compiling (DEFMETHOD INITIALIZE-INTERNAL-SLOT-FUNCTIONS ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/classes.fasl written > ; compilation finished in 0:00:01 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > cache.lisp" (written 12 APR 2007 04:47:31 AM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DEFUN MAKE-CACHE-TABLE ...) > ; compiling (DEFUN GET-CACHE ...) > ; compiling (DEFUN MAKE-FINALIZER ...) > ; compiling (DEFUN SETF-CACHE ...) > ; compiling (DEFSETF GET-CACHE ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/cache.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > serializer.lisp" (written 12 APR 2007 04:47:33 AM): > ; compiling (IN-PACKAGE :ELEPHANT) > ; compiling (DEFUN SERIALIZE ...) > ; compiling (DEFUN DESERIALIZE ...) > ; compiling (DEFGENERIC STRUCT-CONSTRUCTOR ...) > ; compiling (DEFMETHOD STRUCT-CONSTRUCTOR ...) > ; compiling (DEFUN SERIALIZE-TO-BASE64-STRING ...) > ; compiling (DEFUN CONVERT-BUFFER-TO-BASE64-STRING ...) > ; compiling (DEFUN DESERIALIZE-FROM-BASE64-STRING ...) > ; compiling (DEFUN CONVERT-BUFFER-FROM-BASE64-STRING ...) > ; compiling (DEFCONSTANT +RESERVED-DBINFO+ ...) > ; compiling (DEFCONSTANT +ELEPHANT-VERSION+ ...) > ; compiling (DEFCONSTANT +ELEPHANT-SERIALIZER-VERSION+ ...) > ; compiling (DEFUN SERIALIZE-DATABASE-VERSION-KEY ...) > ; compiling (DEFUN SERIALIZE-DATABASE-VERSION-VALUE ...) > ; compiling (DEFUN DESERIALIZE-DATABASE-VERSION-VALUE ...) > ; compiling (DEFUN SERIALIZE-DATABASE-SERIALIZER-VERSION-KEY ...) > ; compiling (DEFUN SERIALIZE-DATABASE-SERIALIZER-VERSION-VALUE ...) > ; compiling (DEFUN DESERIALIZE-DATABASE-SERIALIZER-VERSION-VALUE ...) > ; compiling (DEFUN SERIALIZE-RESERVED-TAG ...) > ; compiling (DEFUN SERIALIZE-SYSTEM-TAG ...) > ; compiling (DEFUN SERIALIZE-SYSTEM-INTEGER ...) > ; compiling (DEFUN DESERIALIZE-SYSTEM-INTEGER ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer.lisp > ; in: DEFUN DESERIALIZE-SYSTEM-INTEGER > ; (ELEPHANT-MEMUTIL:BUFFER-READ-INT32 ELEPHANT::BS) > ; --> BLOCK LET > ; ==> > ; (THE (SIGNED-BYTE 32) > ; (ELEPHANT-MEMUTIL::READ-INT32 > ; (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT- > MEMUTIL::BS) POSITION)) > ; > ; note: type assertion too complex to check: > ; (VALUES (SIGNED-BYTE 32) &REST T). > > ; compiling (DEFUN SLOTS-AND-VALUES ...) > ; compiling (DEFUN STRUCT-SLOTS-AND-VALUES ...) > ; compiling (DECLAIM (TYPE HASH-TABLE ...)) > ; compiling (DEFVAR ARRAY-TYPE-TO-BYTE ...) > ; compiling (DEFVAR BYTE-TO-ARRAY-TYPE ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (SETF (GETHASH # ...) ...) > ; compiling (DEFUN TYPE= ...) > ; compiling (LET (#) ...) > ; compiling (LOOP FOR ...) > ; compiling (DEFUN ARRAY-TYPE-FROM-BYTE ...) > ; compiling (DEFUN BYTE-FROM-ARRAY-TYPE ...) > ; compiling (DEFUN INT-BYTE-SPEC ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/serializer.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > controller.lisp" (written 07 MAY 2007 10:39:33 PM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DEFVAR *ELEPHANT-DATA-STORES* ...) > ; compiling (DEFVAR *ELEPHANT-CONTROLLER-INIT* ...) > ; compiling (DEFUN REGISTER-DATA-STORE-CON-INIT ...) > ; compiling (DEFUN LOOKUP-DATA-STORE-CON-INIT ...) > ; compiling (DEFVAR *DBCONNECTION-SPEC* ...) > ; compiling (DEFVAR *DBCONNECTION-LOCK* ...) > ; compiling (DEFGENERIC GET-CON ...) > ; compiling (DEFINE-CONDITION CONTROLLER-LOST-ERROR ...) > ; compiling (DEFUN SIGNAL-CONTROLLER-LOST-ERROR ...) > ; compiling (DEFMETHOD GET-CON ...) > ; compiling (DEFUN GET-CONTROLLER ...) > ; compiling (DEFUN BUILD-CONTROLLER ...) > ; compiling (DEFUN LOAD-DATA-STORE ...) > ; compiling (DEFUN SATISFY-ASDF-DEPENDENCIES ...) > ; compiling (DEFUN GET-USER-CONFIGURATION-PARAMETER ...) > ; compiling (DEFUN INITIALIZE-USER-PARAMETERS ...) > ; compiling (DEFCLASS STORE-CONTROLLER ...) > ; compiling (DEFMETHOD PRINT-OBJECT ...) > ; compiling (DEFUN CACHE-INSTANCE ...) > ; compiling (DEFUN GET-CACHED-INSTANCE ...) > ; compiling (DEFMETHOD FLUSH-INSTANCE-CACHE ...) > ; compiling (DEFGENERIC DATABASE-VERSION ...) > ; compiling (DEFMETHOD DATABASE-VERSION ...) > ; compiling (DEFUN PRIOR-VERSION-P ...) > ; compiling (DEFPARAMETER *ELEPHANT-UPGRADE-TABLE* ...) > ; compiling (DEFMETHOD UP-TO-DATE-P ...) > ; compiling (DEFMETHOD UPGRADABLE-P ...) > ; compiling (DEFGENERIC UPGRADE ...) > ; compiling (DEFMETHOD UPGRADE ...) > ; compiling (DEFMETHOD INITIALIZE-SERIALIZER ...) > ; compiling (DEFVAR *ALWAYS-CONVERT* ...) > ; compiling (DEFPARAMETER *LEGACY-SYMBOL-CONVERSIONS* ...) > ; compiling (DEFUN ADD-SYMBOL-CONVERSION ...) > ; compiling (DEFUN MAP-LEGACY-SYMBOLS ...) > ; compiling (DEFPARAMETER *LEGACY-PACKAGE-CONVERSIONS* ...) > ; compiling (DEFUN ADD-PACKAGE-CONVERSION ...) > ; compiling (DEFUN MAP-LEGACY-PACKAGE-NAMES ...) > ; compiling (DEFUN MAP-LEGACY-NAMES ...) > ; compiling (DEFUN TRANSLATE-AND-INTERN-SYMBOL ...) > ; compiling (DEFGENERIC OPEN-CONTROLLER ...) > ; compiling (DEFGENERIC CLOSE-CONTROLLER ...) > ; compiling (DEFMETHOD CLOSE-CONTROLLER ...) > ; compiling (DEFGENERIC CONNECTION-IS-INDEED-OPEN ...) > ; compiling (DEFGENERIC NEXT-OID ...) > ; compiling (DEFGENERIC OPTIMIZE-LAYOUT ...) > ; compiling (DEFGENERIC PERSISTENT-SLOT-READER ...) > ; compiling (DEFGENERIC PERSISTENT-SLOT-WRITER ...) > ; compiling (DEFGENERIC PERSISTENT-SLOT-BOUNDP ...) > ; compiling (DEFGENERIC PERSISTENT-SLOT-MAKUNBOUND ...) > ; compiling (DEFUN OPEN-STORE ...) > ; compiling (DEFUN CLOSE-STORE ...) > ; compiling (DEFMACRO WITH-OPEN-STORE ...) > ; compiling (DEFUN ADD-TO-ROOT ...) > ; compiling (DEFUN GET-FROM-ROOT ...) > ; compiling (DEFUN ROOT-EXISTSP ...) > ; compiling (DEFUN REMOVE-FROM-ROOT ...) > ; compiling (DEFUN MAP-ROOT ...) > ; compiling (DEFGENERIC DROP-POBJECT ...) > ; compiling (DEFMETHOD DROP-POBJECT ...) > ; compiling (DEFVAR *RESTRICTED-PROPERTIES* ...) > ; compiling (DEFMETHOD CONTROLLER-PROPERTIES ...) > ; compiling (DEFMETHOD SET-ELE-PROPERTY ...) > ; compiling (DEFMETHOD GET-ELE-PROPERTY ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/controller.fasl written > ; compilation finished in 0:00:01 > STYLE-WARNING: > Implicitly creating new generic function FLUSH-INSTANCE-CACHE. > STYLE-WARNING: Implicitly creating new generic function UP-TO-DATE-P. > STYLE-WARNING: Implicitly creating new generic function UPGRADABLE-P. > STYLE-WARNING: > Implicitly creating new generic function INITIALIZE-SERIALIZER. > STYLE-WARNING: > Implicitly creating new generic function CONTROLLER-PROPERTIES. > STYLE-WARNING: Implicitly creating new generic function SET-ELE- > PROPERTY. > STYLE-WARNING: Implicitly creating new generic function GET-ELE- > PROPERTY. > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > collections.lisp" (written 07 MAY 2007 08:38:29 PM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DEFCLASS PERSISTENT-COLLECTION ...) > ; compiling (DEFUN MAKE-BTREE ...) > ; compiling (DEFGENERIC BUILD-BTREE ...) > ; compiling (DEFCLASS BTREE ...) > ; compiling (DEFGENERIC GET-VALUE ...) > ; compiling (DEFGENERIC (SETF GET-VALUE) ...) > ; compiling (DEFGENERIC REMOVE-KV ...) > ; compiling (DEFGENERIC EXISTSP ...) > ; compiling (DEFMETHOD OPTIMIZE-LAYOUT ...) > ; compiling (DEFGENERIC DROP-BTREE ...) > ; compiling (DEFUN MAKE-INDEXED-BTREE ...) > ; compiling (DEFGENERIC BUILD-INDEXED-BTREE ...) > ; compiling (DEFCLASS INDEXED-BTREE ...) > ; compiling (DEFGENERIC ADD-INDEX ...) > ; compiling (DEFGENERIC GET-INDEX ...) > ; compiling (DEFGENERIC REMOVE-INDEX ...) > ; compiling (DEFGENERIC MAP-INDICES ...) > ; compiling (DEFGENERIC BUILD-BTREE-INDEX ...) > ; compiling (DEFCLASS BTREE-INDEX ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/collections.lisp > ; in: DEFCLASS BTREE-INDEX > ; (DEFCLASS ELEPHANT:BTREE-INDEX (ELEPHANT:BTREE) > ; ((ELEPHANT:PRIMARY :TYPE ELEPHANT:INDEXED-BTREE :READER > ; ELEPHANT:PRIMARY :INITARG :PRIMARY) > ; (ELEPHANT:KEY-FORM :READER ELEPHANT:KEY- > FORM :INITARG :KEY-FORM > ; :INITFORM NIL) > ; (ELEPHANT:KEY-FN :TYPE FUNCTION :ACCESSOR > ELEPHANT:KEY-FN > ; :TRANSIENT T)) > ; (:METACLASS ELEPHANT:PERSISTENT-METACLASS) > ; (:DOCUMENTATION "Secondary index to an indexed-btree.")) > ; --> PROGN EVAL-WHEN LET LOCALLY SB-C::%FUNCALL MULTIPLE-VALUE-BIND > LET > ; --> UNLESS IF > ; ==> > ; (TYPEP #:G30 'ELEPHANT:INDEXED-BTREE) > ; > ; note: can't open-code test of unknown type INDEXED-BTREE > > ; compiling (DEFMETHOD SHARED-INITIALIZE ...) > ; compiling (DEFGENERIC GET-PRIMARY-KEY ...) > ; compiling (DEFMETHOD (SETF GET-VALUE) ...) > ; compiling (DEFMETHOD REMOVE-KV ...) > ; compiling (DEFCLASS CURSOR ...) > ; compiling (DEFGENERIC MAKE-CURSOR ...) > ; compiling (DEFGENERIC MAKE-SIMPLE-CURSOR ...) > ; compiling (DEFGENERIC CURSOR-CLOSE ...) > ; compiling (DEFGENERIC CURSOR-DUPLICATE ...) > ; compiling (DEFGENERIC CURSOR-CURRENT ...) > ; compiling (DEFGENERIC CURSOR-FIRST ...) > ; compiling (DEFGENERIC CURSOR-LAST ...) > ; compiling (DEFGENERIC CURSOR-NEXT ...) > ; compiling (DEFGENERIC CURSOR-PREV ...) > ; compiling (DEFGENERIC CURSOR-SET ...) > ; compiling (DEFGENERIC CURSOR-SET-RANGE ...) > ; compiling (DEFGENERIC CURSOR-GET-BOTH ...) > ; compiling (DEFGENERIC CURSOR-GET-BOTH-RANGE ...) > ; compiling (DEFGENERIC CURSOR-DELETE ...) > ; compiling (DEFGENERIC CURSOR-PUT ...) > ; compiling (DEFCLASS SECONDARY-CURSOR ...) > ; compiling (DEFGENERIC CURSOR-PCURRENT ...) > ; compiling (DEFGENERIC CURSOR-PFIRST ...) > ; compiling (DEFGENERIC CURSOR-PLAST ...) > ; compiling (DEFGENERIC CURSOR-PNEXT ...) > ; compiling (DEFGENERIC CURSOR-PPREV ...) > ; compiling (DEFGENERIC CURSOR-PSET ...) > ; compiling (DEFGENERIC CURSOR-PSET-RANGE ...) > ; compiling (DEFGENERIC CURSOR-PGET-BOTH ...) > ; compiling (DEFGENERIC CURSOR-PGET-BOTH-RANGE ...) > ; compiling (DEFGENERIC CURSOR-NEXT-DUP ...) > ; compiling (DEFGENERIC CURSOR-NEXT-NODUP ...) > ; compiling (DEFGENERIC CURSOR-PNEXT-DUP ...) > ; compiling (DEFGENERIC CURSOR-PNEXT-NODUP ...) > ; compiling (DEFGENERIC CURSOR-PREV-DUP ...) > ; compiling (DEFMETHOD CURSOR-PREV-DUP ...) > ; compiling (DEFGENERIC CURSOR-PREV-NODUP ...) > ; compiling (DEFGENERIC CURSOR-PPREV-DUP ...) > ; compiling (DEFMETHOD CURSOR-PPREV-DUP ...) > ; compiling (DEFGENERIC CURSOR-PPREV-NODUP ...) > ; compiling (DEFMACRO WITH-BTREE-CURSOR ...) > ; compiling (DEFMETHOD DROP-BTREE ...) > ; compiling (DEFUN LISP-COMPARE<= ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/collections.lisp > ; in: DEFUN LISP-COMPARE<= > ; (STRING<= ELEPHANT::A ELEPHANT::B) > ; ==> > ; (SB-KERNEL:STRING<=* SB-C::STRING1 SB-C::STRING2 SB-C::START1 SB- > C::END1 > ; SB-C::START2 SB-C::END2) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a STRING, not a SIMPLE-BASE-STRING. > ; The second argument is a (OR (VECTOR CHARACTER) (VECTOR NIL) > BASE-STRING > ; SYMBOL CHARACTER), not a SIMPLE- > BASE-STRING. > > ; (STRING<= (SYMBOL-NAME ELEPHANT::A) (SYMBOL-NAME ELEPHANT::B)) > ; ==> > ; (SB-KERNEL:STRING<=* SB-C::STRING1 SB-C::STRING2 SB-C::START1 SB- > C::END1 > ; SB-C::START2 SB-C::END2) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SIMPLE-STRING, not a SIMPLE-BASE-STRING. > ; The second argument is a SIMPLE-STRING, not a SIMPLE-BASE-STRING. > > ; (<= (ELEPHANT::OID ELEPHANT::A) (ELEPHANT::OID ELEPHANT::B)) > ; --> OR LET < IF > ; ==> > ; (< SB-C::X SB-C::Y) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The second argument is a REAL, not a DOUBLE-FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a DOUBLE-FLOAT. > ; The second argument is a REAL, not a SINGLE-FLOAT. > ; > ; note: unable to > ; open-code FLOAT to RATIONAL comparison > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > ; The second argument is a REAL, not a RATIONAL. > > ; --> OR LET IF OR = IF > ; ==> > ; (= SB-C::X SB-C::Y) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The second argument is a REAL, not a DOUBLE-FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a DOUBLE-FLOAT. > ; The second argument is a REAL, not a SINGLE-FLOAT. > ; > ; note: unable to > ; open-code FLOAT to RATIONAL comparison > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > ; The second argument is a REAL, not a RATIONAL. > ; > ; note: unable to open code because: The operands might not be the > same type. > > ; (<= ELEPHANT::A ELEPHANT::B) > ; --> OR LET < IF > ; ==> > ; (< SB-C::X SB-C::Y) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The second argument is a REAL, not a DOUBLE-FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a DOUBLE-FLOAT. > ; The second argument is a REAL, not a SINGLE-FLOAT. > ; > ; note: unable to > ; open-code FLOAT to RATIONAL comparison > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > ; The second argument is a REAL, not a RATIONAL. > > ; --> OR LET IF OR = IF > ; ==> > ; (= SB-C::X SB-C::Y) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The second argument is a REAL, not a DOUBLE-FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a DOUBLE-FLOAT. > ; The second argument is a REAL, not a SINGLE-FLOAT. > ; > ; note: unable to > ; open-code FLOAT to RATIONAL comparison > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > ; The second argument is a REAL, not a RATIONAL. > ; > ; note: unable to open code because: The operands might not be the > same type. > > ; (<= (ELEPHANT::OID ELEPHANT::A) (ELEPHANT::OID ELEPHANT::B)) > ; --> OR LET < IF > ; ==> > ; (< SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline float comparison (cost 3) because: > ; The first argument is a REAL, not a DOUBLE-FLOAT. > ; The second argument is a REAL, not a DOUBLE-FLOAT. > ; unable to do inline float comparison (cost 3) because: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The second argument is a REAL, not a SINGLE-FLOAT. > ; etc. > > ; --> OR LET IF OR = IF > ; ==> > ; (= SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-= (cost 10) > ; unable to do inline float comparison (cost 3) because: > ; The first argument is a REAL, not a DOUBLE-FLOAT. > ; The second argument is a REAL, not a DOUBLE-FLOAT. > ; unable to do inline float comparison (cost 3) because: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The second argument is a REAL, not a SINGLE-FLOAT. > > ; (<= ELEPHANT::A ELEPHANT::B) > ; --> OR LET < IF > ; ==> > ; (< SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline float comparison (cost 3) because: > ; The first argument is a REAL, not a DOUBLE-FLOAT. > ; The second argument is a REAL, not a DOUBLE-FLOAT. > ; unable to do inline float comparison (cost 3) because: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The second argument is a REAL, not a SINGLE-FLOAT. > ; etc. > > ; --> OR LET IF OR = IF > ; ==> > ; (= SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-= (cost 10) > ; unable to do inline float comparison (cost 3) because: > ; The first argument is a REAL, not a DOUBLE-FLOAT. > ; The second argument is a REAL, not a DOUBLE-FLOAT. > ; unable to do inline float comparison (cost 3) because: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The second argument is a REAL, not a SINGLE-FLOAT. > > ; compiling (DEFUN LISP-COMPARE-EQUAL ...) > ; compiling (DEFGENERIC MAP-BTREE ...) > ; compiling (DEFMETHOD MAP-BTREE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/collections.lisp > ; in: DEFMETHOD MAP-BTREE (T BTREE) > ; (LET ((ELEPHANT::FN > ; (IF ELEPHANT::COLLECT #'ELEPHANT::COLLECTOR ELEPHANT::FN))) > ; (DECLARE (DYNAMIC-EXTENT #'ELEPHANT::CONTINUE-P > #'ELEPHANT::COLLECTOR)) > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::KEY ELEPHANT::VALUE) > ; (COND > ; (ELEPHANT::VALUE-SET-P > ; (ELEPHANT:CURSOR-SET ELEPHANT::CURS ELEPHANT::VALUE)) > ; ((AND # #) (ELEPHANT:CURSOR-FIRST ELEPHANT::CURS)) > ; ((AND ELEPHANT::FROM-END #) (ELEPHANT:CURSOR-LAST > ELEPHANT::CURS)) > ; (T (IF ELEPHANT::FROM-END # #))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF ELEPHANT::EXISTS? > ; (FUNCALL ELEPHANT::FN ELEPHANT::KEY ELEPHANT::VALUE) > ; (RETURN-FROM ELEPHANT:MAP-BTREE NIL)) > ; (LOOP > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::K ELEPHANT::V) > ; (IF ELEPHANT::FROM-END # #) > ; (DECLARE #) > ; (IF # # #))))) > ; > ; note: ignoring DYNAMIC-EXTENT declaration for free CONTINUE-P > ; > ; note: ignoring DYNAMIC-EXTENT declaration for free COLLECTOR > > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::KEY ELEPHANT::VALUE) > ; (COND > ; (ELEPHANT::VALUE-SET-P > ; (ELEPHANT:CURSOR-SET ELEPHANT::CURS ELEPHANT::VALUE)) > ; ((AND (NOT ELEPHANT::FROM-END) (NULL ELEPHANT::START)) > ; (ELEPHANT:CURSOR-FIRST ELEPHANT::CURS)) > ; ((AND ELEPHANT::FROM-END (NULL ELEPHANT::END)) > ; (ELEPHANT:CURSOR-LAST ELEPHANT::CURS)) > ; (T > ; (IF ELEPHANT::FROM-END > ; (ELEPHANT:CURSOR-SET-RANGE ELEPHANT::CURS ELEPHANT::END) > ; (ELEPHANT:CURSOR-SET-RANGE ELEPHANT::CURS > ELEPHANT::START)))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF ELEPHANT::EXISTS? (FUNCALL ELEPHANT::FN ELEPHANT::KEY > ELEPHANT::VALUE) > ; (RETURN-FROM ELEPHANT:MAP-BTREE NIL)) > ; (LOOP > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::K ELEPHANT::V) > ; (IF ELEPHANT::FROM-END (ELEPHANT:CURSOR-PREV > ELEPHANT::CURS) > ; (ELEPHANT:CURSOR-NEXT ELEPHANT::CURS)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF (AND ELEPHANT::EXISTS? #) > ; (FUNCALL ELEPHANT::FN ELEPHANT::K ELEPHANT::V) (RETURN > NIL))))) > ; --> MULTIPLE-VALUE-CALL > ; ==> > ; #'(LAMBDA > ; (&OPTIONAL (ELEPHANT::EXISTS?) (ELEPHANT::KEY) > (ELEPHANT::VALUE) > ; &REST #:G232) > ; (DECLARE (IGNORE #:G232)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF ELEPHANT::EXISTS? > ; (FUNCALL ELEPHANT::FN ELEPHANT::KEY ELEPHANT::VALUE) > ; (RETURN-FROM ELEPHANT:MAP-BTREE NIL)) > ; (LOOP > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::K ELEPHANT::V) > ; (IF ELEPHANT::FROM-END (ELEPHANT:CURSOR-PREV > ELEPHANT::CURS) > ; (ELEPHANT:CURSOR-NEXT ELEPHANT::CURS)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF (AND ELEPHANT::EXISTS? #) > ; (FUNCALL ELEPHANT::FN ELEPHANT::K ELEPHANT::V) > (RETURN NIL))))) > ; > ; note: ignoring DYNAMIC-EXTENT declaration for free K > ; > ; note: ignoring DYNAMIC-EXTENT declaration for free V > > ; (DEFMETHOD ELEPHANT:MAP-BTREE > ; (ELEPHANT::FN (ELEPHANT:BTREE ELEPHANT:BTREE) &REST > ELEPHANT::ARGS > ; &KEY ELEPHANT::START ELEPHANT::END > ; (ELEPHANT::VALUE NIL ELEPHANT::VALUE-SET-P) > ELEPHANT::FROM-END > ; ELEPHANT::COLLECT &ALLOW-OTHER-KEYS) > ; (LET ((ELEPHANT::END > ; (IF ELEPHANT::VALUE-SET-P ELEPHANT::VALUE > ELEPHANT::END)) > ; (ELEPHANT::RESULTS NIL)) > ; (ELEPHANT:ENSURE-TRANSACTION > ; (:STORE-CONTROLLER (ELEPHANT::GET-CON > ELEPHANT:BTREE) :DEGREE-2 > ; ELEPHANT::*MAP-USING-DEGREE2*) > ; (ELEPHANT:WITH-BTREE-CURSOR (ELEPHANT::CURS > ELEPHANT:BTREE) > ; (FLET # > ; #))) > ; ELEPHANT::RESULTS)) > ; --> PROGN EVAL-WHEN SB-PCL::%DEFMETHOD-EXPANDER PROGN > ; ==> > ; (SB-PCL::LOAD-DEFMETHOD 'STANDARD-METHOD 'ELEPHANT:MAP-BTREE 'NIL > ; (LIST (FIND-CLASS 'T) (FIND-CLASS > 'ELEPHANT:BTREE)) > ; '(ELEPHANT::FN ELEPHANT:BTREE &REST > ELEPHANT::ARGS > ; &KEY ELEPHANT::START ELEPHANT::END > ; (ELEPHANT::VALUE NIL ELEPHANT::VALUE- > SET-P) > ; ELEPHANT::FROM-END ELEPHANT::COLLECT > ; &ALLOW-OTHER-KEYS) > ; (LIST* :FUNCTION > ; (LET* ((SB-PCL::FMF #) (SB- > PCL::MF #)) > ; (SB-MOP:SET-FUNCALLABLE- > INSTANCE-FUNCTION > ; SB-PCL::MF > ; (SB-PCL::METHOD-FUNCTION-FROM- > FAST-FUNCTION > ; SB-PCL::FMF '#)) > ; SB-PCL::MF) > ; '(SB-PCL::PLIST (:ARG-INFO (2 . > T)))) > ; (SB-C:SOURCE-LOCATION)) > ; > ; caught STYLE-WARNING: > ; The variable ARGS is defined but never used. > > ; (FUNCALL ELEPHANT::FN ELEPHANT::K ELEPHANT::V) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; (FUNCALL ELEPHANT::FN ELEPHANT::KEY ELEPHANT::VALUE) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; compiling (DEFGENERIC MAP-INDEX ...) > ; compiling (DEFMETHOD MAP-INDEX ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/collections.lisp > ; in: DEFMETHOD MAP-INDEX (T BTREE-INDEX) > ; (LET ((ELEPHANT::FN > ; (IF ELEPHANT::COLLECT #'ELEPHANT::COLLECTOR ELEPHANT::FN))) > ; (DECLARE (DYNAMIC-EXTENT #'ELEPHANT::COLLECTOR)) > ; (ELEPHANT:ENSURE-TRANSACTION > ; (:STORE-CONTROLLER ELEPHANT::SC :DEGREE-2 ELEPHANT::*MAP- > USING-DEGREE2*) > ; (ELEPHANT:WITH-BTREE-CURSOR (ELEPHANT::CUR ELEPHANT::INDEX) > ; (LABELS (# # # # #) > ; (DECLARE #) > ; (MULTIPLE-VALUE-BIND # # #))))) > ; > ; note: ignoring DYNAMIC-EXTENT declaration for free COLLECTOR > > ; (DEFMETHOD ELEPHANT:MAP-INDEX > ; (ELEPHANT::FN (ELEPHANT::INDEX ELEPHANT:BTREE-INDEX) > &REST > ; ELEPHANT::ARGS &KEY ELEPHANT::START ELEPHANT::END > ; (ELEPHANT::VALUE NIL ELEPHANT::VALUE-SET-P) > ELEPHANT::FROM-END > ; ELEPHANT::COLLECT &ALLOW-OTHER-KEYS) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::ARGS)) > ; (UNLESS > ; (OR (NULL ELEPHANT::START) (NULL ELEPHANT::END) > ; (ELEPHANT::LISP-COMPARE<= ELEPHANT::START > ELEPHANT::END)) > ; (ERROR > ; "map-index called with start = ~A and end = ~A. > Start must be less than or equal to end according to elephant::lisp- > compare<=." > ; ELEPHANT::START ELEPHANT::END)) > ; (LET ((ELEPHANT::SC (ELEPHANT::GET-CON > ELEPHANT::INDEX)) > ; (ELEPHANT::END (OR ELEPHANT::VALUE > ELEPHANT::END)) > ; (ELEPHANT::FROM-END (AND ELEPHANT::FROM-END #)) > ; (ELEPHANT::RESULTS NIL)) > ; (FLET ((ELEPHANT::COLLECTOR # > ; #)) > ; (LET (#) > ; (DECLARE #) > ; (ELEPHANT:ENSURE-TRANSACTION # #))))) > ; --> PROGN EVAL-WHEN SB-PCL::%DEFMETHOD-EXPANDER PROGN > ; ==> > ; (SB-PCL::LOAD-DEFMETHOD 'STANDARD-METHOD 'ELEPHANT:MAP-INDEX 'NIL > ; (LIST (FIND-CLASS 'T) > ; (FIND-CLASS 'ELEPHANT:BTREE-INDEX)) > ; '(ELEPHANT::FN ELEPHANT::INDEX &REST > ELEPHANT::ARGS > ; &KEY ELEPHANT::START ELEPHANT::END > ; (ELEPHANT::VALUE NIL ELEPHANT::VALUE- > SET-P) > ; ELEPHANT::FROM-END ELEPHANT::COLLECT > ; &ALLOW-OTHER-KEYS) > ; (LIST* :FUNCTION > ; (LET* ((SB-PCL::FMF #) (SB- > PCL::MF #)) > ; (SB-MOP:SET-FUNCALLABLE- > INSTANCE-FUNCTION > ; SB-PCL::MF > ; (SB-PCL::METHOD-FUNCTION-FROM- > FAST-FUNCTION > ; SB-PCL::FMF '#)) > ; SB-PCL::MF) > ; '(SB-PCL::PLIST (:ARG-INFO (2 . > T)))) > ; (SB-C:SOURCE-LOCATION)) > ; > ; caught STYLE-WARNING: > ; The variable ARGS is defined but never used. > > ; (FUNCALL ELEPHANT::FN ELEPHANT::SKEY ELEPHANT::VAL > ELEPHANT::PKEY) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; (FUNCALL ELEPHANT::FN ELEPHANT::K ELEPHANT::V ELEPHANT::PK) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; (DEFMETHOD ELEPHANT:MAP-INDEX > ; (ELEPHANT::FN (ELEPHANT::INDEX ELEPHANT:BTREE-INDEX) > &REST > ; ELEPHANT::ARGS &KEY ELEPHANT::START ELEPHANT::END > ; (ELEPHANT::VALUE NIL ELEPHANT::VALUE-SET-P) > ELEPHANT::FROM-END > ; ELEPHANT::COLLECT &ALLOW-OTHER-KEYS) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::ARGS)) > ; (UNLESS > ; (OR (NULL ELEPHANT::START) (NULL ELEPHANT::END) > ; (ELEPHANT::LISP-COMPARE<= ELEPHANT::START > ELEPHANT::END)) > ; (ERROR > ; "map-index called with start = ~A and end = ~A. > Start must be less than or equal to end according to elephant::lisp- > compare<=." > ; ELEPHANT::START ELEPHANT::END)) > ; (LET ((ELEPHANT::SC (ELEPHANT::GET-CON > ELEPHANT::INDEX)) > ; (ELEPHANT::END (OR ELEPHANT::VALUE > ELEPHANT::END)) > ; (ELEPHANT::FROM-END (AND ELEPHANT::FROM-END #)) > ; (ELEPHANT::RESULTS NIL)) > ; (FLET ((ELEPHANT::COLLECTOR # > ; #)) > ; (LET (#) > ; (DECLARE #) > ; (ELEPHANT:ENSURE-TRANSACTION # #))))) > ; --> PROGN EVAL-WHEN SB-PCL::%DEFMETHOD-EXPANDER PROGN > ; ==> > ; (SB-PCL::LOAD-DEFMETHOD 'STANDARD-METHOD 'ELEPHANT:MAP-INDEX 'NIL > ; (LIST (FIND-CLASS 'T) > ; (FIND-CLASS 'ELEPHANT:BTREE-INDEX)) > ; '(ELEPHANT::FN ELEPHANT::INDEX &REST > ELEPHANT::ARGS > ; &KEY ELEPHANT::START ELEPHANT::END > ; (ELEPHANT::VALUE NIL ELEPHANT::VALUE- > SET-P) > ; ELEPHANT::FROM-END ELEPHANT::COLLECT > ; &ALLOW-OTHER-KEYS) > ; (LIST* :FUNCTION > ; (LET* ((SB-PCL::FMF #) (SB- > PCL::MF #)) > ; (SB-MOP:SET-FUNCALLABLE- > INSTANCE-FUNCTION > ; SB-PCL::MF > ; (SB-PCL::METHOD-FUNCTION-FROM- > FAST-FUNCTION > ; SB-PCL::FMF '#)) > ; SB-PCL::MF) > ; '(SB-PCL::PLIST (:ARG-INFO (2 . > T)))) > ; (SB-C:SOURCE-LOCATION)) > ; > ; note: Return type not fixed values, so can't use known return > convention: > ; * > > ; compiling (DEFUN PSET-RANGE-FOR-DESCENDING ...) > ; compiling (DEFUN PPREV-DUP-HACK ...) > ; compiling (DEFMETHOD EMPTY-BTREE-P ...) > ; compiling (DEFUN PRINT-BTREE-ENTRY ...) > ; compiling (DEFUN DUMP-BTREE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/collections.lisp > ; in: DEFUN DUMP-BTREE > ; (FUNCALL ELEPHANT::PRINT-FN ELEPHANT::K ELEPHANT::V) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; (>= (INCF ELEPHANT::I) COUNT) > ; --> OR LET IF OR = IF > ; ==> > ; (= SB-C::X SB-C::Y) > ; > ; note: unable to open code because: The operands might not be the > same type. > > ; (INCF ELEPHANT::I) > ; --> LET* > ; ==> > ; (+ ELEPHANT::I #:G439) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; etc. > > ; (>= (INCF ELEPHANT::I) COUNT) > ; --> OR LET > IF > ; ==> > ; (> SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-> (cost 10) > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a (INTEGER 1), not a FIXNUM. > ; The second argument is a REAL, not a FIXNUM. > > ; compiling (DEFUN PRINT-BTREE-KEY-AND-TYPE ...) > ; compiling (DEFUN BTREE-KEYS ...) > ; compiling (DEFMETHOD BTREE-DIFFER-P ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/collections.lisp > ; in: DEFMETHOD BTREE-DIFFER-P (BTREE BTREE) > ; (1+ ELEPHANT::I) > ; ==> > ; (+ ELEPHANT::I 1) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; etc. > > ; /Users/jensteich/Lisp/elephant/src/elephant/collections.fasl written > ; compilation finished in 0:00:01 > WARNING: > COMPILE-FILE warned while performing # on > #. > STYLE-WARNING: Implicitly creating new generic function EMPTY-BTREE-P. > STYLE-WARNING: Implicitly creating new generic function BTREE-DIFFER- > P. > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > pset.lisp" (written 25 APR 2007 04:28:02 AM): > ; compiling (IN-PACKAGE :ELEPHANT) > ; compiling (DEFCLASS PSET ...) > ; compiling (DEFGENERIC INSERT-ITEM ...) > ; compiling (DEFGENERIC REMOVE-ITEM ...) > ; compiling (DEFGENERIC MAP-PSET ...) > ; compiling (DEFGENERIC FIND-ITEM ...) > ; compiling (DEFGENERIC PSET-LIST ...) > ; compiling (DEFGENERIC BUILD-PSET ...) > ; compiling (DEFGENERIC DROP-PSET ...) > ; compiling (DEFCLASS DEFAULT-PSET ...) > ; compiling (DEFMETHOD BUILD-PSET ...) > ; compiling (DEFUN MAKE-PSET ...) > ; compiling (DEFMETHOD INSERT-ITEM ...) > ; compiling (DEFMETHOD REMOVE-ITEM ...) > ; compiling (DEFMETHOD FIND-ITEM ...) > ; compiling (DEFMETHOD MAP-PSET ...) > ; compiling (DEFMETHOD PSET-LIST ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/pset.lisp > ; in: DEFMETHOD PSET-LIST (DEFAULT-PSET) > ; (FLET ((ELEPHANT::COLLECT (ELEPHANT::ITEM) > ; (PUSH ELEPHANT::ITEM LIST))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::COLLECT)) > ; (ELEPHANT:MAP-BTREE > ; (LAMBDA (ELEPHANT::ITEM ELEPHANT::DC) > ; (DECLARE (IGNORE ELEPHANT::DC)) > ; (PUSH ELEPHANT::ITEM LIST)) > ; (ELEPHANT::PSET-BTREE ELEPHANT:PSET))) > ; > ; note: deleting unused function > ; (FLET COLLECT) > > ; compiling (DEFMETHOD DROP-PSET ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/pset.fasl written > ; compilation finished in 0:00:00 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > classindex-utils.lisp" (written 12 APR 2007 04:47:32 AM): > ; compiling (IN-PACKAGE :ELEPHANT) > ; compiling (DECLAIM (OPTIMIZE # ...)) > ; compiling (DEFPARAMETER *DEFAULT-INDEXED-CLASS-SYNCH-POLICY* ...) > ; compiling (DEFMETHOD CLASS-INDEX-CACHED? ...) > ; compiling (DEFMETHOD DETERMINE-SYNCH-METHOD ...) > ; compiling (DEFMETHOD SET-DB-SYNCH ...) > ; compiling (DEFPARAMETER *DERIVED-INDEX-MARKER* ...) > ; compiling (DEFUN MAKE-DERIVED-NAME ...) > ; compiling (DEFUN DERIVED-NAME? ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex- > utils.lisp > ; in: DEFUN DERIVED-NAME? > ; (LENGTH ELEPHANT::NAME) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (LENGTH ELEPHANT::*DERIVED-INDEX-MARKER*) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (SUBSEQ ELEPHANT::NAME 0 > ; (MIN (LENGTH ELEPHANT::NAME) > ; (LENGTH ELEPHANT::*DERIVED-INDEX-MARKER*))) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a LIST. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a SB-KERNEL:EXTENDED- > SEQUENCE. > > ; (STRING= > ; (SUBSEQ ELEPHANT::NAME 0 > ; (MIN (LENGTH ELEPHANT::NAME) > ; (LENGTH ELEPHANT::*DERIVED-INDEX-MARKER*))) > ; ELEPHANT::*DERIVED-INDEX-MARKER*) > ; ==> > ; (SB-KERNEL:STRING=* SB-C::STRING1 SB-C::STRING2 SB-C::START1 SB- > C::END1 > ; SB-C::START2 SB-C::END2) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (SIMPLE-ARRAY CHARACTER (*)) > ; (SIMPLE-ARRAY NIL (*)) SIMPLE-BASE- > STRING > ; NULL), not a SIMPLE-BASE-STRING. > ; The second argument is a (OR (VECTOR CHARACTER) (VECTOR NIL) > BASE-STRING > ; SYMBOL CHARACTER), not a SIMPLE- > BASE-STRING. > > ; compiling (DEFUN GET-DERIVED-NAME-ROOT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex- > utils.lisp > ; in: DEFUN GET-DERIVED-NAME-ROOT > ; (LENGTH ELEPHANT::*DERIVED-INDEX-MARKER*) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (SUBSEQ ELEPHANT::DNAME (LENGTH ELEPHANT::*DERIVED-INDEX- > MARKER*)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a LIST. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a SB-KERNEL:EXTENDED- > SEQUENCE. > > ; compiling (DEFUN MAKE-SLOT-KEY-FORM ...) > ; compiling (DEFUN READ-SLOT-FOR-INDEX ...) > ; compiling (DEFUN FIND-EFFECTIVE-SLOT-DEF ...) > ; compiling (DEFUN MAKE-DERIVED-KEY-FORM ...) > ; compiling (DEFUN COMPUTE-DERIVED-KEY-RESULT ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex- > utils.lisp > ; in: DEFUN COMPUTE-DERIVED-KEY-RESULT > ; (FUNCALL ELEPHANT::FN ELEPHANT::INSTANCE) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; compiling (DEFCLASS SYNCH-RULE ...) > ; compiling (DEFUN MAKE-SYNCH-RULE ...); in: LAMBDA NIL > ; (POSITION 'ELEPHANT::=> ELEPHANT::RULE-SPEC) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The second argument is a SEQUENCE, not a (OR CONS NULL VECTOR). > > ; (SUBSEQ ELEPHANT::RULE-SPEC 0 (POSITION 'ELEPHANT::=> > ELEPHANT::RULE-SPEC)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a LIST. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a SB-KERNEL:EXTENDED- > SEQUENCE. > > ; (POSITION 'ELEPHANT::=> ELEPHANT::RULE-SPEC) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The second argument is a SEQUENCE, not a (OR CONS NULL VECTOR). > > ; (SUBSEQ ELEPHANT::RULE-SPEC (1+ (POSITION 'ELEPHANT::=> > ELEPHANT::RULE-SPEC))) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a LIST. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a SB-KERNEL:EXTENDED- > SEQUENCE. > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex- > utils.lisp > ; in: DEFUN MAKE-SYNCH-RULE > ; (POSITION 'ELEPHANT::=> ELEPHANT::RULE-SPEC) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The second argument is a SEQUENCE, not a (OR CONS NULL VECTOR). > > ; (SUBSEQ ELEPHANT::RULE-SPEC 0 (POSITION 'ELEPHANT::=> > ELEPHANT::RULE-SPEC)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a LIST. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a SB-KERNEL:EXTENDED- > SEQUENCE. > > ; (POSITION 'ELEPHANT::=> ELEPHANT::RULE-SPEC) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The second argument is a SEQUENCE, not a (OR CONS NULL VECTOR). > > ; (SUBSEQ ELEPHANT::RULE-SPEC (1+ (POSITION 'ELEPHANT::=> > ELEPHANT::RULE-SPEC))) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a LIST. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a SB-KERNEL:EXTENDED- > SEQUENCE. > > ; compiling (DEFPARAMETER *SYNCHRONIZE-RULES* ...) > ; compiling (DEFUN SYNCHRONIZE-CLASS-TO-STORE ...) > ; compiling (DEFUN SYNCH-RULE-APPLICABLE? ...) > ; compiling (DEFUN SIMPLE-MATCH-SET ...) > ; compiling (DEFPARAMETER *PRINT-SYNCH-MESSAGES* ...) > ; compiling (DEFUN APPLY-SYNCH-RULE ...) > ; compiling (DEFUN APPLY-SYNCH-RULES ...) > ; compiling (DEFUN COMPUTE-CLASS-AND-ELE-STATUS ...) > ; compiling (DEFUN DESCRIBE-DB-CLASS-INDEX ...) > ; compiling (DEFUN WIPE-INDEXED-CLASS ...) > ; compiling (DEFUN DUMP-CLASS-INDEX ...) > ; compiling (DEFUN REPORT-INDEXED-CLASSES ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/classindex-utils.fasl > written > ; compilation finished in 0:00:01 > STYLE-WARNING: > Implicitly creating new generic function CLASS-INDEX-CACHED?. > STYLE-WARNING: > Implicitly creating new generic function DETERMINE-SYNCH-METHOD. > STYLE-WARNING: Implicitly creating new generic function SET-DB-SYNCH. > STYLE-WARNING: redefining MAKE-SYNCH-RULE in DEFUN > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > classindex.lisp" (written 06 MAY 2007 12:10:40 AM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DECLAIM (OPTIMIZE # ...)) > ; compiling (DEFGENERIC FIND-CLASS-INDEX ...) > ; compiling (DEFGENERIC FIND-INVERTED-INDEX ...) > ; compiling (DEFGENERIC ENABLE-CLASS-INDEXING ...) > ; compiling (DEFGENERIC DISABLE-CLASS-INDEXING ...) > ; compiling (DEFGENERIC ADD-CLASS-SLOT-INDEX ...) > ; compiling (DEFGENERIC REMOVE-CLASS-SLOT-INDEX ...) > ; compiling (DEFGENERIC ADD-CLASS-DERIVED-INDEX ...) > ; compiling (DEFGENERIC REMOVE-CLASS-DERIVED-INDEX ...) > ; compiling (DEFMETHOD FIND-CLASS-INDEX ...) > ; compiling (DEFMETHOD CLASS-INDEXEDP-BY-NAME ...) > ; compiling (DEFINE-CONDITION PERSISTENT-CLASS-NOT-INDEXED ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex.lisp > ; in: DEFINE-CONDITION PERSISTENT-CLASS-NOT-INDEXED > ; (FORMAT STREAM "Class ~A is not enabled for indexing" > ; (CLASS-NAME (ELEPHANT::UNINDEXED-CLASS-OBJ CONDITION))) > ; ==> > ; (FORMAT SB-C::DEST (FORMATTER "Class ~A is not enabled for > indexing") #:G54) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MEMBER NIL T) STREAM (VECTOR > CHARACTER) > ; (VECTOR NIL) BASE-STRING), not a > STREAM. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MEMBER NIL T) STREAM (VECTOR > CHARACTER) > ; (VECTOR NIL) BASE-STRING), not a > (MEMBER T). > > ; compiling (DEFUN SIGNAL-CLASS-NOT-INDEXED ...) > ; compiling (DEFMETHOD FIND-CLASS-INDEX ...) > ; compiling (DEFUN ENSURE-FINALIZED ...) > ; compiling (DEFUN CACHE-EXISTING-CLASS-INDEX ...) > ; compiling (DEFUN CACHE-NEW-CLASS-INDEX ...) > ; compiling (DEFMETHOD FIND-INVERTED-INDEX ...) > ; compiling (DEFMETHOD FIND-INVERTED-INDEX ...) > ; compiling (DEFMETHOD FIND-INVERTED-INDEX-NAMES ...) > ; compiling (DEFMETHOD CLOSE-CONTROLLER ...) > ; compiling (DEFMETHOD INDEXED-SLOT-WRITER ...) > ; compiling (DEFMETHOD INDEXED-SLOT-MAKUNBOUND ...) > ; compiling (DEFUN NO-INDEXING-NEEDED? ...) > ; compiling (DEFMETHOD ENABLE-CLASS-INDEXING ...) > ; compiling (DEFMETHOD DISABLE-CLASS-INDEXING ...) > ; compiling (DEFMETHOD DISABLE-CLASS-INDEXING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex.lisp > ; in: DEFMETHOD DISABLE-CLASS-INDEXING (PERSISTENT-METACLASS) > ; (RETURN-FROM ELEPHANT:DISABLE-CLASS-INDEXING NIL) > ; > ; note: deleting unreachable code > > ; compiling (DEFMETHOD WIPE-CLASS-INDEXING ...) > ; compiling (DEFMETHOD WIPE-CLASS-INDEXING ...) > ; compiling (DEFMETHOD ADD-CLASS-SLOT-INDEX ...) > ; compiling (DEFMETHOD ADD-CLASS-SLOT-INDEX ...) > ; compiling (DEFMETHOD REMOVE-CLASS-SLOT-INDEX ...) > ; compiling (DEFMETHOD REMOVE-CLASS-SLOT-INDEX ...) > ; compiling (DEFMETHOD ADD-CLASS-DERIVED-INDEX ...) > ; compiling (DEFMETHOD ADD-CLASS-DERIVED-INDEX ...) > ; compiling (DEFMETHOD REMOVE-CLASS-DERIVED-INDEX ...) > ; compiling (DEFMETHOD REMOVE-CLASS-DERIVED-INDEX ...) > ; compiling (DEFGENERIC MAKE-INVERTED-CURSOR ...) > ; compiling (DEFGENERIC MAKE-CLASS-CURSOR ...) > ; compiling (DEFMETHOD MAKE-INVERTED-CURSOR ...) > ; compiling (DEFMETHOD MAKE-INVERTED-CURSOR ...) > ; compiling (DEFMACRO WITH-INVERTED-CURSOR ...) > ; compiling (DEFMETHOD MAKE-CLASS-CURSOR ...) > ; compiling (DEFMETHOD MAKE-CLASS-CURSOR ...) > ; compiling (DEFMACRO WITH-CLASS-CURSOR ...) > ; compiling (DEFUN MAP-CLASS ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex.lisp > ; in: DEFUN MAP-CLASS > ; (FLET ((ELEPHANT::MAP-FN (ELEPHANT::K ELEPHANT::V) > ; (DECLARE (IGNORE ELEPHANT::K)) > ; (FUNCALL ELEPHANT::FN ELEPHANT::V))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::MAP-FN)) > ; (ELEPHANT:MAP-BTREE #'ELEPHANT::MAP-FN ELEPHANT::CLASS- > IDX :COLLECT > ; ELEPHANT::COLLECT)) > ; > ; note: ignoring DYNAMIC-EXTENT declaration for free MAP-FN > > ; (FUNCALL ELEPHANT::FN ELEPHANT::V) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; compiling (DEFUN MAP-INVERTED-INDEX ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex.lisp > ; in: DEFUN MAP-INVERTED-INDEX > ; (FLET ((ELEPHANT::WRAPPER (ELEPHANT::KEY ELEPHANT::VALUE > ELEPHANT::PKEY) > ; (DECLARE (IGNORE ELEPHANT::PKEY)) > ; (FUNCALL ELEPHANT::FN ELEPHANT::KEY ELEPHANT::VALUE))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::WRAPPER)) > ; (IF ELEPHANT::VALUE-P > ; (ELEPHANT:MAP-INDEX #'ELEPHANT::WRAPPER > ELEPHANT::INDEX :VALUE > ; ELEPHANT::VALUE :COLLECT > ELEPHANT::COLLECT) > ; (ELEPHANT:MAP-INDEX #'ELEPHANT::WRAPPER > ELEPHANT::INDEX :START > ; ELEPHANT::START :END > ELEPHANT::END :FROM-END > ; ELEPHANT::FROM-END :COLLECT > ELEPHANT::COLLECT))) > ; > ; note: ignoring DYNAMIC-EXTENT declaration for free WRAPPER > > ; (FUNCALL ELEPHANT::FN ELEPHANT::KEY ELEPHANT::VALUE) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; compiling (DEFGENERIC GET-INSTANCES-BY-CLASS ...) > ; compiling (DEFGENERIC GET-INSTANCE-BY-VALUE ...) > ; compiling (DEFGENERIC GET-INSTANCES-BY-VALUE ...) > ; compiling (DEFGENERIC GET-INSTANCES-BY-RANGE ...) > ; compiling (DEFUN IDENTITY2 ...) > ; compiling (DEFUN IDENTITY3 ...) > ; compiling (DEFMETHOD GET-INSTANCES-BY-CLASS ...) > ; compiling (DEFMETHOD GET-INSTANCES-BY-CLASS ...) > ; compiling (DEFMETHOD GET-INSTANCES-BY-VALUE ...) > ; compiling (DEFMETHOD GET-INSTANCES-BY-VALUE ...) > ; compiling (DEFMETHOD GET-INSTANCE-BY-VALUE ...) > ; compiling (DEFMETHOD GET-INSTANCE-BY-VALUE ...) > ; compiling (DEFMETHOD GET-INSTANCES-BY-RANGE ...) > ; compiling (DEFMETHOD GET-INSTANCES-BY-RANGE ...) > ; compiling (DEFUN DROP-INSTANCES ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/classindex.fasl written > ; compilation finished in 0:00:01 > STYLE-WARNING: > Implicitly creating new generic function CLASS-INDEXEDP-BY-NAME. > STYLE-WARNING: > Implicitly creating new generic function FIND-INVERTED-INDEX-NAMES. > STYLE-WARNING: > Implicitly creating new generic function INDEXED-SLOT-WRITER. > STYLE-WARNING: > Implicitly creating new generic function INDEXED-SLOT-MAKUNBOUND. > STYLE-WARNING: > Implicitly creating new generic function WIPE-CLASS-INDEXING. > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > serializer1.lisp" (written 12 APR 2007 04:47:33 AM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DEFPACKAGE :ELEPHANT-SERIALIZER1 ...) > ; compiling (IN-PACKAGE :ELEPHANT-SERIALIZER1) > ; compiling (DECLAIM (INLINE INT-BYTE-SPEC ...)) > ; compiling (UFFI:DEF-TYPE FOREIGN-CHAR ...) > ; compiling (DEFCONSTANT +FIXNUM+ ...) > ; compiling (DEFCONSTANT +CHAR+ ...) > ; compiling (DEFCONSTANT +SINGLE-FLOAT+ ...) > ; compiling (DEFCONSTANT +DOUBLE-FLOAT+ ...) > ; compiling (DEFCONSTANT +NEGATIVE-BIGNUM+ ...) > ; compiling (DEFCONSTANT +POSITIVE-BIGNUM+ ...) > ; compiling (DEFCONSTANT +RATIONAL+ ...) > ; compiling (DEFCONSTANT +NIL+ ...) > ; compiling (DEFCONSTANT +UCS1-SYMBOL+ ...) > ; compiling (DEFCONSTANT +UCS1-STRING+ ...) > ; compiling (DEFCONSTANT +UCS1-PATHNAME+ ...) > ; compiling (DEFCONSTANT +UCS2-SYMBOL+ ...) > ; compiling (DEFCONSTANT +UCS2-STRING+ ...) > ; compiling (DEFCONSTANT +UCS2-PATHNAME+ ...) > ; compiling (DEFCONSTANT +UCS4-SYMBOL+ ...) > ; compiling (DEFCONSTANT +UCS4-STRING+ ...) > ; compiling (DEFCONSTANT +UCS4-PATHNAME+ ...) > ; compiling (DEFCONSTANT +PERSISTENT+ ...) > ; compiling (DEFCONSTANT +CONS+ ...) > ; compiling (DEFCONSTANT +HASH-TABLE+ ...) > ; compiling (DEFCONSTANT +OBJECT+ ...) > ; compiling (DEFCONSTANT +ARRAY+ ...) > ; compiling (DEFCONSTANT +RESERVED-DBINFO+ ...) > ; compiling (DEFCONSTANT +FILL-POINTER-P+ ...) > ; compiling (DEFCONSTANT +ADJUSTABLE-P+ ...) > ; compiling (DEFVAR *LISP-OBJ-ID* ...) > ; compiling (DEFVAR *CIRCULARITY-HASH* ...) > ; compiling (DEFUN CLEAR-CIRCULARITY-HASH ...) > ; compiling (DEFUN SERIALIZE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer1.lisp > ; in: DEFUN SERIALIZE > ; (CEILING (/ (INTEGER-LENGTH ELEPHANT-SERIALIZER1::NUM) 32)) > ; ==> > ; (CEILING (/ (INTEGER-LENGTH ELEPHANT-SERIALIZER1::NUM) 32) 1) > ; > ; note: unable to > ; convert division by 2^k to shift > ; due to type uncertainty: > ; The first argument is a (RATIONAL 0 536870911/32), not a INTEGER. > > ; (INCF ELEPHANT-SERIALIZER1::*LISP-OBJ-ID*) > ; --> LET* > ; ==> > ; (+ ELEPHANT-SERIALIZER1::*LISP-OBJ-ID* #:G42) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > > ; ==> > ; (+ ELEPHANT-SERIALIZER1::*LISP-OBJ-ID* #:G48) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > > ; ==> > ; (+ ELEPHANT-SERIALIZER1::*LISP-OBJ-ID* #:G75) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > > ; ==> > ; (+ ELEPHANT-SERIALIZER1::*LISP-OBJ-ID* #:G82) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > > ; (ARRAY-RANK ELEPHANT-SERIALIZER1::FROB) > ; > ; note: unable to optimize because: The array rank is not known at > compile time: * > > ; (ARRAY-TOTAL-SIZE ELEPHANT-SERIALIZER1::FROB) > ; > ; note: unable to optimize because: can't tell the rank at compile > time > > ; (ROW-MAJOR-AREF ELEPHANT-SERIALIZER1::FROB ELEPHANT- > SERIALIZER1::I) > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-REF ARRAY > ; (SB-KERNEL:%CHECK-BOUND ARRAY > ; (ARRAY- > TOTAL-SIZE > ; ARRAY) > ; SB- > INT:INDEX)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a ARRAY, not a SIMPLE-STRING. > ; > ; note: unable to > ; avoid runtime dispatch on array element type > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (ARRAY-DIMENSION ELEPHANT-SERIALIZER1::FROB ELEPHANT- > SERIALIZER1::I) > ; > ; note: unable to optimize because: The axis is not constant. > > ; (LABELS ((ELEPHANT-SERIALIZER1::%SERIALIZE (ELEPHANT- > SERIALIZER1::FROB) > ; (DECLARE (OPTIMIZE # #)) > ; (TYPECASE ELEPHANT-SERIALIZER1::FROB > ; (FIXNUM # #) > ; (NULL #) > ; (SYMBOL #) > ; (STRING #) > ; (ELEPHANT:PERSISTENT # # #) > ; (SINGLE-FLOAT # #) > ; (DOUBLE-FLOAT # #) > ; (CHARACTER # #) > ; (PATHNAME #) > ; (INTEGER #) > ; ...))) > ; (ELEPHANT-SERIALIZER1::%SERIALIZE ELEPHANT-SERIALIZER1::FROB) > ; ELEPHANT-SERIALIZER1::BS) > ; > ; note: Return type not fixed values, so can't use known return > convention: > ; * > > ; (ABS ELEPHANT-SERIALIZER1::FROB) > ; --> IF > ; ==> > ; (< SB-C::X 0) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline fixnum comparison (cost 3) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; etc. > > ; --> IF - > ; ==> > ; (SB-KERNEL:%NEGATE SB-C::X) > ; > ; note: forced to do GENERIC-NEGATE (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a (INTEGER * -1), not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; unable to do inline (signed-byte 32) arithmetic (cost 2) > because: > ; The first argument is a (INTEGER * -1), not a (SIGNED-BYTE > 32). > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a (VALUES > ; > (SIGNED-BYTE 32) > ; &REST > T). > > ; (INTEGER-LENGTH ELEPHANT-SERIALIZER1::NUM) > ; > ; note: forced to do full call > ; unable to do inline (unsigned-byte 32) integer-length (cost > 26) because: > ; The first argument is a UNSIGNED-BYTE, not a (UNSIGNED-BYTE > 32). > > ; (< ELEPHANT-SERIALIZER1::FROB 0) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline fixnum comparison (cost 3) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; etc. > > ; (INCF ELEPHANT-SERIALIZER1::*LISP-OBJ-ID*) > ; --> LET* > ; ==> > ; (+ ELEPHANT-SERIALIZER1::*LISP-OBJ-ID* #:G48) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; etc. > > ; ==> > ; (+ ELEPHANT-SERIALIZER1::*LISP-OBJ-ID* #:G75) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; etc. > > ; ==> > ; (+ ELEPHANT-SERIALIZER1::*LISP-OBJ-ID* #:G82) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; etc. > > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER1::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER1::FROB) > ; ELEPHANT-SERIALIZER1::+FILL-POINTER-P+ 0) > ; (IF (ADJUSTABLE-ARRAY-P ELEPHANT-SERIALIZER1::FROB) > ; ELEPHANT-SERIALIZER1::+ADJUSTABLE-P+ 0)) > ; --> LOGIOR > ; ==> > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER1::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER1::FROB) > ; ELEPHANT-SERIALIZER1::+FILL-POINTER-P+ 0)) > ; > ; note: forced to do static-fun Two-arg-ior (cost 53) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 3) > because: > ; The first argument is a INTEGER, not a (SIGNED-BYTE 32). > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > ; (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; ==> > ; (LOGIOR > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER1::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER1::FROB) > ; ELEPHANT-SERIALIZER1::+FILL-POINTER-P+ 0)) > ; (IF (ADJUSTABLE-ARRAY-P ELEPHANT-SERIALIZER1::FROB) > ; ELEPHANT-SERIALIZER1::+ADJUSTABLE-P+ 0)) > ; > ; note: forced to do static-fun Two-arg-ior (cost 53) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; unable to do inline (signed-byte 32) arithmetic (cost 3) > because: > ; The first argument is a INTEGER, not a (SIGNED-BYTE 32). > ; etc. > > ; (INCF ELEPHANT-SERIALIZER1::*LISP-OBJ-ID*) > ; --> LET* > ; ==> > ; (+ ELEPHANT-SERIALIZER1::*LISP-OBJ-ID* #:G42) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; etc. > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+FIXNUM+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT-SERIALIZER1::+NIL+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BYTE-LENGTH ELEPHANT-SERIALIZER1::S) > ; --> ETYPECASE LET COND IF COND IF PROGN * LOGAND > ; ==> > ; SB-C::X > ; > ; note: doing unsigned word to integer coercion (cost 20) > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE > ; (ETYPECASE ELEPHANT-SERIALIZER1::S > ; (BASE-STRING ELEPHANT-SERIALIZER1::+UCS1-SYMBOL+) > ; (STRING ELEPHANT-SERIALIZER1::+UCS4-SYMBOL+)) > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-FLOAT ELEPHANT-SERIALIZER1::FROB > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+SINGLE-FLOAT+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-DOUBLE ELEPHANT-SERIALIZER1::FROB > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+DOUBLE-FLOAT+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT-SERIALIZER1::+CHAR+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+RATIONAL+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT-SERIALIZER1::+CONS+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER1::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER1::FROB) > ; ELEPHANT-SERIALIZER1::+FILL-POINTER-P+ 0) > ; (IF (ADJUSTABLE-ARRAY-P ELEPHANT-SERIALIZER1::FROB) > ; ELEPHANT-SERIALIZER1::+ADJUSTABLE-P+ 0)) > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+ARRAY+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+OBJECT+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-SERIALIZER1::%SERIALIZE > ; (HASH-TABLE-REHASH-THRESHOLD ELEPHANT-SERIALIZER1::FROB)) > ; > ; note: doing float to pointer coercion (cost 13) to FROB > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+HASH-TABLE+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-UINT > ; (SB-BIGNUM:%BIGNUM-REF ELEPHANT-SERIALIZER1::NUM ELEPHANT- > SERIALIZER1::I) > ; ELEPHANT-SERIALIZER1::BS) > ; > ; note: doing unsigned word to integer coercion (cost 20) > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+NEGATIVE-BIGNUM+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+POSITIVE-BIGNUM+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BYTE-LENGTH ELEPHANT-SERIALIZER1::S) > ; --> ETYPECASE LET COND IF COND IF PROGN * LOGAND > ; ==> > ; SB-C::X > ; > ; note: doing unsigned word to integer coercion (cost 20) > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE > ; (ETYPECASE ELEPHANT-SERIALIZER1::S > ; (BASE-STRING ELEPHANT-SERIALIZER1::+UCS1-PATHNAME+) > ; (STRING ELEPHANT-SERIALIZER1::+UCS4-PATHNAME+)) > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER1::+PERSISTENT+ > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BYTE-LENGTH ELEPHANT-SERIALIZER1::FROB) > ; --> ETYPECASE LET COND IF COND IF PROGN * LOGAND > ; ==> > ; SB-C::X > ; > ; note: doing unsigned word to integer coercion (cost 20) > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE > ; (ETYPECASE ELEPHANT-SERIALIZER1::FROB > ; (BASE-STRING ELEPHANT-SERIALIZER1::+UCS1-STRING+) > ; (STRING ELEPHANT-SERIALIZER1::+UCS4-STRING+)) > ; ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; compiling (DEFPARAMETER *TRACE-SERIALIZER* ...) > ; compiling (DEFPARAMETER *TAG-TABLE* ...) > ; compiling (DEFUN ENABLE-SERIALIZER-TRACING ...) > ; compiling (DEFUN DISABLE-SERIALIZER-TRACING ...) > ; compiling (DEFUN PRINT-PRE-DESERIALIZE-TAG ...) > ; compiling (DEFUN PRINT-POST-DESERIALIZE-TAG ...) > ; compiling (DEFUN DESERIALIZE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer1.lisp > ; in: DEFUN DESERIALIZE > ; (OR > ; (ELEPHANT-MEMUTIL:BUFFER-READ-UCS1-STRING ELEPHANT- > SERIALIZER1::BS > ; (ELEPHANT- > MEMUTIL:BUFFER-READ-FIXNUM > ; ELEPHANT- > SERIALIZER1::BS)) > ; "") > ; --> LET IF OR > ; ==> > ; "" > ; > ; note: deleting unreachable code > > ; (OR > ; (ELEPHANT-MEMUTIL:BUFFER-READ-UCS4-STRING ELEPHANT- > SERIALIZER1::BS > ; (ELEPHANT- > MEMUTIL:BUFFER-READ-FIXNUM > ; ELEPHANT- > SERIALIZER1::BS)) > ; "") > ; --> LET IF OR > ; ==> > ; "" > ; > ; note: deleting unreachable code > > ; (SETF (SLOT-VALUE ELEPHANT-SERIALIZER1::O > ; (ELEPHANT-SERIALIZER1::%DESERIALIZE > ; ELEPHANT-SERIALIZER1::BS)) > ; (ELEPHANT-SERIALIZER1::%DESERIALIZE ELEPHANT- > SERIALIZER1::BS)) > ; ==> > ; (SB-PCL::SET-SLOT-VALUE ELEPHANT-SERIALIZER1::O > ; (ELEPHANT-SERIALIZER1::%DESERIALIZE > ; ELEPHANT-SERIALIZER1::BS) > ; (ELEPHANT-SERIALIZER1::%DESERIALIZE > ; ELEPHANT-SERIALIZER1::BS)) > ; > ; note: unable to optimize because: Slot name is not constant. > > ; (MAKE-ARRAY > ; (LOOP ELEPHANT-SERIALIZER1::FOR ELEPHANT-SERIALIZER1::I FIXNUM > ; ELEPHANT-SERIALIZER1::FROM 0 ELEPHANT-SERIALIZER1::BELOW > ; (ELEPHANT-MEMUTIL:BUFFER-READ-INT ELEPHANT-SERIALIZER1::BS) > ; ELEPHANT-SERIALIZER1::COLLECT > ; (ELEPHANT-MEMUTIL:BUFFER-READ-INT ELEPHANT- > SERIALIZER1::BS)) > ; :ELEMENT-TYPE > ; (ELEPHANT::ARRAY-TYPE-FROM-BYTE (LOGAND 63 ELEPHANT- > SERIALIZER1::FLAGS)) > ; :FILL-POINTER > ; (/= 0 > ; (LOGAND ELEPHANT-SERIALIZER1::+FILL-POINTER-P+ > ; ELEPHANT-SERIALIZER1::FLAGS)) > ; :ADJUSTABLE > ; (/= 0 > ; (LOGAND ELEPHANT-SERIALIZER1::+ADJUSTABLE-P+ > ; ELEPHANT-SERIALIZER1::FLAGS))) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MOD 536870911) CONS NULL), not a > INTEGER. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MOD 536870911) CONS NULL), not a > LIST. > > ; (ARRAY-TOTAL-SIZE ELEPHANT-SERIALIZER1::A) > ; > ; note: unable to optimize because: can't tell the rank at compile > time > > ; (SETF (ROW-MAJOR-AREF ELEPHANT-SERIALIZER1::A ELEPHANT- > SERIALIZER1::I) > ; (ELEPHANT-SERIALIZER1::%DESERIALIZE ELEPHANT- > SERIALIZER1::BS)) > ; --> SB-KERNEL:%SET-ROW-MAJOR-AREF > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-SET ARRAY > ; (SB-KERNEL:%CHECK-BOUND ARRAY > ; (ARRAY- > TOTAL-SIZE > ; ARRAY) > ; SB- > INT:INDEX) > ; SB-C::NEW-VALUE) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a ARRAY, not a SIMPLE-STRING. > ; > ; note: unable to > ; avoid runtime dispatch on array element type > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-FLOAT ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK LET ELEPHANT-MEMUTIL::READ-FLOAT BLOCK THE DEREF > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; ==> > ; ALIEN > ; > ; note: doing float to pointer coercion (cost 13) > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-DOUBLE ELEPHANT-SERIALIZER1::BS) > ; --> BLOCK LET ELEPHANT-MEMUTIL::READ-DOUBLE BLOCK THE DEREF > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; ==> > ; ALIEN > ; > ; note: doing float to pointer coercion (cost 13) > > ; compiling (DEFUN DESERIALIZE-BIGNUM ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer1.lisp > ; in: DEFUN DESERIALIZE-BIGNUM > ; (DPB (ELEPHANT-MEMUTIL:BUFFER-READ-UINT ELEPHANT- > SERIALIZER1::BS) > ; ELEPHANT-SERIALIZER1::BYTE-SPEC ELEPHANT-SERIALIZER1::NUM) > ; --> LET > ; ==> > ; (SB-KERNEL:%DPB (ELEPHANT-MEMUTIL:BUFFER-READ-UINT ELEPHANT- > SERIALIZER1::BS) > ; (BYTE-SIZE #:G641) (BYTE-POSITION #:G641) > ; ELEPHANT-SERIALIZER1::NUM) > ; > ; note: unable to > ; convert to inline logical operations > ; due to type uncertainty: > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > (UNSIGNED-BYTE 32) > ; &REST > T). > ; > ; note: unable to > ; convert to inline logical operations > ; due to type uncertainty: > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > (SIGNED-BYTE 32) > ; &REST > T). > > ; (LOOP ELEPHANT-SERIALIZER1::FOR ELEPHANT-SERIALIZER1::I > ; ELEPHANT-SERIALIZER1::FROM 0 ELEPHANT-SERIALIZER1::BELOW > (/ LENGTH 4) > ; ELEPHANT-SERIALIZER1::FOR ELEPHANT-SERIALIZER1::BYTE-SPEC = > ; (ELEPHANT::INT-BYTE-SPEC ELEPHANT-SERIALIZER1::I) > ; ELEPHANT-SERIALIZER1::WITH ...) > ; --> BLOCK LET LET LET SB-LOOP::LOOP-BODY TAGBODY WHEN IF >= IF > ; ==> > ; (< SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline fixnum comparison (cost 4) because: > ; The second argument is a (RATIONAL -134217728 536870911/4), > not a FIXNUM. > > ; (- ELEPHANT-SERIALIZER1::NUM) > ; ==> > ; (SB-KERNEL:%NEGATE ELEPHANT-SERIALIZER1::NUM) > ; > ; note: forced to do GENERIC-NEGATE (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 2) > because: > ; The first argument is a INTEGER, not a (SIGNED-BYTE 32). > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > ; (SIGNED- > BYTE 32) &REST > ; T). > > ; /Users/jensteich/Lisp/elephant/src/elephant/serializer1.fasl written > ; compilation finished in 0:00:01 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > serializer2.lisp" (written 23 APR 2007 04:41:11 AM): > ; compiling (IN-PACKAGE :ELEPHANT) > ; compiling (DEFPACKAGE :ELEPHANT-SERIALIZER2 ...) > ; compiling (IN-PACKAGE :ELEPHANT-SERIALIZER2) > ; compiling (UFFI:DEF-TYPE FOREIGN-CHAR ...) > ; compiling (DEFCONSTANT +FIXNUM32+ ...) > ; compiling (DEFCONSTANT +FIXNUM64+ ...) > ; compiling (DEFCONSTANT +CHAR+ ...) > ; compiling (DEFCONSTANT +SINGLE-FLOAT+ ...) > ; compiling (DEFCONSTANT +DOUBLE-FLOAT+ ...) > ; compiling (DEFCONSTANT +NEGATIVE-BIGNUM+ ...) > ; compiling (DEFCONSTANT +POSITIVE-BIGNUM+ ...) > ; compiling (DEFCONSTANT +RATIONAL+ ...) > ; compiling (DEFCONSTANT +UTF8-STRING+ ...) > ; compiling (DEFCONSTANT +UTF16-STRING+ ...) > ; compiling (DEFCONSTANT +UTF32-STRING+ ...) > ; compiling (DEFCONSTANT +PATHNAME+ ...) > ; compiling (DEFCONSTANT +SYMBOL+ ...) > ; compiling (DEFCONSTANT +PERSISTENT+ ...) > ; compiling (DEFCONSTANT +CONS+ ...) > ; compiling (DEFCONSTANT +HASH-TABLE+ ...) > ; compiling (DEFCONSTANT +OBJECT+ ...) > ; compiling (DEFCONSTANT +ARRAY+ ...) > ; compiling (DEFCONSTANT +STRUCT+ ...) > ; compiling (DEFCONSTANT +CLASS+ ...) > ; compiling (DEFCONSTANT +COMPLEX+ ...) > ; compiling (DEFCONSTANT +SHORT-FLOAT+ ...) > ; compiling (DEFCONSTANT +NIL+ ...) > ; compiling (DEFCONSTANT +RESERVED-DBINFO+ ...) > ; compiling (DEFCONSTANT +FILL-POINTER-P+ ...) > ; compiling (DEFCONSTANT +ADJUSTABLE-P+ ...) > ; compiling (DEFPARAMETER *CIRCULARITY-HASH-QUEUE* ...) > ; compiling (DEFVAR *SERIALIZER-FAST-LOCK* ...) > ; compiling (DEFUN GET-CIRCULARITY-HASH ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer2.lisp > ; in: DEFUN GET-CIRCULARITY-HASH > ; (LENGTH ELEPHANT-SERIALIZER2::*CIRCULARITY-HASH-QUEUE*) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; compiling (DEFUN RELEASE-CIRCULARITY-HASH ...) > ; compiling (DEFPARAMETER *CIRCULARITY-VECTOR-QUEUE* ...) > ; compiling (DEFUN GET-CIRCULARITY-VECTOR ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer2.lisp > ; in: DEFUN GET-CIRCULARITY-VECTOR > ; (LENGTH ELEPHANT-SERIALIZER2::*CIRCULARITY-VECTOR-QUEUE*) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (MAKE-ARRAY 50 :ELEMENT-TYPE T :INITIAL-ELEMENT NIL :FILL- > POINTER 0 > ; :ADJUSTABLE T) > ; --> LET MULTIPLE-VALUE-BIND LET > ; ==> > ; (SB-KERNEL:%DATA-VECTOR-AND-INDEX ARRAY 0) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (VECTOR T), not a SIMPLE-ARRAY. > > ; compiling (DEFUN RELEASE-CIRCULARITY-VECTOR ...) > ; compiling (DEFCONSTANT +2^31+ ...) > ; compiling (DEFCONSTANT +2^32+ ...) > ; compiling (DEFCONSTANT +2^63+ ...) > ; compiling (DEFCONSTANT +2^64+ ...) > ; compiling (DEFUN SERIALIZE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer2.lisp > ; in: DEFUN SERIALIZE > ; (< 536870911 ELEPHANT-SERIALIZER2::+2^63+) > ; ==> > ; ELEPHANT-SERIALIZER2::+2^63+ > ; > ; note: deleting unreachable code > > ; (EQ (< 536870911 ELEPHANT-SERIALIZER2::+2^63+) T) > ; > ; note: deleting unreachable code > > ; (ABS ELEPHANT-SERIALIZER2::FROB) > ; ==> > ; ELEPHANT-SERIALIZER2::FROB > ; > ; note: deleting unreachable code > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+FIXNUM32+ > ; ELEPHANT-SERIALIZER2::BS) > ; ==> > ; ELEPHANT-SERIALIZER2::+FIXNUM32+ > ; > ; note: deleting unreachable code > ; > ; note: deleting unreachable code > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::FROB > ; ELEPHANT-SERIALIZER2::BS) > ; ==> > ; ELEPHANT-SERIALIZER2::FROB > ; > ; note: deleting unreachable code > ; > ; note: deleting unreachable code > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+FIXNUM64+ > ; ELEPHANT-SERIALIZER2::BS) > ; ==> > ; ELEPHANT-SERIALIZER2::+FIXNUM64+ > ; > ; note: deleting unreachable code > ; > ; note: deleting unreachable code > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT64 ELEPHANT-SERIALIZER2::FROB > ; ELEPHANT-SERIALIZER2::BS) > ; ==> > ; ELEPHANT-SERIALIZER2::FROB > ; > ; note: deleting unreachable code > ; > ; note: deleting unreachable code > > ; (REALPART ELEPHANT-SERIALIZER2::FROB) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a COMPLEX, not a (COMPLEX RATIONAL). > > ; (IMAGPART ELEPHANT-SERIALIZER2::FROB) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a COMPLEX, not a (COMPLEX RATIONAL). > > ; (ARRAY-RANK ELEPHANT-SERIALIZER2::FROB) > ; > ; note: unable to optimize because: The array rank is not known at > compile time: * > > ; (ARRAY-TOTAL-SIZE ELEPHANT-SERIALIZER2::FROB) > ; > ; note: unable to optimize because: can't tell the rank at compile > time > > ; (ROW-MAJOR-AREF ELEPHANT-SERIALIZER2::FROB ELEPHANT- > SERIALIZER2::I) > ; --> SB-KERNEL:HAIRY-DATA-VECTOR-REF SB-KERNEL:%CHECK-BOUND > ; ==> > ; (ARRAY-TOTAL-SIZE ARRAY) > ; > ; note: unable to optimize because: can't tell the rank at compile > time > > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-REF ARRAY > ; (SB-KERNEL:%CHECK-BOUND ARRAY > ; (ARRAY- > TOTAL-SIZE > ; ARRAY) > ; SB- > INT:INDEX)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a ARRAY, not a SIMPLE-STRING. > ; > ; note: unable to > ; avoid runtime dispatch on array element type > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (ARRAY-DIMENSION ELEPHANT-SERIALIZER2::FROB ELEPHANT- > SERIALIZER2::I) > ; > ; note: unable to optimize because: The axis is not constant. > > ; (INCF ELEPHANT-SERIALIZER2::LISP-OBJ-ID) > ; --> LET* > ; ==> > ; (+ ELEPHANT-SERIALIZER2::LISP-OBJ-ID #:G175) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > > ; (LENGTH ELEPHANT-SERIALIZER2::SVS) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a (SIMPLE-ARRAY * (*)). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a SEQUENCE, not a VECTOR. > > ; (INCF ELEPHANT-SERIALIZER2::LISP-OBJ-ID) > ; --> LET* > ; ==> > ; (+ ELEPHANT-SERIALIZER2::LISP-OBJ-ID #:G101) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > > ; (LABELS ((ELEPHANT-SERIALIZER2::%NEXT-OBJECT-ID () > ; (INCF ELEPHANT-SERIALIZER2::LISP-OBJ-ID)) > ; (ELEPHANT-SERIALIZER2::%SERIALIZE (ELEPHANT- > SERIALIZER2::FROB) > ; (TYPECASE ELEPHANT-SERIALIZER2::FROB > ; (FIXNUM #) > ; (NULL #) > ; (SYMBOL #) > ; (STRING #) > ; (ELEPHANT:PERSISTENT # # # #) > ; (SINGLE-FLOAT # #) > ; (DOUBLE-FLOAT # #) > ; (STANDARD-OBJECT # #) > ; (INTEGER #) > ; (RATIONAL # # #) > ; ...))) > ; (ELEPHANT-SERIALIZER2::%SERIALIZE ELEPHANT-SERIALIZER2::FROB) > ; (ELEPHANT-SERIALIZER2::RELEASE-CIRCULARITY-HASH > ; ELEPHANT-SERIALIZER2::CIRCULARITY-HASH) > ; ELEPHANT-SERIALIZER2::BS) > ; > ; note: Return type not fixed values, so can't use known return > convention: > ; * > > ; (INCF ELEPHANT-SERIALIZER2::LISP-OBJ-ID) > ; --> LET* > ; ==> > ; (+ ELEPHANT-SERIALIZER2::LISP-OBJ-ID #:G101) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; etc. > > ; ==> > ; (+ ELEPHANT-SERIALIZER2::LISP-OBJ-ID #:G175) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a NUMBER, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; etc. > > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER2::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+FILL-POINTER-P+ 0) > ; (IF (ADJUSTABLE-ARRAY-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+ADJUSTABLE-P+ 0)) > ; --> LOGIOR > ; ==> > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER2::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+FILL-POINTER-P+ 0)) > ; > ; note: forced to do static-fun Two-arg-ior (cost 53) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a T, not a FIXNUM. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 3) > because: > ; The first argument is a T, not a (SIGNED-BYTE 32). > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > ; (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; ==> > ; (LOGIOR > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER2::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+FILL-POINTER-P+ 0)) > ; (IF (ADJUSTABLE-ARRAY-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+ADJUSTABLE-P+ 0)) > ; > ; note: forced to do static-fun Two-arg-ior (cost 53) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 3) > because: > ; The first argument is a INTEGER, not a (SIGNED-BYTE 32). > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > ; (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; (REALPART ELEPHANT-SERIALIZER2::FROB) > ; > ; note: forced to do full call > ; unable to do complex float realpart (cost 3) because: > ; The first argument is a COMPLEX, not a (COMPLEX SINGLE-FLOAT). > ; The result is a (VALUES REAL &OPTIONAL), not a (VALUES > SINGLE-FLOAT &REST > ; T). > ; unable to do complex float realpart (cost 3) because: > ; The first argument is a COMPLEX, not a (COMPLEX DOUBLE-FLOAT). > ; The result is a (VALUES REAL &OPTIONAL), not a (VALUES > DOUBLE-FLOAT &REST > ; T). > > ; (IMAGPART ELEPHANT-SERIALIZER2::FROB) > ; > ; note: forced to do full call > ; unable to do complex float imagpart (cost 3) because: > ; The first argument is a COMPLEX, not a (COMPLEX SINGLE-FLOAT). > ; The result is a (VALUES REAL &OPTIONAL), not a (VALUES > SINGLE-FLOAT &REST > ; T). > ; unable to do complex float imagpart (cost 3) because: > ; The first argument is a COMPLEX, not a (COMPLEX DOUBLE-FLOAT). > ; The result is a (VALUES REAL &OPTIONAL), not a (VALUES > DOUBLE-FLOAT &REST > ; T). > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+FIXNUM32+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-108) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::FROB > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-112) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+FIXNUM32+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT-SERIALIZER2::+NIL+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-348) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+SYMBOL+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-341) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT-SERIALIZER2::+NIL+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-334) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+SYMBOL+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+PERSISTENT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-327) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 > ; (ELEPHANT::OID ELEPHANT-SERIALIZER2::FROB) ELEPHANT- > SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-232) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+PERSISTENT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+SINGLE-FLOAT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-320) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-FLOAT ELEPHANT-SERIALIZER2::FROB > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-124) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+SINGLE-FLOAT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+DOUBLE-FLOAT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-313) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-DOUBLE ELEPHANT-SERIALIZER2::FROB > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-130) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+DOUBLE-FLOAT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+RATIONAL+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-299) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT-SERIALIZER2::+CHAR+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-292) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT-SERIALIZER2::+CONS+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-285) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::IDP > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-220) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::ID > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-216) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT-SERIALIZER2::+CONS+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+COMPLEX+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-271) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+ARRAY+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-257) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::IDP > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-204) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::ID > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-200) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER2::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+FILL-POINTER-P+ 0) > ; (IF (ADJUSTABLE-ARRAY-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+ADJUSTABLE-P+ 0)) > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-250) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::RANK > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-196) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE > ; (LOGIOR > ; (ELEPHANT::BYTE-FROM-ARRAY-TYPE > ; (ARRAY-ELEMENT-TYPE ELEPHANT-SERIALIZER2::FROB)) > ; (IF (ARRAY-HAS-FILL-POINTER-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+FILL-POINTER-P+ 0) > ; (IF (ADJUSTABLE-ARRAY-P ELEPHANT-SERIALIZER2::FROB) > ; ELEPHANT-SERIALIZER2::+ADJUSTABLE-P+ 0)) > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::ID > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+ARRAY+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+STRUCT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-243) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::IDP > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-192) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 (INCF ELEPHANT- > SERIALIZER2::LISP-OBJ-ID) > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-188) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+STRUCT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+HASH-TABLE+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-264) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::IDP > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-212) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::ID > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-208) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-SERIALIZER2::%SERIALIZE > ; (HASH-TABLE-REHASH-THRESHOLD ELEPHANT-SERIALIZER2::FROB)) > ; > ; note: doing float to pointer coercion (cost 13) to FROB > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::ID > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+HASH-TABLE+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+PATHNAME+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-278) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+OBJECT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-306) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::IDP > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-228) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT-SERIALIZER2::ID > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {13075649}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-224) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+OBJECT+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; compiling (DEFUN SERIALIZE-BIGNUM ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer2.lisp > ; in: DEFUN SERIALIZE-BIGNUM > ; (ABS ELEPHANT-SERIALIZER2::FROB) > ; --> IF > ; ==> > ; (< SB-C::X 0) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline fixnum comparison (cost 3) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; etc. > > ; --> IF - > ; ==> > ; (SB-KERNEL:%NEGATE SB-C::X) > ; > ; note: forced to do GENERIC-NEGATE (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a (INTEGER * -1), not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; unable to do inline (signed-byte 32) arithmetic (cost 2) > because: > ; The first argument is a (INTEGER * -1), not a (SIGNED-BYTE > 32). > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a (VALUES > ; > (SIGNED-BYTE 32) > ; &REST > T). > > ; (INTEGER-LENGTH ELEPHANT-SERIALIZER2::NUM) > ; > ; note: forced to do full call > ; unable to do inline (unsigned-byte 32) integer-length (cost > 26) because: > ; The first argument is a UNSIGNED-BYTE, not a (UNSIGNED-BYTE > 32). > > ; (CEILING (/ (INTEGER-LENGTH ELEPHANT-SERIALIZER2::NUM) 32)) > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL TRUNCATE > LET > ; --> VALUES > ; ==> > ; (- SB-C::X SB-C::RES) > ; > ; note: forced to do GENERIC-- (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (RATIONAL 0 536870911/32), not a > FIXNUM. > ; The result is a (VALUES (RATIONAL -16777215 536870911/32) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The first argument is a (RATIONAL 0 536870911/32), not a > (SIGNED-BYTE 32). > ; The result is a (VALUES (RATIONAL -16777215 536870911/32) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL FUNCTION > IF > ; --> AND IF AND IF PLUSP > ; ==> > ; (> NUMBER 0) > ; > ; note: forced to do GENERIC-> (cost 10) > ; unable to do inline fixnum comparison (cost 3) because: > ; The first argument is a (RATIONAL 0 536870911/32), not a > FIXNUM. > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a (RATIONAL 0 536870911/32), not a > FIXNUM. > ; etc. > > ; (< ELEPHANT-SERIALIZER2::FROB 0) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline fixnum comparison (cost 3) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; etc. > > ; (LDB (BYTE 32 (* 32 ELEPHANT-SERIALIZER2::I)) ELEPHANT- > SERIALIZER2::NUM) > ; --> SB-KERNEL:%LDB LOGAND > ; ==> > ; (ASH INT (- SB-C::POSN)) > ; > ; note: forced to do full call > ; unable to do inline ASH (cost 3) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The second argument is a (INTEGER -536870880 0), not a > (UNSIGNED-BYTE 29). > ; The result is a (VALUES UNSIGNED-BYTE &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; unable to do inline ASH (cost 3) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The second argument is a (INTEGER -536870880 0), not a > (UNSIGNED-BYTE 29). > ; The result is a (VALUES UNSIGNED-BYTE &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; etc. > > ; --> SB-KERNEL:%LDB > ; ==> > ; (LOGAND (ASH INT (- SB-C::POSN)) (ASH 4294967295 (- SB-C::SIZE > 32))) > ; > ; note: forced to do static-fun Two-arg-and (cost 53) > ; unable to do inline (unsigned-byte 32) arithmetic (cost 2) > because: > ; The first argument is a UNSIGNED-BYTE, not a (UNSIGNED-BYTE > 32). > ; unable to do inline (unsigned-byte 32) arithmetic (cost 2) > because: > ; The first argument is a UNSIGNED-BYTE, not a (SIGNED-BYTE 32). > ; etc. > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+NEGATIVE-BIGNUM+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {11811611}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-477) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-UINT32 ELEPHANT- > SERIALIZER2::NEEDED > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {11811611}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-481) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-UINT32 > ; (LDB (BYTE 32 (* 32 ELEPHANT-SERIALIZER2::I)) ELEPHANT- > SERIALIZER2::NUM) > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {11811611}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-493) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-UINT32 ELEPHANT- > SERIALIZER2::NEEDED > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+NEGATIVE-BIGNUM+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+POSITIVE-BIGNUM+ > ; ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {11811611}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-500) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; compiling (DEFPARAMETER *TRACE-DESERIALIZER* ...) > ; compiling (DEFPARAMETER *TAG-TABLE* ...) > ; compiling (DEFUN ENABLE-DESERIALIZER-TRACING ...) > ; compiling (DEFUN DISABLE-DESERIALIZER-TRACING ...) > ; compiling (DEFUN PRINT-PRE-DESERIALIZE-TAG ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer2.lisp > ; in: DEFUN PRINT-PRE-DESERIALIZE-TAG > ; (ASSOC ELEPHANT-SERIALIZER2::TAG ELEPHANT-SERIALIZER2::*TAG- > TABLE*) > ; --> BLOCK LET COND IF COND IF COND PROGN IF SB-IMPL::ASSOC-GUTS DO > BLOCK > ; --> LET TAGBODY TAGBODY WHEN IF AND IF AND > ; ==> > ; (EQL SB-IMPL::ITEM (CAAR SB-IMPL::ALIST)) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a T, not a SINGLE-FLOAT. > ; The second argument is a T, not a SINGLE-FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a T, not a DOUBLE-FLOAT. > ; The second argument is a T, not a DOUBLE-FLOAT. > ; > ; note: forced to do GENERIC-EQL (cost 10) > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a T, not a FIXNUM. > ; The second argument is a T, not a FIXNUM. > > ; compiling (DEFUN PRINT-POST-DESERIALIZE-TAG ...) > ; compiling (DEFUN DESERIALIZE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer2.lisp > ; in: DEFUN DESERIALIZE > ; (>= ELEPHANT-SERIALIZER2::ID > ; (FILL-POINTER ELEPHANT-SERIALIZER2::CIRCULARITY-VECTOR)) > ; --> OR LET IF OR = IF > ; ==> > ; (= SB-C::X SB-C::Y) > ; > ; note: unable to > ; open-code FLOAT to RATIONAL comparison > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > ; > ; note: unable to open code because: The operands might not be the > same type. > > ; --> OR LET > IF > ; ==> > ; (> SB-C::X SB-C::Y) > ; > ; note: unable to > ; open-code FLOAT to RATIONAL comparison > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > > ; (CEILING > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) ELEPHANT- > SERIALIZER2::REHASH-THRESHOLD)) > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL FUNCTION > IF > ; --> AND IF NOT IF ZEROP > ; ==> > ; (= REM 0) > ; > ; note: unable to > ; open-code FLOAT to RATIONAL comparison > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > ; > ; note: unable to open code because: The operands might not be the > same type. > > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL FUNCTION > IF > ; --> AND IF AND IF PLUSP > ; ==> > ; (> NUMBER 0) > ; > ; note: unable to > ; open-code FLOAT to RATIONAL comparison > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > > ; (+ ELEPHANT-SERIALIZER2::SIZE 10) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) ELEPHANT- > SERIALIZER2::REHASH-THRESHOLD) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a RATIONAL. > ; The second argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a FLOAT. > ; The second argument is a NUMBER, not a RATIONAL. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a SINGLE-FLOAT. > ; The second argument is a NUMBER, not a DOUBLE-FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a DOUBLE-FLOAT. > ; The second argument is a NUMBER, not a SINGLE-FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; The second argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; The second argument is a NUMBER, not a REAL. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > ; The second argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > ; The second argument is a NUMBER, not a REAL. > ; > ; note: unable to > ; convert x/2^k to shift > ; due to type uncertainty: > ; The first argument is a NUMBER, not a INTEGER. > ; The second argument is a NUMBER, not a INTEGER. > > ; (CEILING > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) ELEPHANT- > SERIALIZER2::REHASH-THRESHOLD)) > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL TRUNCATE > LET > ; --> VALUES > ; ==> > ; (- SB-C::X SB-C::RES) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > > ; (* > ; (CEILING > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) > ; ELEPHANT-SERIALIZER2::REHASH-THRESHOLD)) > ; ELEPHANT-SERIALIZER2::REHASH-SIZE) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The second argument is a NUMBER, not a FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The second argument is a NUMBER, not a (COMPLEX SINGLE-FLOAT). > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The second argument is a NUMBER, not a (COMPLEX DOUBLE-FLOAT). > ; > ; note: unable to > ; convert x*2^k to shift > ; due to type uncertainty: > ; The second argument is a NUMBER, not a INTEGER. > > ; (CEILING > ; (* > ; (CEILING > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) > ; ELEPHANT-SERIALIZER2::REHASH-THRESHOLD)) > ; ELEPHANT-SERIALIZER2::REHASH-SIZE)) > ; ==> > ; (CEILING > ; (* > ; (CEILING > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) > ; ELEPHANT-SERIALIZER2::REHASH-THRESHOLD)) > ; ELEPHANT-SERIALIZER2::REHASH-SIZE) > ; 1) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a REAL, not a FLOAT. > ; > ; note: unable to > ; convert division by 2^k to shift > ; due to type uncertainty: > ; The first argument is a REAL, not a INTEGER. > > ; (SETF (SLOT-VALUE ELEPHANT-SERIALIZER2::O > ; (ELEPHANT-SERIALIZER2::%DESERIALIZE > ; ELEPHANT-SERIALIZER2::BS)) > ; (ELEPHANT-SERIALIZER2::%DESERIALIZE ELEPHANT- > SERIALIZER2::BS)) > ; ==> > ; (SB-PCL::SET-SLOT-VALUE ELEPHANT-SERIALIZER2::O > ; (ELEPHANT-SERIALIZER2::%DESERIALIZE > ; ELEPHANT-SERIALIZER2::BS) > ; (ELEPHANT-SERIALIZER2::%DESERIALIZE > ; ELEPHANT-SERIALIZER2::BS)) > ; > ; note: unable to optimize because: Slot name is not constant. > > ; (MAKE-ARRAY > ; (LOOP ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::I FIXNUM > ; ELEPHANT-SERIALIZER2::FROM 0 ELEPHANT-SERIALIZER2::BELOW > ; (ELEPHANT-MEMUTIL:BUFFER-READ-INT32 ELEPHANT- > SERIALIZER2::BS) > ; ELEPHANT-SERIALIZER2::COLLECT > ; (ELEPHANT-SERIALIZER2::%DESERIALIZE ELEPHANT- > SERIALIZER2::BS)) > ; :ELEMENT-TYPE > ; (ELEPHANT::ARRAY-TYPE-FROM-BYTE (LOGAND 31 ELEPHANT- > SERIALIZER2::FLAGS)) > ; :FILL-POINTER > ; (/= 0 > ; (LOGAND ELEPHANT-SERIALIZER2::+FILL-POINTER-P+ > ; ELEPHANT-SERIALIZER2::FLAGS)) > ; :ADJUSTABLE > ; (/= 0 > ; (LOGAND ELEPHANT-SERIALIZER2::+ADJUSTABLE-P+ > ; ELEPHANT-SERIALIZER2::FLAGS))) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MOD 536870911) CONS NULL), not a > INTEGER. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a (OR (MOD 536870911) CONS NULL), not a > LIST. > > ; (ARRAY-TOTAL-SIZE ELEPHANT-SERIALIZER2::A) > ; > ; note: unable to optimize because: can't tell the rank at compile > time > > ; (SETF (ROW-MAJOR-AREF ELEPHANT-SERIALIZER2::A ELEPHANT- > SERIALIZER2::I) > ; (ELEPHANT-SERIALIZER2::%DESERIALIZE ELEPHANT- > SERIALIZER2::BS)) > ; --> SB-KERNEL:%SET-ROW-MAJOR-AREF SB-KERNEL:HAIRY-DATA-VECTOR-SET > ; --> SB-KERNEL:%CHECK-BOUND > ; ==> > ; (ARRAY-TOTAL-SIZE ARRAY) > ; > ; note: unable to optimize because: can't tell the rank at compile > time > > ; --> SB-KERNEL:%SET-ROW-MAJOR-AREF > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-SET ARRAY > ; (SB-KERNEL:%CHECK-BOUND ARRAY > ; (ARRAY- > TOTAL-SIZE > ; ARRAY) > ; SB- > INT:INDEX) > ; SB-C::NEW-VALUE) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a ARRAY, not a SIMPLE-STRING. > ; > ; note: unable to > ; avoid runtime dispatch on array element type > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (FUNCALL (ELEPHANT:STRUCT-CONSTRUCTOR ELEPHANT- > SERIALIZER2::TYPEDESIG)) > ; --> SB-C::%FUNCALL THE > ; ==> > ; (SB-KERNEL:%COERCE-CALLABLE-TO-FUN FUNCTION) > ; > ; note: unable to > ; optimize away possible call to FDEFINITION at runtime > ; due to type uncertainty: > ; The first argument is a (OR FUNCTION SYMBOL), not a FUNCTION. > > ; (SETF (SLOT-VALUE ELEPHANT-SERIALIZER2::O ELEPHANT- > SERIALIZER2::NAME) > ; ELEPHANT-SERIALIZER2::VALUE) > ; ==> > ; (SB-PCL::SET-SLOT-VALUE ELEPHANT-SERIALIZER2::O ELEPHANT- > SERIALIZER2::NAME > ; ELEPHANT-SERIALIZER2::VALUE) > ; > ; note: unable to optimize because: Slot name is not constant. > > ; (AREF ELEPHANT-SERIALIZER2::CIRCULARITY-VECTOR ELEPHANT- > SERIALIZER2::ID) > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-REF/CHECK-BOUNDS ARRAY SB-INT:INDEX) > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (LABELS ((ELEPHANT-SERIALIZER2::LOOKUP-ID (ELEPHANT- > SERIALIZER2::ID) > ; (IF (>= ELEPHANT-SERIALIZER2::ID #) NIL > ; (AREF ELEPHANT-SERIALIZER2::CIRCULARITY-VECTOR > ; ELEPHANT-SERIALIZER2::ID))) > ; (ELEPHANT-SERIALIZER2::ADD-OBJECT (ELEPHANT- > SERIALIZER2::OBJECT) > ; (VECTOR-PUSH-EXTEND ELEPHANT-SERIALIZER2::OBJECT > ; ELEPHANT-SERIALIZER2::CIRCULARITY- > VECTOR 50) > ; (1- (FILL-POINTER ELEPHANT-SERIALIZER2::CIRCULARITY- > VECTOR))) > ; (ELEPHANT-SERIALIZER2::%DESERIALIZE (ELEPHANT- > SERIALIZER2::BS) > ; (DECLARE > ; (TYPE ELEPHANT-MEMUTIL:BUFFER-STREAM ELEPHANT- > SERIALIZER2::BS)) > ; (LET (#) > ; (DECLARE # #) > ; (COND # # # # # # # # # # # ...)))) > ; (ETYPECASE ELEPHANT-SERIALIZER2::BUF-STR > ; (NULL (RETURN-FROM ELEPHANT-SERIALIZER2::DESERIALIZE NIL)) > ; (ELEPHANT-MEMUTIL:BUFFER-STREAM > ; (LET (#) > ; (ELEPHANT-SERIALIZER2::RELEASE-CIRCULARITY-VECTOR > ; ELEPHANT-SERIALIZER2::CIRCULARITY-VECTOR) > ; ELEPHANT-SERIALIZER2::RESULT)))) > ; > ; note: Return type not fixed values, so can't use known return > convention: > ; * > > ; (>= ELEPHANT-SERIALIZER2::ID > ; (FILL-POINTER ELEPHANT-SERIALIZER2::CIRCULARITY-VECTOR)) > ; --> OR LET > IF > ; ==> > ; (> SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-> (cost 10) > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a REAL, not a FIXNUM. > > ; (+ ELEPHANT-SERIALIZER2::SIZE 10) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a T, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a T, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; etc. > > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) ELEPHANT- > SERIALIZER2::REHASH-THRESHOLD) > ; > ; note: forced to do static-fun Two-arg-/ (cost 53) > ; unable to do inline float arithmetic (cost 12) because: > ; The first argument is a NUMBER, not a DOUBLE-FLOAT. > ; The second argument is a T, not a DOUBLE-FLOAT. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > DOUBLE-FLOAT > ; > &REST T). > ; unable to do inline float arithmetic (cost 12) because: > ; The first argument is a NUMBER, not a SINGLE-FLOAT. > ; The second argument is a T, not a SINGLE-FLOAT. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > SINGLE-FLOAT > ; > &REST T). > > ; (CEILING > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) ELEPHANT- > SERIALIZER2::REHASH-THRESHOLD)) > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL TRUNCATE > LET > ; ==> > ; (SB-KERNEL:%UNARY-TRUNCATE SB-C::X) > ; > ; note: forced to do full call > ; unable to do inline float truncate (cost 5) because: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > ; (UNSIGNED- > BYTE 32) > ; &REST T). > ; unable to do inline float truncate (cost 5) because: > ; The first argument is a REAL, not a SINGLE-FLOAT. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > ; (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL TRUNCATE > LET > ; --> VALUES > ; ==> > ; (- SB-C::X SB-C::RES) > ; > ; note: forced to do GENERIC-- (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a REAL, not a FIXNUM. > ; The second argument is a INTEGER, not a FIXNUM. > ; The result is a (VALUES REAL &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The first argument is a REAL, not a (SIGNED-BYTE 32). > ; The second argument is a INTEGER, not a (SIGNED-BYTE 32). > ; The result is a (VALUES REAL &OPTIONAL), not a (VALUES > (SIGNED-BYTE 32) > ; &REST > T). > ; etc. > > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL FUNCTION > IF > ; --> AND IF AND IF PLUSP > ; ==> > ; (> NUMBER 0) > ; > ; note: forced to do GENERIC-> (cost 10) > ; unable to do inline fixnum comparison (cost 3) because: > ; The first argument is a REAL, not a FIXNUM. > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a REAL, not a FIXNUM. > ; etc. > > ; --> CEILING BLOCK MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL FUNCTION > IF > ; --> VALUES > ; ==> > ; (+ SB-KERNEL::TRU 1) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; etc. > > ; (* > ; (CEILING > ; (/ (+ ELEPHANT-SERIALIZER2::SIZE 10) > ; ELEPHANT-SERIALIZER2::REHASH-THRESHOLD)) > ; ELEPHANT-SERIALIZER2::REHASH-SIZE) > ; > ; note: forced to do GENERIC-* (cost 30) > ; unable to do inline fixnum arithmetic (cost 4) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; The second argument is a T, not a FIXNUM. > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The first argument is a INTEGER, not a (SIGNED-BYTE 32). > ; The second argument is a T, not a (SIGNED-BYTE 32). > ; The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES > (SIGNED-BYTE 32) > ; > &REST T). > ; etc. > > ; (COMPLEX ELEPHANT-SERIALIZER2::RPART ELEPHANT- > SERIALIZER2::IPART) > ; > ; note: forced to do full call > ; unable to do inline complex single-float creation (cost 5) > because: > ; The first argument is a T, not a SINGLE-FLOAT. > ; The second argument is a T, not a SINGLE-FLOAT. > ; The result is a (VALUES > ; (OR RATIONAL (COMPLEX RATIONAL) (COMPLEX > DOUBLE-FLOAT) > ; (COMPLEX SINGLE-FLOAT)) > ; &OPTIONAL), not a (VALUES (COMPLEX SINGLE- > FLOAT) &REST > ; T). > ; unable to do inline complex double-float creation (cost 5) > because: > ; The first argument is a T, not a DOUBLE-FLOAT. > ; The second argument is a T, not a DOUBLE-FLOAT. > ; The result is a (VALUES > ; (OR RATIONAL (COMPLEX RATIONAL) (COMPLEX > DOUBLE-FLOAT) > ; (COMPLEX SINGLE-FLOAT)) > ; &OPTIONAL), not a (VALUES (COMPLEX DOUBLE- > FLOAT) &REST > ; T). > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-BYTE ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK LET INCF LET* FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12748BC1}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-671) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-FLOAT ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK LET SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C:: > %FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12748BC1}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-606) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK LET ELEPHANT-MEMUTIL::READ-FLOAT BLOCK THE DEREF > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; ==> > ; ALIEN > ; > ; note: doing float to pointer coercion (cost 13) to "" > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-DOUBLE ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK LET SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C:: > %FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12748BC1}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-610) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; --> BLOCK LET ELEPHANT-MEMUTIL::READ-DOUBLE BLOCK THE DEREF > ; --> SB-ALIEN-INTERNALS:EXTRACT-ALIEN-VALUE SB-ALIEN- > INTERNALS:NATURALIZE > ; ==> > ; ALIEN > ; > ; note: doing float to pointer coercion (cost 13) to "" > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-BYTE ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK LET INCF LET* FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12748BC1}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-652) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-INT32 ELEPHANT-SERIALIZER2::BS) > ; --> BLOCK LET SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C:: > %FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12748BC1}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-647) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFUN DESERIALIZE-BIGNUM ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/serializer2.lisp > ; in: DEFUN DESERIALIZE-BIGNUM > ; (DPB (ELEPHANT-MEMUTIL:BUFFER-READ-UINT ELEPHANT- > SERIALIZER2::BS) > ; ELEPHANT-SERIALIZER2::BYTE-SPEC ELEPHANT-SERIALIZER2::NUM) > ; --> LET > ; ==> > ; (SB-KERNEL:%DPB (ELEPHANT-MEMUTIL:BUFFER-READ-UINT ELEPHANT- > SERIALIZER2::BS) > ; (BYTE-SIZE #:G769) (BYTE-POSITION #:G769) > ; ELEPHANT-SERIALIZER2::NUM) > ; > ; note: unable to > ; convert to inline logical operations > ; due to type uncertainty: > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > (UNSIGNED-BYTE 32) > ; &REST > T). > ; > ; note: unable to > ; convert to inline logical operations > ; due to type uncertainty: > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > (SIGNED-BYTE 32) > ; &REST > T). > > ; (DEFUN ELEPHANT-SERIALIZER2::DESERIALIZE-BIGNUM > ; (ELEPHANT-SERIALIZER2::BS LENGTH ELEPHANT- > SERIALIZER2::POSITIVE) > ; (DECLARE (TYPE ELEPHANT-MEMUTIL:BUFFER-STREAM ELEPHANT- > SERIALIZER2::BS) > ; (TYPE FIXNUM LENGTH) (TYPE BOOLEAN ELEPHANT- > SERIALIZER2::POSITIVE)) > ; (LET ((ELEPHANT::INT-BYTE-SPEC (BYTE 32 0))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::INT-BYTE-SPEC) > ; (IGNORABLE ELEPHANT::INT-BYTE-SPEC)) > ; (LOOP ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::I > ; ELEPHANT-SERIALIZER2::FROM 0 ELEPHANT-SERIALIZER2::BELOW > ; (/ LENGTH 4) ELEPHANT-SERIALIZER2::FOR > ; ELEPHANT-SERIALIZER2::BYTE-SPEC = > ; (BYTE 32 (* 32 ELEPHANT-SERIALIZER2::I)) ELEPHANT- > SERIALIZER2::WITH > ; ...))) > ; --> PROGN EVAL-WHEN SB-IMPL::%DEFUN LOCALLY SB-C::%FUNCALL > ; --> MULTIPLE-VALUE-BIND LET UNLESS IF TYPEP IF MEMBER BLOCK LET DO > BLOCK > ; --> LET TAGBODY TAGBODY LET WHEN IF SB-IMPL::SATISFIES-THE-TEST > LET COND > ; --> IF COND IF COND PROGN FUNCALL > ; ==> > ; (SB-C::%FUNCALL FUNCTION #:G804 #:G805) > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a T, not a SINGLE-FLOAT. > ; The second argument is a T, not a SINGLE-FLOAT. > ; > ; note: unable to > ; optimize > ; due to type uncertainty: > ; The first argument is a T, not a DOUBLE-FLOAT. > ; The second argument is a T, not a DOUBLE-FLOAT. > > ; (LOOP ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::I > ; ELEPHANT-SERIALIZER2::FROM 0 ELEPHANT-SERIALIZER2::BELOW > (/ LENGTH 4) > ; ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::BYTE-SPEC = > ; (BYTE 32 (* 32 ELEPHANT-SERIALIZER2::I)) ELEPHANT- > SERIALIZER2::WITH ...) > ; --> BLOCK LET LET LET SB-LOOP::LOOP-BODY TAGBODY WHEN IF >= IF > ; ==> > ; (< SB-C::X SB-C::Y) > ; > ; note: forced to do GENERIC-< (cost 10) > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The second argument is a (RATIONAL -134217728 536870911/4), > not a FIXNUM. > > ; (- ELEPHANT-SERIALIZER2::NUM) > ; ==> > ; (SB-KERNEL:%NEGATE ELEPHANT-SERIALIZER2::NUM) > ; > ; note: forced to do GENERIC-NEGATE (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a INTEGER, not a FIXNUM. > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > FIXNUM &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 2) > because: > ; The first argument is a INTEGER, not a (SIGNED-BYTE 32). > ; The result is a (VALUES INTEGER &OPTIONAL), not a (VALUES > ; (SIGNED- > BYTE 32) &REST > ; T). > > ; (* 32 ELEPHANT-SERIALIZER2::I) > ; --> * > ; ==> > ; (ASH SB-C::X 5) > ; > ; note: forced to do full call > ; unable to do inline ASH (cost 2) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES UNSIGNED-BYTE &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; unable to do inline ASH (cost 3) because: > ; The first argument is a UNSIGNED-BYTE, not a (UNSIGNED-BYTE > 32). > ; The result is a (VALUES UNSIGNED-BYTE &OPTIONAL), not a > (VALUES > ; > (UNSIGNED-BYTE > ; 32) > ; > &REST T). > ; etc. > > ; (LOOP ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::I > ; ELEPHANT-SERIALIZER2::FROM 0 ELEPHANT-SERIALIZER2::BELOW > (/ LENGTH 4) > ; ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::BYTE-SPEC = > ; (BYTE 32 (* 32 ELEPHANT-SERIALIZER2::I)) ELEPHANT- > SERIALIZER2::WITH ...) > ; --> BLOCK LET LET LET SB-LOOP::LOOP-BODY TAGBODY > ; --> SB-LOOP::LOOP-REALLY-DESETQ SETQ THE 1+ > ; ==> > ; (+ ELEPHANT-SERIALIZER2::I 1) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a UNSIGNED-BYTE, not a FIXNUM. > ; The result is a (VALUES (INTEGER 1) &OPTIONAL), not a > (VALUES FIXNUM > ; &REST > T). > ; etc. > > ; (DEFUN ELEPHANT-SERIALIZER2::DESERIALIZE-BIGNUM > ; (ELEPHANT-SERIALIZER2::BS LENGTH ELEPHANT- > SERIALIZER2::POSITIVE) > ; (DECLARE (TYPE ELEPHANT-MEMUTIL:BUFFER-STREAM ELEPHANT- > SERIALIZER2::BS) > ; (TYPE FIXNUM LENGTH) (TYPE BOOLEAN ELEPHANT- > SERIALIZER2::POSITIVE)) > ; (LET ((ELEPHANT::INT-BYTE-SPEC (BYTE 32 0))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::INT-BYTE-SPEC) > ; (IGNORABLE ELEPHANT::INT-BYTE-SPEC)) > ; (LOOP ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::I > ; ELEPHANT-SERIALIZER2::FROM 0 ELEPHANT-SERIALIZER2::BELOW > ; (/ LENGTH 4) ELEPHANT-SERIALIZER2::FOR > ; ELEPHANT-SERIALIZER2::BYTE-SPEC = > ; (BYTE 32 (* 32 ELEPHANT-SERIALIZER2::I)) ELEPHANT- > SERIALIZER2::WITH > ; ...))) > ; --> PROGN EVAL-WHEN SB-IMPL::%DEFUN LOCALLY SB-C::%FUNCALL > ; --> MULTIPLE-VALUE-BIND LET UNLESS IF TYPEP IF MEMBER BLOCK LET DO > BLOCK > ; --> LET TAGBODY TAGBODY LET WHEN IF SB-IMPL::SATISFIES-THE-TEST > LET COND > ; --> IF COND IF COND PROGN FUNCALL > ; ==> > ; (SB-C::%FUNCALL FUNCTION #:G804 #:G805) > ; > ; note: forced to do GENERIC-EQL (cost 10) > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a T, not a FIXNUM. > ; The second argument is a T, not a FIXNUM. > > ; /Users/jensteich/Lisp/elephant/src/elephant/serializer2.fasl written > ; compilation finished in 0:00:01 > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > unicode2.lisp" (written 04 JUN 2007 11:14:04 PM): > ; compiling (IN-PACKAGE :ELEPHANT-SERIALIZER2) > ; compiling (DECLAIM (OPTIMIZE # ...)) > ; compiling (DEFUN SERIALIZE-STRING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/unicode2.lisp > ; in: DEFUN SERIALIZE-STRING > ; (CHAR STRING 0) > ; --> AREF > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-REF/CHECK-BOUNDS ARRAY SB-INT:INDEX) > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > > ; compiling (DEFUN SERIALIZE-TO-UTF8 ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/unicode2.lisp > ; in: DEFUN SERIALIZE-TO-UTF8 > ; (CHAR STRING ELEPHANT-SERIALIZER2::I) > ; --> AREF > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-REF/CHECK-BOUNDS ARRAY SB-INT:INDEX) > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+UTF8-STRING+ > ; ELEPHANT-SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12AB5729}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-55) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT- > SERIALIZER2::CHARACTERS > ; ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12AB5729}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-59) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (LET ((ELEPHANT-SERIALIZER2::NEEDED > ; (+ ELEPHANT-SERIALIZER2::SIZE ELEPHANT- > SERIALIZER2::CHARACTERS))) > ; (DECLARE (TYPE FIXNUM ELEPHANT-SERIALIZER2::NEEDED)) > ; (WHEN (> ELEPHANT-SERIALIZER2::NEEDED ELEPHANT- > SERIALIZER2::ALLOCATED) > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT- > SERIALIZER2::BSTREAM > ; ELEPHANT- > SERIALIZER2::NEEDED)) > ; (ETYPECASE STRING > ; (SIMPLE-STRING > ; (LOOP ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::I > FIXNUM > ; ELEPHANT-SERIALIZER2::FROM 0 ELEPHANT- > SERIALIZER2::BELOW > ; ELEPHANT-SERIALIZER2::CHARACTERS DO > ; (LET # > ; # > ; # > ; #))) > ; (STRING > ; (LOOP ELEPHANT-SERIALIZER2::FOR ELEPHANT-SERIALIZER2::I > FIXNUM > ; ELEPHANT-SERIALIZER2::FROM 0 ELEPHANT- > SERIALIZER2::BELOW > ; ELEPHANT-SERIALIZER2::CHARACTERS DO > ; (LET # > ; # > ; # > ; #)))) > ; (SETF (ELEPHANT-MEMUTIL:BUFFER-STREAM-SIZE ELEPHANT- > SERIALIZER2::BSTREAM) > ; ELEPHANT-SERIALIZER2::NEEDED) > ; (ELEPHANT-SERIALIZER2::SUCCEED)) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (SETF (UFFI:DEREF-ARRAY ELEPHANT-SERIALIZER2::BUFFER > '(:ARRAY :UNSIGNED-CHAR) > ; (+ ELEPHANT-SERIALIZER2::I > ; ELEPHANT-SERIALIZER2::SIZE)) > ; ELEPHANT-SERIALIZER2::CODE) > ; --> LET* MULTIPLE-VALUE-BIND LET > ; ==> > ; (SB-ALIEN-INTERNALS:%SET-DEREF #:G67 #:G69 #:G68) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; ==> > ; (SB-ALIEN-INTERNALS:%SET-DEREF #:G63 #:G65 #:G64) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT- > SERIALIZER2::CHARACTERS > ; ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+UTF8-STRING+ > ; ELEPHANT-SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; compiling (DEFUN SERIALIZE-TO-UTF16LE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/unicode2.lisp > ; in: DEFUN SERIALIZE-TO-UTF16LE > ; (CHAR-CODE (SCHAR STRING ELEPHANT-SERIALIZER2::I)) > ; > ; note: deleting unreachable code > > ; (* ELEPHANT-SERIALIZER2::I 2) > ; ==> > ; ELEPHANT-SERIALIZER2::I > ; > ; note: deleting unreachable code > > ; (ELEPHANT-SERIALIZER2::FAIL) > ; > ; note: deleting unreachable code > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+UTF16-STRING+ > ; ELEPHANT-SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12CC0951}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-102) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT- > SERIALIZER2::CHARACTERS > ; ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12CC0951}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-106) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT- > SERIALIZER2::BSTREAM > ; ELEPHANT- > SERIALIZER2::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from NEEDED > > ; (INCF ELEPHANT-SERIALIZER2::SIZE (* ELEPHANT- > SERIALIZER2::CHARACTERS 2)) > ; --> LET* FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12CC0951}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-130) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (SETF (UFFI:DEREF-ARRAY ELEPHANT-SERIALIZER2::BUFFER > '(:ARRAY :UNSIGNED-CHAR) > ; (+ (* ELEPHANT-SERIALIZER2::I 2) > ; ELEPHANT-SERIALIZER2::SIZE)) > ; (LDB (BYTE 8 8) ELEPHANT-SERIALIZER2::CODE)) > ; --> LET* MULTIPLE-VALUE-BIND LET > ; ==> > ; (SB-ALIEN-INTERNALS:%SET-DEREF #:G110 #:G112 #:G111) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (SETF (UFFI:DEREF-ARRAY ELEPHANT-SERIALIZER2::BUFFER > '(:ARRAY :UNSIGNED-CHAR) > ; (+ (* ELEPHANT-SERIALIZER2::I 2) > ; ELEPHANT-SERIALIZER2::SIZE 1)) > ; (LDB (BYTE 8 0) ELEPHANT-SERIALIZER2::CODE)) > ; --> LET* MULTIPLE-VALUE-BIND LET > ; ==> > ; (SB-ALIEN-INTERNALS:%SET-DEREF #:G114 #:G116 #:G115) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT- > SERIALIZER2::CHARACTERS > ; ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+UTF16-STRING+ > ; ELEPHANT-SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; compiling (DEFUN SERIALIZE-TO-UTF32LE ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/unicode2.lisp > ; in: DEFUN SERIALIZE-TO-UTF32LE > ; (ERROR "Invalid unicode code type") > ; ==> > ; "Invalid unicode code type" > ; > ; note: deleting unreachable code > ; > ; note: deleting unreachable code > > ; (CHAR-CODE (SCHAR STRING ELEPHANT-SERIALIZER2::I)) > ; > ; note: deleting unreachable code > > ; (+ ELEPHANT-SERIALIZER2::SIZE (* 4 ELEPHANT- > SERIALIZER2::CHARACTERS)) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The second argument is a (MOD 2147483641), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354551) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354551) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; (> ELEPHANT-SERIALIZER2::NEEDED ELEPHANT-SERIALIZER2::ALLOCATED) > ; > ; note: forced to do GENERIC-> (cost 10) > ; unable to do inline fixnum comparison (cost 4) because: > ; The first argument is a (INTEGER -536870912 2684354551), not > a FIXNUM. > > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE 0) > ; --> + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (MOD 2147483645), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE 1) > ; --> + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (MOD 2147483645), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; ==> > ; (+ (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) 1) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870911 2684354556) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870911 2684354556) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; etc. > > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE 2) > ; --> + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (MOD 2147483645), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; ==> > ; (+ (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) 2) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870910 2684354557) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870910 2684354557) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; etc. > > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE 3) > ; --> + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (MOD 2147483645), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; ==> > ; (+ (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) 3) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870909 2684354558) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870909 2684354558) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; etc. > > ; (INCF ELEPHANT-SERIALIZER2::SIZE (* ELEPHANT- > SERIALIZER2::CHARACTERS 4)) > ; --> LET* > ; ==> > ; (+ (ELEPHANT-MEMUTIL:BUFFER-STREAM-SIZE #:G203) #:G204) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The second argument is a (MOD 2147483641), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354551) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354551) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+UTF32-STRING+ > ; ELEPHANT-SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12F98001}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-161) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT- > SERIALIZER2::CHARACTERS > ; ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > SETF > ; --> SETQ SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C::%FUNCALL > ; --> # STREAM-SIZE) :TYPE # MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM &OPTIONAL))> :WHERE- > FROM :DECLARED :KIND :GLOBAL-FUNCTION {12F98001}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-165) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (+ ELEPHANT-SERIALIZER2::SIZE (* 4 ELEPHANT- > SERIALIZER2::CHARACTERS)) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the second argument of GENERIC-+ > > ; (INCF ELEPHANT-SERIALIZER2::SIZE (* ELEPHANT- > SERIALIZER2::CHARACTERS 4)) > ; --> LET* > ; ==> > ; (+ (ELEPHANT-MEMUTIL:BUFFER-STREAM-SIZE #:G203) #:G204) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the second argument of GENERIC-+ > > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE 0) > ; --> + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of GENERIC-+ > > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE 1) > ; --> + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of GENERIC-+ > > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE 2) > ; --> + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of GENERIC-+ > > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE 3) > ; --> + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::SIZE) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of GENERIC-+ > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-INT32 ELEPHANT- > SERIALIZER2::CHARACTERS > ; ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; (ELEPHANT-MEMUTIL:BUFFER-WRITE-BYTE ELEPHANT- > SERIALIZER2::+UTF32-STRING+ > ; ELEPHANT-SERIALIZER2::BSTREAM) > ; --> BLOCK ELEPHANT-MEMUTIL::WITH-STRUCT-SLOTS SYMBOL-MACROLET LET > WHEN > ; --> IF PROGN > ; ==> > ; (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM ELEPHANT-MEMUTIL::BS > ; ELEPHANT-MEMUTIL::NEEDED) > ; > ; note: doing signed word to integer coercion (cost 20) from > ELEPHANT-MEMUTIL::NEEDED > > ; compiling (DEFPARAMETER NATIVE-STRING-TYPE ...) > ; compiling (DEFUN COMPATIBLE-UNICODE-SUPPORT-P ...) > ; compiling (DEFGENERIC DESERIALIZE-STRING ...) > ; compiling (DEFMETHOD DESERIALIZE-STRING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/unicode2.lisp > ; in: DEFMETHOD DESERIALIZE-STRING :AROUND (T T) > ; (DEFMETHOD ELEPHANT-SERIALIZER2::DESERIALIZE-STRING :AROUND > ; ((TYPE T) ELEPHANT-SERIALIZER2::BSTREAM &OPTIONAL > ; ELEPHANT-SERIALIZER2::TEMP-STRING) > ; (CALL-NEXT-METHOD)) > ; --> PROGN EVAL-WHEN SB-PCL::%DEFMETHOD-EXPANDER PROGN > ; --> SB-PCL::LOAD-DEFMETHOD SYMBOL-MACROLET > ; --> SB-PCL::FAST-LEXICAL-METHOD-FUNCTIONS > ; --> SB-PCL::BIND-FAST-LEXICAL-METHOD-FUNCTIONS FLET LET > ; --> SB-PCL::BIND-ARGS > ; ==> > ; (LET* ((SB-PCL::.ARGS-TAIL. SB-PCL::.REST-ARG.) > ; (ELEPHANT-SERIALIZER2::TEMP-STRING > ; (WHEN SB-PCL::.ARGS-TAIL. (POP SB-PCL::.ARGS-TAIL.))) > ; (SB-PCL::.DUMMY0. > ; (UNLESS (NULL SB-PCL::.ARGS-TAIL.) > ; (ERROR 'SB-INT:SIMPLE-PROGRAM-ERROR :FORMAT-CONTROL > ; "surplus arguments: ~S" :FORMAT-ARGUMENTS #)))) > ; (DECLARE (IGNORABLE SB-PCL::.ARGS-TAIL. SB-PCL::.DUMMY0.)) > ; (LOCALLY > ; (DECLARE (DISABLE-PACKAGE-LOCKS SB-PCL::%PARAMETER-BINDING- > MODIFIED)) > ; (SYMBOL-MACROLET ((SB-PCL::%PARAMETER-BINDING-MODIFIED #)) > ; (DECLARE (ENABLE-PACKAGE-LOCKS SB-PCL::%PARAMETER-BINDING- > MODIFIED)) > ; (BLOCK ELEPHANT-SERIALIZER2::DESERIALIZE-STRING (CALL-NEXT- > METHOD))))) > ; > ; caught STYLE-WARNING: > ; The variable TEMP-STRING is defined but never used. > > ; compiling (DEFMETHOD DESERIALIZE-STRING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/unicode2.lisp > ; in: DEFMETHOD DESERIALIZE-STRING ((EQL :UTF8) T) > ; (SETF (CHAR STRING ELEPHANT-SERIALIZER2::I) > ; (CODE-CHAR > ; (UFFI:DEREF-ARRAY > ; (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER > ; ELEPHANT-SERIALIZER2::BSTREAM) > ; '(:ARRAY :UNSIGNED-BYTE) > ; (+ ELEPHANT-SERIALIZER2::POS ELEPHANT-SERIALIZER2::I)))) > ; --> SB-KERNEL:%CHARSET SB-KERNEL:%ASET > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-SET/CHECK-BOUNDS ARRAY SB-INT:INDEX > ; SB-C::NEW-VALUE) > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (INCF > ; (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > SERIALIZER2::BSTREAM) > ; LENGTH) > ; --> LET* > ; ==> > ; (+ (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION #:G302) #:G303) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The second argument is a (SIGNED-BYTE 32), not a FIXNUM. > ; The result is a (VALUES (INTEGER -2684354560 2684354558) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -2684354560 2684354558) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-INT32 ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK LET SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C:: > %FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {134BAB81}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-313) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (UFFI:DEREF-ARRAY > ; (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT- > SERIALIZER2::BSTREAM) > ; '(:ARRAY :UNSIGNED-BYTE) > ; (+ ELEPHANT-SERIALIZER2::POS ELEPHANT-SERIALIZER2::I)) > ; ==> > ; (DEREF (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT- > SERIALIZER2::BSTREAM) > ; (+ ELEPHANT-SERIALIZER2::POS ELEPHANT-SERIALIZER2::I)) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFMETHOD DESERIALIZE-STRING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/unicode2.lisp > ; in: DEFMETHOD DESERIALIZE-STRING ((EQL :UTF16LE) T) > ; (ELEPHANT-MEMUTIL:BUFFER-READ-INT32 ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK LET SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C:: > %FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {1175D659}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-402) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 0) > ; --> UFFI:DEREF-ARRAY > ; ==> > ; (DEREF (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT- > SERIALIZER2::BSTREAM) > ; (+ (* ELEPHANT-SERIALIZER2::I 2) ELEPHANT- > SERIALIZER2::POS 0)) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 1) > ; --> UFFI:DEREF-ARRAY > ; ==> > ; (DEREF (ELEPHANT-MEMUTIL:BUFFER-STREAM-BUFFER ELEPHANT- > SERIALIZER2::BSTREAM) > ; (+ (* ELEPHANT-SERIALIZER2::I 2) ELEPHANT- > SERIALIZER2::POS 1)) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (INCF > ; (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > SERIALIZER2::BSTREAM) > ; (* LENGTH 2)) > ; --> LET* FUNCALL SB-C::%FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {1175D659}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-397) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; compiling (DEFMETHOD DESERIALIZE-STRING ...) > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/unicode2.lisp > ; in: DEFMETHOD DESERIALIZE-STRING ((EQL :UTF32LE) T) > ; (SETF (CHAR STRING ELEPHANT-SERIALIZER2::I) > ; (CODE-CHAR ELEPHANT-SERIALIZER2::CODE)) > ; --> SB-KERNEL:%CHARSET SB-KERNEL:%ASET > ; ==> > ; (SB-KERNEL:HAIRY-DATA-VECTOR-SET/CHECK-BOUNDS ARRAY SB-INT:INDEX > ; SB-C::NEW-VALUE) > ; > ; note: unable to > ; optimize > ; because: > ; Upgraded element type of array is not known at compile time. > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 0) > ; --> UFFI:DEREF-ARRAY DEREF + + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (MOD 2147483645), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 1) > ; --> UFFI:DEREF-ARRAY DEREF + + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (MOD 2147483645), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; --> UFFI:DEREF-ARRAY DEREF + > ; ==> > ; (+ (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) 1) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870911 2684354556) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870911 2684354556) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; etc. > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 2) > ; --> UFFI:DEREF-ARRAY DEREF + + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (MOD 2147483645), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; --> UFFI:DEREF-ARRAY DEREF + > ; ==> > ; (+ (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) 2) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870910 2684354557) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870910 2684354557) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; etc. > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 3) > ; --> UFFI:DEREF-ARRAY DEREF + + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (MOD 2147483645), not a FIXNUM. > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -536870912 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; --> UFFI:DEREF-ARRAY DEREF + > ; ==> > ; (+ (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) 3) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 1) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870909 2684354558) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The first argument is a (INTEGER -536870912 2684354555), not > a FIXNUM. > ; The result is a (VALUES (INTEGER -536870909 2684354558) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; etc. > > ; (INCF > ; (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > SERIALIZER2::BSTREAM) > ; (* LENGTH 4)) > ; --> LET* > ; ==> > ; (+ (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION #:G487) #:G488) > ; > ; note: forced to do GENERIC-+ (cost 10) > ; unable to do inline fixnum arithmetic (cost 2) because: > ; The second argument is a (INTEGER -2147483648 2147483644), > not a FIXNUM. > ; The result is a (VALUES (INTEGER -2684354560 2684354555) > ; &OPTIONAL), not a (VALUES FIXNUM > &REST T). > ; unable to do inline (signed-byte 32) arithmetic (cost 5) > because: > ; The result is a (VALUES (INTEGER -2684354560 2684354555) > ; &OPTIONAL), not a (VALUES (SIGNED- > BYTE 32) &REST > ; T). > ; etc. > > ; (ELEPHANT-MEMUTIL:BUFFER-READ-INT32 ELEPHANT- > SERIALIZER2::BSTREAM) > ; --> BLOCK LET SETF LET* MULTIPLE-VALUE-BIND LET FUNCALL SB-C:: > %FUNCALL > ; --> # MEMUTIL::BUFFER-STREAM-POSITION) :TYPE # (FUNCTION (FIXNUM ELEPHANT-MEMUTIL:BUFFER-STREAM) (VALUES FIXNUM > &OPTIONAL))> :WHERE-FROM :DECLARED :KIND :GLOBAL-FUNCTION {12F41DA9}> > ; --> LET LET SB-KERNEL:%INSTANCE-SET > ; ==> > ; (THE FIXNUM #:ONCE-ONLY-494) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 0) > ; --> UFFI:DEREF-ARRAY DEREF + + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of GENERIC-+ > > ; (SETF ELEPHANT-SERIALIZER2::CODE > ; (DPB (ELEPHANT-SERIALIZER2::NEXT-BYTE 0) (BYTE 8 24) 0)) > ; --> SETQ > ; ==> > ; (THE # > ; (DPB (ELEPHANT-SERIALIZER2::NEXT-BYTE 0) (BYTE 8 24) 0)) > ; > ; note: doing unsigned word to integer coercion (cost 20), for: > ; the first argument of CHECK-FIXNUM > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 1) > ; --> UFFI:DEREF-ARRAY DEREF + + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of GENERIC-+ > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 2) > ; --> UFFI:DEREF-ARRAY DEREF + + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of GENERIC-+ > > ; (ELEPHANT-SERIALIZER2::NEXT-BYTE 3) > ; --> UFFI:DEREF-ARRAY DEREF + + > ; ==> > ; (+ (* ELEPHANT-SERIALIZER2::I 4) ELEPHANT-SERIALIZER2::POS) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the first argument of GENERIC-+ > > ; (INCF > ; (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION ELEPHANT- > SERIALIZER2::BSTREAM) > ; (* LENGTH 4)) > ; --> LET* > ; ==> > ; (+ (ELEPHANT-MEMUTIL::BUFFER-STREAM-POSITION #:G487) #:G488) > ; > ; note: doing signed word to integer coercion (cost 20), for: > ; the second argument of GENERIC-+ > > ; /Users/jensteich/Lisp/elephant/src/elephant/unicode2.fasl written > ; compilation finished in 0:00:01 > WARNING: > COMPILE-FILE warned while performing # on > #. > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/ > migrate.lisp" (written 12 APR 2007 04:47:33 AM): > ; compiling (IN-PACKAGE "ELEPHANT") > ; compiling (DEFGENERIC MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFUN COPY-CINDEX-CONTENTS ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFVAR *INHIBIT-SLOT-COPY* ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFUN COPY-PERSISTENT-OBJECT ...) > ; compiling (DEFUN INHIBIT-INDEXED-SLOT-COPY? ...) > ; compiling (DEFUN COPY-PERSISTENT-SLOTS ...) > ; compiling (DEFMACRO WITH-INHIBITED-SLOT-COPY ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD COPY-BTREE-CONTENTS ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFMETHOD MIGRATE ...) > ; compiling (DEFVAR *OID-HASH* ...) > ; compiling (DEFVAR *OID-STORE* ...) > ; compiling (DEFVAR *OID-SPEC* ...) > ; compiling (DEFVAR *OID-BTREE* ...) > ; compiling (DEFUN SET-OID-SPEC ...) > ; compiling (DEFUN INITIALIZE-MIGRATE-DUPLICATE-DETECTION ...) > ; compiling (DEFUN CLEAR-MIGRATE-DUPLICATE-DETECTION ...) > ; compiling (DEFUN OBJECT-WAS-COPIED-P ...) > ; compiling (DEFUN REGISTER-COPIED-OBJECT ...) > ; compiling (DEFUN RETRIEVE-COPIED-OBJECT ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/migrate.fasl written > ; compilation finished in 0:00:00 > STYLE-WARNING: > Implicitly creating new generic function COPY-BTREE-CONTENTS. > ; compiling file "/Users/jensteich/Lisp/elephant/src/elephant/data- > store-api.lisp" (written 21 APR 2007 07:22:50 PM): > ; compiling (IN-PACKAGE :CL-USER) > ; compiling (DEFMACRO DEFPACKAGE-IMPORT-EXPORTED ...) > ; compiling (DEFPACKAGE-IMPORT-EXPORTED :ELEPHANT-DATA-STORE ...) > > ; /Users/jensteich/Lisp/elephant/src/elephant/data-store-api.fasl > written > ; compilation finished in 0:00:00 > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/collections.lisp > ; in: DEFMETHOD ELEPHANT:MAP-BTREE (T ELEPHANT:BTREE) > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::KEY ELEPHANT::VALUE) > ; (COND > ; (ELEPHANT::VALUE-SET-P > ; (ELEPHANT:CURSOR-SET ELEPHANT::CURS ELEPHANT::VALUE)) > ; ((AND (NOT ELEPHANT::FROM-END) (NULL ELEPHANT::START)) > ; (ELEPHANT:CURSOR-FIRST ELEPHANT::CURS)) > ; ((AND ELEPHANT::FROM-END (NULL ELEPHANT::END)) > ; (ELEPHANT:CURSOR-LAST ELEPHANT::CURS)) > ; (T > ; (IF ELEPHANT::FROM-END > ; (ELEPHANT:CURSOR-SET-RANGE ELEPHANT::CURS ELEPHANT::END) > ; (ELEPHANT:CURSOR-SET-RANGE ELEPHANT::CURS > ELEPHANT::START)))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF ELEPHANT::EXISTS? (FUNCALL ELEPHANT::FN ELEPHANT::KEY > ELEPHANT::VALUE) > ; (RETURN-FROM ELEPHANT:MAP-BTREE NIL)) > ; (LOOP > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::K ELEPHANT::V) > ; (IF ELEPHANT::FROM-END (ELEPHANT:CURSOR-PREV > ELEPHANT::CURS) > ; (ELEPHANT:CURSOR-NEXT ELEPHANT::CURS)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF (AND ELEPHANT::EXISTS? #) > ; (FUNCALL ELEPHANT::FN ELEPHANT::K ELEPHANT::V) (RETURN > NIL))))) > ; --> MULTIPLE-VALUE-CALL > ; ==> > ; #'(LAMBDA > ; (&OPTIONAL (ELEPHANT::EXISTS?) (ELEPHANT::KEY) > (ELEPHANT::VALUE) > ; &REST #:G232) > ; (DECLARE (IGNORE #:G232)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF ELEPHANT::EXISTS? > ; (FUNCALL ELEPHANT::FN ELEPHANT::KEY ELEPHANT::VALUE) > ; (RETURN-FROM ELEPHANT:MAP-BTREE NIL)) > ; (LOOP > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::K ELEPHANT::V) > ; (IF ELEPHANT::FROM-END (ELEPHANT:CURSOR-PREV > ELEPHANT::CURS) > ; (ELEPHANT:CURSOR-NEXT ELEPHANT::CURS)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF (AND ELEPHANT::EXISTS? #) > ; (FUNCALL ELEPHANT::FN ELEPHANT::K ELEPHANT::V) > (RETURN NIL))))) > ; > ; caught WARNING: > ; undefined variable: ELEPHANT::K > > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex.lisp > ; in: DEFUN ELEPHANT:MAP-CLASS > ; (FLET ((ELEPHANT::MAP-FN (ELEPHANT::K ELEPHANT::V) > ; (DECLARE (IGNORE ELEPHANT::K)) > ; (FUNCALL ELEPHANT::FN ELEPHANT::V))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::MAP-FN)) > ; (ELEPHANT:MAP-BTREE #'ELEPHANT::MAP-FN ELEPHANT::CLASS- > IDX :COLLECT > ; ELEPHANT::COLLECT)) > ; > ; caught WARNING: > ; undefined variable: ELEPHANT::MAP-FN > > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/controller.lisp > ; in: DEFUN ELEPHANT:ADD-TO-ROOT > ; (DEFUN ELEPHANT:ADD-TO-ROOT > ; (ELEPHANT::KEY ELEPHANT::VALUE > ; &KEY (ELEPHANT::SC ELEPHANT:*STORE-CONTROLLER*)) > ; "Add an arbitrary persistent thing to the root, so you can > ; retrieve it in a later session. Anything referenced by an > ; object added to the root is considered reachable and thus live" > ; (DECLARE (TYPE ELEPHANT:STORE-CONTROLLER ELEPHANT:STORE- > CONTROLLER)) > ; (ASSERT (NOT (EQ ELEPHANT::KEY ELEPHANT::*ELEPHANT-PROPERTIES- > LABEL*))) > ; (SETF (ELEPHANT:GET-VALUE ELEPHANT::KEY > ; (ELEPHANT:CONTROLLER-ROOT ELEPHANT::SC)) > ; ELEPHANT::VALUE)) > ; --> PROGN EVAL-WHEN > ; ==> > ; (SB-IMPL::%DEFUN 'ELEPHANT:ADD-TO-ROOT > ; (SB-INT:NAMED-LAMBDA ELEPHANT:ADD-TO-ROOT > ; (ELEPHANT::KEY > ELEPHANT::VALUE &KEY > ; (ELEPHANT::SC > ; ELEPHANT:*STORE- > CONTROLLER*)) > ; (DECLARE > ; (TYPE ELEPHANT:STORE- > CONTROLLER > ; ELEPHANT:STORE- > CONTROLLER)) > ; (BLOCK ELEPHANT:ADD-TO-ROOT > ; (ASSERT (NOT #)) > ; (SETF (ELEPHANT:GET-VALUE > ; ELEPHANT::KEY #) > ; ELEPHANT::VALUE))) > ; "Add an arbitrary persistent thing to the root, > so you can > ; retrieve it in a later session. Anything referenced by an > ; object added to the root is considered reachable and thus live" > ; 'NIL (SB-C:SOURCE-LOCATION)) > ; > ; caught WARNING: > ; undefined variable: ELEPHANT:STORE-CONTROLLER > > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/collections.lisp > ; in: DEFMETHOD ELEPHANT:MAP-BTREE (T ELEPHANT:BTREE) > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::KEY ELEPHANT::VALUE) > ; (COND > ; (ELEPHANT::VALUE-SET-P > ; (ELEPHANT:CURSOR-SET ELEPHANT::CURS ELEPHANT::VALUE)) > ; ((AND (NOT ELEPHANT::FROM-END) (NULL ELEPHANT::START)) > ; (ELEPHANT:CURSOR-FIRST ELEPHANT::CURS)) > ; ((AND ELEPHANT::FROM-END (NULL ELEPHANT::END)) > ; (ELEPHANT:CURSOR-LAST ELEPHANT::CURS)) > ; (T > ; (IF ELEPHANT::FROM-END > ; (ELEPHANT:CURSOR-SET-RANGE ELEPHANT::CURS ELEPHANT::END) > ; (ELEPHANT:CURSOR-SET-RANGE ELEPHANT::CURS > ELEPHANT::START)))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF ELEPHANT::EXISTS? (FUNCALL ELEPHANT::FN ELEPHANT::KEY > ELEPHANT::VALUE) > ; (RETURN-FROM ELEPHANT:MAP-BTREE NIL)) > ; (LOOP > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::K ELEPHANT::V) > ; (IF ELEPHANT::FROM-END (ELEPHANT:CURSOR-PREV > ELEPHANT::CURS) > ; (ELEPHANT:CURSOR-NEXT ELEPHANT::CURS)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF (AND ELEPHANT::EXISTS? #) > ; (FUNCALL ELEPHANT::FN ELEPHANT::K ELEPHANT::V) (RETURN > NIL))))) > ; --> MULTIPLE-VALUE-CALL > ; ==> > ; #'(LAMBDA > ; (&OPTIONAL (ELEPHANT::EXISTS?) (ELEPHANT::KEY) > (ELEPHANT::VALUE) > ; &REST #:G232) > ; (DECLARE (IGNORE #:G232)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF ELEPHANT::EXISTS? > ; (FUNCALL ELEPHANT::FN ELEPHANT::KEY ELEPHANT::VALUE) > ; (RETURN-FROM ELEPHANT:MAP-BTREE NIL)) > ; (LOOP > ; (MULTIPLE-VALUE-BIND > ; (ELEPHANT::EXISTS? ELEPHANT::K ELEPHANT::V) > ; (IF ELEPHANT::FROM-END (ELEPHANT:CURSOR-PREV > ELEPHANT::CURS) > ; (ELEPHANT:CURSOR-NEXT ELEPHANT::CURS)) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::EXISTS? ELEPHANT::K > ELEPHANT::V)) > ; (IF (AND ELEPHANT::EXISTS? #) > ; (FUNCALL ELEPHANT::FN ELEPHANT::K ELEPHANT::V) > (RETURN NIL))))) > ; > ; caught WARNING: > ; undefined variable: ELEPHANT::V > > > ; file: /Users/jensteich/Lisp/elephant/src/elephant/classindex.lisp > ; in: DEFUN ELEPHANT:MAP-INVERTED-INDEX > ; (FLET ((ELEPHANT::WRAPPER (ELEPHANT::KEY ELEPHANT::VALUE > ELEPHANT::PKEY) > ; (DECLARE (IGNORE ELEPHANT::PKEY)) > ; (FUNCALL ELEPHANT::FN ELEPHANT::KEY ELEPHANT::VALUE))) > ; (DECLARE (DYNAMIC-EXTENT ELEPHANT::WRAPPER)) > ; (IF ELEPHANT::VALUE-P > ; (ELEPHANT:MAP-INDEX #'ELEPHANT::WRAPPER > ELEPHANT::INDEX :VALUE > ; ELEPHANT::VALUE :COLLECT > ELEPHANT::COLLECT) > ; (ELEPHANT:MAP-INDEX #'ELEPHANT::WRAPPER > ELEPHANT::INDEX :START > ; ELEPHANT::START :END > ELEPHANT::END :FROM-END > ; ELEPHANT::FROM-END :COLLECT > ELEPHANT::COLLECT))) > ; > ; caught WARNING: > ; undefined variable: ELEPHANT::WRAPPER > > ; > ; caught WARNING: > ; These variables are undefined: > ; ELEPHANT::K ELEPHANT::MAP-FN ELEPHANT:STORE-CONTROLLER > ELEPHANT::V ELEPHANT::WRAPPER > ; > ; compilation unit finished > ; caught 6 WARNING conditions > ; caught 7 STYLE-WARNING conditions > ; printed 505 notes > NIL > CL-USER> (use-package :elephant) > T > CL-USER> (in-package :elephant-user) > # > ELE-USER>_______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Sat May 17 01:37:52 2008 From: read at robertlread.net (Robert L. Read) Date: Fri, 16 May 2008 20:37:52 -0500 Subject: [elephant-devel] Data Collection Management/Prevalence. In-Reply-To: <60D907E5-DC83-48C5-BF05-DA9A2109FAC4@media.mit.edu> References: <61950.88.73.247.43.1210701819.squirrel@mail.stardawn.org> <13D6C445-BBFA-4AA6-A6AB-54A75EE4B1E4@media.mit.edu> <1210773234.23948.47.camel@penguin.yourdomain.com> <1210821332.23948.110.camel@penguin.yourdomain.com> <8A9B375E3FCA40B1A3887976B80A9399@killer> <1210907177.29423.45.camel@penguin.yourdomain.com> <3A11F6AA4E744DD783A8307AE9732C2A@killer> <60D907E5-DC83-48C5-BF05-DA9A2109FAC4@media.mit.edu> Message-ID: <1210988272.29423.165.camel@penguin.yourdomain.com> On Fri, 2008-05-16 at 11:26 -0400, Ian Eslick wrote: > Prevalence is much, much faster because you don't have to flush data > structures on each commit, so cl-prevalence performance with > Elephant's data and transaction abstractions would be a really nice > design point. I wonder if we would get some of this benefit from a > Rucksack adaptation? I'd like to take this opportunity to explain something. We already have a "prevalence-ish" auxilliary system in the contrib/rread/dcm directory. This implements what you might call "prevalence"-style caching --- everything is stored in a hashtable, and any writes or additions are written immediately to Elephant (which ends up doing the write I/O) before control is returned. It's also thread-safe, I think. I wrote it right after I got the CL-SQL backend working. That is in fact part of the reason that I never worried about making the CL-SQL backend faster -- the caching took care of almost all of my needs. I didn't mind paying for the writes, since each one typically was in response to a human being clicking a browser button, and the writes were certainly faster than that. I called this system "DCM" for Data Collection Managment. In fact it implements what you might call a Tier 2 or "Business Object" cache. It writes objects directly to btrees, and creates its own keys. In a way, it does what Ian's class-based persistence does. However, I haven't touted because it isn't very good. It has the following drawbacks: 1) I was relatively new to LISP when I wrote it, 2) It makes no use of Ian's new stuff (which is newer than DCM), 3) It uses SBCL-specific locking, 4) It does not have a lot of tests, 5) It does not allow limitations to the cache size --- it assumes you have enough memory for those classes for which you use the in-memory caching strategy, 6) I think its use of btrees could be pretty bad, 7) it might not run now, as it has not been under test for a while. However, the one thing it does really well, that I don't think we have at the current level, is object-level caching. (If I am wrong about that, please enlighten me.) I haven't looked at Rucksack yet, but I would venture that if we wanted to write a Prevalence-style system, we could examine DCM and either improve it or take some of its approach as a starting point. One way in which this would differ from Prevalence is that this a "write-through cache", not a "journaling transaction system." A true journaling system would have less I/O and would allow more control of checkpointing strategy. One stylistic point that I'm undecided on is which of these styles is better: 1) You have an explicit "manager" or "director" that is responsible for Create, Read, Update Delete (CRUD) operations on a class of objects. The managed class is not itself inherently persistent; it is persisted when you call "register" on the managed object via the "manager". When you instantiate a manager, you specify a caching strategy by subclassing a manager class that follows a particular strategy. 2) You use the MOP to make a class really intelligent, and let it be re-defclassed with different settings, and you implement lots of slot keywords to say which slots are transient, persistent, etc., and in general think of the "class", when treated as an actual data value as it is in lisp, as responsible for caching and persistence, thus doing the same job that the "manager" does in the other model. What Elephant has now is the latter; DCM is the former. DCM will be familiar to your typical Java/C# software engineer. Given all of the discussion around performance, it is hard for me to personally sort out how important performance really is, and how best to get it, not because it is particularly hard, but because we don't seem to have anybody with an urgent use-case, and because object-level caching is so effective (at least for me.) So I'll go out on a limb and say that offering object-level caching is the single biggest performance enhancement we make for the most common cases. If we agree with that, then we can start to imagine how we would most like to implement this. I personally don't have any objection to the "manager" pattern as a discrete object. However, I think most of our users are happier with the "defpclass" approach. So I think the ideal situation would be to expand the "defpclass" macro to allow one to specify a caching strategy, and the parameters that control such a strategy. Whether this should be slot-based or object based or some combination, I'm not sure. It ought to be quite simple to create some performance tests that clarify the performance of this approach. However, I don't know if this is more important than a native-lisp backend, or a query-language. For the next year at least I am working at a job rather than working on my lisp application; and even then I was happy with the performance I was getting out of DCM. So I personally don't have performance need that drives anything. I wish I knew how many new users we would have from better performance vs. a native-lisp backend vs. a query-language, or what our existing users would prefer. From eslick at media.mit.edu Sat May 17 03:22:46 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Fri, 16 May 2008 23:22:46 -0400 Subject: [elephant-devel] Discussions Message-ID: <1B4EEC8D-D4DF-4FFC-8708-1F9EBA384C23@media.mit.edu> Robert said: > I'll go out on a limb and say that offering object-level caching is > the single biggest performance enhancement we make for the most common > cases. A clarifying question. How did you ensure ACID properties in the DCM scenario in the presence of threading? Without letting BDB or sql know about the reads that you've done, you can't tell if a prior transaction has clobbered on data that you are currently using because the reads are directly from memory. e.g. You can easily read the old 'balance' on the checking account, do your computing while someone else has written that same object, then write back an incorrect value. Rucksack tracks changes by versioning objects in memory and rolling back newer versions when older versions are committed. This is a copy- on-write model which keeps everything in memory during the transaction, but then writes the txn log and a version of the object to disk, updating the in-memory 'valid' version as appropriate. Leslie had a good related e-mail on this topic a few days ago: > I don't know what the best decision might be here. > But I have a use case that might help; it has the following > features: > > * I access the slots of two persistent objects. > > * The number of the slots and the times requested > together produce very bad performance (think seconds) > even with PM txn caching (for comparison, BDB is about > three times faster) > > * The environment is multi-threaded (web server), but the > slots won't be changed by any other process. > > * Ideally the slots would be cached only for this one > function and the functions called by it (and only > per-invocation, i.e. slot caches get refreshed right at > the beginning of the function). > > * This is currently the only place in my app where I would > need the performance advantages of slot caching. In all > other places ACID is highly preferred and speed is sufficient. > > * The desired behaviour can be somewhat modelled by CLSQL's > OO interface: > > - get the objects from the DB at the beginning > > - work with those in-memory objects > > - write back the values to the DB at the end of the process > > The difference is that I don't want the whole object (other slot > values of it might be changed from outside!) but only a few > selected slots. I think we can basically do this today. A refresh command simply reads from the DB for all cached slots (in a transaction this is thread safe and avoids the aforementioned problem). You operate on the cached data, nothing happens in the transaction, at the end you do a 'save' and those cached slots get written to disk. I think this meets leslie's use case and I think it's an hour or two to implement on top of what is already there. > However, I don't know if this is more important than a native-lisp > backend, or a query-language. For the next year at least I am working > at a job rather than working on my lisp application; and even then I was > happy with the performance I was getting out of DCM. So I personally > don't have performance need that drives anything. I wish I knew how > many new users we would have from better performance vs. a native- lisp > backend vs. a query-language, or what our existing users would prefer. My two dollars on this topic is that the most interesting thing to improve adoption and overall utility is a lisp-only backend to get going with. The most interesting value to the current users, including myself, is a query system that manages and abstracts some of the performance query hacks that today you have to write yourself in lisp, often over and over. I think of the query system, by the way, as a DSL (domain specific language) extension of lisp, not a SQL syntax. So it's not an either or, it's exactly what Lisp was meant to do, enable linguistic abstraction that makes thinking about a given problem easier. That's what I think when I hear 'lisp as the query language'. Rucksack strikes me as the best way to start on the lisp-only front, because so much is there. It's a non-trivial port/adaptation so someone needs to be willing to put in a week or two (at least) of serious effort. I think we may also be able to change it so that it only writes a transaction log and doesn't write the underlying DB unless something is flushed from the cache. What I like about Rucksack for a more prevalence style model (and maybe I'm misreading this and it's not flushing objects to disk on each write) is that it already implements versioning as its transaction model, which gets around fine-grained locking performance problems. If we add in Robert's DCM ideas about having a cache instead of the whole DB in memory, then we could imagine writing flushed objects to disk and effectively incrementally syncing the memory objects to disk rather than having to do a full snapshot every so often. Regards, Ian Ian From eslick at media.mit.edu Sat May 17 15:54:39 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 17 May 2008 11:54:39 -0400 Subject: [elephant-devel] Cached objects Message-ID: <20D9470B-D704-4305-8A8A-0FBC21741DED@media.mit.edu> As a quick study, I'm prototyping the following model for cached slots. Turns out to be fairly easy to implement. Objects that have cached slots have an extra boolean that indicates whether the cached slots are caching or not. When you turn on caching, all read writes are strictly to memory, when you turn caching off they behave like normal slots. You can save the contents to disk without turning off caching. ;; Can you think of the right names for the refresh & save commands? (refresh instance) - refresh all cached slots on the instance from the store (enable-caching instance) - tell the object to perform cached reads/ writes (save instance) - push all slots to disk (disable-caching instance) - tell the object to behave like a standard instance. Flushes to disk. *cached-instance-default-mode* (will set the caching mode for new instances automatically, will not effect existing instances) ;; Convenience macro (with-caching (:type instance list) all cached slots of the instances are cached, followed by a save on exit ) Transaction Behavior: --------------------- If you use the discipline of using with-caching before the first read of a cached instance's slot inside with-transaction, then you get the following cool behavior in a multi-threaded scenario: - txn one reads the cached values (setting read locks) - txn one does various ops in memory - txn one then commits the instance values (grabbing write locks) - txn two in running in parallel and does refresh, grabbin read locks - txn two commits after txn one and is aborted and restarted. - txn two is guaranteed to get the latest data by virtue of the refresh. In this mode of operation, I believe that we can guarantee that the ACID properties are maintained at transaction boundaries. There are caveats however: Complications with Indexing: ---------------------------- - Slot indexing is a problem. Completely cached slots will be out of sync with their index because you can't keep the index in-sync with the cached slot, so the API is broken (add instance, do query, new instance not in query result!). This can be fixed with a write- through mode on the indexed slots. Tradeoff Requests for Comment: ------------------------------ - Reads and writes could set a dirty bit on the instance. I'm debating this one. The nice thing is if all you do is read an object in a transaction, then you don't cancel other parallel transactions that are only reading. However setting dirty on each slot access isn't terribly cheap. Easy enough to profile this when we have some tests. For now, my guess is that the probability of two threads operating on the same objects is low given the web-oriented usage of elephant. - Profliferation of modes. I can now see 4 modes of operation - is this too complicated? no caching, all-in-memory, write-through caching of everything, write-through caching of indexed/cached slots. Perhaps we have a standard mode (all or nothing, no indexing allowed) and provide an advanced section for people with more complex requirements. There are also performance issues (lots of power eats lots of performance as Alex has been reminding us:) - Would anyone like to volunteer to review the API and to test this? It's fairly orthogonal to everything else in the system, so shouldn't interfere with other testing/use that is happening. Leslie? Thank you, Ian From read at robertlread.net Sat May 17 20:23:19 2008 From: read at robertlread.net (Robert L. Read) Date: Sat, 17 May 2008 15:23:19 -0500 Subject: [elephant-devel] Discussions In-Reply-To: <1B4EEC8D-D4DF-4FFC-8708-1F9EBA384C23@media.mit.edu> References: <1B4EEC8D-D4DF-4FFC-8708-1F9EBA384C23@media.mit.edu> Message-ID: <1211055799.29423.219.camel@penguin.yourdomain.com> On Fri, 2008-05-16 at 23:22 -0400, Ian Eslick wrote: > Robert said: > > > I'll go out on a limb and say that offering object-level caching is > > the single biggest performance enhancement we make for the most > common > > cases. > > A clarifying question. How did you ensure ACID properties in the DCM > scenario in the presence of threading? Without letting BDB or sql > know about the reads that you've done, you can't tell if a prior > transaction has clobbered on data that you are currently using because > the reads are directly from memory. > > e.g. You can easily read the old 'balance' on the checking account, do > your computing while someone else has written that same object, then > write back an incorrect value. I'm assuming what I consider to be a standard 3-tiered model, consisting or a Presentation Layer, and Business Logic Layer, and Persistence Layer. Follwing the Law of Demeter, the Presentation Layer is not allowed to talk to the Persistence Layer. An operation like making a journal entry in an accounting system, or more simply even just decrementing an account, is implemented by a method on the Business Logic Layer or Domain Object Layer (a "Manager" in the DCM model). Such a method is normally called a "Business Rule." Generally, Business Rules tend to demand atomicity, be loggable, and change the underlying store by calls to the Persistence Layer. How the Domain Object Layer enforces ACID rules is its own business. In practice, this would typically be done by obtaining locks on the Domain Object(s) that are being changed. Since in this model is there is exactly one official copy of the object in memory at any time, this is straightforward. If I'm decrementing a counter, the code in the Domain Object Layer obtains a lock on the conter. If I am debiting one account and crediting another in a financial transaction, I obtain a lock on both accounts. If I'm doing something more complicated, I might have to lock the entire General Ledger, but in general that should not be necessary. In this model, the Persistence Layer doesn't help you with ACID properties. This works if you assume that Domain Object Layer is the ONLY thing that can touch the database. From an organization point of view, the Manager is the central point of contact for a collection of objects (like accounts), and nobody is allowed to permanently manipulate an account except via operations defined on the Manager. The Manager can therefore be responsible for concurrency control. Concurrency at the domain object level could be done via versioning/journaling as in Rucksack, or via locks. I have never used the former, but it sounds like a great idea. > > Rucksack tracks changes by versioning objects in memory and rolling > back newer versions when older versions are committed. This is a copy- > on-write model which keeps everything in memory during the > transaction, but then writes the txn log and a version of the object > to disk, updating the in-memory 'valid' version as appropriate. > > Leslie had a good related e-mail on this topic a few days ago: > > > I don't know what the best decision might be here. > > But I have a use case that might help; it has the following > > features: > > > > * I access the slots of two persistent objects. > > > > * The number of the slots and the times requested > > together produce very bad performance (think seconds) > > even with PM txn caching (for comparison, BDB is about > > three times faster) > > > > * The environment is multi-threaded (web server), but the > > slots won't be changed by any other process. > > > > * Ideally the slots would be cached only for this one > > function and the functions called by it (and only > > per-invocation, i.e. slot caches get refreshed right at > > the beginning of the function). > > > > * This is currently the only place in my app where I would > > need the performance advantages of slot caching. In all > > other places ACID is highly preferred and speed is sufficient. > > > > * The desired behaviour can be somewhat modelled by CLSQL's > > OO interface: > > > > - get the objects from the DB at the beginning > > > > - work with those in-memory objects > > > > - write back the values to the DB at the end of the process > > > > The difference is that I don't want the whole object (other slot > > values of it might be changed from outside!) but only a few > > selected slots. > > I think we can basically do this today. A refresh command simply > reads from the DB for all cached slots (in a transaction this is > thread safe and avoids the aforementioned problem). You operate on > the cached data, nothing happens in the transaction, at the end you do > a 'save' and those cached slots get written to disk. > I think this meets leslie's use case and I think it's an hour or two > to implement on top of what is already there. > > > However, I don't know if this is more important than a native-lisp > > backend, or a query-language. For the next year at least I am > working > > at a job rather than working on my lisp application; and even then > I was > > happy with the performance I was getting out of DCM. So I personally > > don't have performance need that drives anything. I wish I knew how > > many new users we would have from better performance vs. a native- > lisp > > backend vs. a query-language, or what our existing users would > prefer. > > My two dollars on this topic is that the most interesting thing to > improve adoption and overall utility is a lisp-only backend to get > going with. The most interesting value to the current users, > including myself, is a query system that manages and abstracts some of > the performance query hacks that today you have to write yourself in > lisp, often over and over. Good. I'll support whatever gets to $5 first. :->. > > I think of the query system, by the way, as a DSL (domain specific > language) extension of lisp, not a SQL syntax. So it's not an either > or, it's exactly what Lisp was meant to do, enable linguistic > abstraction that makes thinking about a given problem easier. That's > what I think when I hear 'lisp as the query language'. Right. LISP generally does that by the addition of symbols to the language, not the taking away of them. So the full power of LISP remains available, in addition to new macros/functions etc. that are convenient. > > Rucksack strikes me as the best way to start on the lisp-only front, > because so much is there. It's a non-trivial port/adaptation so > someone needs to be willing to put in a week or two (at least) of > serious effort. I agree with this. > > I think we may also be able to change it so that it only writes a > transaction log and doesn't write the underlying DB unless something > is flushed from the cache. What I like about Rucksack for a more > prevalence style model (and maybe I'm misreading this and it's not > flushing objects to disk on each write) is that it already implements > versioning as its transaction model, which gets around fine-grained > locking performance problems. If we add in Robert's DCM ideas about > having a cache instead of the whole DB in memory, then we could > imagine writing flushed objects to disk and effectively incrementally > syncing the memory objects to disk rather than having to do a full > snapshot every so often. This does sound exciting. I guess however that Rucksack offers two quite different things: A pre-exising Btree implementation, and an in-memory versioning concurrency control model. I assume that you are mentioning these together because the Rucksack Btree by itself would not give us an ACID transaction model, as we currently have with the other backends. > > Regards, > Ian > > > > > > Ian > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Sat May 17 20:37:25 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 17 May 2008 16:37:25 -0400 Subject: [elephant-devel] Discussions In-Reply-To: <1211055799.29423.219.camel@penguin.yourdomain.com> References: <1B4EEC8D-D4DF-4FFC-8708-1F9EBA384C23@media.mit.edu> <1211055799.29423.219.camel@penguin.yourdomain.com> Message-ID: <5BFF0E05-F411-4FB9-8C6C-DA48E6DD4D81@media.mit.edu> > > In this model, the Persistence Layer doesn't help you with ACID > properties. This works if you assume that Domain Object Layer is the > ONLY thing that can touch the database. From an organization point of > view, the Manager is the central point of contact for a collection of > objects (like accounts), and nobody is allowed to permanently > manipulate > an account except via operations defined on the Manager. The Manager > can therefore be responsible for concurrency control. > > Concurrency at the domain object level could be done via > versioning/journaling as in Rucksack, or via locks. I have never used > the former, but it sounds like a great idea. > I hadn't realized that you were relying on the Manager to lock objects, and that only one can be used at a time when 'checked out'. >> > This does sound exciting. I guess however that Rucksack offers two > quite different things: A pre-exising Btree implementation, and an > in-memory versioning concurrency control model. I assume that you are > mentioning these together because the Rucksack Btree by itself would > not > give us an ACID transaction model, as we currently have with the other > backends. That is correct. The data store is responsible for ensuring, within a transaction body, all of the ACID properties for any access to persistent objects (unless you are using caching, see prior e-mail). Ian From eslick at media.mit.edu Sat May 17 21:34:04 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sat, 17 May 2008 17:34:04 -0400 Subject: [elephant-devel] Cached slots Message-ID: <04C0587A-590F-4FE6-B24D-8687F6247C6B@media.mit.edu> I just pushed a patch implementing the cached-slot model I described earlier. I haven't fully exercised it, but it basically works. You can simply tell an instance what you want it to do with slots by setting (setf (cache-mode instance) :all-cached) Other options are :none-cached, :write-through and :index-write-through. Index cached slots are not supported, but will be easy to add later. Feel free to play - Leslie I'd love to see if this fits your use case. This may give us many of the benefits of prevalence / managed objects. If you used the all-cached option, then you can have a very long running transaction without exhausting, for example, BDB cache memory or locks. Ian From killerstorm at newmail.ru Sun May 18 10:05:59 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Sun, 18 May 2008 13:05:59 +0300 Subject: [elephant-devel] Cached objects References: <20D9470B-D704-4305-8A8A-0FBC21741DED@media.mit.edu> Message-ID: <55AEFCC85F9A4DCF8D1DE796459D41B6@killer> IE> If you use the discipline of using with-caching before the first read IE> of a cached instance's slot inside with-transaction, then you get the IE> following cool behavior in a multi-threaded scenario: IE> - txn one reads the cached values (setting read locks) IE> - txn one does various ops in memory IE> - txn one then commits the instance values (grabbing write locks) IE> - txn two in running in parallel and does refresh, grabbin read locks IE> - txn two commits after txn one and is aborted and restarted. IE> In this mode of operation, I believe that we can guarantee that the IE> ACID properties are maintained at transaction boundaries. how is that? both txn one and two grab read locks, so they don't block each other, right? then txn one modifies slots, and txn two immidiately sees changes, that breaks isolation, doesn't this? only way to avoid collisions is to make even read locks exclusive, but that will totally kill paralellism -- even read-only transactions working with same objects will not be running concurrently. i believe that "the right way" to do such stuff is thread/transaction local cache -- i.e. each transactions bounds *slot-cache* (e.g. hash-table), and all slot reads/writes work with it. slot reads/modifications performance will be somewhat inferior, but it won't break concurrency. IE> - Slot indexing is a problem. Completely cached slots will be out of IE> sync with their index because you can't keep the index in-sync with IE> the cached slot, so the API is broken (add instance, do query, new IE> instance not in query result!). This can be fixed with a write- IE> through mode on the indexed slots. partially this can be fixed by doing caching on btree level rather than on slot level. From leslie.polzer at gmx.net Sun May 18 13:05:58 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sun, 18 May 2008 15:05:58 +0200 (CEST) Subject: [elephant-devel] Problem on the upgrade path Message-ID: <61504.88.73.220.54.1211115958.squirrel@mail.stardawn.org> Loading a BDB database created by stable with the new branch gives me: debugger invoked on a UNBOUND-SLOT in thread #: The slot DB-BDB::INDICES is unbound in the object #. Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL. restarts (invokable by number or by possibly-abbreviated name): 0: [USE-VALUE ] Return a value as the slot-value. 1: [STORE-VALUE] Store and return a value as the slot-value. 2: [ABORT ] Exit debugger, returning to top level. ((SB-PCL::FAST-METHOD SLOT-UNBOUND (T T T)) # # # # DB-BDB::INDICES) 0] :ba 0: ((SB-PCL::FAST-METHOD SLOT-UNBOUND (T T T)) # # # # DB-BDB::INDICES) 1: ((SB-PCL::FAST-METHOD ELEPHANT::PERSISTENT-SLOT-READER (DB-BDB::BDB-STORE-CONTROLLER T T)) # # # # #) 2: (SLOT-VALUE # DB-BDB::INDICES) 3: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER (DB-BDB::BDB-INDEXED-BTREE T)) #)[:EXTERNAL] 4: ((LAMBDA (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB-PCL::.ARG1. SB-INT:&MORE SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.)) # # # (ELEPHANT::OID ELEPHANT::SPEC ELEPHANT:CACHE-MODE DB-BDB::INDICES-CACHE) -304096854 4) 5: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AROUND (ELEPHANT:PERSISTENT-OBJECT T)) # # # #)[:EXTERNAL] 6: ((LAMBDA (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB-INT:&MORE SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.)) # # # -304096814 4) 7: ((SB-PCL::FAST-METHOD MAKE-INSTANCE (CLASS)) # # #)[:EXTERNAL] 8: ((LAMBDA ()) #) 9: ((SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-BDB::BDB-STORE-CONTROLLER T)) # # # #)[:EXTERNAL] 10: ((SB-PCL::FAST-METHOD ELEPHANT::OPEN-CONTROLLER (DB-BDB::BDB-STORE-CONTROLLER)) #)[:EXTERNAL] 11: ((LAMBDA (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB-INT:&MORE SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.)) # # # -304096650 6) 12: (ELEPHANT:OPEN-STORE (:BDB "/home/sky/projects/mystic/backup-world.bdb/"))[:EXTERNAL] 13: (MYSTIC-ENGINE:START)[:EXTERNAL] 14: (SB-INT:SIMPLE-EVAL-IN-LEXENV (MYSTIC-ENGINE:START) #) 15: (INTERACTIVE-EVAL (MYSTIC-ENGINE:START)) 16: (SB-ACLREPL::REP-ONE) 17: (SB-ACLREPL::REPL)[:EXTERNAL] 18: ((LAMBDA (SB-ACLREPL::NOPRINT)) NIL) 19: ((LAMBDA ())) 20: ((LAMBDA ()))[:EXTERNAL] 21: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX #) 22: (SB-IMPL::TOPLEVEL-REPL NIL) 23: (SB-IMPL::TOPLEVEL-INIT) 24: ((LABELS SB-IMPL::RESTART-LISP)) Amending SHARED-INITIALIZE :AFTER in bdb-collections.lisp to read: (defmethod shared-initialize :after ((instance bdb-indexed-btree) slot-names &rest rest) (declare (ignore slot-names rest)) (unless (slot-boundp instance 'indices) (setf (indices instance) (make-hash-table))) (setf (indices-cache instance) (indices instance))) seems to do the job. How I hate uninitialized slots. The next issue I encountered was the absence of ADD-CLASS-DERIVED-INDEX. So this is really gone now? What's the replacement? I'm going to answer your messages regarding caching later today, Ian. Leslie From eslick at media.mit.edu Sun May 18 13:33:34 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sun, 18 May 2008 09:33:34 -0400 Subject: [elephant-devel] Problem on the upgrade path In-Reply-To: <61504.88.73.220.54.1211115958.squirrel@mail.stardawn.org> References: <61504.88.73.220.54.1211115958.squirrel@mail.stardawn.org> Message-ID: <8045AD89-89BC-4DCA-AD5A-1CB3E8C4EEA3@media.mit.edu> That's interesting. I'll have to look into that further because any existing database should either have the btree already or create it on startup... yes, add-class-derived-index has been deprecated in favor of a defclass form. There is a more detailed e-mail I sent out on this awhile back with some of the tradeoffs and the rationale behind the decisions. ((slot1 :accessor slot1) (slot2 :accessor slot2) (dslot :accessor dslot :derived-fn 'my-fn :slots-deps (slot1 slot2))) my-fn is the usual derived index key function. The property that I wanted most is that classes in the store will always be updated to match the current defclass definition. Thanks, Ian On May 18, 2008, at 9:05 AM, Leslie P. Polzer wrote: > > Loading a BDB database created by stable with the new branch gives me: > > debugger invoked on a UNBOUND-SLOT in thread # thread" {A725821}>: > The slot DB-BDB::INDICES is unbound in the object # BTREE oid:-3>. > > Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL. > > restarts (invokable by number or by possibly-abbreviated name): > 0: [USE-VALUE ] Return a value as the slot-value. > 1: [STORE-VALUE] Store and return a value as the slot-value. > 2: [ABORT ] Exit debugger, returning to top level. > > ((SB-PCL::FAST-METHOD SLOT-UNBOUND (T T T)) > # > # > # > # > DB-BDB::INDICES) > 0] :ba > > 0: ((SB-PCL::FAST-METHOD SLOT-UNBOUND (T T T)) > # > # > # > # > DB-BDB::INDICES) > 1: ((SB-PCL::FAST-METHOD ELEPHANT::PERSISTENT-SLOT-READER > (DB-BDB::BDB-STORE-CONTROLLER T T)) > # > # > # > # > #) > 2: (SLOT-VALUE # DB-BDB::INDICES) > 3: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER > (DB-BDB::BDB-INDEXED-BTREE T)) #) > [:EXTERNAL] > 4: ((LAMBDA > (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB- > PCL::.ARG1. > SB-INT:&MORE SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE- > COUNT.)) > # > # > # > (ELEPHANT::OID ELEPHANT::SPEC ELEPHANT:CACHE-MODE DB-BDB::INDICES- > CACHE) > -304096854 > 4) > 5: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AROUND > (ELEPHANT:PERSISTENT-OBJECT T)) > # > # > # > #)[:EXTERNAL] > 6: ((LAMBDA > (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB- > INT:&MORE > SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.)) > # > # > # > -304096814 > 4) > 7: ((SB-PCL::FAST-METHOD MAKE-INSTANCE (CLASS)) > # > # > #) > [:EXTERNAL] > 8: ((LAMBDA ()) #) > 9: ((SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION > (DB-BDB::BDB-STORE-CONTROLLER T)) > # > # > # > #)[:EXTERNAL] > 10: ((SB-PCL::FAST-METHOD ELEPHANT::OPEN-CONTROLLER > (DB-BDB::BDB-STORE-CONTROLLER)) #) > [:EXTERNAL] > 11: ((LAMBDA > (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB- > INT:&MORE > SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.)) > # > # > # world.bdb/> > -304096650 > 6) > 12: (ELEPHANT:OPEN-STORE (:BDB "/home/sky/projects/mystic/backup- > world.bdb/"))[:EXTERNAL] > 13: (MYSTIC-ENGINE:START)[:EXTERNAL] > 14: (SB-INT:SIMPLE-EVAL-IN-LEXENV (MYSTIC-ENGINE:START) # LEXENV>) > 15: (INTERACTIVE-EVAL (MYSTIC-ENGINE:START)) > 16: (SB-ACLREPL::REP-ONE) > 17: (SB-ACLREPL::REPL)[:EXTERNAL] > 18: ((LAMBDA (SB-ACLREPL::NOPRINT)) NIL) > 19: ((LAMBDA ())) > 20: ((LAMBDA ()))[:EXTERNAL] > 21: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX #) > 22: (SB-IMPL::TOPLEVEL-REPL NIL) > 23: (SB-IMPL::TOPLEVEL-INIT) > 24: ((LABELS SB-IMPL::RESTART-LISP)) > > Amending SHARED-INITIALIZE :AFTER in bdb-collections.lisp to read: > > (defmethod shared-initialize :after ((instance bdb-indexed-btree) > slot-names > &rest rest) > (declare (ignore slot-names rest)) > (unless (slot-boundp instance 'indices) > (setf (indices instance) (make-hash-table))) > (setf (indices-cache instance) (indices instance))) > > seems to do the job. How I hate uninitialized slots. > > The next issue I encountered was the absence of ADD-CLASS-DERIVED- > INDEX. > So this is really gone now? What's the replacement? > > I'm going to answer your messages regarding caching later today, Ian. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Sun May 18 13:33:48 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sun, 18 May 2008 09:33:48 -0400 Subject: [elephant-devel] Cached objects In-Reply-To: <55AEFCC85F9A4DCF8D1DE796459D41B6@killer> References: <20D9470B-D704-4305-8A8A-0FBC21741DED@media.mit.edu> <55AEFCC85F9A4DCF8D1DE796459D41B6@killer> Message-ID: On May 18, 2008, at 6:05 AM, Alex Mizrahi wrote: > IE> If you use the discipline of using with-caching before the first > read > IE> of a cached instance's slot inside with-transaction, then you > get the > IE> following cool behavior in a multi-threaded scenario: > IE> - txn one reads the cached values (setting read locks) > IE> - txn one does various ops in memory > IE> - txn one then commits the instance values (grabbing write locks) > > IE> - txn two in running in parallel and does refresh, grabbin read > locks > IE> - txn two commits after txn one and is aborted and restarted. > > IE> In this mode of operation, I believe that we can guarantee that > the > IE> ACID properties are maintained at transaction boundaries. > > how is that? both txn one and two grab read locks, so they don't > block each other, right? > then txn one modifies slots, and txn two immidiately sees changes, > that breaks isolation, doesn't this? > only way to avoid collisions is to make even read locks exclusive, > but that will totally kill paralellism -- even read-only > transactions working with same objects will not be running > concurrently. You are quite right! It's been a long week, so please bear with me on this. :) I was thinking about my original write-through mode, which would catch this cases as the writes will block the writer (because all read locks were grabbed at the beginning of the session). You wouldn't want to use caching mode in the presence of high degrees of sharing anyway - so if we grabbing write locks (i.e. exclusive read locks) just for the cached objects the cost wouldn't be too high in concurrency but we'd make sure these semi-independent operations were isolated. Any shared objects, especially those rarely written, should be handled in a write-through mode. I'm sure there are still holes. Would you like to help think through a policy that we could throw into a macro wrapper to make this easier. I'd like to have one or two out-of-the-box use cases for this and leave the rest to the advanced user. Hmmm...you could also engineer around some of the above tradeoffs by implementing a policy where a thread or slot is used to ensure that any thread that will side effect a set of fully cached objects first has to grab an exclusive lock on a designated slot (essentially using the DB to grab a semaphore, but ostensibly at a lower cost in time and complexity) prior to the side effecting operations in memory. > i believe that "the right way" to do such stuff is thread/ > transaction local cache -- i.e. each transactions bounds *slot- > cache* (e.g. hash-table), and all slot reads/writes work with it. > slot reads/modifications performance will be somewhat inferior, but > it won't break concurrency. I agree with you here. However, while they are constant time, most hash ops are rather expensive (it was the biggest bottleneck in the serializer) and clearing or allocating hashes on each transaction is really expensive in my experience. If you only had a dozen or so slots, even an alist would be faster... I think the usual tradeoff is a 20-deep alist search is the same as a single hash fetch. > IE> - Slot indexing is a problem. Completely cached slots will be > out of > IE> sync with their index because you can't keep the index in-sync > with > IE> the cached slot, so the API is broken (add instance, do query, new > IE> instance not in query result!). This can be fixed with a write- > IE> through mode on the indexed slots. > > partially this can be fixed by doing caching on btree level rather > than on slot level. Please explain further, I'm not sure I follow you here. Thank you for the input! Ian > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Sun May 18 13:48:38 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sun, 18 May 2008 15:48:38 +0200 (CEST) Subject: [elephant-devel] Cached slots In-Reply-To: <04C0587A-590F-4FE6-B24D-8687F6247C6B@media.mit.edu> References: <04C0587A-590F-4FE6-B24D-8687F6247C6B@media.mit.edu> Message-ID: <63508.88.73.220.54.1211118518.squirrel@mail.stardawn.org> > Feel free to play - Leslie I'd love to see if this fits your use > case. I took a look at the API, and it seems to be exactly the thing I need (and likely beyond that!). Not sure about the naming, though. I'm quite amazed at the stuff you implemented in the last few weeks. Now I only need DBPM support and freedom from coarse bugs in the new branch. Leslie From leslie.polzer at gmx.net Sun May 18 13:52:10 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sun, 18 May 2008 15:52:10 +0200 (CEST) Subject: [elephant-devel] Problem on the upgrade path In-Reply-To: <8045AD89-89BC-4DCA-AD5A-1CB3E8C4EEA3@media.mit.edu> References: <61504.88.73.220.54.1211115958.squirrel@mail.stardawn.org> <8045AD89-89BC-4DCA-AD5A-1CB3E8C4EEA3@media.mit.edu> Message-ID: <63662.88.73.220.54.1211118730.squirrel@mail.stardawn.org> > That's interesting. I'll have to look into that further because any > existing database should either have the btree already or create it on > startup... Either the approach from my last mail was faulty, or some other thing is afoul; the next open-store threw a deserialization error related to this: 0: (ELEPHANT::DESERIALIZE #S(ELEPHANT-MEMUTIL:BUFFER-STREAM :BUFFER # :SIZE 76 :POSITION 76 :LENGTH 80) # NIL) 1: ((SB-PCL::FAST-METHOD ELEPHANT::PERSISTENT-SLOT-READER (DB-BDB::BDB-STORE-CONTROLLER T T)) # # # # #) 2: (SLOT-VALUE # DB-BDB::INDICES) 3: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER (DB-BDB::BDB-INDEXED-BTREE T)) #)[:EXTERNAL] [...] > yes, add-class-derived-index has been deprecated in favor of a > defclass form. There is a more detailed e-mail I sent out on this > awhile back with some of the tradeoffs and the rationale behind the > decisions. I remember the discussion, but I could't remember anything about the new interface. Thanks for explaining! > ((slot1 :accessor slot1) > (slot2 :accessor slot2) > (dslot :accessor dslot :derived-fn 'my-fn :slots-deps (slot1 slot2))) > > my-fn is the usual derived index key function. Very good, the old API was quite clumsy to work with. Leslie From eslick at media.mit.edu Sun May 18 16:58:33 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sun, 18 May 2008 12:58:33 -0400 Subject: [elephant-devel] Problem on the upgrade path In-Reply-To: <63662.88.73.220.54.1211118730.squirrel@mail.stardawn.org> References: <61504.88.73.220.54.1211115958.squirrel@mail.stardawn.org> <8045AD89-89BC-4DCA-AD5A-1CB3E8C4EEA3@media.mit.edu> <63662.88.73.220.54.1211118730.squirrel@mail.stardawn.org> Message-ID: Ok, I have a few minutes now to look into this. You're using SBCL - on what platform? -Ian On May 18, 2008, at 9:52 AM, Leslie P. Polzer wrote: > >> That's interesting. I'll have to look into that further because any >> existing database should either have the btree already or create it >> on >> startup... > > Either the approach from my last mail was faulty, or some other thing > is afoul; the next open-store threw a deserialization error related to > this: > > 0: (ELEPHANT::DESERIALIZE > #S(ELEPHANT-MEMUTIL:BUFFER-STREAM > :BUFFER # (* > (UNSIGNED > 8 > ))> > :SIZE 76 > :POSITION 76 > :LENGTH 80) > # > NIL) > 1: ((SB-PCL::FAST-METHOD ELEPHANT::PERSISTENT-SLOT-READER > (DB-BDB::BDB-STORE-CONTROLLER T T)) > # > # > # > # > #) > 2: (SLOT-VALUE # DB-BDB::INDICES) > 3: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER > (DB-BDB::BDB-INDEXED-BTREE T)) #) > [:EXTERNAL] > > [...] > > >> yes, add-class-derived-index has been deprecated in favor of a >> defclass form. There is a more detailed e-mail I sent out on this >> awhile back with some of the tradeoffs and the rationale behind the >> decisions. > > I remember the discussion, but I could't remember anything > about the new interface. Thanks for explaining! > > >> ((slot1 :accessor slot1) >> (slot2 :accessor slot2) >> (dslot :accessor dslot :derived-fn 'my-fn :slots-deps (slot1 >> slot2))) >> >> my-fn is the usual derived index key function. > > Very good, the old API was quite clumsy to work with. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Sun May 18 17:40:23 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Sun, 18 May 2008 19:40:23 +0200 (CEST) Subject: [elephant-devel] Problem on the upgrade path In-Reply-To: References: <61504.88.73.220.54.1211115958.squirrel@mail.stardawn.org> <8045AD89-89BC-4DCA-AD5A-1CB3E8C4EEA3@media.mit.edu> <63662.88.73.220.54.1211118730.squirrel@mail.stardawn.org> Message-ID: <62790.88.73.220.54.1211132423.squirrel@mail.stardawn.org> > Ok, I have a few minutes now to look into this. You're using SBCL - > on what platform? -Ian [sky at wintermute ~]% sbcl --version SBCL 1.0.15 [sky at wintermute ~]% uname -a Linux wintermute 2.6.24-rt #1 SMP PREEMPT RT Wed May 7 16:54:52 CEST 2008 i686 AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ AuthenticAMD GNU/Linux From eslick at media.mit.edu Sun May 18 18:38:30 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Sun, 18 May 2008 14:38:30 -0400 Subject: [elephant-devel] Problem on the upgrade path In-Reply-To: <62790.88.73.220.54.1211132423.squirrel@mail.stardawn.org> References: <61504.88.73.220.54.1211115958.squirrel@mail.stardawn.org> <8045AD89-89BC-4DCA-AD5A-1CB3E8C4EEA3@media.mit.edu> <63662.88.73.220.54.1211118730.squirrel@mail.stardawn.org> <62790.88.73.220.54.1211132423.squirrel@mail.stardawn.org> Message-ID: Well I can't reproduce this in Allegro/Mac/32-bit. I'll try SBCL Mac on 1.0.16 (there were some other UFFI probs to diagnose there anyway) and see where I get. I may not get back to this until Tuesday or Wednesday if I can't figure it out quickly. Thanks, Ian On May 18, 2008, at 1:40 PM, Leslie P. Polzer wrote: > >> Ok, I have a few minutes now to look into this. You're using SBCL - >> on what platform? -Ian > > [sky at wintermute ~]% sbcl --version > SBCL 1.0.15 > > [sky at wintermute ~]% uname -a > Linux wintermute 2.6.24-rt #1 SMP PREEMPT RT Wed May 7 16:54:52 CEST > 2008 i686 AMD > Athlon(tm) 64 X2 Dual Core Processor 4200+ AuthenticAMD GNU/Linux > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From gtarcea at umich.edu Sun May 18 21:37:05 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Sun, 18 May 2008 17:37:05 -0400 Subject: [elephant-devel] Change to docstrings.lisp Message-ID: <2B0C98C2-95E8-4FEA-8D8F-C922407DE33D@umich.edu> Hi, I've started trying to make some changes to the documentation. I've run into a couple of problems. Attached is a patch to docstrings.lisp. On later versions of SBCL it appears that finalize- inheritance needs to be called before introspecting a class. The attached patch adds one line to texinfo-inferred-body: (sb-mop:finalize-inheritance (find-class name)) The documentation still doesn't completely build for me, but this is at least a step further (no lisp errors, I believe the next set are tex errors). FYI: I've never used darcs before, so if there is a better (or even correct way...) to format the patch please let me know. I used darcs record and darcs send to create the patch. Thanks, Glenn -------------- next part -------------- A non-text attachment was scrubbed... Name: docstrings.patch Type: application/octet-stream Size: 13589 bytes Desc: not available URL: From read at robertlread.net Sun May 18 23:26:59 2008 From: read at robertlread.net (Robert L. Read) Date: Sun, 18 May 2008 18:26:59 -0500 Subject: [elephant-devel] Change to docstrings.lisp In-Reply-To: <2B0C98C2-95E8-4FEA-8D8F-C922407DE33D@umich.edu> References: <2B0C98C2-95E8-4FEA-8D8F-C922407DE33D@umich.edu> Message-ID: <1211153219.29423.258.camel@penguin.yourdomain.com> That is the correct way to do it, thank you. I will apply the patch today or tomorrow. On Sun, 2008-05-18 at 17:37 -0400, Glenn Tarcea wrote: > Hi, > > I've started trying to make some changes to the documentation. I've > run into a couple of problems. Attached is a patch to > docstrings.lisp. On later versions of SBCL it appears that finalize- > inheritance needs to be called before introspecting a class. The > attached patch adds one line to texinfo-inferred-body: > > (sb-mop:finalize-inheritance (find-class name)) > > The documentation still doesn't completely build for me, but this is > at least a step further (no lisp errors, I believe the next set are > tex errors). > > FYI: I've never used darcs before, so if there is a better (or even > correct way...) to format the patch please let me know. I used darcs > record and darcs send to create the patch. > > Thanks, > > Glenn > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Tue May 20 03:26:41 2008 From: read at robertlread.net (Robert L. Read) Date: Mon, 19 May 2008 22:26:41 -0500 Subject: [elephant-devel] Change to docstrings.lisp In-Reply-To: <2B0C98C2-95E8-4FEA-8D8F-C922407DE33D@umich.edu> References: <2B0C98C2-95E8-4FEA-8D8F-C922407DE33D@umich.edu> Message-ID: <1211254001.29423.290.camel@penguin.yourdomain.com> Dear Glenn, I have applied your patch to elephant-unstable. Thank you. Unfortunately, I can not build the documentation right now due to an error in compiling defpackage in sb-posix, therefore I cannot verify that that works. What version of SBCL are you using (I'm on 1.0.13). Are the tex errors simply dangling references in the documentation? Or perhaps they are caused by another version incompatibility? On Sun, 2008-05-18 at 17:37 -0400, Glenn Tarcea wrote: > Hi, > > I've started trying to make some changes to the documentation. I've > run into a couple of problems. Attached is a patch to > docstrings.lisp. On later versions of SBCL it appears that finalize- > inheritance needs to be called before introspecting a class. The > attached patch adds one line to texinfo-inferred-body: > > (sb-mop:finalize-inheritance (find-class name)) > > The documentation still doesn't completely build for me, but this is > at least a step further (no lisp errors, I believe the next set are > tex errors). > > FYI: I've never used darcs before, so if there is a better (or even > correct way...) to format the patch please let me know. I used darcs > record and darcs send to create the patch. > > Thanks, > > Glenn > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From gtarcea at umich.edu Tue May 20 03:46:05 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Mon, 19 May 2008 23:46:05 -0400 Subject: [elephant-devel] Change to docstrings.lisp In-Reply-To: <1211254001.29423.290.camel@penguin.yourdomain.com> References: <2B0C98C2-95E8-4FEA-8D8F-C922407DE33D@umich.edu> <1211254001.29423.290.camel@penguin.yourdomain.com> Message-ID: <3BBD9029-7B40-4F6F-B948-77F6E6F32CB0@umich.edu> Hi Robert, I am on SBCL 1.0.15 on intel based Mac OSX. I built it with threading enabled. I get the following error when I go to build the documentation (make - d all): Formatting node Variable Index... Formatting node Colophon... make: *** [docs] Error 1 Got a SIGCHLD; 1 unreaped children. Reaping losing child 0x0030c6a0 PID 510 make: *** [docs] Error 1 Removing child 0x0030c6a0 PID 510 from chain. It looks like it is running the following command: makeinfo -v --html --css-include=style.css --force elephant.texinfo Also, the following example other errors appear during this processing: /Users/gtarcea/Documents/workspace/darcs/elephant-unstable/doc// reference.texinfo:88: @include `includes/fun-elephant-enable-class- indexing.texinfo': No such file or directory. And indeed that file does not exist. I apologize that I haven't had a chance to look into this more. Unfortunately with working during the day and evening classes I probably won't be able to look further into this until Thursday-ish. Glenn On May 19, 2008, at 11:26 PM, Robert L. Read wrote: > Dear Glenn, > I have applied your patch to elephant-unstable. Thank you. > > Unfortunately, I can not build the documentation right now due to an > error in compiling defpackage in sb-posix, therefore I cannot verify > that that works. What version of SBCL are you using (I'm on 1.0.13). > Are the tex errors simply dangling references in the > documentation? Or > perhaps they are caused by another version incompatibility? > > On Sun, 2008-05-18 at 17:37 -0400, Glenn Tarcea wrote: >> Hi, >> >> I've started trying to make some changes to the documentation. I've >> run into a couple of problems. Attached is a patch to >> docstrings.lisp. On later versions of SBCL it appears that finalize- >> inheritance needs to be called before introspecting a class. The >> attached patch adds one line to texinfo-inferred-body: >> >> (sb-mop:finalize-inheritance (find-class name)) >> >> The documentation still doesn't completely build for me, but this is >> at least a step further (no lisp errors, I believe the next set are >> tex errors). >> >> FYI: I've never used darcs before, so if there is a better (or even >> correct way...) to format the patch please let me know. I used darcs >> record and darcs send to create the patch. >> >> Thanks, >> >> Glenn >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > From gtarcea at umich.edu Mon May 26 02:33:02 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Sun, 25 May 2008 22:33:02 -0400 Subject: [elephant-devel] Documentation Build Problems and Questions Message-ID: I have a few questions on the document builds. I've finally completed a build of the documentation, including generating a PDF file, but I had to change a few files to get the document build to complete. Having not used TeX before, and because I hacked things to get it working, I'm going to describe my changes, rather than send out an incorrect patch. The first issue I ran into was that makeinfo was failing. I was able to resolve that by removing the following includes in the reference.texinfo file: @subsection Direct Class Index Manipulation @include includes/fun-elephant-find-class-index.texinfo @include includes/fun-elephant-find-inverted-index.texinfo @include includes/fun-elephant-make-class-cursor.texinfo @include includes/macro-elephant-with-class-cursor.texinfo @include includes/fun-elephant-make-inverted-cursor.texinfo @include includes/macro-elephant-with-inverted-cursor.texinfo and @subsection Dynamic Indexing API @include includes/fun-elephant-enable-class-indexing.texinfo @include includes/fun-elephant-disable-class-indexing.texinfo @include includes/fun-elephant-add-class-slot-index.texinfo @include includes/fun-elephant-remove-class-slot-index.texinfo @include includes/fun-elephant-add-class-derived-index.texinfo @include includes/fun-elephant-remove-class-derived-index.texinfo Next, I tried to build the PDF file, but I kept getting pdfetex failures. It couldn't find the 'image' file IndexedBtrees1.pdf and ArchDiagram.pdf. The file user-guide.texinfo refers to IndexedBtrees1, but it it (correctly) referring to the png file. The same with the ArchDiagram reference in elephant-design.texinfo. If I copy the IndexedBtrees.png file to IndexedBtrees.pdf and ArchDiagram.png to ArchDiagram.pdf then the system completes. The @image commands look correct (example: @image{ArchDiagram,,4.5in,[Architecture Diagram],png}) Even if I change the ",png" to ",.png" that doesn't make a difference, it is still looking for a .pdf file. Is anyone else able to build the documentation, without having to rename the .png files to .pdf files? I'm building on MacOSX (Tiger). Right now I don't have access to another system to try this on, but I am wondering if there is an issue with my pdfetex command not correctly parsing the @image. Thanks, Glenn From killerstorm at newmail.ru Mon May 26 07:55:46 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Mon, 26 May 2008 10:55:46 +0300 Subject: [elephant-devel] unstable postmodern Message-ID: i've implemented stuff required to make it fly this weekend (due to lack of internet connectivity), but apparently i've messed something: Did 407 checks. Pass: 384 (94%) Skip: 1 ( 0%) Fail: 22 ( 5%) i hope that would be not hard to fix, so soon we'll see it flying.. btw, oid->schema-id for default data structures is a real WTF! From eslick at media.mit.edu Mon May 26 10:11:26 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 26 May 2008 06:11:26 -0400 Subject: [elephant-devel] unstable postmodern In-Reply-To: References: Message-ID: <239D86D2-3106-441A-A392-39983B534AE6@media.mit.edu> On May 26, 2008, at 3:55 AM, Alex Mizrahi wrote: > i've implemented stuff required to make it fly this weekend > (due to lack of internet connectivity), but apparently i've messed > something: > > Did 407 checks. > Pass: 384 (94%) > Skip: 1 ( 0%) > Fail: 22 ( 5%) > > i hope that would be not hard to fix, so soon we'll see it flying.. > > btw, oid->schema-id for default data structures is a real WTF! > Yeah, especially the btree schema ID default stuff. If you can think of a better way to handle/name things there I'd be very open to something cleaner! Ian From eslick at media.mit.edu Mon May 26 13:42:25 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 26 May 2008 09:42:25 -0400 Subject: [elephant-devel] Documentation Build Problems and Questions In-Reply-To: References: Message-ID: Thanks Glen, Some of these are issues caused by a change in the APIs in unstable - I haven't built the documentation locally in quite some time. I'll probably need to change some of the docs to accommodate this (like the class indexing section). Not sure about the png/pdf problem, but if the change you made works - then just do it again using 'darcs mv' so the source control gets the message. If you can get it to work for you so you can work on the documentation content, that should be enough for now. Robert or I can sit down with you and hash through this when some time opens up. One thing that would be helpful is putting together a TODO list that identifies the things that need to be documented or fixed: 1) Update all class indexing docs, especially derived indices (dropped explicit API, hierarchical indexed slots) 2) Document new slot options in defpclass 3) Document associations 4) Document cached operation 5) Revamp database synchronization 6) Discussion of upgrade and schemas 7) Add section for postmodern backend 8) section on multi-threaded testing and update testing section in general 9) Update 'multi-threading' considerations 10) Maybe a section on building the docs! :) Ian On May 25, 2008, at 10:33 PM, Glenn Tarcea wrote: > I have a few questions on the document builds. I've finally > completed a build of the documentation, including generating a PDF > file, but I had to change a few files to get the document build to > complete. Having not used TeX before, and because I hacked things to > get it working, I'm going to describe my changes, rather than send > out an incorrect patch. > > The first issue I ran into was that makeinfo was failing. I was able > to resolve that by removing the following includes in the > reference.texinfo file: > > @subsection Direct Class Index Manipulation > > @include includes/fun-elephant-find-class-index.texinfo > @include includes/fun-elephant-find-inverted-index.texinfo > @include includes/fun-elephant-make-class-cursor.texinfo > @include includes/macro-elephant-with-class-cursor.texinfo > @include includes/fun-elephant-make-inverted-cursor.texinfo > @include includes/macro-elephant-with-inverted-cursor.texinfo > > and > > @subsection Dynamic Indexing API > > @include includes/fun-elephant-enable-class-indexing.texinfo > @include includes/fun-elephant-disable-class-indexing.texinfo > @include includes/fun-elephant-add-class-slot-index.texinfo > @include includes/fun-elephant-remove-class-slot-index.texinfo > @include includes/fun-elephant-add-class-derived-index.texinfo > @include includes/fun-elephant-remove-class-derived-index.texinfo > > Next, I tried to build the PDF file, but I kept getting pdfetex > failures. It couldn't find the 'image' file IndexedBtrees1.pdf and > ArchDiagram.pdf. The file user-guide.texinfo refers to > IndexedBtrees1, but it it (correctly) referring to the png file. The > same with the ArchDiagram reference in elephant-design.texinfo. If I > copy the IndexedBtrees.png file to IndexedBtrees.pdf and > ArchDiagram.png to ArchDiagram.pdf then the system completes. > > The @image commands look correct (example: > @image{ArchDiagram,,4.5in,[Architecture Diagram],png}) > > Even if I change the ",png" to ",.png" that doesn't make a > difference, it is still looking for a .pdf file. > > Is anyone else able to build the documentation, without having to > rename the .png files to .pdf files? I'm building on MacOSX (Tiger). > Right now I don't have access to another system to try this on, but > I am wondering if there is an issue with my pdfetex command not > correctly parsing the @image. > > Thanks, > > Glenn > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From leslie.polzer at gmx.net Mon May 26 10:02:12 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 26 May 2008 12:02:12 +0200 (CEST) Subject: [elephant-devel] Time-boxed releases Message-ID: <63408.88.73.233.220.1211796132.squirrel@mail.stardawn.org> This topic was addressed in a side-note by Robert. I think it's a very good idea to have, say, monthly releases. What do you think about it? We could start this with the release of 1.0. Leslie From leslie.polzer at gmx.net Mon May 26 10:15:11 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Mon, 26 May 2008 12:15:11 +0200 (CEST) Subject: [elephant-devel] Documentation Build Problems and Questions In-Reply-To: References: Message-ID: <63713.88.73.233.220.1211796911.squirrel@mail.stardawn.org> > The first issue I ran into was that makeinfo was failing. I was able > to resolve that by removing the following includes in the > reference.texinfo file: Got that, too. "Solved" by your instructions. > Is anyone else able to build the documentation, without having to > rename the .png files to .pdf files? I'm building on MacOSX (Tiger). > Right now I don't have access to another system to try this on, but I > am wondering if there is an issue with my pdfetex command not > correctly parsing the @image. Yeah, works for me with TeXLive. Leslie From eslick at media.mit.edu Mon May 26 18:49:25 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Mon, 26 May 2008 14:49:25 -0400 Subject: [elephant-devel] Time-boxed releases In-Reply-To: <63408.88.73.233.220.1211796132.squirrel@mail.stardawn.org> References: <63408.88.73.233.220.1211796132.squirrel@mail.stardawn.org> Message-ID: Hi Leslie, I'd be open to a time-driven release process, but perhaps on a longer timeframe. A proper Elephant release is actually quite a chore, given the 36 permutations of lisp + data store + OS that we support. Also developer time is highly variable and the rate of change outside of one of the major overhauls we just went through isn't that high (bug fixes, small feature enhancements). If we allowed 8 weeks for small fixes to accumulate, then split the branches into incremental release & development branch we could reliably have quarterly incremental releases. We could move up the schedule to accommodate, for example, a major bug fix or feature addition. Thoughts? Ian On May 26, 2008, at 6:02 AM, Leslie P. Polzer wrote: > > This topic was addressed in a side-note by Robert. > > I think it's a very good idea to have, say, monthly releases. > > What do you think about it? We could start this with the > release of 1.0. > > Leslie > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From gtarcea at umich.edu Mon May 26 21:28:25 2008 From: gtarcea at umich.edu (Glenn Tarcea) Date: Mon, 26 May 2008 17:28:25 -0400 Subject: [elephant-devel] Documentation Build Problems and Questions In-Reply-To: References: Message-ID: <3CBAA6A2-4E41-44D8-AF5A-E2BF1AE4F9B6@umich.edu> I'll take this list as a starting point and send out a list with items I know about. Areas I know I'd like to document a little more include just a tad bit more on using BDB (ie, parameters you can pass, and a reference to tell people to look at DB_CONFIG), as well as running multiple versions of Elephant (ie, Alex's ASDF registry example). As a side note, I'm interested in helping out on the native lisp back end or the query system. I'm not sure what the next steps are, and I won't really have time to contribute more than documentation and tests until the middle of June. Thanks, Glenn On May 26, 2008, at 9:42 AM, Ian Eslick wrote: > Thanks Glen, > > Some of these are issues caused by a change in the APIs in unstable > - I haven't built the documentation locally in quite some time. > I'll probably need to change some of the docs to accommodate this > (like the class indexing section). > > Not sure about the png/pdf problem, but if the change you made > works - then just do it again using 'darcs mv' so the source > control gets the message. > > If you can get it to work for you so you can work on the > documentation content, that should be enough for now. Robert or I > can sit down with you and hash through this when some time opens up. > > One thing that would be helpful is putting together a TODO list > that identifies the things that need to be documented or fixed: > > 1) Update all class indexing docs, especially derived indices > (dropped explicit API, hierarchical indexed slots) > 2) Document new slot options in defpclass > 3) Document associations > 4) Document cached operation > 5) Revamp database synchronization > 6) Discussion of upgrade and schemas > 7) Add section for postmodern backend > 8) section on multi-threaded testing and update testing section in > general > 9) Update 'multi-threading' considerations > 10) Maybe a section on building the docs! :) > > Ian > > On May 25, 2008, at 10:33 PM, Glenn Tarcea wrote: > >> I have a few questions on the document builds. I've finally >> completed a build of the documentation, including generating a PDF >> file, but I had to change a few files to get the document build to >> complete. Having not used TeX before, and because I hacked things >> to get it working, I'm going to describe my changes, rather than >> send out an incorrect patch. >> >> The first issue I ran into was that makeinfo was failing. I was >> able to resolve that by removing the following includes in the >> reference.texinfo file: >> >> @subsection Direct Class Index Manipulation >> >> @include includes/fun-elephant-find-class-index.texinfo >> @include includes/fun-elephant-find-inverted-index.texinfo >> @include includes/fun-elephant-make-class-cursor.texinfo >> @include includes/macro-elephant-with-class-cursor.texinfo >> @include includes/fun-elephant-make-inverted-cursor.texinfo >> @include includes/macro-elephant-with-inverted-cursor.texinfo >> >> and >> >> @subsection Dynamic Indexing API >> >> @include includes/fun-elephant-enable-class-indexing.texinfo >> @include includes/fun-elephant-disable-class-indexing.texinfo >> @include includes/fun-elephant-add-class-slot-index.texinfo >> @include includes/fun-elephant-remove-class-slot-index.texinfo >> @include includes/fun-elephant-add-class-derived-index.texinfo >> @include includes/fun-elephant-remove-class-derived-index.texinfo >> >> Next, I tried to build the PDF file, but I kept getting pdfetex >> failures. It couldn't find the 'image' file IndexedBtrees1.pdf and >> ArchDiagram.pdf. The file user-guide.texinfo refers to >> IndexedBtrees1, but it it (correctly) referring to the png file. >> The same with the ArchDiagram reference in elephant- >> design.texinfo. If I copy the IndexedBtrees.png file to >> IndexedBtrees.pdf and ArchDiagram.png to ArchDiagram.pdf then the >> system completes. >> >> The @image commands look correct (example: >> @image{ArchDiagram,,4.5in,[Architecture Diagram],png}) >> >> Even if I change the ",png" to ",.png" that doesn't make a >> difference, it is still looking for a .pdf file. >> >> Is anyone else able to build the documentation, without having to >> rename the .png files to .pdf files? I'm building on MacOSX >> (Tiger). Right now I don't have access to another system to try >> this on, but I am wondering if there is an issue with my pdfetex >> command not correctly parsing the @image. >> >> Thanks, >> >> Glenn >> >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > > From leslie.polzer at gmx.net Tue May 27 08:13:41 2008 From: leslie.polzer at gmx.net (Leslie P. Polzer) Date: Tue, 27 May 2008 10:13:41 +0200 (CEST) Subject: [elephant-devel] Time-boxed releases In-Reply-To: References: <63408.88.73.233.220.1211796132.squirrel@mail.stardawn.org> Message-ID: <61207.88.73.218.157.1211876021.squirrel@mail.stardawn.org> > Also developer time is highly variable and the rate of change outside > of one of the major overhauls we just went through isn't that high > (bug fixes, small feature enhancements). If we allowed 8 weeks for > small fixes to accumulate, then split the branches into incremental > release & development branch we could reliably have quarterly > incremental releases. We could move up the schedule to accommodate, > for example, a major bug fix or feature addition. > > Thoughts? I'd be happy with the "quarterly releases and emergency fixes" scheme. Leslie From ocorrain at gmail.com Tue May 27 19:41:57 2008 From: ocorrain at gmail.com (Tiarnan O'Corrain) Date: Tue, 27 May 2008 20:41:57 +0100 Subject: [elephant-devel] Error while compiling memutil.lisp Message-ID: Hi-- I'm getting the following when compiling elephant (both 0.9 and 0.9.1): Error in function ALIEN::%PARSE-ALIEN-TYPE: Unknown alien type: ELEPHANT-MEMUTIL::UNSIGNED-CHAR CMUCL 19d, with UFFI 1.6.0, running on Linux x86 2.6.18 any ideas? Tiarn?n From eslick at media.mit.edu Tue May 27 19:59:06 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Tue, 27 May 2008 15:59:06 -0400 Subject: [elephant-devel] Error while compiling memutil.lisp In-Reply-To: References: Message-ID: CMUCL is an un-officially supported lisp for Elephant. You are the first user who has indicated you were using it since I started working on the project two years ago! I'd recommend using SBCL. If you want to spend a little time debugging this, look in src/memutil/ memutil.lisp for the UFFI type definitions for the various FFI functions to manage memory. It may be that it doesn't recognize unsigned-char vs. the keyword version :unsigned-char? Ian On May 27, 2008, at 3:41 PM, Tiarnan O'Corrain wrote: > Hi-- > > I'm getting the following when compiling elephant (both 0.9 and > 0.9.1): > > Error in function ALIEN::%PARSE-ALIEN-TYPE: > Unknown alien type: ELEPHANT-MEMUTIL::UNSIGNED-CHAR > > CMUCL 19d, with UFFI 1.6.0, running on Linux x86 2.6.18 > > any ideas? > > Tiarn?n > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From read at robertlread.net Wed May 28 00:24:09 2008 From: read at robertlread.net (Robert L. Read) Date: Tue, 27 May 2008 19:24:09 -0500 Subject: [elephant-devel] unstable postmodern In-Reply-To: References: Message-ID: <1211934249.29423.584.camel@penguin.yourdomain.com> Great! This is the almost the last piece we need before the upcoming release. On Mon, 2008-05-26 at 10:55 +0300, Alex Mizrahi wrote: > i've implemented stuff required to make it fly this weekend > (due to lack of internet connectivity), but apparently i've messed > something: > > Did 407 checks. > Pass: 384 (94%) > Skip: 1 ( 0%) > Fail: 22 ( 5%) > > i hope that would be not hard to fix, so soon we'll see it flying.. > > btw, oid->schema-id for default data structures is a real WTF! > > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From killerstorm at newmail.ru Wed May 28 09:26:03 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Wed, 28 May 2008 12:26:03 +0300 Subject: [elephant-devel] serializer not thread safe! Message-ID: obviously broken code: (defun get-circularity-hash () "Get a clean hash for object serialization" (declare (type fixnum *circularity-initial-hash-size*)) (if (= 0 (length *circularity-hash-queue*)) (make-hash-table :test 'eq :size *circularity-initial-hash-size*) (ele-with-fast-lock (*serializer-fast-lock*) (vector-pop *circularity-hash-queue*)))) concurrent call could happen between check and pop, thus invalidating the check. (defun get-cicularity-hash () (or (ele-with-fast-lock (*serializer-fast-lock*) (and (plusp (length *circularity-hash-queue*)) (vector-pop *circularity-hash-queue*))) (make-hash-table :test 'eq :size *circularity-initial-hash-size*))) From eslick at media.mit.edu Wed May 28 12:26:48 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 28 May 2008 08:26:48 -0400 Subject: [elephant-devel] serializer not thread safe! In-Reply-To: References: Message-ID: <73799BD3-8913-4E56-981A-32544CCFD3F4@media.mit.edu> Good catch! I'll promote that. I think the original intention was to minimize locking, but you are correct in observing that the corner case of length = 1 can lead to errors. Thanks, Ian On May 28, 2008, at 5:26 AM, Alex Mizrahi wrote: > obviously broken code: > > (defun get-circularity-hash () > "Get a clean hash for object serialization" > (declare (type fixnum *circularity-initial-hash-size*)) > (if (= 0 (length *circularity-hash-queue*)) > (make-hash-table :test 'eq :size *circularity-initial-hash-size*) > (ele-with-fast-lock (*serializer-fast-lock*) > (vector-pop *circularity-hash-queue*)))) > > concurrent call could happen between check and pop, thus > invalidating the > check. > > (defun get-cicularity-hash () > (or > (ele-with-fast-lock (*serializer-fast-lock*) > (and (plusp (length *circularity-hash-queue*)) > (vector-pop *circularity-hash-queue*))) > (make-hash-table :test 'eq :size *circularity-initial-hash- > size*))) > > > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From killerstorm at newmail.ru Fri May 30 14:12:04 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Fri, 30 May 2008 17:12:04 +0300 Subject: [elephant-devel] Re: serializer not thread safe! References: <73799BD3-8913-4E56-981A-32544CCFD3F4@media.mit.edu> Message-ID: IE> Good catch! I'll promote that. I think the original intention was to IE> minimize locking, but you are correct in observing that the corner IE> case of length = 1 can lead to errors. there are (at least) two more places like this -- one few lines below, and one in memutil.