From yarek.kowalik at gmail.com Fri Dec 5 20:22:41 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Fri, 5 Dec 2008 12:22:41 -0800 Subject: [elephant-devel] SBCL compilation warnings Message-ID: I get several warnings in SBCL 1.0.20 that are related to use of dynamic-extent. For example, here it complains about 'COLLECT' is an indefined variable: (defun get-query-instances (constraints) "Get a list of instances according to the query constraints" (declare (dynamic-extent constraints)) (let ((list nil)) (flet ((collect (inst) (push inst list))) (declare (dynamic-extent collect)) (map-class-query #'collect constraints)))) There are 5 warnings for patterns just like the one above. Should (dynamic-extent collect) be in fact (dynamic-extent #'collect) ? Yarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsynnott at gmail.com Wed Dec 10 20:38:32 2008 From: rsynnott at gmail.com (Robert Synnott) Date: Wed, 10 Dec 2008 20:38:32 +0000 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table Message-ID: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com> Hi, >From my experimenting, it seems that values are never removed from the blob table. So, for instance, if I have an object, and I delete it with drop-instances/drop-pobject, the blob rows representing the slot values never go away. Am I right about this? Is there anything I can do about it short of db migration? Thanks Rob From eslick at media.mit.edu Wed Dec 10 22:15:38 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Wed, 10 Dec 2008 17:15:38 -0500 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table In-Reply-To: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com> References: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com> Message-ID: Have you updated from elephant-unstable recently? I found a bug where indices were not properly updated that I think was in the core elephant code. However there may be a bug specific to postmodern. Ian On Dec 10, 2008, at 3:38 PM, Robert Synnott wrote: > Hi, > From my experimenting, it seems that values are never removed from the > blob table. So, for instance, if I have an object, and I delete it > with drop-instances/drop-pobject, the blob rows representing the slot > values never go away. Am I right about this? Is there anything I can > do about it short of db migration? > Thanks > Rob > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From rsynnott at gmail.com Thu Dec 11 04:12:13 2008 From: rsynnott at gmail.com (Robert Synnott) Date: Thu, 11 Dec 2008 04:12:13 +0000 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table In-Reply-To: References: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com> Message-ID: <24f203480812102012h5e233c06qc6cb427cca5be89f@mail.gmail.com> 2008/12/10 Ian Eslick : > Have you updated from elephant-unstable recently? I found a bug where > indices were not properly updated that I think was in the core > elephant code. However there may be a bug specific to postmodern. > > Ian > Nope, this isn't the problem, or at least I haven't seen this happening. My problem is that blobs never seem to get deleted. The postmodern backend appears to store all slot values (and btree values, etc) in a great big table called blob. So, if I do something like, say, (defpclass myclass () ((slot1 :accessor slot1 :initarg :slot1))) (defvar *lots-of-objects* (with-transaction () (loop for i from 1 to 10000 collect (make-instance 'myclass :slot1 i)))) (loop for i in *lots-of-objects* do (drop-pobject i)) then the slot definitions in the slot table are removed, but the actual slot _values_, which live in the blob table, are retained. Is this intended behaviour? It has become a problem for me, as I'm left with a huge blob table full mostly of things which have been deleted or replaced; (setf (slot1 myobject) 4) has the same effect where slot1 in myobject already has a value. I realise I could migrate to free up space, but I'd really like to avoid the downtime if at all possible. One thing that did occur to me was that I could record the current maximum bid in the blob table, then go through each tree table, including root, and also the slots table, recording blob id references in each one. Then take the entire set of bids from the blob table; anything present in the blob table but not referred to in any of the trees or slots table, and with bid less than or equal to the maximum bid when the procedure was started, should be obsolete, and deleting it should be safe. Does this sound reasonable, or am I missing something? I already tried it on a non-production database, and it seemed to work; I couldn't find any problems afterwards, but I'd like to be sure I'm not doing anything silly. Thanks Rob From reddaly at gmail.com Thu Dec 11 06:35:38 2008 From: reddaly at gmail.com (Red Daly) Date: Wed, 10 Dec 2008 22:35:38 -0800 Subject: [elephant-devel] BDB hanging with many stalled threads In-Reply-To: References: <63086.88.73.215.209.1224143021.squirrel@mail.stardawn.org> <61901.88.73.196.243.1224318455.squirrel@mail.stardawn.org> Message-ID: As it turns out, this solution has also turned out to be inadequate. For the past few months I have been experiencing the same sort of issues I originally described. Every few days I find SBCL taking up 100% of the CPU and a hundred Hunchentoot threads stalled in the web server. When a thread is killed in lisp, the database demands recovery. the BDB error file does not display anything unusual, and I have a db_deadlock and db_checkpoint command executing every minute to resolve deadlocks and checkpoint the database. Once again this is with the release version of elephant and BDB 4.5. I suppose the most helpful thing would be a program that replicates this behavior, which I will try to get to at some point. Best, Red Daly On Thu, Oct 23, 2008 at 1:10 AM, Red Daly wrote: > This appears to have resolved the problem. The server has been running for > the past several days with no database crashes. The only unusual behavior > is an occasional condition being raised that looks like this: > > > Thanks for the fix. This remaining issue does not seem to happen too much > so I can live with it. It simply belongs in the documentary record. > > Red > > > On Sat, Oct 18, 2008 at 1:27 AM, Leslie P. Polzer > wrote: > >> >> > Unfortunately the standalone deadlock utility provided by BDB does not >> seem >> > to resolve this. I just tried running it with all these hung threads >> and >> > somehow the result was a database that needs recovery. db_stat also >> > indicates the number of deadlocks is 0. >> > >> > This seems to indicate that the problem is not caused by deadlocks. >> What do >> > you think? >> >> Let's make sure. After opening the store controller run >> >> (db-bdb::db-env-set-lock-detect >> (elephant::controller-environment *store-controller*) >> db-bdb::DB_LOCK_DEFAULT) >> >> and check whether this still occurs. >> >> Leslie >> >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From killerstorm at newmail.ru Thu Dec 11 10:06:35 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Thu, 11 Dec 2008 12:06:35 +0200 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table References: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com> <24f203480812102012h5e233c06qc6cb427cca5be89f@mail.gmail.com> Message-ID: RS> then the slot definitions in the slot table are removed, but the RS> actual slot _values_, which live in the blob table, are retained. Is RS> this intended behaviour? yep, it is. it sort of sucks, i know. whether or not it becomes a problem depends on what kind of data you store in slots. blobs get reused (at least they should..), so, for example, all slots with value 4 will refer to same blob id, and you can consider bloated blob table as a cache of some kind in this case :). if you have constant inflow of unique data that becomes a problem, indeed. RS> avoid the downtime if at all possible. One thing that did occur to me RS> was that I could record the current maximum bid in the blob table, RS> then go through each tree table, including root, and also the slots RS> table, recording blob id references in each one. Then take the entire RS> set of bids from the blob table; anything present in the blob table RS> but not referred to in any of the trees or slots table, and with bid RS> less than or equal to the maximum bid when the procedure was started, RS> should be obsolete, and deleting it should be safe. Does this sound RS> reasonable, or am I missing something? yep, right in general, some details seem to be wrong. first, you do not need to go through _all_ tables, but only columns that store blob ids. there is some metainformation attached to btrees, key-type and value-type. if key-type is :object, blob ids are in qi column, if value-type is :object, value column contains blob ids. then there are performance considerations -- if you have lots of data, allocating temporary storage for all blob ids might be painful. if you run cleanup procedure more-or-less frequently, there will be more alive bids than dead ones, so you can optimize it ruling out alive bids early. for example, if you work mostly with objects, you can first find the difference between blob and slots tables (because most alive bids would be in slots): SELECT blob.bid FROM blob LEFT JOIN slots ON blob.bid = slots.value WHERE slots.value IS NULL (warning: totally untested). this would return all dead bids and bids that are referenced from other tables.then you can go through rest of tables, pulling blob ids from them and eliminating ones you see in that data set, in the end you end up with only dead ones. then, you do not actually need recordning maximum bid if you're working within a single transaction -- postgresql's transactional engine will do this for you. (it works in similar way, btw -- ignores rows that have txn id larger than current txn id). From henrik at evahjelte.com Thu Dec 11 10:45:27 2008 From: henrik at evahjelte.com (Henrik Hjelte) Date: Thu, 11 Dec 2008 11:45:27 +0100 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table In-Reply-To: References: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com> <24f203480812102012h5e233c06qc6cb427cca5be89f@mail.gmail.com> Message-ID: <50e8e4f60812110245l5386d27hf1a00e441f3024d5@mail.gmail.com> I agree with Alex, just one perhaps obvious comment: I would have one transaction around cleaning up one blob-id, not a big transaction around cleaning up the whole blob table. If you do several transactions it doesn't lock up much, so it should be able to execute at the same time as a database is in production. I hope your code can be added to the postmodern backend. Thanks, Henrik From killerstorm at newmail.ru Thu Dec 11 12:56:43 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Thu, 11 Dec 2008 14:56:43 +0200 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table References: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com><24f203480812102012h5e233c06qc6cb427cca5be89f@mail.gmail.com> <50e8e4f60812110245l5386d27hf1a00e441f3024d5@mail.gmail.com> Message-ID: HH> I agree with Alex, just one perhaps obvious comment: I would have one HH> transaction around cleaning up one blob-id, not a big transaction HH> around cleaning up the whole blob table. If you do several HH> transactions it doesn't lock up much, so it HH> should be able to execute at the same time as a database is in HH> production. i think with PostgreSQL's concurrency model (MVCC) locks will be of no harm -- iirc only concurrent updates of same values block one another. here we are not doing concurrent updates, and so it should go fine in one transaction, probably will work way faster too. i won't bet on it, though, it's better to test to be sure From rsynnott at gmail.com Thu Dec 11 17:09:53 2008 From: rsynnott at gmail.com (Robert Synnott) Date: Thu, 11 Dec 2008 17:09:53 +0000 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table In-Reply-To: <50e8e4f60812110245l5386d27hf1a00e441f3024d5@mail.gmail.com> References: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com> <24f203480812102012h5e233c06qc6cb427cca5be89f@mail.gmail.com> <50e8e4f60812110245l5386d27hf1a00e441f3024d5@mail.gmail.com> Message-ID: <24f203480812110909s1ba4f6cbs929fd30e015c941b@mail.gmail.com> 2008/12/11 Henrik Hjelte : > I agree with Alex, just one perhaps obvious comment: I would have one > transaction around cleaning up one blob-id, not a big transaction > around cleaning up the whole blob table. If you do several > transactions it doesn't lock up much, so it > should be able to execute at the same time as a database is in production. > > I hope your code can be added to the postmodern backend. > > Thanks, > Henrik > > ________ I can certainly provide it as a contrib, but as it stands, it is not useful in all circumstances; it keeps track of the live bids using a huge array of booleans, and for a sufficiently big database memory would become a problem. It suits my database, however. I have two other minor worries about this technique. First, it would seem possible that if, say, we have slot1 in object1 which was set to :some-opt at some time in the past, but is now totally unused by any object, the blob row containing that would be considered dead; if someone comes along while the cleanup is running and sets some slot to :some-opt, then it will be in use again, but potentially get deleted anyway, because it will have already been deemed dead. This isn't really a problem for me; due to the nature of my dataset the chances of this happening are remote, but I can imagine it'd be an issue for some people. Secondly, there's the possibility that somebody could put a bid in, say, a serialized list. This would NOT be detected by the procedure I've outlined, and the thing with that bid would be deleted if not referred to elsewhere. Again, this isn't a problem for me, but somebody else might potentially be doing it. Beyond those, though, in the normal case I am correct to assume that a blob entry can be considered totally unused if not referred to in either slots or any tree? Thanks Rob From yarek.kowalik at gmail.com Thu Dec 11 18:56:40 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Thu, 11 Dec 2008 10:56:40 -0800 Subject: [elephant-devel] Help with an error on app startup/initialization Message-ID: I've converted my weblocks app to use elephant. One of the first things I do is to initialize some variables. One of the variables is a "demo" item, that I use as a placeholder until the user choses something more specific. I'm having trouble initializing the dummy. Here is my code: (defun init-dummy-item () (let* ((ref-type 0) (ref-id "dummy-item")) (setf *dummy-item* (unless-use (awhen (elephant:get-instances-by-value 'product-item 'reference-id ref-id) (find-if (lambda (item) (eql (product-item-reference-type item) ref-type)) it)) (make-instance 'product-item :reference-type ref-type :reference-id ref-id :title "No Item selected" :url "/pub/static/dummy.html" :image-url "/pub/images/dummy.png"))))) (The UNLESS-USE macro simply does unless, and if results are nil uses the second sexp.) The problem is that I get an error (see trace below) when calling GET-INSTANCES-BY-VALUE. What am I doing wrong? The nearest that I can gather is that GET-VALUE is called with the secondd argument as NIL, but it's supposed to be something reasonable, i.e a controller-index-table held by the controler. For whatever raeson that's NIL. What's the deal here? Am I not alowed to call the get-instances-by-* methods until I create at least one item for each class I'm querying about? Yarek THE ERROR: There is no applicable method for the generic function # when called with arguments ((FASHION-ORIGAMI::PRODUCT-ITEM . FASHION-ORIGAMI::REFERENCE-ID) NIL). [Condition of type SIMPLE-ERROR] Restarts: 0: [TERMINATE-THREAD] Terminate this thread (#) Backtrace: 0: ((LAMBDA (SWANK-BACKEND::DEBUGGER-LOOP-FN)) #) 1: (SWANK::DEBUG-IN-EMACS #) 2: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) # #) 3: (SWANK::CALL-WITH-REDIRECTED-IO # #) 4: (SWANK::CALL-WITH-CONNECTION # #) 5: (SWANK:INVOKE-SLIME-DEBUGGER #) 6: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) # #) 7: (INVOKE-DEBUGGER #) 8: (INVOKE-DEBUGGER #)[:EXTERNAL] 9: ((SB-PCL::FAST-METHOD HUNCHENTOOT:MAYBE-INVOKE-DEBUGGER (T)) # # #) 10: (SIGNAL #)[:EXTERNAL] 11: (ERROR #)[:EXTERNAL] 12: ((FLET #:LAMBDA43) #) 13: ((FLET #:LAMBDA43) #)[:EXTERNAL] 14: (SIGNAL #)[:EXTERNAL] 15: (ERROR "~@")[:EXTERNAL] Locals: SB-DEBUG::ARG-0 = 3 SB-DEBUG::ARG-1 = "~@ # #)[:EXTERNAL] Locals: SB-DEBUG::ARG-0 = 5 SB-DEBUG::ARG-1 = : SB-DEBUG::ARG-2 = : SB-DEBUG::ARG-3 = # 17: (ELEPHANT::ENSURE-SLOT-DEF-INDEX # #) Locals: SB-DEBUG::ARG-0 = : SB-DEBUG::ARG-1 = : 18: ((SB-PCL::FAST-METHOD ELEPHANT:FIND-INVERTED-INDEX (ELEPHANT:PERSISTENT-METACLASS T)) # # # #)[:EXTERNAL] Locals: SB-DEBUG::ARG-0 = : SB-DEBUG::ARG-1 = : SB-DEBUG::ARG-2 = : SB-DEBUG::ARG-3 = : SB-DEBUG::ARG-4 = : 19: (ELEPHANT:MAP-INVERTED-INDEX # # #)[:EXTERNAL] Locals: SB-DEBUG::ARG-0 = : SB-DEBUG::ARG-1 = : SB-DEBUG::ARG-2 = : SB-DEBUG::ARG-3 = : 20: (FASHION-ORIGAMI::INIT-DUMMY-ITEM) Locals: #:G137 = : ANAPHORA:IT = : -------------- next part -------------- An HTML attachment was scrubbed... URL: From eslick at media.mit.edu Thu Dec 11 21:19:10 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Thu, 11 Dec 2008 16:19:10 -0500 Subject: [elephant-devel] Help with an error on app startup/initialization In-Reply-To: References: Message-ID: Which version of elephant are you using? Which backend are you using? If BDB, which version? I presume *store-controller* open and reachable from the package you are in? Looks like the elephant store isn't in the proper state (e.g. was closed and *store-controller* was not zeroed out). On Dec 11, 2008, at 1:56 PM, Yarek Kowalik wrote: > I've converted my weblocks app to use elephant. One of the first > things I do is to initialize some variables. One of the variables > is a "demo" item, that I use as a placeholder until the user choses > something more specific. I'm having trouble initializing the > dummy. Here is my code: > > (defun init-dummy-item () > (let* ((ref-type 0) > (ref-id "dummy-item")) > (setf *dummy-item* > (unless-use (awhen (elephant:get-instances-by-value > 'product-item 'reference-id ref-id) > (find-if (lambda (item) > (eql (product-item-reference-type > item) ref-type)) > it)) > (make-instance > 'product-item > :reference-type ref-type > :reference-id ref-id > :title "No Item selected" > :url "/pub/static/dummy.html" > :image-url "/pub/images/dummy.png"))))) > > (The UNLESS-USE macro simply does unless, and if results are nil > uses the second sexp.) > > The problem is that I get an error (see trace below) when calling > GET-INSTANCES-BY-VALUE. What am I doing wrong? > > The nearest that I can gather is that GET-VALUE is called with the > secondd argument as NIL, but it's supposed to be something > reasonable, i.e a controller-index-table held by the controler. For > whatever raeson that's NIL. What's the deal here? Am I not alowed > to call the get-instances-by-* methods until I create at least one > item for each class I'm querying about? > > Yarek > > THE ERROR: > > There is no applicable method for the generic function > # > when called with arguments > ((FASHION-ORIGAMI::PRODUCT-ITEM > . FASHION-ORIGAMI::REFERENCE-ID) > NIL). > [Condition of type SIMPLE-ERROR] > > Restarts: > 0: [TERMINATE-THREAD] Terminate this thread (# worker-4" RUNNING {10029A0321}>) > > Backtrace: > 0: ((LAMBDA (SWANK-BACKEND::DEBUGGER-LOOP-FN)) # #) {1003F6A4C9}>) > 1: (SWANK::DEBUG-IN-EMACS #) > 2: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) # SWANK:SWANK-DEBUGGER-HOOK> #) > 3: (SWANK::CALL-WITH-REDIRECTED-IO # {10037B86D1}> #) > 4: (SWANK::CALL-WITH-CONNECTION # > #) > 5: (SWANK:INVOKE-SLIME-DEBUGGER #) > 6: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) # SWANK:SWANK-DEBUGGER-HOOK> #) > 7: (INVOKE-DEBUGGER #) > 8: (INVOKE-DEBUGGER #)[:EXTERNAL] > 9: ((SB-PCL::FAST-METHOD HUNCHENTOOT:MAYBE-INVOKE-DEBUGGER (T)) > # # # {1002E3A1A1}>) > 10: (SIGNAL #)[:EXTERNAL] > 11: (ERROR #)[:EXTERNAL] > 12: ((FLET #:LAMBDA43) #) > 13: ((FLET #:LAMBDA43) #)[:EXTERNAL] > 14: (SIGNAL #)[:EXTERNAL] > 15: (ERROR "~@ function ~2I~_~S~\n ~I~_when called with arguments > ~2I~_~S.~:>")[:EXTERNAL] > Locals: > SB-DEBUG::ARG-0 = 3 > SB-DEBUG::ARG-1 = "~@ generic function ~2I~_~S~\n .. > 16: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) # argument> # # ELEPHANT:GET-VALUE (3)>)[:EXTERNAL] > Locals: > SB-DEBUG::ARG-0 = 5 > SB-DEBUG::ARG-1 = : > SB-DEBUG::ARG-2 = : > SB-DEBUG::ARG-3 = # VALUE (3)> > 17: (ELEPHANT::ENSURE-SLOT-DEF-INDEX # > #) > Locals: > SB-DEBUG::ARG-0 = : > SB-DEBUG::ARG-1 = : > 18: ((SB-PCL::FAST-METHOD ELEPHANT:FIND-INVERTED-INDEX > (ELEPHANT:PERSISTENT-METACLASS T)) # > # # # argument>)[:EXTERNAL] > Locals: > SB-DEBUG::ARG-0 = : > SB-DEBUG::ARG-1 = : > SB-DEBUG::ARG-2 = : > SB-DEBUG::ARG-3 = : > SB-DEBUG::ARG-4 = : > 19: (ELEPHANT:MAP-INVERTED-INDEX # > # #)[:EXTERNAL] > Locals: > SB-DEBUG::ARG-0 = : > SB-DEBUG::ARG-1 = : > SB-DEBUG::ARG-2 = : > SB-DEBUG::ARG-3 = : > 20: (FASHION-ORIGAMI::INIT-DUMMY-ITEM) > Locals: > #:G137 = : > ANAPHORA:IT = : > > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From yarek.kowalik at gmail.com Thu Dec 11 21:50:53 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Thu, 11 Dec 2008 13:50:53 -0800 Subject: [elephant-devel] Help with an error on app startup/initialization In-Reply-To: References:

Message-ID: BDB 4.5 for back-end. I'm using elephant-unstable, pulled last week. I'm elephant this via Weblocks, so I'm assuming the store is open by the time the init-session for Weblocks app is called. Maybe I should check on that - it's possible that the store is not in fully open state? Yarek On Thu, Dec 11, 2008 at 1:19 PM, Ian Eslick wrote: > Which version of elephant are you using? > Which backend are you using? If BDB, which version? > > I presume *store-controller* open and reachable from the package you > are in? > > Looks like the elephant store isn't in the proper state (e.g. was > closed and *store-controller* was not zeroed out). > > On Dec 11, 2008, at 1:56 PM, Yarek Kowalik wrote: > > > I've converted my weblocks app to use elephant. One of the first > > things I do is to initialize some variables. One of the variables > > is a "demo" item, that I use as a placeholder until the user choses > > something more specific. I'm having trouble initializing the > > dummy. Here is my code: > > > > (defun init-dummy-item () > > (let* ((ref-type 0) > > (ref-id "dummy-item")) > > (setf *dummy-item* > > (unless-use (awhen (elephant:get-instances-by-value > > 'product-item 'reference-id ref-id) > > (find-if (lambda (item) > > (eql (product-item-reference-type > > item) ref-type)) > > it)) > > (make-instance > > 'product-item > > :reference-type ref-type > > :reference-id ref-id > > :title "No Item selected" > > :url "/pub/static/dummy.html" > > :image-url "/pub/images/dummy.png"))))) > > > > (The UNLESS-USE macro simply does unless, and if results are nil > > uses the second sexp.) > > > > The problem is that I get an error (see trace below) when calling > > GET-INSTANCES-BY-VALUE. What am I doing wrong? > > > > The nearest that I can gather is that GET-VALUE is called with the > > secondd argument as NIL, but it's supposed to be something > > reasonable, i.e a controller-index-table held by the controler. For > > whatever raeson that's NIL. What's the deal here? Am I not alowed > > to call the get-instances-by-* methods until I create at least one > > item for each class I'm querying about? > > > > Yarek > > > > THE ERROR: > > > > There is no applicable method for the generic function > > # > > when called with arguments > > ((FASHION-ORIGAMI::PRODUCT-ITEM > > . FASHION-ORIGAMI::REFERENCE-ID) > > NIL). > > [Condition of type SIMPLE-ERROR] > > > > Restarts: > > 0: [TERMINATE-THREAD] Terminate this thread (# > worker-4" RUNNING {10029A0321}>) > > > > Backtrace: > > 0: ((LAMBDA (SWANK-BACKEND::DEBUGGER-LOOP-FN)) # > #) {1003F6A4C9}>) > > 1: (SWANK::DEBUG-IN-EMACS #) > > 2: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) # > SWANK:SWANK-DEBUGGER-HOOK> #) > > 3: (SWANK::CALL-WITH-REDIRECTED-IO # > {10037B86D1}> #) > > 4: (SWANK::CALL-WITH-CONNECTION # > > #) > > 5: (SWANK:INVOKE-SLIME-DEBUGGER #) > > 6: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) # > SWANK:SWANK-DEBUGGER-HOOK> #) > > 7: (INVOKE-DEBUGGER #) > > 8: (INVOKE-DEBUGGER #)[:EXTERNAL] > > 9: ((SB-PCL::FAST-METHOD HUNCHENTOOT:MAYBE-INVOKE-DEBUGGER (T)) > > # # # > {1002E3A1A1}>) > > 10: (SIGNAL #)[:EXTERNAL] > > 11: (ERROR #)[:EXTERNAL] > > 12: ((FLET #:LAMBDA43) #) > > 13: ((FLET #:LAMBDA43) #)[:EXTERNAL] > > 14: (SIGNAL #)[:EXTERNAL] > > 15: (ERROR "~@ > function ~2I~_~S~\n ~I~_when called with arguments > > ~2I~_~S.~:>")[:EXTERNAL] > > Locals: > > SB-DEBUG::ARG-0 = 3 > > SB-DEBUG::ARG-1 = "~@ > generic function ~2I~_~S~\n .. > > 16: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) # > argument> # # > ELEPHANT:GET-VALUE (3)>)[:EXTERNAL] > > Locals: > > SB-DEBUG::ARG-0 = 5 > > SB-DEBUG::ARG-1 = : > > SB-DEBUG::ARG-2 = : > > SB-DEBUG::ARG-3 = # > VALUE (3)> > > 17: (ELEPHANT::ENSURE-SLOT-DEF-INDEX # > > #) > > Locals: > > SB-DEBUG::ARG-0 = : > > SB-DEBUG::ARG-1 = : > > 18: ((SB-PCL::FAST-METHOD ELEPHANT:FIND-INVERTED-INDEX > > (ELEPHANT:PERSISTENT-METACLASS T)) # > > # # # > argument>)[:EXTERNAL] > > Locals: > > SB-DEBUG::ARG-0 = : > > SB-DEBUG::ARG-1 = : > > SB-DEBUG::ARG-2 = : > > SB-DEBUG::ARG-3 = : > > SB-DEBUG::ARG-4 = : > > 19: (ELEPHANT:MAP-INVERTED-INDEX # > > # #)[:EXTERNAL] > > Locals: > > SB-DEBUG::ARG-0 = : > > SB-DEBUG::ARG-1 = : > > SB-DEBUG::ARG-2 = : > > SB-DEBUG::ARG-3 = : > > 20: (FASHION-ORIGAMI::INIT-DUMMY-ITEM) > > Locals: > > #:G137 = : > > ANAPHORA:IT = : > > > > > > > > _______________________________________________ > > elephant-devel site list > > elephant-devel at common-lisp.net > > http://common-lisp.net/mailman/listinfo/elephant-devel > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yarek.kowalik at gmail.com Thu Dec 11 22:16:10 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Thu, 11 Dec 2008 14:16:10 -0800 Subject: [elephant-devel] Help with an error on app startup/initialization In-Reply-To: References:

Message-ID: Yep, store gets opened on application start, before session-init is called. Also, moving init-dummy to open-store :after pushes the problem further, then I get a BDB-DB-ERROR when calling (elephant::get-db-schemas (weblocks-elephant::elephant-controller *my-elephant-store*) 'product-item) - this was optained running code from REPL trying reproduce the error I get when using (weblocks-elephant:count-persisten-objects *my-elephant-store* 'product-item). Yarek Bad type argument: BDB-DB-ERROR [Condition of type SIMPLE-TYPE-ERROR] Restarts: 0: [RETRY] Retry SLIME REPL evaluation request. 1: [ABORT] Return to SLIME's top level. 2: [TERMINATE-THREAD] Terminate this thread (#) Backtrace: 0: (MAKE-CONDITION BDB-DB-ERROR)[:EXTERNAL] Locals: SB-DEBUG::ARG-0 = 3 SB-DEBUG::ARG-1 = BDB-DB-ERROR 1: (ERROR BDB-DB-ERROR)[:EXTERNAL] Locals: SB-DEBUG::ARG-0 = 3 SB-DEBUG::ARG-1 = BDB-DB-ERROR 2: ((SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-BDB::BDB-STORE-CONTROLLER T)) ..)[:EXTERNAL] Locals: SB-DEBUG::ARG-0 = 10 SB-DEBUG::ARG-1 = : SB-DEBUG::ARG-2 = : SB-DEBUG::ARG-3 = # SB-DEBUG::ARG-4 = # 3: (ELEPHANT::MAP-BTREE-VALUES #) [No Locals] 4: (ELEPHANT::GET-DB-SCHEMAS # PRODUCT-ITEM) Locals: SB-DEBUG::ARG-0 = # SB-DEBUG::ARG-1 = PRODUCT-ITEM 5: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ELEPHANT::GET-DB-SCHEMAS (WEBLOCKS-ELEPHANT::ELEPHANT-CONTROLLER *FASHION-ORIGAMI-ELEPHANT-STORE*) 'PRODUCT-ITEM) #) Locals: SB-DEBUG::ARG-0 = (ELEPHANT::GET-DB-SCHEMAS ..) SB-DEBUG::ARG-1 = # 6: (SWANK::EVAL-REGION "(elephant::get-db-schemas (weblocks-elephant::elephant-controller *fashion-origami-elephant-store*) 'product-item)\n") Locals: SB-DEBUG::ARG-0 = "(elephant::get-db-schemas (weblocks-elephant::elephant-controller *fashion.. Yarek On Thu, Dec 11, 2008 at 1:50 PM, Yarek Kowalik wrote: > BDB 4.5 for back-end. > > I'm using elephant-unstable, pulled last week. > > I'm elephant this via Weblocks, so I'm assuming the store is open by the > time the init-session for Weblocks app is called. Maybe I should check on > that - it's possible that the store is not in fully open state? > > Yarek > > > On Thu, Dec 11, 2008 at 1:19 PM, Ian Eslick wrote: > >> Which version of elephant are you using? >> Which backend are you using? If BDB, which version? >> >> I presume *store-controller* open and reachable from the package you >> are in? >> >> Looks like the elephant store isn't in the proper state (e.g. was >> closed and *store-controller* was not zeroed out). >> >> On Dec 11, 2008, at 1:56 PM, Yarek Kowalik wrote: >> >> > I've converted my weblocks app to use elephant. One of the first >> > things I do is to initialize some variables. One of the variables >> > is a "demo" item, that I use as a placeholder until the user choses >> > something more specific. I'm having trouble initializing the >> > dummy. Here is my code: >> > >> > (defun init-dummy-item () >> > (let* ((ref-type 0) >> > (ref-id "dummy-item")) >> > (setf *dummy-item* >> > (unless-use (awhen (elephant:get-instances-by-value >> > 'product-item 'reference-id ref-id) >> > (find-if (lambda (item) >> > (eql (product-item-reference-type >> > item) ref-type)) >> > it)) >> > (make-instance >> > 'product-item >> > :reference-type ref-type >> > :reference-id ref-id >> > :title "No Item selected" >> > :url "/pub/static/dummy.html" >> > :image-url "/pub/images/dummy.png"))))) >> > >> > (The UNLESS-USE macro simply does unless, and if results are nil >> > uses the second sexp.) >> > >> > The problem is that I get an error (see trace below) when calling >> > GET-INSTANCES-BY-VALUE. What am I doing wrong? >> > >> > The nearest that I can gather is that GET-VALUE is called with the >> > secondd argument as NIL, but it's supposed to be something >> > reasonable, i.e a controller-index-table held by the controler. For >> > whatever raeson that's NIL. What's the deal here? Am I not alowed >> > to call the get-instances-by-* methods until I create at least one >> > item for each class I'm querying about? >> > >> > Yarek >> > >> > THE ERROR: >> > >> > There is no applicable method for the generic function >> > # >> > when called with arguments >> > ((FASHION-ORIGAMI::PRODUCT-ITEM >> > . FASHION-ORIGAMI::REFERENCE-ID) >> > NIL). >> > [Condition of type SIMPLE-ERROR] >> > >> > Restarts: >> > 0: [TERMINATE-THREAD] Terminate this thread (#> > worker-4" RUNNING {10029A0321}>) >> > >> > Backtrace: >> > 0: ((LAMBDA (SWANK-BACKEND::DEBUGGER-LOOP-FN)) #> > #) {1003F6A4C9}>) >> > 1: (SWANK::DEBUG-IN-EMACS #) >> > 2: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) #> > SWANK:SWANK-DEBUGGER-HOOK> #) >> > 3: (SWANK::CALL-WITH-REDIRECTED-IO #> > {10037B86D1}> #) >> > 4: (SWANK::CALL-WITH-CONNECTION # >> > #) >> > 5: (SWANK:INVOKE-SLIME-DEBUGGER #) >> > 6: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) #> > SWANK:SWANK-DEBUGGER-HOOK> #) >> > 7: (INVOKE-DEBUGGER #) >> > 8: (INVOKE-DEBUGGER #)[:EXTERNAL] >> > 9: ((SB-PCL::FAST-METHOD HUNCHENTOOT:MAYBE-INVOKE-DEBUGGER (T)) >> > # # #> > {1002E3A1A1}>) >> > 10: (SIGNAL #)[:EXTERNAL] >> > 11: (ERROR #)[:EXTERNAL] >> > 12: ((FLET #:LAMBDA43) #) >> > 13: ((FLET #:LAMBDA43) #)[:EXTERNAL] >> > 14: (SIGNAL #)[:EXTERNAL] >> > 15: (ERROR "~@> > function ~2I~_~S~\n ~I~_when called with arguments >> > ~2I~_~S.~:>")[:EXTERNAL] >> > Locals: >> > SB-DEBUG::ARG-0 = 3 >> > SB-DEBUG::ARG-1 = "~@> > generic function ~2I~_~S~\n .. >> > 16: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) #> > argument> # #> > ELEPHANT:GET-VALUE (3)>)[:EXTERNAL] >> > Locals: >> > SB-DEBUG::ARG-0 = 5 >> > SB-DEBUG::ARG-1 = : >> > SB-DEBUG::ARG-2 = : >> > SB-DEBUG::ARG-3 = #> > VALUE (3)> >> > 17: (ELEPHANT::ENSURE-SLOT-DEF-INDEX # >> > #) >> > Locals: >> > SB-DEBUG::ARG-0 = : >> > SB-DEBUG::ARG-1 = : >> > 18: ((SB-PCL::FAST-METHOD ELEPHANT:FIND-INVERTED-INDEX >> > (ELEPHANT:PERSISTENT-METACLASS T)) # >> > # # #> > argument>)[:EXTERNAL] >> > Locals: >> > SB-DEBUG::ARG-0 = : >> > SB-DEBUG::ARG-1 = : >> > SB-DEBUG::ARG-2 = : >> > SB-DEBUG::ARG-3 = : >> > SB-DEBUG::ARG-4 = : >> > 19: (ELEPHANT:MAP-INVERTED-INDEX # >> > # #)[:EXTERNAL] >> > Locals: >> > SB-DEBUG::ARG-0 = : >> > SB-DEBUG::ARG-1 = : >> > SB-DEBUG::ARG-2 = : >> > SB-DEBUG::ARG-3 = : >> > 20: (FASHION-ORIGAMI::INIT-DUMMY-ITEM) >> > Locals: >> > #:G137 = : >> > ANAPHORA:IT = : >> > >> > >> > >> > _______________________________________________ >> > elephant-devel site list >> > elephant-devel at common-lisp.net >> > http://common-lisp.net/mailman/listinfo/elephant-devel >> >> >> _______________________________________________ >> elephant-devel site list >> elephant-devel at common-lisp.net >> http://common-lisp.net/mailman/listinfo/elephant-devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yarek.kowalik at gmail.com Thu Dec 11 22:23:48 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Thu, 11 Dec 2008 14:23:48 -0800 Subject: [elephant-devel] Is it safe to use the same elephant store in two different processes? Message-ID: I'm trying to access BDB 4.5 backed elephant store (elephant-unstable) from two different lisp processes using the elephant package. Is this a safe thing to do? Yarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From yarek.kowalik at gmail.com Thu Dec 11 22:36:21 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Thu, 11 Dec 2008 14:36:21 -0800 Subject: [elephant-devel] Is it safe to use the same elephant store in two different processes? In-Reply-To: References: Message-ID: I'm guessing it is not... as that's what's causing some of my data problems.... I thought BDBs can handle multi-process access. Yarek On Thu, Dec 11, 2008 at 2:23 PM, Yarek Kowalik wrote: > I'm trying to access BDB 4.5 backed elephant store (elephant-unstable) from > two different lisp processes using the elephant package. Is this a safe > thing to do? > > Yarek > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yarek.kowalik at gmail.com Thu Dec 11 23:03:36 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Thu, 11 Dec 2008 15:03:36 -0800 Subject: [elephant-devel] Is it safe to use the same elephant store in two different processes? In-Reply-To: References: Message-ID: I've read section 4.13 in the doc, and it appears that it's possible to use two processes. However, when I launch two processes reading the same database, the first process trows errors (see trace below). Something is messing up the access/confing for the first process. I was hoping I would not be running into this problem, since I am running two processes on purpose: one is running standard HTTP server, the other SSL for secure longins, etc. I'm using BDB as a means of communicating the user login state: the user logs in on SSL side, sets up a pass tokens in the cookie, redirects to HTTP, and there the server fetches authorisation code from DB using the token. THis bug, then, stops me cold. Please help. Yarek Bad type argument: BDB-DB-ERROR [Condition of type SIMPLE-TYPE-ERROR] Restarts: 0: [RETRY] Retry SLIME REPL evaluation request. 1: [ABORT] Return to SLIME's top level. 2: [TERMINATE-THREAD] Terminate this thread (#) Backtrace: 0: (MAKE-CONDITION BDB-DB-ERROR)[:EXTERNAL] 1: (ERROR BDB-DB-ERROR)[:EXTERNAL] 2: ((SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION (DB-BDB::BDB-STORE-CONTROLLER T)) ..)[:EXTERNAL] 3: (ELEPHANT::MAP-BTREE-VALUES #) 4: (ELEPHANT::GET-DB-SCHEMAS # PRODUCT-SET) 5: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ELEPHANT::GET-DB-SCHEMAS (WEBLOCKS-ELEPHANT::ELEPHANT-CONTROLLER *FASHION-ORIGAMI-ELEPHANT-STORE*) 'PRODUCT-SET) #) 6: (SWANK::EVAL-REGION "(elephant::get-db-schemas (weblocks-elephant::elephant-controller *fashion-origami-elephant-store*) 'product-set)\n") On Thu, Dec 11, 2008 at 2:36 PM, Yarek Kowalik wrote: > I'm guessing it is not... as that's what's causing some of my data > problems.... I thought BDBs can handle multi-process access. > > Yarek > > > On Thu, Dec 11, 2008 at 2:23 PM, Yarek Kowalik wrote: > >> I'm trying to access BDB 4.5 backed elephant store (elephant-unstable) >> from two different lisp processes using the elephant package. Is this a >> safe thing to do? >> >> Yarek >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eslick at media.mit.edu Fri Dec 12 00:42:29 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Thu, 11 Dec 2008 19:42:29 -0500 Subject: [elephant-devel] Is it safe to use the same elephant store in two different processes? In-Reply-To: References: Message-ID: Multiple lisp processes (in a shared memory environment) should be safe, but I'm not sure if it's tested (anyone here done this?). I believe they have to use the same BDB environment, so you may need to do some checking on the proper open-store arguments to join vs. create an environment. Some defaults in weblocks or elephant may interfere with this. bdb-controller.lisp - open-controller definition shows you what arguments are passed to BDB when it is opened. Regardless, please upgrade to BDB 4.7. Most of us using unstable are on 4.6 or 4.7 now. I'd rather not track down issues associated with non-compatible upgrades (it should work, but testing coverage is getting tougher with all the system combinations of backends, BDB versions, DB versions, etc). Ian On Dec 11, 2008, at 5:23 PM, Yarek Kowalik wrote: > I'm trying to access BDB 4.5 backed elephant store (elephant- > unstable) from two different lisp processes using the elephant > package. Is this a safe thing to do? > > Yarek > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From eslick at media.mit.edu Fri Dec 12 00:44:51 2008 From: eslick at media.mit.edu (Ian Eslick) Date: Thu, 11 Dec 2008 19:44:51 -0500 Subject: [elephant-devel] Is it safe to use the same elephant store in two different processes? In-Reply-To: References: Message-ID: Take a peek at the issues around sharing a DB environment in the BDB docs and look at the open-controller args and see where you get. I won't be able to look into this for a few days but can answer the occasional question (probably best to e-mail me offline of the mailing list until we resolve this. Be great if anyone else has hacked through this, though. We should document the assumptions. Ian On Dec 11, 2008, at 6:03 PM, Yarek Kowalik wrote: > I've read section 4.13 in the doc, and it appears that it's possible > to use two processes. However, when I launch two processes reading > the same database, the first process trows errors (see trace below). > > Something is messing up the access/confing for the first process. > > I was hoping I would not be running into this problem, since I am > running two processes on purpose: one is running standard HTTP > server, the other SSL for secure longins, etc. I'm using BDB as a > means of communicating the user login state: the user logs in on SSL > side, sets up a pass tokens in the cookie, redirects to HTTP, and > there the server fetches authorisation code from DB using the token. > > THis bug, then, stops me cold. Please help. > > Yarek > > > Bad type argument: > BDB-DB-ERROR > [Condition of type SIMPLE-TYPE-ERROR] > > Restarts: > 0: [RETRY] Retry SLIME REPL evaluation request. > 1: [ABORT] Return to SLIME's top level. > 2: [TERMINATE-THREAD] Terminate this thread (# RUNNING {1002CAA181}>) > > Backtrace: > 0: (MAKE-CONDITION BDB-DB-ERROR)[:EXTERNAL] > 1: (ERROR BDB-DB-ERROR)[:EXTERNAL] > 2: ((SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION (DB- > BDB::BDB-STORE-CONTROLLER T)) ..)[:EXTERNAL] > 3: (ELEPHANT::MAP-BTREE-VALUES #) > 4: (ELEPHANT::GET-DB-SCHEMAS # lisp/projects/fashion-origami/data/> PRODUCT-SET) > 5: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ELEPHANT::GET-DB-SCHEMAS > (WEBLOCKS-ELEPHANT::ELEPHANT-CONTROLLER *FASHION-ORIGAMI-ELEPHANT- > STORE*) 'PRODUCT-SET) #) > 6: (SWANK::EVAL-REGION "(elephant::get-db-schemas (weblocks- > elephant::elephant-controller *fashion-origami-elephant-store*) > 'product-set)\n") > > On Thu, Dec 11, 2008 at 2:36 PM, Yarek Kowalik > wrote: > I'm guessing it is not... as that's what's causing some of my data > problems.... I thought BDBs can handle multi-process access. > > Yarek > > > On Thu, Dec 11, 2008 at 2:23 PM, Yarek Kowalik > wrote: > I'm trying to access BDB 4.5 backed elephant store (elephant- > unstable) from two different lisp processes using the elephant > package. Is this a safe thing to do? > > Yarek > > > _______________________________________________ > elephant-devel site list > elephant-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel From rsynnott at gmail.com Fri Dec 12 01:52:08 2008 From: rsynnott at gmail.com (Robert Synnott) Date: Fri, 12 Dec 2008 01:52:08 +0000 Subject: [elephant-devel] Is it safe to use the same elephant store in two different processes? In-Reply-To: References: Message-ID: <24f203480812111752l66469a76jd5995ec4e3c2dbfc@mail.gmail.com> 2008/12/11 Yarek Kowalik : > I've read section 4.13 in the doc, and it appears that it's possible to use > two processes. However, when I launch two processes reading the same > database, the first process trows errors (see trace below). > > Something is messing up the access/confing for the first process. > > I was hoping I would not be running into this problem, since I am running > two processes on purpose: one is running standard HTTP server, the other SSL > for secure longins, etc. I'm using BDB as a means of communicating the user > login state: the user logs in on SSL side, sets up a pass tokens in the > cookie, redirects to HTTP, and there the server fetches authorisation code > from DB using the token. > If that's all you need it for, you can actually probably run both servers in the same process. Rob From yarek.kowalik at gmail.com Fri Dec 12 03:40:33 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Thu, 11 Dec 2008 19:40:33 -0800 Subject: [elephant-devel] Is it safe to use the same elephant store in two different processes? Message-ID: <4941dd33.14b48c0a.63db.ffffd1df@mx.google.com> I thought about using one process, but there doesn't seem a way to do this in Weblocks. I'll try Ian's sugestion. Yarek -----Original Message----- From: Robert Synnott Sent: Thursday, December 11, 2008 5:52 PM To: Elephant bugs and development Subject: Re: [elephant-devel] Is it safe to use the same elephant store in two different processes? 2008/12/11 Yarek Kowalik : > I've read section 4.13 in the doc, and it appears that it's possible to use > two processes. However, when I launch two processes reading the same > database, the first process trows errors (see trace below). > > Something is messing up the access/confing for the first process. > > I was hoping I would not be running into this problem, since I am running > two processes on purpose: one is running standard HTTP server, the other SSL > for secure longins, etc. I'm using BDB as a means of communicating the user > login state: the user logs in on SSL side, sets up a pass tokens in the > cookie, redirects to HTTP, and there the server fetches authorisation code > from DB using the token. > If that's all you need it for, you can actually probably run both servers in the same process. Rob _______________________________________________ elephant-devel site list elephant-devel at common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel From henrik at evahjelte.com Fri Dec 12 10:28:20 2008 From: henrik at evahjelte.com (Henrik Hjelte) Date: Fri, 12 Dec 2008 11:28:20 +0100 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table In-Reply-To: <24f203480812110909s1ba4f6cbs929fd30e015c941b@mail.gmail.com> References: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com> <24f203480812102012h5e233c06qc6cb427cca5be89f@mail.gmail.com> <50e8e4f60812110245l5386d27hf1a00e441f3024d5@mail.gmail.com> <24f203480812110909s1ba4f6cbs929fd30e015c941b@mail.gmail.com> Message-ID: <50e8e4f60812120228o39e8935bx80e967b4194b98e3@mail.gmail.com> On Thu, Dec 11, 2008 at 6:09 PM, Robert Synnott wrote: > I have two other minor worries about this technique. First, it would > seem possible that if, say, we have slot1 in object1 which was set to > :some-opt at some time in the past, but is now totally unused by any > object, the blob row containing that would be considered dead; if > someone comes along while the cleanup is running and sets some slot to > :some-opt, then it will be in use again, but potentially get deleted > anyway, because it will have already been deemed dead. This isn't > really a problem for me; due to the nature of my dataset the chances > of this happening are remote, but I can imagine it'd be an issue for > some people This will not be a problem if the cleanup code is kept in a postgres transaction. The new slot update to :some-opt (after the delete) will result in a new entry in the blob table (from the stored procedure sp_ensure_bid). > > Secondly, there's the possibility that somebody could put a bid in, > say, a serialized list. This would NOT be detected by the procedure > I've outlined, and the thing with that bid would be deleted if not > referred to elsewhere. Again, this isn't a problem for me, but > somebody else might potentially be doing it. It would be a fair punishment. The blob table and bids should be considered internal to db_postmodern, you are not supposed to mess with it or use these values in your application code. > > Beyond those, though, in the normal case I am correct to assume that a > blob entry can be considered totally unused if not referred to in > either slots or any tree? Yes. /Henrik Hjelte From killerstorm at newmail.ru Fri Dec 12 16:15:19 2008 From: killerstorm at newmail.ru (Alex Mizrahi) Date: Fri, 12 Dec 2008 18:15:19 +0200 Subject: [elephant-devel] Postmodern backend; are values ever removed from the blob table References: <24f203480812101238n3a647d2ahe5b284727e96abf2@mail.gmail.com><24f203480812102012h5e233c06qc6cb427cca5be89f@mail.gmail.com><50e8e4f60812110245l5386d27hf1a00e441f3024d5@mail.gmail.com> <24f203480812110909s1ba4f6cbs929fd30e015c941b@mail.gmail.com> Message-ID: RS> I have two other minor worries about this technique. First, it would RS> seem possible that if, say, we have slot1 in object1 which was set to RS> :some-opt at some time in the past, but is now totally unused by any RS> object, the blob row containing that would be considered dead; if RS> someone comes along while the cleanup is running and sets some slot to RS> :some-opt, then it will be in use again, but potentially get deleted RS> anyway, because it will have already been deemed dead. RS> This isn't really a problem for me; due to the nature of my dataset the RS> chances of this happening are remote, but I can imagine it'd be an RS> issue for some people. yep, this might be a problem. unfortunately i do not see general solution to this problem without modifying blob table and likely introducing considerable overhead. first possible solution would be to mark blobs with a serial numbers on each sp_ensure_bid call, even if it finds it in table. then garbage blob collector could just filter out ones that were touched after it have started working. unfortunately this adds overhead of writinng data into database. another solution is to do a garbage collection in mark & sweep style -- first mark all objects white, then go through the table and mark reachable black. at same time, if sp_ensure_bid sees white blob, it marks it black too. then in final stage delete all white objects. a race condition during sweeping stage will be handled by postgresql's transactions -- if conflict happens, one of transactions will fail and would be restarted. disadvatage of this solution is that it will require large amounts of writing during collection. however it also does not require any temporary storage and works entirely on database side, and sp_ensure_bid performance is not impacted when no collection is going on. and totally different approach would be to delete blobs as soon as they get unused, via triggers or something. RS> Secondly, there's the possibility that somebody could put a bid in, RS> say, a serialized list. i think postmodern backend itself never does this, and as blobs are internal to backend, neither should users mess with blobs. RS> Beyond those, though, in the normal case I am correct to assume that a RS> blob entry can be considered totally unused if not referred to in RS> either slots or any tree? i guess so. blob ids might be also temporarily kept in memory, but only within a single transaction, so that shouldn't be an issue. neither blob ids are cached. From yarek.kowalik at gmail.com Fri Dec 12 20:52:26 2008 From: yarek.kowalik at gmail.com (Yarek Kowalik) Date: Fri, 12 Dec 2008 12:52:26 -0800 Subject: [elephant-devel] Is it safe to use the same elephant store in two different processes? In-Reply-To: References: