From tfb at OCF.Berkeley.EDU Tue Jun 1 19:57:05 2004 From: tfb at OCF.Berkeley.EDU (Thomas F. Burdick) Date: Tue, 1 Jun 2004 12:57:05 -0700 Subject: [cells-devel] RTs for unbound cells Message-ID: <16572.57233.668375.854843@famine.OCF.Berkeley.EDU> [ not sure if the c-l.net lists are back up yet, one way to find out ] My jerk of a boss is making me write regression tests today; sometimes being self-employed sucks. So, I added to the basic suite of RTs I had for unbound cells, which are attached below. Rather than translate them from using RT (a super light-weight test harness, included with SBCL), I just wrote a po man's RT at the top of the file. These also use md-slot-makunbound, which is new since my last patch: (defun md-slot-makunbound (self slot-spec) (setf (md-slot-value self slot-spec) +unbound+) (slot-makunbound self slot-spec)) Now, back to those lovely database RTs, oh boy! -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: unbound-tests.lisp URL: From tfb at OCF.Berkeley.EDU Tue Jun 1 20:09:33 2004 From: tfb at OCF.Berkeley.EDU (Thomas F. Burdick) Date: Tue, 1 Jun 2004 13:09:33 -0700 Subject: [cells-devel] defmodel Message-ID: <16572.57981.72217.478678@famine.OCF.Berkeley.EDU> So, when I send my local changes to cells, I included a change to defmodel that removed :accessor, :reader, and :writer from the generated DEFCLASS, since DEFMODEL turns around and redefines them anyways. Well, it turns out that that tickles a buglet in some versions of SBCL, which causes it to emit scary-sounding warnings. I think it's fixed in 0.8.11, but it might be best to just forget that change. From ktilton at nyc.rr.com Wed Jun 2 03:09:35 2004 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Tue, 01 Jun 2004 23:09:35 -0400 Subject: [cells-devel] Re: defmodel In-Reply-To: <16572.57981.72217.478678@famine.OCF.Berkeley.EDU> References: <16572.57981.72217.478678@famine.OCF.Berkeley.EDU> Message-ID: <40BD44EF.2060504@nyc.rr.com> Thomas F. Burdick wrote: >So, when I send my local changes to cells, I included a change to >defmodel that removed :accessor, :reader, and :writer from the >generated DEFCLASS, since DEFMODEL turns around and redefines them >anyways. Well, it turns out that that tickles a buglet in some >versions of SBCL, which causes it to emit scary-sounding warnings. I >think it's fixed in 0.8.11, but it might be best to just forget that >change. > Already gone. :) Thanks for the RTs. Cells II looks good, should ship RSN. kt -- Home? http://tilton-technology.com Cells? http://www.common-lisp.net/project/cells/ Cello? http://www.common-lisp.net/project/cello/ Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film Your Project Here! http://alu.cliki.net/Industry%20Application From ktilton at nyc.rr.com Tue Jun 8 01:51:34 2004 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Mon, 07 Jun 2004 21:51:34 -0400 Subject: [cells-devel] Cells II Functional Requirements Message-ID: <40C51BA6.3010903@nyc.rr.com> Cells II seems to be OK, and is back at work running Cello as well as passing its regression tests. It will be released when I figure out whether to put everything up on the Cells or Cello project. I plan from here out to focus more on developing a commercial title (using Cello) than on porting Cello. This may mean I will not even look at OS X until the commercial title is ready to go and I want the OS X version (I will). This also means I want a very simple way to share my ongoing work on Cello/Cells and the easiest will be to have one cvs repository I update periodically. Anyway, for now, here is the spec which guided the Cells II rewrite: Suppose the application is at some steady state S in which all Cells (slots of instances) are either inputs (what I used to call c-variable) or have been computed from other Cells. Now the application polls a socket or OS event stream and gets a new input, which leads to some cell X being assigned a new value. Suppose also the new value for X is different from the prior value (according to a user-definable test). We now have not just a write operation but also a semantically meaningful change to X, so we also consider a new state S+ to have been established, with X being the only Cell to have reached state S+. Now suppose there exists some cell A which depends only on cell B which depends only on X. Suppose also that when cell B gets recomputed to reach state S+, the change test for B's new value indicates "no change". Let us then say B is dependent on the change to X but /unaffected/ by it. At this point, all data points other than X are obsolete. Formally: dX establishes state S+; X is by definition affected by dX, and reaches S+ trivially; all state unaffected by dX is considered to be at state S+; all other state is still at S. Cells is responsible for automatically ensuring all data points reach state S+ within these constraints: [] completeness: all non-lazy cells affected by X will be recomputed [] consistency: once state S+ exists, computations should involve values only from state S+. Note that this means state S++ cannot be allowed to arise during the transition from S to S+. [] effciency: only cells affected by X get recomputed, only once, and only those cells which change will be scheduled for output. [] useability: no re-entry of cell computation code, mainly so that programmers need not worry about it. kt From ktilton at nyc.rr.com Tue Jun 8 02:07:01 2004 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Mon, 07 Jun 2004 22:07:01 -0400 Subject: [cells-devel] Re: [cello-devel] Cells II Functional Requirements In-Reply-To: <40C51BA6.3010903@nyc.rr.com> References: <40C51BA6.3010903@nyc.rr.com> Message-ID: <40C51F45.4000905@nyc.rr.com> Kenny Tilton wrote: > Cells II seems to be OK, and is back at work running Cello as well as > passing its regression tests. It will be released when I figure out > whether to put everything up on the Cells or Cello project. I plan > from here out to focus more on developing a commercial title (using > Cello) than on porting Cello. This may mean I will not even look at OS > X until the commercial title is ready to go and I want the OS X > version (I will). This also means I want a very simple way to share my > ongoing work on Cello/Cells and the easiest will be to have one cvs > repository I update periodically. > > Anyway, for now, here is the spec which guided the Cells II rewrite: > > Suppose the application is at some steady state S in which all Cells > (slots of instances) are either inputs (what I used to call c-variable) > or have been computed from other Cells. > > Now the application polls a socket or OS event stream and gets a new > input, which leads to some cell X being assigned a new value. > > Suppose also the new value for X is different from the prior value > (according to a user-definable test). We now have not just a write > operation but also a semantically meaningful change to X, so we also > consider a new state S+ to have been established, with X being the > only Cell to have reached state S+. > > Now suppose there exists some cell A which depends only on cell B > which depends only on X. Suppose also that when cell B gets recomputed > to reach state S+, the change test for B's new value indicates "no > change". Let us then say B is dependent on the change to X but > /unaffected/ by it. should be "A and B are dependent on but unaffected by the change to X". kt From michael at naunton.us Tue Jun 8 02:54:18 2004 From: michael at naunton.us (Michael Naunton) Date: Tue, 08 Jun 2004 02:54:18 -0000 Subject: [cells-devel] Cells II Functional Requirements In-Reply-To: <40C51BA6.3010903@nyc.rr.com> References: <40C51BA6.3010903@nyc.rr.com> Message-ID: <1086667306.2427.336.camel@micron> So, (where = is an arbitrary function of) B = X A = B I can see how to gate on B such that A remains quiescent. What about: B = X C = B A = X+C+B After X changes, A and B need to move to S+. How do you get the ordering between A and B such that A only gets recomputed once? I could just recompute A, recomputing B recursively as needed, but surely C must be marked as "possibly needing recomputation" on dX, i.e. we must traverse all recursive dependants of X. Is the recursive step needed, or am I missing something obvious here? -- Michael On Mon, 2004-06-07 at 21:51, Kenny Tilton wrote: > Cells II seems to be OK, and is back at work running Cello as well as > passing its regression tests. It will be released when I figure out > whether to put everything up on the Cells or Cello project. I plan from > here out to focus more on developing a commercial title (using Cello) > than on porting Cello. This may mean I will not even look at OS X until > the commercial title is ready to go and I want the OS X version (I > will). This also means I want a very simple way to share my ongoing work > on Cello/Cells and the easiest will be to have one cvs repository I > update periodically. > > Anyway, for now, here is the spec which guided the Cells II rewrite: > > Suppose the application is at some steady state S in which all Cells > (slots of instances) are either inputs (what I used to call c-variable) > or have been computed from other Cells. > > Now the application polls a socket or OS event stream and gets a new > input, which leads to some cell X being assigned a new value. > > Suppose also the new value for X is different from the prior value > (according to a user-definable test). We now have not just a write > operation but also a semantically meaningful change to X, so we also > consider a new state S+ to have been established, with X being the > only Cell to have reached state S+. > > Now suppose there exists some cell A which depends only on cell B which > depends only on X. Suppose also that when cell B gets recomputed to > reach state S+, the change test for B's new value indicates "no change". > Let us then say B is dependent on the change to X but /unaffected/ by it. > > At this point, all data points other than X are obsolete. Formally: > > dX establishes state S+; > X is by definition affected by dX, and reaches S+ trivially; > all state unaffected by dX is considered to be at state S+; > all other state is still at S. > > Cells is responsible for automatically ensuring all data points reach > state S+ within these constraints: > > [] completeness: all non-lazy cells affected by X will be recomputed > > [] consistency: once state S+ exists, computations should involve > values only from state S+. Note that this means state S++ cannot be allowed > to arise during the transition from S to S+. > > [] effciency: only cells affected by X get recomputed, only once, and > only those cells which change will be scheduled for output. > > [] useability: no re-entry of cell computation code, mainly so that > programmers need not worry about it. > > > kt > > > > _______________________________________________ > cells-devel site list > cells-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/cells-devel From ktilton at nyc.rr.com Tue Jun 8 04:45:13 2004 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Tue, 08 Jun 2004 00:45:13 -0400 Subject: [cells-devel] Cells II Functional Requirements In-Reply-To: <1086667306.2427.336.camel@micron> References: <40C51BA6.3010903@nyc.rr.com> <1086667306.2427.336.camel@micron> Message-ID: <40C54459.9070501@nyc.rr.com> Michael Naunton wrote: >So, (where = is an arbitrary function of) > >B = X >A = B > >I can see how to gate on B such that A remains quiescent. > >What about: > >B = X >C = B >A = X+C+B > >After X changes, A and B need to move to S+. How do you get the >ordering between A and B such that A only gets recomputed once? I could >just recompute A, recomputing B recursively as needed, but surely C must >be marked as "possibly needing recomputation" on dX, i.e. we must >traverse all recursive dependants of X. > >Is the recursive step needed, or am I missing something obvious here? > Yes and no. Original cells (until last year when I did RoboCells and it started to bite me in the ass) violated the new rules and would indeed recalculate A twice, the first time using values from obsolete Cells still to be recalculated. So it is an astute observation. But it just takes a little engineering to avoid that, which I will describe shortly. But first FWIW the worst case is actually worse than A=X+C+B. It would be something like: :a (c? (if X C B)) In other words, we cannot use dependency info from state S to do lookahead and calculate things in the right order, because the new value of X or some dependency can cause other rules to branch differently and go after unanticipated other state. So there is some hair-raising engineering involved, which in part involves a rather heavy-handed trick: each input or make-unbound is assigned a sequentially increasing numeric ID. Eventually I will switch to get-internal-real-time and maybe start to have some fun with real time programming. Each cell, when it gets recalculated or when it is determined to be unaffected by an input, gets stamped with the current state ID. If it gets recalculated /and/ turns out to have changed, it gets stamped as changed. If it turns out to be unaffected, its "changed" flag will be nil. Each access checks that any cell mediating the slot is current, if not it runs down the dependency graph looking to see if anyone is (a) more current and (b) changed. If so, it recalculates. if not, as described above it gets stamped as current and unchanged. As we return back up the dependency graph, these results trigger other recalculations where appropriate. Finally the slot access returns the value requested. By stamping unaffected Cells as current, we ensure that getting to S+ touches each node at most once. In Cells Classic the model I had in mind was dataflow, where data got pushed along dependency "wires" to dependent cells which then recalculated and possibly pushed data further along the dependency graph. That is still the upshot in Cells II, but now the model is "Is everybody current with state S+?". We begin with the answer "no" only for the direct dependents of X, and set about recalculating them. If the first dependent is A and it turns out A uses B and B also uses X, there is no problem because the access to B (during the calculation of A) will JIT come up with a current value for B. recalculating B and then A will add their dependents to the list of cells known to require recalculation, but those are deferred until after the original calculation of A is complete (to avoid re-entrance into A should B just kick off recalculations willy-nilly and some user H (for "higher up" the dependency graph) of B turns out to be interested in A as well as B. So calculations just calculate, deferring notification of dependencies and outputs (ne echos) until the calculation has completed, to avoid re-entrance. Slot access always checks that any computed cell is current, satisfying that constraint. I should mention that a correspondent does not like the constraint that state S++ not arise until all state has reached state S+. Certainly a program could work correctly without satisfying that constraint. One can also abuse gotos and still come up with a correct program. Cells certainly worked magnificently for big applications while violating almost all the new rules flagrantly. But I see no loss of expressive power with these new constraints, so why not go with the correctness guarantee implicit in never using obsolete state or missing state changes by jumping from S to S++? kenny -- Home? http://tilton-technology.com Cells? http://www.common-lisp.net/project/cells/ Cello? http://www.common-lisp.net/project/cello/ Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film Your Project Here! http://alu.cliki.net/Industry%20Application From ktilton at nyc.rr.com Sun Jun 13 18:58:06 2004 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sun, 13 Jun 2004 14:58:06 -0400 Subject: [cells-devel] Update: Cells II, Cells/Tk, Cello Message-ID: <40CCA3BE.6050506@nyc.rr.com> Cello has paused while OpenGL kicks my butt over "picking", in which one has OpenGL determine what in a scene has been clicked (actually, more generally) what in a scene is where. But I have had this working before, so I just have to figure out which monkey broke that. The real pause comes from: Cells/Tk. I think this will be a nice quicky alternative to Cello, and certainly not require so many libraries. Meanwhile, on "Notes From All Over" front, this laugher from the book I bought last night for Maya, the killer 3D animation and modeling app: Maya's architecture can be explained [in] a single line: "nodes with attributes which are connected" Small world, eh? :) And yes, they speak of the "dependency graph". kenny -- Home? http://tilton-technology.com Cells? http://www.common-lisp.net/project/cells/ Cello? http://www.common-lisp.net/project/cello/ Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film Your Project Here! http://alu.cliki.net/Industry%20Application From ktilton at nyc.rr.com Sun Jun 20 12:50:20 2004 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sun, 20 Jun 2004 08:50:20 -0400 Subject: [cells-devel] Various announcements Message-ID: <40D5880C.9060200@nyc.rr.com> -- I just uploaded ck200406.zip and .sig to the cells ftp area. contents next, but here's a scoop: "ck" is short for Code Kenny. Anything found in such an upload is simply my working code base. I have to back off the surprising amount of time it takes to worry about Cells/Cello qua open source projects (maintaining ASD files I do not use for actual work, sorting out things which break under Lispworks) and worry about making money for a while. FYI, the LPR files are text files one can read to see which files get loaded in which order, and which projects depend on which other projects. btw, if anyone sends in patches to make the code compatible with some other environment I will work those into my code base, i am just not going to push that myself at this point. -- I for one will be doing Cello under the Cells project on common-lisp.net until further notice. As I feared, too much overlap. -- the version of Cells in this zip is what I have been calling Cells II, because there was a fundamental re-working of some crucial internals. Vanilla semantics of Cells is unchanged, and edge cases should behave better. Internals themselves are insanely better. And for the first time I came up with a somewhat formal spec for Cells which guided the internals rewrite, which I will be sharing RSN. -- included also in the zip is Cells-LTk, derived from Peter Hersh's LTk. I started with the idea of wrapping LTk, but that was going to be a mess, so I just subsituted a widget framework based on Cells for the original. I will look next week at whether Cello can be simply given a Tk backend. -- the included Cello uses display lists for everything, and runs five times faster. This work has made FTGL (FreeType in OpenGL) misbehave quite a bit, but I have some ideas as to why. kenny -- Home? http://tilton-technology.com Cells? http://www.common-lisp.net/project/cells/ Cello? http://www.common-lisp.net/project/cello/ Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film Your Project Here! http://alu.cliki.net/Industry%20Application