From edi at agharta.de Fri Mar 10 21:25:43 2006 From: edi at agharta.de (Edi Weitz) Date: Fri, 10 Mar 2006 22:25:43 +0100 Subject: [rdnzl-devel] Type lookup problems, and solutions. In-Reply-To: <11600-64036@sneakemail.com> (Dan Muller's message of "Fri, 24 Feb 2006 19:34:03 -0500") References: <16468-90626@sneakemail.com> <11600-64036@sneakemail.com> Message-ID: On Fri, 24 Feb 2006 19:34:03 -0500, "Dan Muller" wrote: > On further reflection, I've decided to work on this as a separate > package layered on top of RDNZL, so that I can try implementing some > other features that I'd like, related to the management of package > use-lists and the timing of type resolution, which would be very > intrusive to incorporate into RDNZL. That sounds interesting. Maybe you can make this available somewhere once it's finished. If you want, we can also add it as some kind of "contrib" folder to RDNZL. From edi at agharta.de Fri Mar 10 21:24:32 2006 From: edi at agharta.de (Edi Weitz) Date: Fri, 10 Mar 2006 22:24:32 +0100 Subject: [rdnzl-devel] Type lookup problems, and solutions. In-Reply-To: <16468-90626@sneakemail.com> (Dan Muller's message of "Thu, 23 Feb 2006 21:32:25 -0500") References: <16468-90626@sneakemail.com> Message-ID: Hi Dan! As I already said in private email: sorry for the long delay. On Thu, 23 Feb 2006 21:32:25 -0500, "Dan Muller" wrote: > Remember that problem I was having importing some types? I'm not > sure why, but on my system, System.Type.GetTypes is more picky than > RDNZL seems to expect: > > ;;; > ;;; To demonstrate these failing tests, start with a clean Lisp image, > ;;; then load RDNZL, then load this file. Assemblies can't be > ;;; unloaded except by unloading their containing application domain, > ;;; and you can't unload the default application domain. > ;;; > ;;; It's possible that the tests are sensitive to the assemblies > ;;; available on my machine, but I can't find an indication in > ;;; Microsoft's documentation of why that might be the case. > ;;; > (in-package :cl-user) > (use-package :rdnzl) There's a RDNZL-USER package for this kind of experimenting, BTW. > (import-assembly "System.Windows.Forms") > > ;; This fails, because the system doesn't find the type. > ;; Comment it out and try reloading to move on to the next test. > (new "System.Windows.Forms.Form") > > ;; This fails, too. Same problem. > ;; Comment it out and try reloading to move on to the next test. > (import-type "System.Windows.Forms.Form") > > ;; I also tried direct calls to System.Type.GetType with various > ;; abbreviations of the fully qualified type name, leaving off > ;; trailing pieces of the full assembly name. None of these succeeded. > > ;; This succeeds. > (import-type "System.Windows.Forms.Form" > (load-assembly "System.Windows.Forms")) > > ;; This succeeds, because RDNZL now remembers what assembly it found > ;; Form in. > (let ((form > ;; This fails, because the system doesn't find the type. > (new "System.Windows.Forms.Form"))) > (invoke form "Dispose")) > > Well, the recommended way to do it is this one: (import-types "System.Windows.Forms" "Form") That should succeed. > I spent a lot of time poking around in RDNZL and the .NET framework, > and reading documentation. The documentation is for 2.0, FWIW, but I > was careful to look only at classes and methods that were available > in 1.1. What follows is a rather lengthy discussion of some possible > improvements/fixes I've come up with. This goes rather beyond fixing > the immediate problem, but please give it some consideration. > > - Have IMPORT-TYPE always retrieve and cache the assembly-qualified > name of the type. > > Downside: This will include the exact version. This means that in a > delivered application involving a saved image, there is no way to > use an updated version of a used assembly. (Haven't tested for this, > though.) Might be an acceptable short-term fix, though. I don't like that. Being able to "cleanly" deliver applications was one of my main goals when developing RDNZL. > - Have IMPORT-ASSEMBLY supply the assembly to IMPORT-TYPE. Good > idea, since it's currently inconsistent about this vis-a-vis > IMPORT-TYPES. But it doesn't address the problem described above. > > - In MAKE-TYPE-FROM-NAME, do the following. If the name is > assembly-qualified (contains an unquoted paren), then use > System.Type.GetType() as before. Otherwise, call > System.Reflection.Assembly.GetType() on each assembly returned by > System.AppDomain.CurrentDomain.GetAssemblies(). The search should be > exhaustive in order to detect ambiguities, which should be signaled > as errors. In IMPORT-TYPE, don't accept assembly-qualified names, > and cache the actual type object instead of the fully-qualified > name. During shutdown, discard these type object references. During > initialization, do the lookups again. (Changes to assemblies that > introduce ambiguities will cause an error during initialization.) > > Downsides: Names given to RDNZL for resolution must not be > assembly-qualified. (Or, perhaps better, detect these and don't > cache them.) It is not possible for the system to deal predictably > with like-named types in multiple assemblies. All loaded assemblies > would be searched, even if they were not loaded via RDNZL. The > latter problem might be a liability for advanced uses, > e.g. involving dynamic assemblies that are being built by the > application. > > I tried creating a C# console application that used two assemblies > with conflicting type names. You get an error only if you actually > reference the conflicting type name. The error documentation > explains how C# allows this problem to be worked around, using a > compiler switch that assigns a prefix to all namespaces in a given > assembly: > > http://msdn2.microsoft.com/en-us/library/64wh5743.aspx > > AFAIK, implementing a similar solution requires work on the > compiler's part. I have not noticed any direct support for this > aliasing in the .NET framework. So a similar solution in RDNZL would > require tracking assemblies. However, this might be desirable in > order to address the other problem mentioned above. > > If RDNZL would track loaded assemblies, only those assemblies would > be searched for type name matches. If we associate an optional alias > with an assembly (e.g. via an optional argument to LOAD-ASSEMBLY), > then that assembly would only be searched if the type name includes > the prefix, and only after removing the prefix. > > As a further enhancement, I'd like to actually get rid of the need > for importing entirely. It doesn't seem unreasonable to search for a > type regardless of whether it has been imported or not, provided > that it is then always cached in *TYPE-HASH*. This way *TYPE-HASH* > builds up information about all types referenced via RDNZL. Users > can still make calls to primitives like System.Type.GetType() if > they ever want to bypass such activity. This is also true for > LOAD-ASSEMBLY, if it starts tracking assemblies. Of course, the > aliasing wouldn't work if you bypass RDNZL. > > I have prototyped some of the low-level pieces needed for these > changes, entirely in Lisp using RDNZL. I didn't want to put more > work into it unless I'm sure there's interest in including it in > RDNZL. (Or at least, I'll do the work differently if there isn't.) > I'm particularly uncertain about issues related to saving and > loading Lisp images, something I've never tried out, although I > think I understand the basic concerns there. I'm not sure I fully understand where you're heading at the moment. I think it's obvious that RDNZL can't quite work like, say, a C# compiler due to Lisp's dynamic nature and due to the loose coupling between Lisp and the .NET runtime. If you can come up with something that's more versatile without complicating matters for RDNZL developers, that's fine for me. Backwards compatibility is not really an issue as I /think/ we don't have more than half a dozen users anyway... :) Being able to save and re-load images and to deliver applications is an important issue, though. As well as performance. I'm not advocating pre-mature optimization but I think one should shy away from solutions that are slow by design. I'm sorry if my answer isn't as detailed as you expected - I'm still too busy with other things to think about this ATM. Maybe it'd be helpful if you could demonstrate how the existing RDNZL examples would look like after applying the changes you want to have. Thanks, Edi. From s8ctxw402 at sneakemail.com Sat Mar 25 15:36:21 2006 From: s8ctxw402 at sneakemail.com (Dan Muller) Date: Sat, 25 Mar 2006 10:36:21 -0500 Subject: [rdnzl-devel] CL-CLR, an alternative way of working with RDNZL Message-ID: <30659-18086@sneakemail.com> I recently put together a system that I layered on top of RDNZL, in order to experiment with some alternative ways of interacting with CLR/.NET code. I'm hoping that some of the ideas could be incorporated into RDNZL. Edi's too busy to look it over right now, and he suggested that I offer for perusal here on the mailing list. Aside from making type lookup a little easier, this system tries to make your .NET-using Lisp code a bit more Lisp-like. Performance is shabby because of the layering over RDNZL; much, if not all of this, could eventually be fixed. Nonetheless, I've been using this system successfully for the past few weeks as a vehicle for exploring Direct3D programming on a WinXP system, using the LispWorks trial edition of Lisp. In order to get reasonable performance, I had to use CL-CLR:DEFINE-CLR-CALL (a thin veneer over DEFINE-RDNZL-CALL) for most calls in the rendering loop. This system *should* work with any Lisp that is supported by RDNZL. The package includes several of Microsoft's Direct3D Tutorial programs, translated to Lisp. If you have ASDF, RDNZL, and Direct3D installed, you ought to be able to run the examples with a minimum of fuss. You can download it from here: http:\\..................\cl-clr.zip Comments are welcome. You are also encouraged to laugh at the excessively long TODO.txt file. -- Dan Muller ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From s8ctxw402 at sneakemail.com Sat Mar 25 16:27:33 2006 From: s8ctxw402 at sneakemail.com (Dan Muller) Date: Sat, 25 Mar 2006 11:27:33 -0500 Subject: [rdnzl-devel] CL-CLR, an alternative way of working with RDNZL In-Reply-To: <30659-18086@sneakemail.com> References: <30659-18086@sneakemail.com> Message-ID: <16910-25028@sneakemail.com> Hmm, that didn't work quite right. Looks like Sneakemail censored my Web server's DNS name. Here's the download location again in slightly mangled form: http://spookydistance-dot-com/cl-clr.zip Dan Muller s8ctxw402-at-sneakemail.com |RDNZL-devel/via Sneakemail| wrote: > I recently put together a system that I layered on top of RDNZL, in > order to experiment with some alternative ways of interacting with > CLR/.NET code. I'm hoping that some of the ideas could be > incorporated into RDNZL. Edi's too busy to look it over right now, > and he suggested that I offer for perusal here on the mailing list. > > Aside from making type lookup a little easier, this system tries to > make your .NET-using Lisp code a bit more Lisp-like. Performance is > shabby because of the layering over RDNZL; much, if not all of this, > could eventually be fixed. Nonetheless, I've been using this system > successfully for the past few weeks as a vehicle for exploring > Direct3D programming on a WinXP system, using the LispWorks trial > edition of Lisp. In order to get reasonable performance, I had to use > CL-CLR:DEFINE-CLR-CALL (a thin veneer over DEFINE-RDNZL-CALL) for > most calls in the rendering loop. > > This system *should* work with any Lisp that is supported by RDNZL. > > The package includes several of Microsoft's Direct3D Tutorial > programs, translated to Lisp. If you have ASDF, RDNZL, and Direct3D > installed, you ought to be able to run the examples with a minimum of > fuss. > > You can download it from here: http:\\..................\cl-clr.zip > > Comments are welcome. You are also encouraged to laugh at the > excessively long TODO.txt file. > > -- > Dan Muller > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > _______________________________________________ > rdnzl-devel mailing list > rdnzl-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/rdnzl-devel > > > -- Dan ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From s8ctxw402 at sneakemail.com Wed Mar 29 02:31:41 2006 From: s8ctxw402 at sneakemail.com (Dan Muller) Date: Tue, 28 Mar 2006 21:31:41 -0500 Subject: [rdnzl-devel] CL-CLR, an alternative way of working with RDNZL In-Reply-To: <16910-25028@sneakemail.com> References: <30659-18086@sneakemail.com> <16910-25028@sneakemail.com> Message-ID: <27456-73324@sneakemail.com> Hi, it's just me talking to myself again. I've discovered some shortcomings of System.Type.InvokeMember, which CL-CLR depends on, when it comes to passing null arguments. Also, there's a bug in CL-CLR with respect to ByRef types (which doesn't surprise me, since I hadn't looked at them in detail yet). It will probably take me a few days to figure out how to work around them, while preserving some of the unique features of CL-CLR that I'm most interested in. If you do take a look at CL-CLR (and I still hope to get feedback on some of the ideas talked about in the README.txt file), be aware of these two problems: - If you pass a null value as an argument, there is no type information for InvokeMember to use in resolving among overloaded members. If an ambiguity results, the call will fail. - ByRef parameters (ref or out in C#) are not copied back out. RDNZL containers carry quasi-static type information with them, and so a container representing a null object reference does, too. In CL-CLR, I'm experimenting with using the types derived from the object values, and it effectively ignores the types embedded in the containers. This has the interesting effect of making overloaded methods act much like Lisp methods (even more so than RDNZL already does), potentially run-time polymorphic on all parameters. And like Lisp methods, if you pass null (in analogy to NIL) for an argument, the only type it will really match is Object. But, interestingly, the default binding behavior in InvokeMember seems to allow a null argument to match anything at all, as long as there's no ambiguity -- in other words, as long as that particular parameter isn't overloaded. RDNZL has some interesting problems of its own in handing ByRef parameters, mainly because REF modifies containers in-place. Assuming that some-type is a type object for SomeType, and MyStaticMethod returns a new value through its parameter: (let* ((a (rdnzl:new some-type)) (b a)) (rdnzl:invoke "MyHandyClass" "MyStaticMethod" (ref a)) (print a) (print b)) After the call to INVOKE, both a *and* b will reference the new object. Other even stranger things can happen: (let ((a (rdnzl:new some-type))) (invoke a "AnInstanceMethod" (ref a))) This fails entirely, because a becomes temporarily of type SomeType&, and the lookup of AnInstanceMethod fails on that type. A possible strategy for fixing this in RDNZL: Make REF a macro taking a SETF-able form. It expands to a form that creates an instance of a RDNZL REFERENCE class, containing a reference to the result of evaluating the form, and a closure that will SETF the original form when called with a new value. INVOKE et al must detect these parameters, arrange to *clone* the input continer (with a new GCHandle value!), and use the cloned container to receive the result. Finally, after the member call, pass the new value to the closure. CL-CLR is still available here for now: http://64.91.250.185/cl-clr.zip -- Dan Muller ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From lisp at spikeisland.com Wed Mar 29 10:40:57 2006 From: lisp at spikeisland.com (lisp) Date: Wed, 29 Mar 2006 11:40:57 +0100 Subject: [rdnzl-devel] CL-CLR, an alternative way of working with RDNZL In-Reply-To: <27456-73324@sneakemail.com> References: <30659-18086@sneakemail.com> <16910-25028@sneakemail.com> <27456-73324@sneakemail.com> Message-ID: <1143628857.28073.257805081@webmail.messagingengine.com> Hi Dan I'm listening :) Sadly I've not had anytime to devote to RDNZL stuff recently - but I'll certainly take a look and give you some feedback as soon as I get the opportunity. I have an experimental local version of RDNZL with some performance oriented mods to reduce consing on the lisp and .NET sides - optimising the boxing/unboxing, specialisations for different numbers of arguments in event callbacks etc. I've been trying to get the performance up to a level where it is viable to use for high volumes of calls and events (the DataGridView in virtual mode is one example). This is still all rather hacked together and needs tidying up. - Dominic On Tue, 28 Mar 2006 21:31:41 -0500, "Dan Muller" said: > Hi, it's just me talking to myself again. > > I've discovered some shortcomings of System.Type.InvokeMember, which > CL-CLR depends on, when it comes to passing null arguments. Also, > there's a bug in CL-CLR with respect to ByRef types (which doesn't > surprise me, since I hadn't looked at them in detail yet). > > It will probably take me a few days to figure out how to work around > them, while preserving some of the unique features of CL-CLR that I'm > most interested in. If you do take a look at CL-CLR (and I still hope > to get feedback on some of the ideas talked about in the README.txt > file), be aware of these two problems: > > - If you pass a null value as an argument, there is no type > information for InvokeMember to use in resolving among overloaded > members. If an ambiguity results, the call will fail. > > - ByRef parameters (ref or out in C#) are not copied back out. > > RDNZL containers carry quasi-static type information with them, and so > a container representing a null object reference does, too. In CL-CLR, > I'm experimenting with using the types derived from the object values, > and it effectively ignores the types embedded in the containers. This > has the interesting effect of making overloaded methods act much like > Lisp methods (even more so than RDNZL already does), potentially run- > time polymorphic on all parameters. And like Lisp methods, if you pass > null (in analogy to NIL) for an argument, the only type it will really > match is Object. But, interestingly, the default binding behavior in > InvokeMember seems to allow a null argument to match anything at all, > as long as there's no ambiguity -- in other words, as long as that > particular parameter isn't overloaded. > > RDNZL has some interesting problems of its own in handing ByRef > parameters, mainly because REF modifies containers in-place. Assuming > that some-type is a type object for SomeType, and MyStaticMethod > returns a new value through its parameter: > > (let* ((a (rdnzl:new some-type)) (b a)) (rdnzl:invoke > "MyHandyClass" "MyStaticMethod" (ref a)) (print a) (print b)) > > After the call to INVOKE, both a *and* b will reference the new > object. Other even stranger things can happen: > > (let ((a (rdnzl:new some-type))) (invoke a "AnInstanceMethod" > (ref a))) > > This fails entirely, because a becomes temporarily of type SomeType&, > and the lookup of AnInstanceMethod fails on that type. > > A possible strategy for fixing this in RDNZL: Make REF a macro taking > a SETF-able form. It expands to a form that creates an instance of a > RDNZL REFERENCE class, containing a reference to the result of > evaluating the form, and a closure that will SETF the original form > when called with a new value. INVOKE et al must detect these > parameters, arrange to *clone* the input continer (with a new GCHandle > value!), and use the cloned container to receive the result. Finally, > after the member call, pass the new value to the closure. > > > CL-CLR is still available here for now: > http://64.91.250.185/cl-clr.zip > > -- > Dan Muller > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > _______________________________________________ > rdnzl-devel mailing list rdnzl-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/rdnzl-devel