From avodonosov at yandex.ru Sat Feb 25 19:23:06 2012 From: avodonosov at yandex.ru (Anton Vodonosov) Date: Sat, 25 Feb 2012 23:23:06 +0400 Subject: [Bordeaux-threads-devel] test failures Message-ID: <584191330197786@web40.yandex.ru> Hello. In the scope of this project https://github.com/cl-test-grid/cl-test-grid I and Robert Goldman run tests of CL libraries on various CL implementations. Bordeaux threads tests fails on some implementations, sometimes due to buts in the tests I suppose. You may find reports here http://common-lisp.net/project/cl-test-grid/ Probably the most convenient report to overview a single library status is this one: http://common-lisp.net/project/cl-test-grid/pivot_lib-lisp_ql.html Between the test results for the recent version of quicklisp and the previous one, bordeaux-threads has only one regression: sbcl-1.0.54.45-a2bef14-macosx-x64. Probably it's due to some bug (like race conditions) in the tests, because the Qicklisp Blog doesn't list bordeaux-threads between updated projects in the last distro update, and if it's true, than the results are from the same bordeaux-threads version. http://blog.quicklisp.org/2012/02/february-dist-update-now-available.html Best regards, - Anton From vsedach at gmail.com Tue Feb 28 00:32:59 2012 From: vsedach at gmail.com (Vladimir Sedach) Date: Mon, 27 Feb 2012 19:32:59 -0500 Subject: [Bordeaux-threads-devel] test failures In-Reply-To: <584191330197786@web40.yandex.ru> References: <584191330197786@web40.yandex.ru> Message-ID: First I want to say that this is the first I've heard of the cl-test-grid project, and it is really cool. You should blog about it and let more people know. About testing CLISP and bordeaux-threads, is there a way to test and report results for a custom compiled version of CLISP? It's had thread support for some time now, but needs to be compiled with it turned on. Vladimir On Sat, Feb 25, 2012 at 2:23 PM, Anton Vodonosov wrote: > Hello. > > In the scope of this project https://github.com/cl-test-grid/cl-test-grid > I and Robert Goldman run tests of CL libraries on various CL implementations. > > Bordeaux threads tests fails on some implementations, sometimes due to > buts in the tests I suppose. > > You may find reports here > http://common-lisp.net/project/cl-test-grid/ > > Probably the most convenient report to overview a single library > status is this one: > http://common-lisp.net/project/cl-test-grid/pivot_lib-lisp_ql.html > > Between the test results for the recent version of quicklisp > and the previous one, bordeaux-threads has only one regression: > sbcl-1.0.54.45-a2bef14-macosx-x64. > > Probably it's due to some bug (like race conditions) in the tests, > because the Qicklisp Blog doesn't list bordeaux-threads > between updated projects in the last distro update, and if it's true, > than the results are from the same bordeaux-threads > version. http://blog.quicklisp.org/2012/02/february-dist-update-now-available.html > > Best regards, > - Anton > > _______________________________________________ > Bordeaux-threads-devel mailing list > Bordeaux-threads-devel at common-lisp.net > http://lists.common-lisp.net/cgi-bin/mailman/listinfo/bordeaux-threads-devel From avodonosov at yandex.ru Wed Feb 29 00:38:29 2012 From: avodonosov at yandex.ru (Anton Vodonosov) Date: Wed, 29 Feb 2012 04:38:29 +0400 Subject: [Bordeaux-threads-devel] test failures In-Reply-To: References: <584191330197786@web40.yandex.ru> Message-ID: <167201330475910@web66.yandex.ru> 28.02.2012, 04:32, "Vladimir Sedach" : > First I want to say that this is the first I've heard of the > cl-test-grid project, and it is really cool. You should blog about it > and let more people know. Hello. Thanks for the positive judgment. The project is in quite early stage, there are things to do to make it more useful. But I already reported about this project I think to all the mailing lists of the libraries having test failures. > About testing CLISP and bordeaux-threads, is there a way to test and > report results for a custom compiled version of CLISP? It's had thread > support for some time now, but needs to be compiled with it turned on. If you want to run the tests yourself, it's easy, the instructions are on the github page. As of today, CLISP is unable to submit test results to the server automatically, because of broken drakma; the easiest way to work this around is to use another lisp (e.g. SBCL, CCL) after the CLISP tests are completed, to submit the results. An example how to do this is at the last line of run.sh.sample. If you can't afford yourself to run the tests, then I will keep in mind the possibility to run bordeaux-threads on multi-threaded CLISP. I am subscribed to the clisp-devel and was aware that the multi-threading is almost completed, except for synchronizing internal CLISP data structures; but somehow missed the point when it shaped to clearly specified constraint: perform all defclass, defmethod and other CLOS definitions before spawning threads. Also I wasn't aware that bordeaux-threads already supports CLISP. So, thanks for the information. I can't switch to updating my CLISP immediately, because there are relatively simple, but very worthwhile things to implement in cl-test-grid: I am finishing a function to automatically find regressions between two quicklisp releases (would be also good to search for regressions between to versions of the same compiler, but I'll leave it for latter); next it's necessary to collect result not as just ok/fail for a whole library, but detailed to particular failed tests - otherwise we can't distinguish regression between library version where 1 test fails and 3 tests fails; this is already done for the tests based on RT framework; 7 more frameworks to do; then it's necessary to add more libraries; then I want to make running the tests more convenient... So, as I said, I will keep in mind the possibility to use multithreaded CLISP, but will postpone it to the next time when I will update CLISP on my machine. Is there anyone who have time to check the bordeaux-threads failed tests? Is it a bug in tests? Best regards, - Anton From vsedach at gmail.com Wed Feb 29 16:42:43 2012 From: vsedach at gmail.com (Vladimir Sedach) Date: Wed, 29 Feb 2012 11:42:43 -0500 Subject: [Bordeaux-threads-devel] test failures In-Reply-To: <167201330475910@web66.yandex.ru> References: <584191330197786@web40.yandex.ru> <167201330475910@web66.yandex.ru> Message-ID: I run a multithreaded CLISP build and will probably have some time to do the tests and see what needs to be fixed in drakma next week. Vladimir On Tue, Feb 28, 2012 at 7:38 PM, Anton Vodonosov wrote: > 28.02.2012, 04:32, "Vladimir Sedach" : >> First I want to say that this is the first I've heard of the >> cl-test-grid project, and it is really cool. You should blog about it >> and let more people know. > > Hello. Thanks for the positive judgment. The project is in quite early stage, > there are things to do to make it more useful. But I already reported about this > project I think to all the mailing lists of the libraries having test failures. > >> About testing CLISP and bordeaux-threads, is there a way to test and >> report results for a custom compiled version of CLISP? It's had thread >> support for some time now, but needs to be compiled with it turned on. > > If you want to run the tests yourself, it's easy, the instructions are on the > github page. As of today, CLISP is unable to submit test results > to the server automatically, because of broken drakma; the easiest way > to work this around is to use another lisp (e.g. SBCL, CCL) after the > CLISP tests are completed, to submit the results. An example how > to do this is at the last line of run.sh.sample. > > If you can't afford yourself to run the tests, then I will keep in mind the possibility > to run bordeaux-threads on multi-threaded CLISP. I am subscribed to the > clisp-devel and was aware that the multi-threading is almost completed, > except for synchronizing internal CLISP data structures; but somehow > missed the point when it shaped to clearly specified constraint: perform > all defclass, defmethod and other CLOS definitions before spawning > threads. Also I wasn't aware that bordeaux-threads already supports > CLISP. So, thanks for the information. > > I can't switch to updating my CLISP immediately, because there are > relatively simple, but very worthwhile things to implement in > cl-test-grid: I am finishing a function to automatically > find regressions between two quicklisp releases (would be > also good to search for regressions between to versions of > the same compiler, but I'll leave it for latter); next it's necessary > to collect result not as just ok/fail for a whole library, but detailed to > particular failed tests - otherwise we can't distinguish regression > between library version where 1 test fails and 3 tests fails; this > is already done for the tests based on RT framework; 7 more > frameworks to do; then it's necessary to add more libraries; > then I want to make running the tests more convenient... > So, as I said, I will keep in mind the possibility to use multithreaded CLISP, > but will postpone it to the next time when I will update CLISP on my machine. > > > Is there anyone who have time to check the bordeaux-threads > failed tests? Is it a bug in tests? > > Best regards, > - Anton From avodonosov at yandex.ru Wed Feb 29 16:48:36 2012 From: avodonosov at yandex.ru (Anton Vodonosov) Date: Wed, 29 Feb 2012 20:48:36 +0400 Subject: [Bordeaux-threads-devel] test failures In-Reply-To: References: <584191330197786@web40.yandex.ru> <167201330475910@web66.yandex.ru> Message-ID: <701971330534116@web113.yandex.ru> 29.02.2012, 20:42, "Vladimir Sedach" : > see what needs to be fixed in drakma next week. It's fixed already, should work in the next quicklisp release