From tcr at freebits.de Tue Nov 15 15:42:07 2005 From: tcr at freebits.de (Tobias C. Rittweiler) Date: Tue, 15 Nov 2005 16:42:07 +0100 Subject: [cl-utilities-devel] making EXTREMUM return multiple values -- and request for a function EXTREMA. Message-ID: <8764qtzzn4.fsf@GNUlot.localnet> Hi Peter, I'd like to suggest making EXTREMUM return all /equivalent/ extrema as multiple values, such that for instance: CL-USER> (extremum '((A . 3) (B . 1) (C . 2) (D . 1)) #'< :key #'cdr) (D . 1) (B . 1) My more restrictive (featurewise) version based on parts of your code but that I hacked together to fit my particular needs (no :start, :end keywords, only works for lists), looks like this: ;;; Inspired by www.cliki.net/EXTREMUM), but this function returns ;;; all extrema of sequence (if being equal) as multiple values. (defun extremum (sequence predicate &key (key #'identity)) (let* ((smallest-elements (list (first sequence))) (smallest-key (funcall key (first smallest-elements)))) (map nil #'(lambda (x) (let ((x-key (funcall key x))) (cond ((funcall predicate x-key smallest-key) (setq smallest-elements (list x)) (setq smallest-key x-key)) ;; both elements are considered equal if the predicate ;; returns false for (PRED A B) and (PRED B A) ((not (funcall predicate smallest-key x-key)) (push x smallest-elements))))) (rest sequence)) (apply #'values smallest-elements))) Similiarly, I'd like to suggest a new function EXTREMA which returns the N "topmost" extrema: CL-USER> (extrema 1 '(3 1 2 1) #'>) (3) CL-USER> (extrema 2 '(3 1 2 1) #'>) (3 2) CL-USER> (extrema 2 '(3 1 2 1) #'<) (1 1) CL-USER> (extrema 1 '((A . 3) (B . 1) (C . 2) (D . 1)) #'> :key #'cdr) ((A . 3)) CL-USER> (extrema 2 '((A . 3) (B . 1) (C . 2) (D . 1)) #'< :key #'cdr) ((D . 1) (B . 1)) My version -- which can almost certainly be written many times simpler -- is: (defun push-rotate-chop (item array &key (start 0) (end (length array))) (loop with saved-item = item for i from start to (1- end) do (rotatef saved-item (aref array i)))) ; CL-USER> (let ((array (make-array 5 :initial-contents '(1 2 3 4 5)))) ; (push-rotate-chop 'a array) array) ; #(A 1 2 3 4) ; CL-USER> (let ((array (make-array 5 :initial-contents '(1 2 3 4 5)))) ; (push-rotate-chop 'a array :start 1 :end 4) array) ; #(1 A 2 3 5) (defun extrema (n list predicate &key (key #'identity)) (let ((smallest-elements (make-array n)) (smallest-keys (make-array n)) (real-length 1)) (flet ((free-slot-p (x) (not x))) (setf (aref smallest-elements 0) (first list) (aref smallest-keys 0) (funcall key (first list))) (map nil #'(lambda (x) (let ((x-key (funcall key x))) (loop for key-idx from 0 for key across smallest-keys do (when (or (free-slot-p key) (funcall predicate x-key key) ; x-key < key (not (funcall predicate key x-key))) ; x-key = key (when (< real-length n) (incf real-length)) (push-rotate-chop x-key smallest-keys :start key-idx :end real-length) (push-rotate-chop x smallest-elements :start key-idx :end real-length) (loop-finish))))) (rest list)) (coerce (subseq smallest-elements 0 real-length) 'list)))) Well, you'll hopefully get inspired. :-) -t From sketerpot at gmail.com Tue Nov 15 23:35:37 2005 From: sketerpot at gmail.com (Peter Scott) Date: Tue, 15 Nov 2005 17:35:37 -0600 Subject: [cl-utilities-devel] Re: making EXTREMUM return multiple values -- and request for a function EXTREMA. In-Reply-To: <8764qtzzn4.fsf@GNUlot.localnet> References: <8764qtzzn4.fsf@GNUlot.localnet> Message-ID: <7e267a920511151535t76bc34b0u3cb0b1e6fcc1025d@mail.gmail.com> On 11/15/05, Tobias C. Rittweiler wrote: > I'd like to suggest making EXTREMUM return all /equivalent/ extrema as > multiple values, such that for instance: > > CL-USER> (extremum '((A . 3) (B . 1) (C . 2) (D . 1)) #'< :key #'cdr) > (D . 1) > (B . 1) That makes a lot of sense from a semantics perspective, and you've given me a lot to think about. However, I believe that this would be a better way to do it: * The EXTREMUM function would behave the way it does now. This is efficient and its correctness is immediately obvious from the source code. Using EXTREMUM indicates that you want only one answer, and you aren't particularly concerned about duplicates. This should be explicitly spelled out in the documentation (which I should probably get around to finishing sometime soon...). * The EXTREMA function will return a list of all equivalent extrema (by your definition of equality). This name and return type emphasizes that it is returning all the extrema, and nothing that is not an extremum. * The N-MOST-EXTREME function will behave like your EXTREMA function: it will return the n most extreme values in a sequence. > My more restrictive (featurewise) version based on parts of your code > but that I hacked together to fit my particular needs (no :start, :end > keywords, only works for lists), looks like this: > > ;;; Inspired by www.cliki.net/EXTREMUM), but this function returns > ;;; all extrema of sequence (if being equal) as multiple values. > (defun extremum (sequence predicate &key (key #'identity)) > (let* ((smallest-elements (list (first sequence))) > (smallest-key (funcall key (first smallest-elements)))) > (map nil > #'(lambda (x) > (let ((x-key (funcall key x))) > (cond ((funcall predicate x-key smallest-key) > (setq smallest-elements (list x)) > (setq smallest-key x-key)) > ;; both elements are considered equal if the predicate > ;; returns false for (PRED A B) and (PRED B A) > ((not (funcall predicate smallest-key x-key)) > (push x smallest-elements))))) > (rest sequence)) > (apply #'values smallest-elements))) Here's my slightly less hackish code, based on yours: (defun extremum (sequence predicate &key (key #'identity) (start 0) end) (setf sequence (subseq sequence start end)) (let* ((smallest-elements (list (elt sequence 0))) (smallest-key (funcall key (elt smallest-elements 0)))) (map nil #'(lambda (x) (let ((x-key (funcall key x))) (cond ((funcall predicate x-key smallest-key) (setq smallest-elements (list x)) (setq smallest-key x-key)) ;; both elements are considered equal if the predicate ;; returns false for (PRED A B) and (PRED B A) ((not (funcall predicate smallest-key x-key)) (push x smallest-elements))))) (subseq sequence 1)) (apply #'values smallest-elements))) This works with all proper sequences and accepts :start and :end arguments. It's also a little slower than yours, and it doesn't perform bounds checking (although that would be easy; I have a macro which handles all that for cl-utilities). These work as you would expect: (extremum '(3 2 1 1 2 1) #'<) (extremum #(3 2 1 1 2 1) #'<) (extremum #(3 2 1 1 2 1) #'< :end 4) (extremum #(3 2 1 1 2 1) #'< :start 3 :end 4) (extremum '(3 2 1 1 2 1) #'< :end 4) The complexity of that code bothers me, though. I worry about complex code; it gives me feelings of looming bugs that are about to bite me---or worse, bite someone else. Perhaps I can find a cleaner way to write it if I put some more thought into it. > Similiarly, I'd like to suggest a new function EXTREMA which returns the > N "topmost" extrema: > > CL-USER> (extrema 1 '(3 1 2 1) #'>) > (3) > CL-USER> (extrema 2 '(3 1 2 1) #'>) > (3 2) > CL-USER> (extrema 2 '(3 1 2 1) #'<) > (1 1) > > CL-USER> (extrema 1 '((A . 3) (B . 1) (C . 2) (D . 1)) #'> :key #'cdr) > ((A . 3)) > CL-USER> (extrema 2 '((A . 3) (B . 1) (C . 2) (D . 1)) #'< :key #'cdr) > ((D . 1) (B . 1)) Pretty good idea. > My version -- which can almost certainly be written many times > simpler -- is: > [snip] As I was understanding this, I came up with a mental model: it simply returned the first n elements of the sequence sorted with a certain key and predicate. This immediately suggested a many times simpler version: (defun extrema (n sequence predicate &key (key #'identity)) (subseq (sort (copy-seq sequence) predicate :key key) 0 n)) One thing you might want to consider is the stability of your extrema function, in the sense that quecksort is "unstable". (my-extrema 2 '((A . 3) (B . 1) (C . 2) (D . 1)) #'< :key #'cdr) => ((B . 1) (D . 1)) (your-extrema 2 '((A . 3) (B . 1) (C . 2) (D . 1)) #'< :key #'cdr) => ((D . 1) (B . 1)) My extrema function is stable in this case (by accident, since I didn't ask for a stable sort) and yours is unstable. This can sometimes be bad. I'll have to make -STABLE and regular versions. My extrema function conses about twice as much as yours, but it's a little faster on ACL 6.2 trial. > Well, you'll hopefully get inspired. :-) I have been. And once I get my big load of homework and exams worked off, I'll inspire myself to actually make some changes to cl-utilities. You've been a big help, thanks. -Peter From tcr at freebits.de Mon Nov 21 21:13:27 2005 From: tcr at freebits.de (Tobias C. Rittweiler) Date: Mon, 21 Nov 2005 22:13:27 +0100 Subject: [cl-utilities-devel] Re: making EXTREMUM return multiple values -- and request for a function EXTREMA. In-Reply-To: <7e267a920511151535t76bc34b0u3cb0b1e6fcc1025d@mail.gmail.com> (Peter Scott's message of "Tue, 15 Nov 2005 17:35:37 -0600") References: <8764qtzzn4.fsf@GNUlot.localnet> <7e267a920511151535t76bc34b0u3cb0b1e6fcc1025d@mail.gmail.com> Message-ID: <877jb17lhk.fsf@GNUlot.localnet> Peter Scott writes: > On 11/15/05, Tobias C. Rittweiler wrote: > > I'd like to suggest making EXTREMUM return all /equivalent/ extrema as > > multiple values, such that for instance: > > > > CL-USER> (extremum '((A . 3) (B . 1) (C . 2) (D . 1)) #'< :key #'cdr) > > (D . 1) > > (B . 1) > > That makes a lot of sense from a semantics perspective, and you've > given me a lot to think about. However, I believe that this would be a > better way to do it: > > * The EXTREMUM function would behave the way it does now. This is > efficient and its correctness is immediately obvious from the source > code. Using EXTREMUM indicates that you want only one answer, and you > aren't particularly concerned about duplicates. This should be > explicitly spelled out in the documentation (which I should probably > get around to finishing sometime soon...). > > * The EXTREMA function will return a list of all equivalent extrema > (by your definition of equality). This name and return type emphasizes > that it is returning all the extrema, and nothing that is not an > extremum. > > * The N-MOST-EXTREME function will behave like your EXTREMA function: > it will return the n most extreme values in a sequence. I arrived to agreeing with this scheme. The main argument for me, that you were certainly hinting at but didn't get spelled out as explicitely as to make it immediately apparent (to me anyway), is that EXTREMUM will invariantly return /one/ value and EXTREMA will invariantly return a /list/ of values: This way (ie. not having EXTREMUM return multiple values) a reader doesn't need to mentally ask herself whether the very invocation of EXTREMUM does deliberately ignore the possibility of multiple equivalent extrema or whether the author was just being sloppy. Whereas with EXTREMA returning a list, a writer will probably always map through that list---making the case whether one extremum or several ones pretty much equivalent. > Here's my slightly less hackish code, based on yours: > > [...] > > The complexity of that code bothers me, though. I worry about complex > code; it gives me feelings of looming bugs that are about to bite > me---or worse, bite someone else. Perhaps I can find a cleaner way to > write it if I put some more thought into it. It were quite cool if the boilerplate for dealing with the right subsequence &c could be *easily* hided like in the following way: (loop with smallest-elements = (list (elt sequence 0)) with smallest-key = (funcall key (elt sequence 0)) for x across sequence from (1+ start) to end do (let ((x-key (funcall key x))) (cond ((funcall predicate x-key smallest-key) ; x-key < smallest-key (setq smallest-elements (list x)) (setq smallest-key x-key)) ((not (funcall predicate smallest-key x-key)) ; x-key = smallest-key (push x smallest-elements)))) finally (return smallest-elements)) Ah, well. :-) > > Similiarly, I'd like to suggest a new function EXTREMA which returns the > > N "topmost" extrema: > > > > [...] > > > > CL-USER> (extrema 1 '((A . 3) (B . 1) (C . 2) (D . 1)) #'> :key #'cdr) > > ((A . 3)) > > CL-USER> (extrema 2 '((A . 3) (B . 1) (C . 2) (D . 1)) #'< :key #'cdr) > > ((D . 1) (B . 1)) > > Pretty good idea. > > > My version -- which can almost certainly be written many times > > simpler -- is: [snip] > > As I was understanding this, I came up with a mental model: it simply > returned the first n elements of the sequence sorted with a certain > key and predicate. This immediately suggested a many times simpler > version: > > (defun extrema (n sequence predicate &key (key #'identity)) > (subseq (sort (copy-seq sequence) predicate :key key) > 0 n)) Yes. Well, I initially had a (FIRST (SORT ...)) in my code, but then needed all topmost equivalent elements, so I wrote something like: (let* ((list-with-long-name (foo ...)) (sorted-list-with-long-name (sort list-with-long-name ...)) (first-entry (first sorted-list-with-long-name))) (topmost-elements-if (lambda (x) (= x (compute-something first-entry))) sorted-list-with-long-name :key #'compute-something)) Which I considered to be so ugly that I went off and adapted your EXTREMUM function to return all equivalent extrema as multiple values. Well, then I found out that what I really needed is to always get the N most extreme values, thus I wrote my version of EXTREMA and, as you'll probably understand, my head was just immersed into how I wrote EXTREMUM (which I wanted to evolutionize to fit my new needs), such that I totally forgot my beginnings with (FIRST (SORT ...)) and ended up with the code of my last mail. > [...] My extrema function is stable in this case (by accident, since I > didn't ask for a stable sort) and yours is unstable. This can > sometimes be bad. I'll have to make -STABLE and regular versions. I'd make that a keyword parameter :STABLE, maybe? > I have been. And once I get my big load of homework and exams worked > off, I'll inspire myself to actually make some changes to > cl-utilities. You've been a big help, thanks. I have to thank you. Keep up the work. -t