From tritchey at mac.com Thu Apr 16 12:28:26 2009 From: tritchey at mac.com (Timothy Ritchey) Date: Thu, 16 Apr 2009 08:28:26 -0400 Subject: [mel-base-devel] multipart messages and pop3 Message-ID: I'm having an issue extracting the parts of messages, particularly when trying to work with pop3 folders. I am wondering if I am doing something wrong. I am trying to extract any text/plain, or text/plain alternative from emails. The following works when I pass it an imaps folder, but mel:parts hangs when I pass it a pop3 folder. When I break it, it is hanging out in compute-bodystructure-using-folder, waiting on a stream. Am I grabbing messages the right way? (defun find-plain-text (message) (let ((parts (mel:parts message)) (body "Unable to process email contents")) (dolist (p parts) (multiple-value-bind (a b) (mel:content-type p) (if (and (equal a :TEXT) (equal b :PLAIN)) (let ((b (mel:part-body-string p))) (handler-case (setf body (mel:decode-quoted-printable b)) (t () (setf body b)))) (progn (when (and (equal a :MULTIPART) (equal b :ALTERNATIVE)) (find-plain-text p)))))) body)) (defun check-email (folder) (let* ((messages (mel:messages folder))) (dolist (message messages) (format t "~a~%" (find-plain-text message))))) I can call the following on the same message from the pop3 folder, and get the raw contents fine. (defmethod body (message) (with-output-to-string (str) (with-open-stream (stream (mel:message-body-stream message)) (loop for c = (read-char stream nil nil) while c do (write-char c str))))) Any help would be appreciated. - T From tritchey at mac.com Thu Apr 16 14:06:18 2009 From: tritchey at mac.com (Timothy Ritchey) Date: Thu, 16 Apr 2009 10:06:18 -0400 Subject: [mel-base-devel] multipart messages and pop3 In-Reply-To: References: Message-ID: <1E7B02A2-E0D3-499D-9C1F-B4D32344116E@mac.com> Thank you. I was getting the sense that it was hanging waiting on something from the server, so your suggestion of pulling into a local maildir makes sense, and gives me a direction to push. On a side note, is there a way to detect whether a text body needs decode-quoted-printable run on it? Right now I naively run it on every message, and then just fallback to the raw text if it fails. - T On Apr 16, 2009, at 9:52 AM, Jochen Schmidt wrote: > Am 16.04.2009 um 14:28 schrieb Timothy Ritchey : > >> I'm having an issue extracting the parts of messages, particularly >> when trying to work with pop3 folders. I am wondering if I am doing >> something wrong. I am trying to extract any text/plain, or text/plain >> alternative from emails. The following works when I pass it an imaps >> folder, but mel:parts hangs when I pass it a pop3 folder. When I >> break >> it, it is hanging out in compute-bodystructure-using-folder, waiting >> on a stream. >> >> Am I grabbing messages the right way? > > It could well be a bug - I've not tested it, but actually it may be > a better idea to fetch the messages from the POP3 server into a > local maildir folder and then use that for further processing. POP3 > is quite braindead and doesn't really lean to well accessing > messages randomly. Due to the lazy caching nature of mel-base there > would probably be multiple fetches of a single mail in POP3. > > What you try to do seems to be what mel-base does in FIND-VIEWABLE- > PART (see multiparts.lisp). > > ciao > Jochen > >> >> (defun find-plain-text (message) >> (let ((parts (mel:parts message)) >> (body "Unable to process email contents")) >> (dolist (p parts) >> (multiple-value-bind (a b) (mel:content-type p) >> (if (and (equal a :TEXT) (equal b :PLAIN)) >> (let ((b (mel:part-body-string p))) >> (handler-case >> (setf body (mel:decode-quoted-printable b)) >> (t () >> (setf body b)))) >> (progn >> (when (and (equal a :MULTIPART) (equal b :ALTERNATIVE)) >> (find-plain-text p)))))) >> body)) >> >> (defun check-email (folder) >> (let* ((messages (mel:messages folder))) >> (dolist (message messages) >> (format t "~a~%" (find-plain-text message))))) >> >> I can call the following on the same message from the pop3 folder, >> and >> get the raw contents fine. >> >> (defmethod body (message) >> (with-output-to-string (str) >> (with-open-stream (stream (mel:message-body-stream message)) >> (loop for c = (read-char stream nil nil) >> while c do (write-char c str))))) >> >> Any help would be appreciated. >> >> - T >> >> >> _______________________________________________ >> mel-base-devel mailing list >> mel-base-devel at common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/mel-base-devel > > -- > Jochen Schmidt > CRISPYLOGICS > Uhlandstr. 9, 90408 N?rnberg > > Fon: +49 (0)911 517 999-82 > Fax: +49 (0)911 517 999-83 > > mailto:info at crispylogics.com > http://www.crispylogics.com From js at codeartist.org Thu Apr 16 14:25:53 2009 From: js at codeartist.org (Jochen Schmidt) Date: Thu, 16 Apr 2009 16:25:53 +0200 Subject: [mel-base-devel] multipart messages and pop3 In-Reply-To: <1E7B02A2-E0D3-499D-9C1F-B4D32344116E@mac.com> References: <1E7B02A2-E0D3-499D-9C1F-B4D32344116E@mac.com> Message-ID: <01682051-46DE-4E8D-BCF0-B00E063E69D5@codeartist.org> -- Jochen Schmidt CRISPYLOGICS Uhlandstr. 9, 90408 N?rnberg Fon: +49 (0)911 517 999-82 Fax: +49 (0)911 517 999-83 mailto:info at crispylogics.com http://www.crispylogics.com Am 16.04.2009 um 16:06 schrieb Timothy Ritchey : > Thank you. I was getting the sense that it was hanging waiting on > something from the server, so your suggestion of pulling into a local > maildir makes sense, and gives me a direction to push. > > On a side note, is there a way to detect whether a text body needs > decode-quoted-printable run on it? Right now I naively run it on every > message, and then just fallback to the raw text if it fails. There is a header called Content-Transfer-Encoding. If it is "quoted- printable" then the body of the part is encoded using quoted printables. ciao, Jochen -- Jochen Schmidt CRISPYLOGICS Uhlandstr. 9, 90408 N?rnberg Fon: +49 (0)911 517 999-82 Fax: +49 (0)911 517 999-83 mailto:info at crispylogics.com http://www.crispylogics.com From jsc at crispylogics.com Thu Apr 16 13:33:22 2009 From: jsc at crispylogics.com (Jochen Schmidt) Date: Thu, 16 Apr 2009 13:33:22 -0000 Subject: [mel-base-devel] multipart messages and pop3 In-Reply-To: References: Message-ID: -- Jochen Schmidt CRISPYLOGICS Uhlandstr. 9, 90408 N?rnberg Fon: +49 (0)911 517 999-82 Fax: +49 (0)911 517 999-83 mailto:info at crispylogics.com http://www.crispylogics.com Am 16.04.2009 um 14:28 schrieb Timothy Ritchey : > I'm having an issue extracting the parts of messages, particularly > when trying to work with pop3 folders. I am wondering if I am doing > something wrong. I am trying to extract any text/plain, or text/plain > alternative from emails. The following works when I pass it an imaps > folder, but mel:parts hangs when I pass it a pop3 folder. When I break > it, it is hanging out in compute-bodystructure-using-folder, waiting > on a stream. > > Am I grabbing messages the right way? > > (defun find-plain-text (message) > (let ((parts (mel:parts message)) > (body "Unable to process email contents")) > (dolist (p parts) > (multiple-value-bind (a b) (mel:content-type p) > (if (and (equal a :TEXT) (equal b :PLAIN)) > (let ((b (mel:part-body-string p))) > (handler-case > (setf body (mel:decode-quoted-printable b)) > (t () > (setf body b)))) > (progn > (when (and (equal a :MULTIPART) (equal b :ALTERNATIVE)) > (find-plain-text p)))))) > body)) > > (defun check-email (folder) > (let* ((messages (mel:messages folder))) > (dolist (message messages) > (format t "~a~%" (find-plain-text message))))) > > I can call the following on the same message from the pop3 folder, and > get the raw contents fine. > > (defmethod body (message) > (with-output-to-string (str) > (with-open-stream (stream (mel:message-body-stream message)) > (loop for c = (read-char stream nil nil) > while c do (write-char c str))))) > > Any help would be appreciated. > > - T > > > _______________________________________________ > mel-base-devel mailing list > mel-base-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/mel-base-devel From jsc at crispylogics.com Thu Apr 16 13:52:46 2009 From: jsc at crispylogics.com (Jochen Schmidt) Date: Thu, 16 Apr 2009 13:52:46 -0000 Subject: [mel-base-devel] multipart messages and pop3 In-Reply-To: References: Message-ID: Am 16.04.2009 um 14:28 schrieb Timothy Ritchey : > I'm having an issue extracting the parts of messages, particularly > when trying to work with pop3 folders. I am wondering if I am doing > something wrong. I am trying to extract any text/plain, or text/plain > alternative from emails. The following works when I pass it an imaps > folder, but mel:parts hangs when I pass it a pop3 folder. When I break > it, it is hanging out in compute-bodystructure-using-folder, waiting > on a stream. > > Am I grabbing messages the right way? It could well be a bug - I've not tested it, but actually it may be a better idea to fetch the messages from the POP3 server into a local maildir folder and then use that for further processing. POP3 is quite braindead and doesn't really lean to well accessing messages randomly. Due to the lazy caching nature of mel-base there would probably be multiple fetches of a single mail in POP3. What you try to do seems to be what mel-base does in FIND-VIEWABLE- PART (see multiparts.lisp). ciao Jochen > > (defun find-plain-text (message) > (let ((parts (mel:parts message)) > (body "Unable to process email contents")) > (dolist (p parts) > (multiple-value-bind (a b) (mel:content-type p) > (if (and (equal a :TEXT) (equal b :PLAIN)) > (let ((b (mel:part-body-string p))) > (handler-case > (setf body (mel:decode-quoted-printable b)) > (t () > (setf body b)))) > (progn > (when (and (equal a :MULTIPART) (equal b :ALTERNATIVE)) > (find-plain-text p)))))) > body)) > > (defun check-email (folder) > (let* ((messages (mel:messages folder))) > (dolist (message messages) > (format t "~a~%" (find-plain-text message))))) > > I can call the following on the same message from the pop3 folder, and > get the raw contents fine. > > (defmethod body (message) > (with-output-to-string (str) > (with-open-stream (stream (mel:message-body-stream message)) > (loop for c = (read-char stream nil nil) > while c do (write-char c str))))) > > Any help would be appreciated. > > - T > > > _______________________________________________ > mel-base-devel mailing list > mel-base-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/mel-base-devel -- Jochen Schmidt CRISPYLOGICS Uhlandstr. 9, 90408 N?rnberg Fon: +49 (0)911 517 999-82 Fax: +49 (0)911 517 999-83 mailto:info at crispylogics.com http://www.crispylogics.com