In preparation for a new release of my news reader, I have implemented categories support in the DB, and rewritten the Python feeds loader to use threads, and to clean up old news items.

The threaded feeds loader is more than 10x faster than the old one (obviously it depends on the number of feeds), and seems to perform well. If you feel adventurous, you can try it now. To use it, you also need the updated SQL schema in preparation for categories and the corresponding feed management page.

The next release will feature an optional frames-based interface, categories, search, and OPML export/import. It will take some time though, what with work and vacations coming up. Let me know if you have any features you would like to see implemented.

Russell learns that you don't always save any real money on Ebay. I am learning the same thing these days, luckily without spending as much money as Russell.

My P800 screen is broken, so I'm looking for a new phone since repairing the P800 will cost almost 200 euros. As much as I love my P800, I'm pretty tired of having to carry a brick in my pocket without even being able to read email or surf the web (my company's cellular subscription has GPRS and MMS disabled), and since I have lots of SonyEricsson gear (USB data and power cables, etc.) I am trying to buy a T600 (which is the smallest modern phone around), or a T610.

I have been tracking Ebay auctions for a couple of weeks, and I guess today I'll go to the store downstairs and buy a new T610. T600s are pretty rare, and with all the noob ebayers out there who start bidding as soon as the auction is up, and shippinh, they end up costing around 100 euro which is too much for a GPRS-less used phone.

T610s on Ebay go for 130-140 euros plus shipping, and since they're being discontinued for the T630, the shops that are still carrying them have lowered their prices to 170, which gets you a brand new phone with a 2-yr warranty without the risk of being scammed, and without having to wait for the seller to wake up and ship the item.

I guess Ebay is ok if you're looking for something special, or very rare (I just bougth a Swatch bag which isn't available to the public), otherwise I guess you'd make a better deal buying new from a store.

From time to time I receive an inquiry about the status of of my FeedOnFeeds rewrite, so today I took a few hours to add the main missing bits. For those of you who have never heard of FeedOnFeeds, Simple Aggregator is a web-based news aggregator. It supports Atom feeds, CDF, and the nine different versions of RSS thanks to Mark Pilgrim's Universal Feed Parser. There are still a few things missing (a frames view, OPML import/export, maybe categories), but it's quite usable and highly customizable if you know a bit of HTML and/or PHP.

It features a few improvements over its ancestor:

• a better SQL schema
• separation of code and presentation using HTML templates
• cleaner code, the web interface is composed of 3 PHP scripts, 3 templates, and two PEAR-compliant PHP classes (DB access and templating)
• a Python command line feeds loader making full use of Mark Pilgrim's Universal Feed Parser last-modified and etag support to reduce bandwidth usage
• better looking (IMHO), and more user-friendly interface

It has few requirements:

• a recent (2.2+) version of Python
• the MySQLdb module
• Mark Pilgrim's Universal Feed Parser
• MySQL 4.0+ with InnoDB support
• a recent PHP with the mysql extension
• the PEAR base class in your PHP include_path

If you want to give it a try, version 1.0 is available for download. Please send me any comments, patches, or features suggestions.

This entry expands on the subject of line endings in email (and news) messages, which I introduced in my previous entry on SMIME. In my (brief) experience working with mail and news messages, there are three different contexts involving line endings:

• transmission over the wire by SMTP (or NNTP)
• MIME canonicalization
• local handling of messages (storage, mail applications, etc.)

Transmission by SMTP RFC2822 specifies CR+LF line endings for on-the-wire transmission by SMTP.

MIME canonicalization RFC2049 specifies that CR+LFs are used in the canonicalization of MIME body parts BEFORE applying transfer encoding. It further states that

The output of the encoders may have to pass through one or more additional steps prior to being transmitted as a message. As such, the output of the encoder may not be conformant with the formats specified by RFC 822. In particular, once again it may be appropriate for the converter's output to be expressed using local newline conventions rather than using the standard RFC 822 CRLF delimiters.

In this context then, the appropriate linefeeds for an email message depend on the steps to perform after MIME canonicalization, and the local line endings convention.

Local handling It is apparent from the previous excerpt from RFC2049 that local handling of mail messages has to follow the local operating system's line endings convention. The most typical examples of this context are storing messages (eg in mailbox files or Maildirs), and piping/passing messages to mail handling programs like qmail-inject or Openssl's smime command. Googling on this context reveals the complexity often surrounding the question of proper line endings in email and news messages, and produces a few interesting links such as Life With Qmail: G.11. Carriage Return/Linefeed (CRLF) line breaks don't work, PHP bug #15841, a thread on the ietf- 822 list, and a thread on the Usenet Article Standard Update mailing list.

An illustrious victim of the line endings complexity I had to struggle with these past few days is Openssl's smime command, which has the nasty habit of outputting only the pkcs7 payload with CR+LF line endings when clear signing a message. The Python email package also seems to have a few problems with line endings, cf. bug #975330 submitted by Anders Hammarquist, who gave the very interesting Real-world email handling in python presentation at EuroPython 2004.