Last night Italy suffered a massive blackout, so this morning I came back from the lake Maggiore to find my server dead. The server, which runs all our mail and web sites, was (yes, was) a very old Compaq Deskpro with a Celeron 300 and twin 4Gb disks configured as a software RAID-1 array. I liked it because it was a very silent and low- consumption machine with no CPU fan, and I don't need more power than that to manage a few mailboxes, a couple of web sites, and my home LAN.

I knew it was in bad shape, but I always managed to reboot it in the past, the very few times I needed to- Today I tried everything, but the bios screen refuses to come up, and no beeps escape from the speaker at boot time.

So I wasted all day trying to let my desktop's twin machine (an Athlon 750 PC I built three years ago) see my server's two disks, and remount the RAID array. The new machine's BIOS screws up the disks geometry, and if I attach both of them on the two IDE channels, I see only the first one. Weird.

So I burned a mini-CD with tomsrtbt, transferred everything on a 20Gb disk, rebuilt the array, hotraidadded one of the two 4Gb disks, reconfigured LILO, rebooted, and LILO started spitting out error messages. Reconfigured, double checked everything, LILO comes up with a checksum error. Hmmm I'm getting old for this, I remember when it was fun but it's not anymore.

In the end I installed Grub, took the 4Gb disk off the array, fscked to death the 20Gb disk who got corrupted in the meantime, and the server is back up. Tomorrow I will buy a second 20Gb or 40Gb disk and add it to the RAID array.

The only good thing to come out of this is that I learned something more about Grub, and I really like it. If things get tough, Grub is your friend. A few useful links if you want to switch from LILO to Grub, boot a software RAID partition from Grub, convert a running system to software RAID (mainly geared towards Debian users, Slackware users may find this more useful).

My fiber optic link where this site is usually served from is still down, something to do with the Catalyst in the basement who serves the whole building, I have called a few times but all I'm getting is it will be fixed RSN. If I knew how to lockpick the rack, and I had not spent the whole day fighting disks and LILO, I would be tempted to connect my laptop and try to bring it up myself (but maybe it's not so easy anymore to reboot a Cisco and get enable permissions). Luckily I have still my ADSL link, who promptly resumed service as soon as power came back. I have switched the DNS to the ADSL address, so this site will slowly resurface on the Web, but it will be pretty slow until the fiber optic link is working again.

A few days ago I finally got fed up with spam, so I decided to install a spam filter on my server. After reading around a bit, I settled on Spambayes, which (apart from being written in Python) looks a very solid and well maintained project.

I don't use Outlook (I run Linux both on my server and on my desktops), so using Spambayes Outlook plugin was not an option. Since I'm using Maildir as the storage format for both SMTP and IMAP, I initially tried the Spambayes IMAP filter. Unfortunately, the filter is still in its early stage of development, and the IMAP protocol varies significantly among different server implementations. The main problems I had with the IMAP filter were its marking all new messages as read after processing (this is apparently due to my IMAP server lack of support for an obscure IMAP command), and its frequent crashes.

So after a few hours of monitoring the IMAP filter's activity, I decided to change my approach. Reading around a bit, I discovered that the venerable procmail (which I used a lot until five or six years ago) now natively supports Maildirs.

A .qmail forward file, a .procmailrc recipe and a cron job later, I had a flawless Spambayes setup. In the past 3 days, Spambayes has worked admirably with minimal training, intercepting 99% of all spam and generating zero false positives. Definitely recommended.

My .qmail file simply passes everything along to procmail for delivery:

| preline /usr/bin/procmail

My .procmailrc recipe looks like this:

PATH=$HOME/bin:/usr/bin:/bin:/usr/local/bin:.
MAILDIR=$HOME/Maildir
DEFAULT=$MAILDIR/
LOGFILE=$HOME/procmail.log
LOCKFILE=$HOME/.lockmail
:0 fw
| /usr/bin/sb_filter.py
:0
* ^X-SpamBayes-Classification: spam
.INBOX.spambayes.spam/
:0
* ^X-SpamBayes-Classification: unsure
.INBOX.spambayes.unsure/

Notice how the trailing slash in the DEFAULT delivery identifies a Maildir storage. The rest is pretty self-explanatory, apart maybe from the folder namespace, which is the one used by default by my IMAP server:

• the first directive instructs procmail to feed the message to sb_filter.py
• sb_filter processes the message and adds the X-SpamBayes-Classification header with two values, the first one marking the message as either spam/ham/unsure, the second one displaying the exact numeric spam rating (from 0 to 1)
• the second and third directives match the header on its first value for spam and unsure, and deliver the message to the appropriate Maildir
• if a message is not matched by the second or third directive, it "falls off" the chain and gets delivered to DEFAULT, which in this case is my inbox

To train the filter, I run a cron job every half hour that looks into two Maildir folders for spam and ham messages (the following lines are ofc a single crontab line):

0,30 * * * * /usr/bin/sb_mboxtrain.py -d /home/ludo/.spambayes_hammie.db 
-g /home/ludo/Maildir/.INBOX.spambayes.train_ham/ 
-s /home/ludo/Maildir/.INBOX.spambayes.train_spam/ 
-n >/dev/null 2>&1

Meaning, every half hour cron runs sb_mboxtrain, instructing it to use the .spambayes_hammie.db (previously created with sb_filter.py -n), and to fetch ham messages from the .INBOX.spambayes.train_ham Maildir, and spam messages from the .INBOX.spambayes.train_spam Maildir.

The Maildir directories where spam/unsure messages get delivered, and where you deposit messages to train SpamBayes, can be created either from your mail client or with the command-line utility maildirmake, provided with qmail and courier.

The last piece of information you need before running this setup, is a .spambayesrc file in your home directory. Mine contains the following lines:

[Storage]
persistent_use_database = True
persistent_storage_file = ~/.spambayes_hammie.db

That's all, efficient and reliable spam protection in 5 minutes or so.

Tonight, while trying to scan very old (100 yrs or so) negatives on my cheap ScanJet 3400c, I managed to read the very good article The Mad Hatter meets the MCSE, mentioned on a Slashdot post.

As I wrote in a past entry, I saw Sun Rays and MadHatter at Sun's Milan offices a few weeks ago, and I was deeply impressed. This article examines in detail the impact of introducing a Lintel architecture in the Enterprise.

What I liked most from tonight's quick read of this article (gonna read and summarize it for the big shots tomorrow at the office) are a few well-made points:

• a Linux integration in the enteprise is a potential and very expensive failure, if it has to adapt to a Windows-based infrastructure
• a Unix-based architecture is unlikely to evolve in Windows or Mainframe centric environments due to cultural differences between the two worlds
• Sun's Unix Business Architecture (UBA) has a tremendous potential and could change the way enterprises design and manage their IT infrastructures

Definitely a recommended reading.