Sysadmin research

Lately, I've been doing a lot of sysadmin research (and work). In just a few days, I've installed spam filtering engines in several forms, email antivirus, a mail filtering gateway and a multi domain MX server (all virtual accounts) managed by a web application.

warning: this is a big one!

Anti spam and Anti virus mail filtering

First of all, I've installed mail filtering systems on two servers, with antivirus and spam detection. There are many ways to skin a cat and the most widely used ways to skin this cat are, at the moment, clamav and spamassassin respectively for antivirus and spam detection. Another aparently good spam detection software is dspam, which has a different aproach than spamassassin but is said to be much faster and offers a different set of features.

One of my servers was, until recently, running a netqmail installation with qmail-scanner running messages through clamav. It was a recent setup that was running well, despite my lack of experience with qmail. qmail-scanner was hard to get working with clamav due to some permission problems originated by bad distribution packaging. This particular software package filters every message, eventually holding attachments for quarantine and changes the messages a bit (headers and/or body) before returning them to the MTA (this is important!). Essentially it works as a transparent filter integrated with the MTA and that's the way I want it to run.

Integration of spamassassin/clamav with MTAs

Due to the different integration aproaches of dspam and spamassassin, I've decided to use spamassassin, and having it integrated with qmail-scanner was really easy, mostly because qmail-scanner supports it explicitly and enabling spam scanning is only a matter of changing a couple of configuration switches. So for this server, things were going well, I had transparent spam detection and antivirus scanning of incoming mail.

On the other server the running MTA is a postfix 2.2.10, so qmail-scanner would not be adequate (although not impossible). To get things working quickly, I did a simple procmail rule to filter my email through spamassassin's spamc/spamd and it's working. I left system wide autolearn with sensible score values (0.0 < no autolearn < 12.0) and per user bayesian databases can be used. Not the ideal solution, but it works.

:0fw: spamassassin.lock
| /usr/bin/spamc

To get a nice scalable solution for postfix MTA with spamassassin and clamav antivirus, I googled a while and found amavisd-new. amavis is a mail filtering tool that accepts email like an MTA with no queues and runs a chain of programs on each message. It's by no standards a new tool but I've never stumbled upon it before.

Mail filtering gateways

This solution is very apealing for mail filtering gateways, where there's a host that acts as a domain MX, receives email, filters it through antivirus and antispam software and just redirects sane mail (or all mail, depending on the defined policy) to the internal mail servers. What's good about this solution is it's scalability: The MX MTA can be a cluster of servers, the antivirus can be another cluster and the antispam filters can be a third cluster. Such setup can scale to massive processing power and fault tolerance with cheap hardware.
Following this idea, I decided to build such a beast on a virtual server, based on CentOS 4.4. I'm not a fan of CentOS, but it's currently the chosen Linux distribution at where I work, so it seemed appropriate. This mail filtering gateway is built from the following components:
- mta: postfix
- mail filter: amavisd-new
- antispam: spamassassin
- antivirus: clamav
I'll give some feedback on the process one I find enough time to finish it.

Multi domain MX server with virtual accounts

At work, I am currently responsible for building a multi domain MX server with virtual accounts. It has to rely on an LDAP directory for accounting and authentication, supply POP access and provide easy management.

Setup

The Linux distribution is automatically chosen - CentOS 4.4 - and I chose Postfix for the MTA service, openldap for LDAP service, cyrus-sasl for authentication library (hell), dovecot for POP access and Jamm for management service.

It's been fun to build an elaborated MX server such as this, but not everything went well. As far as I was able to debug, cyrus-sasl isn't talking to openldap and there aren't many alternatives to do authenticated relay on postfix. Also, Jamm installation was incredibly complicated, time consuming and, at the end, impossible.

The bad

Starting with Jamm; It's a java web application that deploys on tomcat. Although I never did this before, it shouldn't be that hard, but for CentOS 4.4, it is. There are no official tomcat rpms, so I had to use a third party mirror and install something around 76 packages to get tomcat working. After that, I was unable to build the Jamm application (due to some unmet versioning requirements, I guess) and couldn't deploy it. Sad but true. Why doesn't CentOS include a tomcat/java tools distribution??
One way to solve this is to deploy Jamm on another server, perhaps on a Linux distribution better suited for tomcat. Another way is to use phamm - an alternative interface written in php with even more features than Jamm.
About cyrus-sasl... the story isn't pretty either. There are no alternatives (which don't imply recompilation of postfix and manual integration with another sasl library) and the ldap authentication plugin isn't talking to openldap. I'll have to get this working one way or the other.

The good

On the upside, LDAP is working well and makes management potentially so simple! Dovecot is a fresh, actively maintained IMAP/POP implementation and Postfix is very rich in features that make things such as aliasing, virtual domains, tls negotiations very easy to setup.

Migrating IMAP/POP service

As a sidenote, for migrations between IMAP implementations, Dovecot needs some tweaking for compatibility with courier-imap namespaces. courier-imap aparently uses an implicit INBOX namespace that needs to be defined in dovecot configuration, otherwise it won't find existing folders. As a reference this is, at least, for dovecot 1.0rc10.
The following namespace should be defined in addition to the default_env

# Courier-imap compatibility
namespace private {
        separator = .
        prefix = INBOX.
        inbox = yes
}

Service monitoring tool

Time to wrap up with a service watchdog tool: monit. monit monitors and acts on configured processes and/or files. Processes can be restarted if they somehow die, if thei're using up more than a specified resource limit, files and directories can be monitored for changes, etc. Events are, of course, notified and the system can monitor remote services via authenticated SSL tunnels (for example). Finally, there's a cool web interface to access all this lovely information. I have to say, I was impressed by the website and the tool looks damn good!

That's it for now,
Cheers.