SMTP Connections: How to handle large loads
I recently came across several blog posts about the declining state of E-mail due to spam. Specifically these:
- Spam Volumes at accidental-DOS Levels
- Another War We Aren't Winning: Us vs Spam
- E-mail as we know it is doomed
I've been running E-mail servers for myself and others for over 10 years now and I have to agree that with the current version of SMTP we all use, there isn't much that can be done about spam that isn't already being done. If you've got some RBLs, SPF, anti-virus, and a decent spam filter setup, there isn't much more you can do. Sure you can get +/- another percentage point, but you won't really find a solution that is 100% effective 100% of the time. It just isn't possible with the current standards.
However, these articles also discuss another issue that is overlooked by everyone who doesn't run a large E-mail system. By large I'll say over 1,000 users. Around that point you start to run into problems being able to handle the shear number of incoming SMTP connections. Note that you may hit this point long before 1,000 users depending on hardware and your personal traffic levels. But I digress.
I haven't found any Open Source solution to this problem, but I've come pretty close. TrafficControl is a commercial product from MailChannels that is built upon the good Open Source base of Apache and mod_perl. TrafficControl does two things for you:
- It uses async I/O to proxy incoming SMTP connections. This has allowed me to handle 10x the number of connections on the exact same hardware
- Allows you to configure throttling rules based on RBLs, Operating System, etc. and choke off the bandwidth to suspicious sending servers. This reduces a great deal of spam as most bots simply move on if they are not getting go flow. After a configurable time period it will remove the choke hold and allow the message to continue on normally. Servers that follow the SMTP RFC have no problem with this as they hang on during the process.
I've been using it for about a year now and couldn't imagine trying to run a large E-mail system without it now. I encourage you to check it out if you see you are having a similar problem.