Tryst With Spam - II
After the "Great Spam Debacle", I left spam-fighting for a few months. But I kept reading about spam a lot. I finally reached the conclusion that the best way to stop spam was at SMTP level itself. Abhijit Wekhande, a friend of mine gave me chance to test my new reasoning. He had a FreeBSD server, on which he wanted to start email hosting. Meanwhile, Dimakh also had got another mail server requirement. Now, with 2 servers, I had to get it right and maintenance free. I thought of a proper strategy. As before the mail server was to be qmail.
First, I used RBLs extensively. To allow POP-before-SMTP users, I used a wrapper to check whether RELAYCLIENT variable was set and if set remove RBLSMTPD for that IP. Using proper RBLs I stopped around 50-60% of spam (or probable spam). 20-30% more was stopped by checking for RDNS of the connecting IP and rejecting mail if not present. Very soon a lot of spam was stopped with a very negligible overhead.
Now for fine-tuning. I wrote a few shell/perl/python scripts to do various other tasks. I found that simple things like stopping early talkers, banner delays, limiting maximum number of recipients per message, rejecting mails with attachments having specific extensions, greylisting etc. were very effective in blocking a surprising large number of spams. Again as before, the load increased was very negligible. (A simple script that the average load on the FreeBSD server since last 3 months was just 0.0031).
The final topping on the cake was installing spamassassin and clamav and invoking it via. simscan. I choose this program over qmail-scanner because it was written in C and was faster than qmail-scanner and due to it being able to reject mail over certain threshold score at SMTP level itself.
Lesson I learnt in all these processes were,
- most of the spam should be rejected at SMTP level.
- heavy tools like spamassassin and clamav should be made to work on mails after filtering was done by smaller programs before.
- it is possible to filter around 90-95% spam using simple tools and finding a proper strategy.
All this was achieved because of qmail's inherent abilities to allow chaining of filters written in any language. It was great to write a prototype filter in shell, see its results and rewrite it properly in perl or python and just make a drop-in replacement.