Spam Cleaning with the Big Boys

Share it on Twitter  
Share it on Facebook  
Share it on Google+
Share it on Linked in  
You think you've got spam problems with a hundred or so spam messages a day? Try being an ISP or a business where on a good day you don't get more than a one hundred thousand spam mails a day.

You can't believe it's that much. Think again. How bad is it? Ferris Research, a San Francisco- and London-based email and groupware analysis firm, says that 30% of inbound email is spam at ISPs, while at companies, spam accounts for 15% to 20% of inbound email. "In 2002," Ferris says, "the total cost of spam to corporate organizations in the United States was $8.9 billion."

Since that Ferris study, things have only gotten worse. According to ISP and business mail administrators I've spoken with, ISP inbound mail is now up to 50% junk mail, while corporations e-mail servers are up to a rather horrifying 30%. In addition, it's only going to get worse, the Coalition against Unsolicited Bulk Email, Australia, estimates that spam's volume is doubling every 4.5 months.

So what can you do, when your network bandwidth is eaten alive by spammers, your users are screaming for relief, and your mail server hard drives are always running close to their limit? What most ISPs and companies are doing is deploying gateway anti-spam programs.

Specifically, ISPs tend to deploy SpamAssassin, a powerful open source mail-filtering program, while businesses tend to use commercial programs like Brightmail Anti-Spam 5.1 or MailFrontier Anti-Spam Gateway 2.1.

You could, of course, install client-based programs like Norton AntiSpam 2004 or Qurb 2, but that's not a great idea for two reasons.

The first is simply that client-based approaches cost more, much more, per user than a server-based solution. The other, and really the more important reason, is that supporting them will cost you even more in terms of help desk time.

Thus, while client based solutions are fine for individual users or even small businesses, they simply don't scale well for ISPs or medium to big businesses.

Spam Protection at the Gateway

Regardless of the gateway program, this kind of software always has some things in common. They must run on boxes of their own. You can't run them on the same box as the mail server no matter whether you're running Sendmail or Exchange 2000.

Next, you should take the memory requirements for any given program, and double it on your production machines. It's not that they won't run properly, they will. However, you'll need every KB of RAM you can get these programs to quickly weed out the bad mail. These processes take up a lot of RAM, and remember, the spam load is only going to get higher, much higher, in the coming year.

To only keep the mail moving in a timely fashion, you'll also need as fast a connection as you can get between your spam killer and your Internet gateway and e-mail server. If you've been thinking about moving to Gigabit Ethernet, well, beating spam is a better reason than most for the upgrade.

Page 2: Users have a say too.

You only need to give your users some control over the spam process. One man's spam is another man's steak. There are two basic ways to approach this. One is to simply deliver the spam to the user's client mailbox and set it up so that they can look in their spam mailbox whenever they want.

There are two problems with this approach. The first is that your internal network is still going to be clogged up by spam traffic. The other is that if you're going to go ahead and send it along, perhaps an inexpensive or open source client-based solution like POPFile, would serve your users better.

The other way of handling it is to simply keep the most recent spam mail in a server-based folder for users to look in if they suspect that they've missed a very important message. Again, it's not ideal; in this case, you're losing valuable server disk space to spam.

But, it's not as if you have much of a choice. Even with the best Bayesian filters, even individually tweaked anti-spam settings will still misidentify one message in a hundred. That's not so bad, when it means that a user will see a small fraction of junk e-mail instead of the flood they're used to, but every now and again, a message that needed to get through, a false-positive, will come by. It's for those valuable, but mis-identified messages that you need to give users some mechanism to look at their spam mail.

You'll also need to plan to constantly look at how well your spam protection is actually working. In my experience, SpamAssassin particularly needs a constant eye on it lest it start letting more spam through while increasing its false positive rate. This is also true of the other programs, but with the commercial products, changes in spam patterns are usually reflected in these programs' regular updates.

The choice is yours. SpamAssassin will run great, if you have someone constantly managing it. The others require less attention but you must get a long-term support contract to be safe. To me, the key factor is your network or e-mail administrator's level of expertise. If they're already comfortable working with complex procmail or the like scripting, SpamAssassin is probably the better idea. If they're still stumbling around Exchange's graphical interface, it will be more cost effective to go with a commercial program.

Regardless of how you update and manage your spam program, the simple truth is that you simply can't set them up once and forget about them. Just as spammers are always changing the way they send spam, you must constantly be on the alert for these changes and adjust your spam filters according. Yes, it's a pain, but there's no choice in the matter.

Consider, two years ago, if an e-mail came in with a valid "From" header you could safely assume that it was a perfectly fine e-mail. Today, with forged headers being a part of every spammer's toolbox, only a fool would assume that just because the "From" field right looks OK is any reason to think that the mail isn't spam.

You also need to keep your users informed of the ways they can slow down spam. For example, encourage them not to put their real e-mail addresses on public Web sites or postings. Instead, a format like will let you any human reader know that chances are Joe can be reached at, while bot programs that collect addresses from the Web will faithfully collect the bogus address.

At the same time, though, some user-based anti-spam ideas actually do more harm than good. For example, sending out fake 'bounce' or 'notice of spam' messages to spammers won't do much good. In the first case, with fake headers being all the rage, sending someone a note falsely telling them that their message didn't arrive or that their message is spam is highly unlikely to actually reach the real sender. All it will do is eat up more network traffic and annoy the innocent user on the other end of the Internet line. And, even if the message does get to a spammer, why in the world do you think they'd care? Spamming relies upon sheer volume. Its senders already know that their success rate is going to be in the 0.01 range per message sent.

No, the only real answer is to install a gateway side server program to stop spam and constantly manage it. You can forget about a magic anti-spam program or law coming along and re-setting the e-mail server clock to 1997. It's not going to happen, and if you want to keep your users happy and your e-mail costs down, you should put a server-based solution in sooner rather than later.


Loading Comments...