Those who know Tucows probably know us as either a) a download site or b) a domain name company. Both of these are of course true, however providing email services to ISPs and hosting companies is now a big part of our business and one of our focuses going forward. We provide both a fully hosted Email Service, where we host webmail, SMTP, POP, IMAP, filtering, etc. and an Email Defense Service where we do the spam and virus filtering, and then forward the clean mail to our customer's mail server. We currently have millions of paid-for mailboxes. In an effort to create more awareness about what we do and to generate some discussion on the spam topic, I wanted to give you some insight into what we're doing and get your opinions/thoughts on what you're seeing and hearing from your customers.
Since September 2006, we've seen a 100% increase in email attacks and spam hitting our email services. In August we had just over 1 billion email connections to our hosted Email Service and Email Defense Service systems, which was relatively ‘normal'. However, what has happened since then is something that I don't think we or anyone else has accurately projected. Steadily increasing since September, November connections topped out at around 2 billion. This certainly kept our 24X7 Abuse Team and our systems hopping. What we saw is certainly in line with what everyone else providing email services has seen, although few of the other big players publish their numbers. A sample stat is that the Anti-Phishing Working Group (APWG) reports that the number of distinct spoof Web sites rose 52% in October 2006 to a record-shattering 37,444, up from 24,565 a month earlier.
In order to try to keep up with the mounting attacks, we added more IP based filtering at both the network and application layer to block connections at the door, worked diligently to improve filtering rules/ techniques and spent $1 million on our email infrastructure.
Even after doing that though, we're not yet totally happy with how well we're defending against attack. Although we are definitely blocking a ton of spam and keeping many people happy, because of the significant overall volume increase and new tactics employed by spammers with image spam, many end users are seeing more spam in their inbox than they were used to.
One question I have is what is an acceptable accuracy rate? Do end users expect 96% catch-rates with zero false-positives OR do they base their acceptance on how many spam get through (not the percentage that are caught). The ‘industry' generally only talks about catch-rates and accuracy, but more and more I think that end users only really care about how much spam gets through to their inbox and everyone has their own personal threshold. The people I've talked to tell me that they don't care nor do they find it acceptable that the spammer has launched their annual fall spam campaign and this will result in their mailbox having 10 spam instead of the normal 5 spam. Sure, deleting another 5 messages isn't a big deal to some, but at the end of the day most people just want it to go away. For me personally, I have about 100 messages a day that are put in my spam quarantine, but if 5 messages get through the filter, I'm not happy.
Something almost all end users don’t realize is that we're blocking a lot more than they see even if they have a spam quarantine. Even though it looks to me as if filtering caught 100 messages today, in fact for
every 100 put into quarantine many more have been blocked right at the gate because of IP filtering/connection management mechanisms. I can tell you that on average about 52% of connections are blocked by the IP filters/connection management techniques versus 21% of connections that is blocked by the content filters. A good chunk of these blocked connections won't be directed at the mailboxes we host, but are rather Directory Harvest Attacks and other attacks directed at the domain. The fact that the service they use is doing much more than is visible - again, they probably don't care. However, the cost of filtering mail is only increasing and the more we move toward blocking mail at the door and not saving everything in a quarantine, the less visible spam filtering value end users will have in what the service provider is
doing for them and that's assuming that the end user even looks at their quarantine today.
If it's true that end users really only care about how much spam gets through to their inbox, then we all have some work to do. If people care, we should do a much better job of educating (and for Tucows it will need to start with many of you, our partners). We want to be able to demonstrate to those of you that only outsource your filtering to Tucows that we are doing a hell of a lot of work to protect your email infrastructure by giving you visibility into all the attacks that we're blocking. And after that, maybe you’ll think about outsourcing the pain of it all (email and filtering).
So tell us your thoughts…
What is an acceptable accuracy rate?
Are your customers noticing this influx? How are you dealing?