A couple weeks ago, the SORBS spamtrap list picked up a few Hotmail and Gmail servers, and a Yahoo mailing list server. This lead to me getting complaints that legitimate mail was bouncing. I’m all for letting the mail get blocked, because it’s the only way that large companies like Google and Microsoft will be forced to fix problems. Unfortunately, the end users don’t see it this way. They think because Hotmail user A can’t get mail to our user B, it’s a problem with our system and we need to fix it. Ignoring the fact that thousands of other ISPs who use the same RBL are also blocking mail from those people. Long story short, I was forced to remove the spamtrap RBL (by using all of the separate SORBS RBLs instead of the composite list) — the mail started flowing again and the complaints stopped.

This is leading to conversations on the general merit of RBLs in general, and whether or not we should use them because it’s allowing someone else to control whether or not mail gets to our users. Of course the people raising these questions do not have to listen to the end user complaints. People want all their mail and no spam, which of course is impossible.

Currently, between several different RBLs, we reject about 130,000 messages per day (~80% of the total daily mail volume) at the MTA level. Should we turn them off, everybody would notice. There are no other spam filtering techniques that have done as much to reduce our spam overall as RBLs. Sure, we could throw a million content filters at it, but that takes a lot of horsepower to run, and probably would not be as effective. I put more stock in RBLs than I do in content filtering. The only legitimate alternative to using RBLs at the MTA level is using them in SpamAssassin where they are ranked with scores based on the RBL’s reliability and such. However, performing the RBL checks in SpamAssassin also introduces a lot more delays in message delivery (and of course, if someone sends an e-mail and the other person doesn’t have it in less than a minute people call and complain too!)

Life would be so much easier if there was a secure and spam-resistant alternative to SMTP, but that won’t be happening anytime soon.

So it’s been quite some time since I posted anything, mainly because I’ve been rather busy on a project at work. It’s a fairly complex system that we used Ruby on Rails for. I had never used Rails before (or Ruby for that matter!) but it was easy to pick up (I suppose having coded in a dozen languages or so at one time or another makes picking up new ones easier…) Rails really excels at taking care of the grunt work (db access/mapping especially) and lets you focus on what you’re trying to accomplish.

I did all of my coding using Eclipse, with Ruby Develpment Tools, Subclipse (for Subversion access), and most importantly the RadRails plugin. For the most part is was a good experience. There were a few times when renaming a file or performing miscellaneous actions that Eclipse locked up on me (at home and at work) but restarting Eclipse worked ok. The only feature missing that would have been nice is Rails debugging. It’s in the works in RadRails, but it’s a ways off.

More about Ruby on Rails can be found elsewhere.

I found the book Agile Web Development with Rails invaluable during the entire process. (Programming Ruby is another fine choice, too!)