Re: RBL for spam filtering

From: Brian Clapper (bmc@WillsCreek.com)
Date: Mon Apr 23 2001 - 08:13:18 PDT


On 23 April, 2001, at 14:14 (+0100)
Justin Mason <jm@jmason.org> wrote:

> Joseph S Barrera III said:
>
> > Any thoughts on using RBL for spam filtering on FoRK?
>
> I've been hacking recently on a spam filter in perl, based on the system I
> use myself. Pre-alpha code at http://spamassassin.taint.org/ .
>
> The spammers seem to be moving too quickly for the RBL nowadays. Out of
> 517 messages I've been spammed with since the start of the year, only 7
> were in the RBL; that's 1.3%.
>
> The open-relays list is a bit more successful, at 37% of spams hitting
> that rule. But it's more prone to friendly fire.
>
> Mind you, I've been using rbl.maps.vix.com, and I've just noticed that
> the current MAPS RBL seems to live at blackholes.mail-abuse.org. Maybe
> the maps.vix.com zone is no longer updated, in which case I should go
> stick my head in a brown bag.

I'd concur with Justin's comments. Though I haven't been keeping hard
statistics, I can provide some personal observations.

First, some background. I use MAPS RBL, RSS, and DUL on my mail server. (I
don't use the RBL for any other purpose; specifically, I don't block any
web sites.) I also keep my own blacklist for persistently obnoxious mail
servers that MAPS doesn't blacklist. My mail server serves email for three
personal domains. Two of those domains belong to me and my wife; one
belongs to a friend for whom I'm relaying mail at the moment. I also
maintain several permanent forwarding accounts for family members. Finally,
I also host a few private, extremely low-volume mailing lists, one of which
has more than 200 members from 94 distinct domains.

I have a quick-and-dirty perl script that scrapes my email log nightly,
summarizes all the rejected messages, and mails me the summary. This
permits me to gauge (a) how much spam is getting stopped at the door, and
(b) whether or not any legitimate email is being rejected. Without such a
script, I'd be wary of putting any anti-spam rules in place.

The mail server typically receives between 50 and 200 incoming email
connections per day, including rejects. The bulk of those messages are
intended for me. Of the total number of messages received, the server
rejects anywhere from 5 to 20 incoming email messages because they match
one of the MAPS rules. A rejected message almost always matches an RSS
entry. Once a week or so, the server will reject a message because it
matches a DUL entry or my personal blacklist. I think I've seen maybe two
RBL matches in the six months since I put the MAPS rules in place.

Each day, between 5 and 20 spam messages leak through anyway. Typically,
between one-third and one-half of these messages come from the FoRK list.
The remaining messages are from sites that aren't in any of the MAPS
databases and aren't in my personal blacklist.

Since I've had the rules in place, I've seen one "innocent" get caught: My
father's ISP's email server ended up in the RSS list, and all email from
him was getting rejected. I put an exception in place for his ISP, allowing
mail from him to come through until his ISP get its act together.

Overall, I'd say the MAPS databases nuke perhaps half the spam coming into
my site. They're effective enough that they're worth leaving in place,
especially since the false hit rate is extremely low for my site.

However, I clearly don't run a high-volume site, so my experiences may not
apply to much larger (and more diverse) mailing lists such as FoRK.

Brian Clapper, bmc@WillsCreek.com



This archive was generated by hypermail 2b29 : Sun Apr 29 2001 - 20:26:05 PDT