This post is actually about U2’s new record “How to Dismantle an Atomic Bomb” which is currently spreading like wildfire on certain well-known P2P networks. The problem, apart from the obvious copyright infringements, is that the record hasn’t even been released yet. It’s due for release the 22nd of November. However, a copy of the album dissappeared at a photo shoot and since then there’s been intense speculation about whether the band would bring forward the release date. No decision has been made as yet. More info at the reg..
Category: technology
The Federal Communications Commission (FCC) have decided that individual states cannot impose additional restrictions on VoIP service providers. This follows an attempt by the Minneapolis public utilities commission to force Vonage to abide by the same rules as existing telephony service providers. The FCC overruled deeming that this stance was “inconsistent with the FCC’s deregulatory policies”. More information on the reg. This is a fascinating story as the implications of this ruling are unclear. The FCC’s policies to-date regarding VoIP are supportive but not coherent. It’s very much a wait-and-see approach rather than a strategy promoting adoption of VoIP while reasonably compensating existing operators for the user of their network. This kind of sustainable policy is required to ensure that VoIP services are deployed in a safe and responsible manner with the reliability and security that users expect.
Putting the brakes on spammers
Pulled this off benezedrine.cx. It a tasty, easy to replicate mechanism for dealing with spammers, safe in the knowledge that you’re slowing down their grubby, stinking little operations in the process. As most spammers get paid by volume this reduces the money they make from slowing down the internet, helping to spread viruses and generally being complete assholes. The author advocates the creation of a tarpit using spamd which is basically an MTA which keeps SMTP relaying connections open but slows responses down to a C-R-A-W-L… Throw in the use of spamassasin for some dynamic spam detection together with the creation of a blacklist for tarpit redirection using information from an authoritative site like spews.org and you have a reliable system that kicks the majority of spammers where it hurts. The original text is shown below.
To quote Bill Hicks: “just trying to plant seeds”
Introduction
I don’t like getting spam. The problem is not detecting it automatically, that works very well with tools like SpamAssassin and bmf . Even though I can automatically delete spam without reading it, the spammers still successfully deliver their mails and get paid by volume. I want to hurt them. They should not be able to deliver their mails, and waste as much of their resources as possible attempting to do so.
Tarpits
Tarpits like spamd are fake SMTP servers, which accept connections but don’t deliver mail. Instead, they keep the connections open and reply very slowly. If the peer is patient enough to actually complete the SMTP dialogue (which will take ten minutes or more), the tarpit returns a ‘temporary error’ code (4xx), which indicates that the mail could not be delivered successfully and that the sender should keep the mail in his queue and retry again later. If he does, the same procedure repeats. Until, after several attempts, wasting both his queue space and socket handles for several days, he gives up. The resources I have to waste to do this are minimal.
If the sender is badly configured, an uncooperative recipient might actually delay his entire queue handling for several minutes each time he connects to the tarpit. And many spammers use badly configured open relays.
Obviously, I only want known spammers to get connected to my tarpit instead of my real MTA.
Blacklists
I can use an externally maintained list of spammers like spews.org to redirect senders to the tarpit selectively. But such lists may be either to slow to include new spamming hosts, or too aggressive for my taste. Some blacklists will not only include single hosts, but entire networks that contain a single spamming host, willingly hurting innocent customers of an ISP to pressure the ISP to terminate the spammer. The blacklist maintainers document such policies, and if I agree with them, it’s my decision to block mail from such networks by using their blacklist.
But even if I’m comfortable with blocking mail from innocent bystanders and use the most aggressive blacklists combined, there will still be spammers getting mails delivered to me through newly discovered open relays. Those spam mails will of course be detected by my spam filters, so I’d like to use these IP addresses to build my own blacklist.
Building my own blacklist
Assume I have the following procmail configuration in place to detect (and file) spam:
:0fw
| /usr/local/bin/bmf -m maildir -p
:0:
* ^X-Spam-Status: Yes
in-x-spam
:0fw
| /usr/local/bin/spamc
:0:
* ^X-Spam-Status: Yes
in-x-spam
Each incoming mail is piped through the two spam detectors. If either one of them classifies the mail as spam, the message gets stored in a separate file. I could delete them instead, but I might want to check the mails for false positives every once in a while. Once the classifiers are tuned right, there will be almost no false positives, and almost all spam is detected. I’m reaching 99.95% accuracy here, with maybe 0.01% false positives, which is fine for me.
Analyzing Received: headers
I’m using one additional tool, relaydb , to build a database of all hosts that send me mail. This is done after the classification by the spam detectors, so I can tell the database whether the sender was sending spam or legitimate mail.
I add the following parts to my procmail configuration:
:0fw
| /usr/local/bin/bmf -m maildir -p
:0c
* ^X-Spam-Status: Yes
| /home/dhartmei/bin/relaydb -b
:0:
* ^X-Spam-Status: Yes
in-x-spam
:0fw
| /usr/local/bin/spamc
:0c
* ^X-Spam-Status: Yes
| /home/dhartmei/bin/relaydb -b
:0:
* ^X-Spam-Status: Yes
in-x-spam
:0c
| /home/dhartmei/bin/relaydb -w
So, detected spam gets piped through relaydb -b (blacklist), and legitimate mail through relaydb -w (whitelist). Note that only copies of mails get piped through relaydb, the program never modifies or drops a mail. All it does is build a database of hosts that sent me mail, counting spam and legitimate mail from each one.
relaydb traverses all Received: headers in a mail from top (nearest relay) to bottom. It only acts on valid numerical IP addresses in [] brackets, which is the only reliable part. And it’s only reliable when I trust the previous relay in the chain, as spammers often add fake Received: headers. So relaydb starts with the top-most relay in the header and consults its database to see whether it is a known host, and if so, whether it sent me legitimate mail before. If that’s the case, it increases the respective counter (spam or legitimate, as told through the -b/-w option) for that host and continues with the next relay found in the header. If the relay is a known spammer, traversal ends, as further headers cannot be trusted.
After I run this setup for a while, relaydb has built both a blacklist and a whitelist. One important detail is that a legitimate mail has more weight than than a spam mail. I regularly receive spam through mailing lists. Of course, I don’t consider the mailing list server a spamming host. Yet, each spam I receive through it will increase the spam counter for that server. Therefore, relaydb only reports hosts as blacklisted when their spam counter is at least three times as high as the counter for legitimate mail (and the factor can be adjusted, of course). So a relay doesn’t get blacklisted as long as it sends me legitimate mail to compensate for spam it sends, which covers mailing list servers. But if I get a spam from a host that never sent me anything before, that will cause it to get blacklisted immediately (1 >= 0*3).
Completing the puzzle
Now I’m building my own blacklist, based on the evidence I’ve seen myself, classified by my own spam detector configuration. The only politics involved in someone getting blacklisted are my own, I don’t have to trust a third party to make fair decisions.
And I use this blacklist to redirect hosts to the tarpit, using pf and some cronjobs:
$ pfctl -sn
rdr inet proto tcp from to any port 25 -> 127.0.0.1 port 8025
$ relaydb -lb | pfctl -t spammers -T replace -f -
This requires a recent OpenBSD -current system.
Instead of just loading the relaydb blacklist to redirect to spamd, I could combine it with spews. Or I can use the whitelist to prevent hosts which have sent me legitimate mail before from getting redirected to spamd due to a spews listing, etc. There are many interesting combinations.
And how well does it work?
I’m getting several dozen connections redirected to the tarpit per hour, and most peers waste about ten minutes per connection, and retry several times, for multiple days. The impact on my own resources is minimal.
Best of all, I regularly get spam through a mailing list and the sender (not the mailing list server!) gets blacklisted. Then the same spammer connects to me directly, too, as it harvested my address like the one of the mailing list. And it gets stuck in the tarpit. For long. And many times.
Remember, I’m doing all of this not to reduce the amount of incoming spam. That gets detected and filed very reliably, anyway. The sole purpose is to hurt the spammers. And I’m thoroughly enjoying watching my spamd log now, as I’m perfectly sure that each of those connections comes from a spammer who has spammed me before.
“Spam me once, shame on you. Spam me twice, shame on me.” 🙂
If you have questions or comments, write to daniel@benzedrine.cx . And all you spammers harvesting email addresses from pages like this, please spam me. My trap is awaiting you.
Thanks to benzedrine and fif3. Also thannks to my mate Kieran for pointing me towards the original link. Cheers!
Design By Contract in C
Charlie Mills creates a Design-By-Contract library for C (which could equally be used for C++ with minor changes) in his most recent OnLamp article. DBC views functions and methods as contractual agreements between the functional caller and the object/module providing the function. Charlie’s implementation is a really neat idea using Object Constraint Language (OCL)to describe:
- function preconditions
- function postconditions
- type and function invariants
The implementation is hacked up using Ruby and Racc and is available here.. I’m currently playing around with DBC for Java using iContracts and I’ll post the inevitable success stories here soon…