Ella Rocks - All About Spam

How Accurate is your Spam Filter?

2008-02-04T15:04:00.000-08:00

When researching spam filters, you'll frequently see reference to accuracy. Is a filter 90% accurate? 95% accurate? 99% accurate?

Here's how you can test the accuracy of spam filtering with whatever email system you're using:

In the morning, when you get into the office or fire up your computer at home, clear your spam folder. (i.e. Microsoft’s Junk e-Mail folder, Ella Review Spam etc.)

Jot down the number of messages you have in your inbox. [e.g. 523]

During the course of the day, don't delete any messages (good or spam). Leave them in the inbox or the spam folder

The next morning, determine the “denominator” – the total number of messages you received since yesterday morning

a. Take the total number of messages in your inbox (including messages you consider to be spam), [e.g. 584 total messages in your inbox]

b. Plus the number of messages in your spam folder, [e.g. 87 messages in your spam folder]

c. then subtract the total number of messages that you wrote down in step 2 above [(584+87)-523 = 148]

Then determine the “numerator”

a. the number of inbox messages that you consider to be spam. [e.g. 9 inbox messages considered to be spam]

b. plus the number of any “false positive” messages (if any) that the filter might have misclassified (good messages in your spam folder) [e.g. 1 message]

Now divide the number of messages misclassified by your filter (the numerator) by the total number of messages you received. [10/148 = 6.76%]

Subtract the result from 100%. [100% – 6.76% = 93.24%]

This simple method should give you a true reflection of the accuracy of your spam filter.

...and it keeps on comin'

2006-12-01T19:18:00.000-08:00

The past couple of months have seen a REAL spike in spam - and as such, a spike in installs of the FREE and Trial versions of Ella (oh yeah, sales too). We again consulted with Erik Schmidt, and he pointed out that since August, spam volume has increased roughly 300%. (I sure wish I would have kept that link). Ella is still keeping up - and that's good - in my account today, 301 spam came in and Ella missed just 1 - that's over 99.5% accuracy. You go girl.

The only significant content difference in spam is those silly gif/jpg images that are typical with stock come-ons. Of all of the spam I have ever seen - those seem to be the ones least likely to have a user actually click on- but hey, maybe I'm a jaded anti-spam guy.

Weird uptick

2006-09-16T22:57:00.000-07:00

Over the past couple of months, spam has been increasing, and as such, there has been an odd uptick in usage of Ella. We have been seeing a steady increase in downloads, installations and sales of Ella. Our SEO ace, Erik Schmidt, is convinced that there is a new strain of spam and that Ella is either the only, or just one of the few that can handle this. It certainly good for our ego, but in a way it makes a little sense. By using a unique training profile for every single user, Ella is an elusive target for the spammers. Even if he is wrong and it is a phase of some astrological sign - I'm ok with it. I just want to see more people benefitting from a solid solution to what seems to a be a problem with no clear end.

Spam, beginning to drift

2006-03-30T01:09:00.000-08:00

Spam seems to be beginning to drift. Not necessarily drift away, but from a volume and an importance point of view. My impression is that the CAN-Spam laws have caused porn spam to virtually disappear (at least in my sphere of influence) and much of what we get is severe repetition. In my opinion, this is a good thing - spam is becoming less of a viable revenue source. The less revenue associated with it, the fewer players, and eventually a falloff in businesses that support it.

February 2006 Spam Categories

2006-02-04T18:15:00.000-08:00

Here is a breakdown of 100 spam messages that I received over the last couple of days. The distinct lack of porn might indicate that the CAN Spam activities might be working on them - but certainly not the viagra ads. Over the last 500 messages - Ella missed two spam - not bad... 99.6%

55 pharma (mostly viagra, but weight loss, fountain of youth cream, pheromones)
13 cheap replica watches
10 oem software
9 mortgage, get out of debt for free
4 virus, fraud, 419
2 cheap degrees
2 gambling
1 cheap sex
1 foreign language
1 get rich quick
1 penny stocks
1 sat dish

What is a White List

2006-01-07T21:52:00.000-08:00

A white list is a anti-spam method that allows the user to receive only mail if the senders name is on the list. In theory, it is like caller-id, except the only calls that are let through are the ones that you have designated on the white list. This would work, except email return addresses can be faked. Granted, spammers would have to know a name that would be on your white list. This isn't as hard as it might sound - in the case of company email, naming conventions are typically used and spammers use automated programs to generate the lists. If two people on the spammers list are from the same domain (i.e. ibm.com) they put one of those names in the return address field.

The other flaw with white lists that you will miss mail you really want because the sender's name was not on your list. Typical systems such as Earthlink's bounce the mail back to the user with a friendly note that asks the sender to fill out a form to prove you are real - usually answering a reading quiz for some graphically mangled letters. I have seen other systems that asked you to type in how many puppies are in the picture. Although this seems somewhat harmless, some people find it a nuisance and not particularly professional.

What is a False Positive?

2006-01-03T18:12:00.001-08:00

Since no anti-spam software will ever be absolutely perfect, there is always a risk that the spam filter will classify a good message as spam. A False Positive is when the filter falsely identifies a good message as spam. For this reason, virtually no spam filter will actually delete mail – it will move it to a folder so you can glance through it and delete it manually. With all of the lawsuit happy people, no company could take the risk of keeping you from getting something sent to you – it would be like having the mailman decide which envelopes you really might want.

How to fact-check suspected scams

2006-01-03T18:12:00.000-08:00

If you suspect something is too good to be true – it probably is. You should validate assertions you get if you have any suspicions that you are being scammed. The most renowned, reliable source of urban legend is http://www.snopes.com. Here is an example of the Citibank phishing scam - http://www.snopes.com/inboxer/scams/citibank3.asp

What is Phishing?

2006-01-03T18:11:00.000-08:00

Phishing is the 21st Century’s version of the con-man. A renowned circus guy once said: “there’s a sucker born every minute” These scam artists pose as someone you might trust (a big brand name or worse yet, a bank), purely for evil means. Typically they send email under the guise of a household name like Amazon, EBay, PayPal or Citibank and use fear tactics to get you to divulge your username and password. Once they have this, they can use this information to fraudulent bilk your account.

One of the oldest phishing scams on the internet was named by the US State Department as “419 scams” – the old Nigerian prince that died and would give you $10M if you would just give him your bank account information and $2000 to bribe the right authorities.

If everybody hates spam, why does it persist?

2006-01-03T17:34:00.000-08:00

Because a few people actually go for these things. They buy a fake Rolex watch, or they try out those drugs that alleged change body parts. Like with any detective work, in order to figure these spammers out, you have to think like one. If only 1 in 100,000 people bought your $25 offer, and you desired to make $10,000 per month – then you need to send out 40,000,000 emails per month (1.33M/day). It sounds like a lot – but since unlike junk mail you get delivered by your mail carrier, it is essentially free to send email. I went looking for stats on spam, but could not find a sufficiently reliable source to reference – nonetheless, I found a several sites that referenced a Newsweek article titled “Crammed with Spam” that claimed AT&T, not the largest, but a sizable email services provider, rejects 10M-12M spam per day.

Want to help in stopping spam? Be sure NEVER to buy anything or respond to spam email. Even responding causes the spammer to confirm that your email address is a valid one – making it a commodity they can sell to other spammers.Secondly – get a good antispam product, so you never have to see it. If you use Outlook or Outlook Express, I am fond of Ella – but then, I am a bit biased – it’s my company. There are other fine products out there – and most of them you can try before you buy.

What is Spam/How is it created?

2006-01-03T17:30:00.000-08:00

Spammers fall into a variety of categories – but my definition is as follows: “People who send unsolicited email to a large number of recipients for the purpose of engaging them to buy something without the a valid method to be removed from the email list”

There are certainly “legitimate” companies that email you unsolicited mail – sometimes you want it, sometimes you don’t. For example your bank will send you notices from time to time about changes in their services and the like. That’s fine – because if I don’t care, I know that if I “unsubscribe” that I will get off the email list AND they won’t sell/trade/barter/give my name to any third party. There are other sources of mass mailings such as Newsletters or vendors (like iTunes) that send you mail because you purchased something through them. Again – you can always safely unsubscribe.

This is totally unlike the companies/individuals that are trying to sell me prescriptions, watches, or a low rate mortgage. They acquired (or used software to guess) your email address in order to lure you into some sort of offer.