Notes & Thoughts

I got tagged…

2009-03-09T01:06:00.001-07:00

Been a while…..

Was tagged by Kurt, in an internet meme about drawing bunnies.

I was most famously (or notoriously) chided in my high school days for “drawing” a landscape piece that looked like a 3 year old artwork.

To preempt any disappointment, i am glad to say that the incident was an indication of my overall interest in drawing,.. nil.

So i tried a 2 minute quick draw, and here’s what i got.

Looks bad huh?

After i stopped laughing for a while, i decided to make another attempt.

This is about as good as i can get!

And to finally continue on this meme, i’m going to tag Alex Eckelberry, Roger Thompson and of course, Randy Abrams.

Minor change

2009-01-05T08:55:00.000-08:00

Away for too long. Added Avira's newly launched blog to my blogroll.

antivirus antimalware antispyware

Back from Amsterdam

2008-05-04T19:00:00.001-07:00

I was in Amsterdam for two events: the AMTSO meeting and the CARO workshop on Packers, Decryptors and Obfuscators.

First, the AMTSO meeting. It was a follow-up to the discussions made in the last few AV industry conferences and gatherings, and the Pro-Term management committee did a lot of work to get the discussions flowing.

What's the AMTSO you ask? The Anti-Malware Testing Standards Organization, or AMTSO, is dedicated to helping improve the objectivity, quality and relevance of anti-malware technology testing.

Open discussions were the main goal of the day, on various subjects ranging on the technical details to the practicality of some of the recommendations that came out of these discussions.

The fact that there were professional testers, publishers and legal representatives gave better and instant feedback to some of the issues that were brought up, instead of delaying them to offline discussions later.

I look forward to the next AMTSO meeting, and the eventual adherence to its recommendations to improve the overall quality of testing of the antimalware products.

The CARO workshop was a great gathering of the experts that are doing the unpacking, the decrypting and the de-obfuscating of files on a day to day basis sharing their insights and lessons learnt from their work.

Kurt Natvig from Norman started off with an opening that made it really hard to follow-up on. Most of the presenters gamely took on that challenge though! Even I was able to understand the challenges and the work required for future research. Fellow attendee from McAfee also agreed on that point.

The program was organized very tightly and had papers that related and continued on the prior papers' work in a very logical manner. This helped to facilitate lots of discussions during the various coffee-break sessions and the dinner sessions.

A few themes were constantly present through the two events. The need to do good for the sake of the computing community being always in the minds and hearts of the folks that attended the events. Everyone was in a position that can and will make product improvements that will impact large groups of IT users.

To facilitate that need, this group of experts are sharing their knowledge and their experience with one another. Do keep in mind that the attendees are working in competing companies, but yet, they share most of their insight to help one another.

To make full use of these kind of gatherings, the attendees practically need to wave their goodbyes to something known as sleep. Discussions continued through the late night, accompanied by the industry number one energy drink: beer. I even got a few action items that i need to work on when i'm back at work.

The CARO workshop and the hosting of the AMTSO were organized by Righard Zwienenberg, from Norman ASA. Thanks to his great work, amidst a personal monumental event, and slight sickness, both events went on fine. Thanks Righard!

Back from AVAR 2007

2007-12-25T07:43:00.001-08:00

I just came back to the States after attending the AVAR 2007 conference in Seoul, and a detour to Singapore to visit my family & friends.

The organizers made a great decision in arranging for the papers with related subjects to be presented within the same session. The whole session on anti-malware testing kept everyone in thinking how such issues affect the industry.

The conference this year had a few papers that focused on hacking or malware on online gaming, which reflected the massive online gaming market in South Korea.

Two other local presenters, one from my company, and is a immediate team member of Jeanette, did a paper on Vista technology, while a representative from KISA showcased the botnet mitigation efforts in Korea.

Another interesting paper was on the research and defense against password stealing trojans in China. It gave a different flavor to what most of the attendees have seen outside of China.

I believe having more and more regional based presentations will make AVAR a much more unique conference as compared to the other major anti virus conferences. Having the local presenters present in a language that they're comfortable in also make good sense, as that will increase the quality of the presentation to the local attendees, but will increase some operational costs on the organizers to provide real-time translations and the equipment required for such a service.

Randy Abrams did a very interesting paper on heuristics. It was unique in the sense that he described a highly technical subject matter in a very easy to understand manner, as the WebSense folks mentioned,.. ".. moved the audience deeply."

It's quite funny to find that oftentimes, i have to go to a conference overseas to meet up with someone else in the company. I finally got a chance to meet up face-to-face with Dan, who's a great guy to be with, and Jaime, who is a fellow Singaporean that moved to Redmond, instead of just shooting each other emails.

Of course, meeting up with members from the various AV companies, and my company's other colleagues, and having a great time doing such discussions make any conference fun, and make the 20+ hour travel a lot more easier to handle.

The post-conference tour was unique in the sense that we got to visit the DMZ. The most memorable thing on that tour was something that was totally unscheduled. It was a group of guys in the thirties-to-fifties, in a volleyball-like court, playing something that's quite similar to Asia's Sepak Takraw, but with a soccer ball. The energy and excitement these folks had in their games totally impressed the tour attendees, who were mostly tired from walking down a path that probably was designed for hobbits rather than typical humans nowadays.

I thank the AhnLab folks that organized this year's conference for being able to provide such a great conference!

All hail XKCD

2007-11-28T01:33:00.001-08:00

Best Comic Indeed.

[nod to Fergie]

Commentary on commentaries

2007-10-25T18:44:00.000-07:00

During my high school days, one of the primary books that i had to cover is William Shakespeare's Macbeth. At that time, one couldn't really grasp the prowess of his words and phrasing that captured every essence in life. I guess you really need life's experiences to complement such classic literature.

I can still easily recite some of the soliloquys from the main character. One of my favorite quotes from Macbeth is in Act V, Scene V, where Macbeth, upon hearing the death of his wife, cited:

"Out, out, brief candle! Life's but a walking shadow, a poor player that struts and frets his hour upon the stage and then is heard no more: it is a tale told by an idiot, full of sound and fury, signifying nothing."

The last line has multiple layers of meaning to it. The layer that i'm focusing on this post is about the comment that Shakespeare made on what constitutes good drama. I believe that the comment applies to sports, and other areas as well.

Being a boy in his teenager years in Singapore, chances are that he would be playing football (aka Soccer in USA), and would have been watching the Big League Soccer from the UK. At that time, the telecast would be a straightforward showing of the football match, with minimal commentary. Similarly, the football World Cup matches would be shown on TV right at the minute the matches started in the host countries. You'd watch the game, have a break during half time, and finish the game right at the final whistle.

Nowadays, it is not uncommon to see a pre-game show that lasts for an hour. Sometimes for big events like the World Cup final, the pre-game show might be three hours for a 90 minute match! A big team of former players, commentators, "experts" and other guests will be sharing their points of view with one another.

Similarly, during the actual match, the commentators will start to cite tons of metrics, for example:

1) how many assists each player make this season

2) how many tackles (successful/failed)

3) how much playing time each player has

4) so on and so forth....

I wonder how all these metrics actually matter. At the end of the day, isn't the one important data point is whether one team scores more goals than the other? Team A can make 1000 assists, has 100 shots at goal, make 100 successful tackles, and ran 10000 yards during the 90 minutes, but if Team B can make their 1 shot at goal count, nothing else matter. The top scorer in the World Cup 2006 was Germany's Miroslav Klose, who had a two goal lead against the next set of top scorers, but the final was between Italy and France.

What all the pre-game/post-game shows and the in-game commentaries have done is to create a huge and profitable market in terms of allowing these non-playing "experts" to give their two cents' worth of opinion in as many media channels as they can.

I recalled that there was a series in probably early 2002 that had two sets of commentators that were spouting different "data" on the same players in the same season. One set would say that Zola passed 212 times in the month of August, while another set said it was 195 times. It became a farce when they started arguing about the definition of a pass from their respective point of view!

Is there any real value added to the game? No. The unnecessary hype of such useless metrics not only do not add value to the game, but rather, shifts people's focus off the most important goal, which is, who score more goals!

This also created some false sense of authority on these "experts". Again, the ones that gained from these "sound and fury" are the ones that go around spreading sound and fury. I pity the players that are doing the actual sports. At least in terms of football players, most top league players are getting rich enough that they can afford to ignore this kind of distraction.

Technorati Tags: Commentary , Shakespeare , Football , Soccer , World Cup , Macbeth

Optimizing for metrics

2007-10-24T18:16:00.001-07:00

While waiting for Sharon yesterday, i was at the company's in-house library and read from cover to cover two to three small but good books. The topics range from various software development models, Mathematical Puzzles (gosh, how i missed doing the math and logic puzzles from Martin Gardner when i was in high school) and one on good management and leadership skills on a quality team!

The first book (Smart and Gets Things Done) that i just completed in about 20 mins was a book by Joel. Some of the chapters do not apply to me since i'm not a hiring manager, but the concise but very detailed chapters are great to incite ideas in my brain. I can see someone already shivering from afar.

An example that he mentioned on Starbucks vs other local cafes was just something that i personally observed recently. Having the chance to visit two similar Mexican fast food places, Chipotle and Qdoba, that have practically the same setup in terms of layout and menu options. They even had the same number of people queuing up for orders but somehow, one had a very quick flow while the other was stuttering along.

The only thing different is that Chipotle has a very optimized process from the point that the customer orders the food to the point where the customer pays for the food. The order is only taken once, and the necessary information is passed down the "service line" in an effective manner. This is something that Qdoba didn't do. First the customer give his order to the first "server", and then to the one that handles the condiments, and then to the cashier eventually.

Three times vs one time. Hmm.. So where's the secret trick that Starbucks and Chipotle do that the others don't? The person taking the order WRITES down all the necessary information in acronyms on the wrapper/cup, and basing on the provided information, the rest of the service line can continue on without impacting the customer.

I wonder why that there's such a big difference in the way the two sets of examples are doing processes that are inherently the same?

One way of looking at it is perhaps the way that the organizations are measuring their staff. What is the critical metric that they should focus on? Is it the eventual customer satisfaction, or the measurement of how long each member along the service line take?

That line of thought link the first book to the book with the mathematical puzzles that i just couldn't remember the title for. A puzzle that was detailed is the well known Prisoner's Dilemma. In this game, as in all game theory, the only concern of each individual player ("prisoner") is maximizing his/her own payoff, without any concern for the other player's payoff.

Since each member along the service line is measured for his own efficiency, the quicker he can move the customer down the line the more rewards he will get. He will not care for whether the downstream gets the necessary information to work with, or worse, whether the customer is even being frustrated by the repeated queries on the same order again.

Would i blame the folks on the service line? No. If their management gives them such a metric for measurement, chances are that the folks will find ways to optimize for the specific metric that gets rewarded, without actually achieving the most significant outcome that is desired. In this case, i believe that the management might want happy customers that will be glad to come back for future purchases, but will they get such repeated purchases? I know that i won't want to spend time in Qdoba. If i want the same kind of food, i'll go to the more effective and efficient Chipotle.

More book reviews in the future. I promise a more regular update on this blog. 8).

Technorati tags: book review, starbucks, fastfood, chipotle, qdoba, joel, notes

Real life testing

2007-10-23T18:00:00.000-07:00

So, what is a test? Among many other definitions, it's "achieve a certain score or rating on a test". Most tests that one will do is constrained to a certain subject (for example, Elementary English), or even just a certain area within a specific subject (Estimating the Airspeed Velocity of an Unladen Swallow in Physics).

Why should we have a limit in the areas that we're tested on? For one, it's to protect the folks that are taking the test. For example, there's no way that an English teacher, while conducting a test in Elementary English, will fail a student in the test just because he has no knowledge of Klingon language.

Another example that i can think of is that one will not fail his Elementary English test because he has a number of errors in his Physics test.

Usually, the passing standard is usually agreed upon before one takes a test. One doesn't necessary get to "agree" on the standard, but at least the standard is known. For example, the passing grade for the driving test here in Washington is 80 out of 100. You don't get to agree on it, but you know it well before you take your driving test.

A controlled environment is quite common in tests as well. You don't get to do your Elementary English test in a Indy 500 racing course; neither do you do your driving test in a high school car park (i hope not!).

Objective testing vs subjective testing is always something that can be argued till kingdom come. Even in our primary example of Elementary English, you can have an objective test using multiple choice questions vs an open and subjective composition test.

Subjective testing scores are further skewed when the examiner has a goal in mind before evaluating the test entries. My high school English teacher usually score a "A" grade for me and a "C" grade for my classmate, basing on the fact that he was one of the students that was weak in English while i was always in the top two for English in the high school. So, we decided to play a prank on the English (he was an expatriate from Norwich, England) teacher and we wrote our usual compositions, gave each other our original work, and copied each other's work 100%.

At the end, I still received a "A" for my friend's usually "C" work, while my friend got a "C" for my usually "A" work! That's not totally scientific, but hey.. that proved a point. If you have a predetermined goal in mind, whatever that you are testing for will be affected.

So unless you are perfect, errors will be found (perhaps a typo, a grammatical error, drove through a red light etc) and you might fail the test. One thing that you shouldn't be justifying will be the fact that you drove through the red light because the light was too dim, or your pencil suddenly made you do a typo. As another well known antivirus researcher has mentioned, "No point in looking at the color of my shirt and scoring my application based on that.". You also shouldn't expect people to mark your test based on irrelevant, or worse, constantly changing baselines during a test.

Most importantly, if you fail, you are held accountable for the failure. You either have to retake the test, take another subject, or fail the whole course altogether. Your actions will be based on your prior actions. As easy as that. Nothing of those comments such as "Wow.. your test is very difficult. Are you sure this is meant for Elementary grades instead of High School levels?", "Hmm.. your questions made me look bad. Let's take these questions out and give me some easier ones to handle.".

So what should you do if you fail? I hope that you don't go crying to your parents. They shouldn't help you to argue with the instructors for a passing grade. They should help you to find out why you have failed, and help you to improve. If you lose the World Series, you don't go crying to your coach and ask the Major League to replay the series again. You lose the series, you figure it out, and move on. Similarly, Zidane lost the World Cup for France. He cried. Even though it was found out later that the Italian defender Marco Materazzi did insult his family, he (and France) took the defeat in a sporting manner. Did he ask for a replay? No.

That's how real life from my life in Singapore has shown me. I don't think the examples above really works that way here now.

What do you think?

Technorati tags: testing, exams, Singapore, observations

Back from AVAR 2006

2006-12-11T12:20:00.000-08:00

I am just back from the AVAR (Association of anti Virus Asia Researchers) 2006 conference that was recently held in Auckland, New Zealand.

The conference had a large variety of papers from various security organizations and companies, and I'm delighted to be one of the presenters in this year's conference.

Some of the papers that interested me the most were:

Eric Chien's (Symantec) presentation on gadgets
Igor Muttik's (McAfee AVERT) presentation on the web of sin
Peter Ferrie's (Symantec) presentation on attacks on Virtual Machine emulators (Slides and Paper on his homepage)
Kimmo Kasslim's (F-Secure) presentation on kernel malware
Maksym Schipka's (Messagelabs) presentation on the prevalence of PE Packers in email traffic

The best part about attending conferences such as the ones by AVAR, Eicar and Virus Bulletin are the informational exchange that the attendees do, either officially during the conference presentations, or socially in the gatherings in the restaurants or pubs. I think it's a surprise for folks to realize how much cooperation there is between the different companies and organizations.

Throughout the conference, i was still doing some final revisions to ensure that i can get my subject matter through within the allocated time. The organizers were pretty strict on the timing of the papers! Thankfully, i think i did my presentation without too much of an issue.

tags: antivirus, AVAR, antimalware

Got some spare time? Let's do some (anti)phishing together

2006-10-22T22:02:00.000-07:00

I'm going to make use of the unexpected and sudden increase in traffic on this blog, due to my previous post, to talk about a volunteer project that I've been doing in my spare time.

The volunteer project that I'm talking about is the Phishing Incident Reporting and Reporting Squad, or PIRT as it is more commonly known.

What this project does is to vet through each and every submission of potential phishing sites and scam mails that are submitted and prepare the necessary information for ISPs, hosting providers, NIC handlers, CERTs, the commercial entities that are being faked, and/or other organizations that need to gather such reports for takedown or legal proceedings.

The gathering point for PIRT is on CastleCops, and is the first public and volunteers-based antiphishing community. The group of handlers are friendly and training is provided to help the newcomers get up to speed in determining whether a site is a phish/scam/exploit/spam or not.

As i go through the submissions, and gathered the necessary information for the reports, I've increased my knowledge of how such exploits work, usage of network tools, and the ways that the hosting sites are exploited or hacked to host such pages. I believe that's the same case for the rest of the handlers too.

Though not a primary focus of PIRT, the queue does get submissions of direct links to malware. Due to my concern as the admin of the release scanning system, i do gather such samples and forward them to the AV vendors for their detection (if they are not being detected at the point of investigation).

Recently, PIRT has hit the 10,000 submissions to Netcraft. Though it is indeed a good landmark to hit, it's also a bad reflection on what the end users are facing on a day to day basis.

It's also an indicator of the amount of daily submissions to PIRT. Though the current handlers are doing their best, the ever-increasing queue make one feel like a member of the Rohirrim defence in Helm's Deep while overlooking the oncoming army of Saruman's Uruk-hai!

Chances are, you will get at least a phish mail once in a while. Even if you do not have the time or knowledge to join the handlers, you can still send the phish mails to PIRT for the folks to take care of. You can also forward the phish mail to pirt (AT) castlecops.com.

By doing either of these actions, you will be helping to reduce the chances that another fellow Internet user will be scammed by the phishing sites.

Using an excerpt from Eric Cartman, in Make Love, Not Warcraft, as he was gathering support from his buddies to fight against the one with no life, "You can just hang around.... or you can sit at your computer and do something that matters.".

I think it applies in this case too. 8)

For more info about PIRT, check out the Castlecops' wiki.

Technorati tags: phishing, antiphishing, PIRT, volunteer, Castlecops

Where's your class? Your integrity?

2006-10-17T20:17:00.000-07:00

Heard about the portable video and music device that came preinstalled with a virus?

Instead of focusing on how and why it was even included in the first place, the company that published a series of video ads, including this one, actual try to divert the blame on the Windows platform!

It's not a matter of which platform that the virus originated. The fact that it's found on the portable player means that there's an issue with how the quality checks, specifically the content check was done. This also indicates that through the manufacturing cycle, the base device from which the image was duplicated to the other devices in the manufacturing run, was connected to a PC that most probably did not have , and i quote their press release, "up to date anti-virus software which is included with most Windows computers".

The press release also show a lack of awareness of how malware works. Focusing on the filename, instead of the actual malware info, might confuse the owners of both the media player and the antivirus program of the same filename!

Is it now open season for Zune to come up with their own ad to highlight this incident, as a direct response to the video ad?

Taking this into perspective, McDonald's in Japan encountered a similar incident just a few days earlier as well.

Indeed, they published a press release (via Babelfish or Google Translate), apologized for it, and did not insinuate that Windows was the cause of their issue.

Furthermore, they provided a very specific fix to their issue, compared to a general set of linkages to trial and/or free versions of anti-virus scanners.

Steve, if you need someone to advise on how to improve your quality checks, feel free to contact me 8).

As i was writing this post, i found that Ed Bott did a similar post as well.

Update (10/17) : Randy did a similar post too on Eset's Threat blog. It's almost 100% exactly the same as my post here, which shouldn't surprise those that know both of us as we have the same experience in managing the same scanning system.

Another Update (10/19) : Add a little clarification on the second paragraph. I did not mean that the company published the ad as a response to this incident. I meant that the conpany had previously published the ad as part of their campaign. It's common knowledge that these ads were published and promoted a long while ago. I do appreciate the comments. Thanks!

Yet Another Update (10/19) : Sunbelt, and many other AV vendors, on the Ravmone.exe trojan.

Technorati tags: antivirus, macdonald's, media devices

detecting test files

2006-10-02T23:51:00.000-07:00

Recently, i came across a set of files that were only flagged by one of the av products as being possibly malicious, specially as a trojan.

Since these were executables, i was able to use the sandbox technologies, Norman and Sunbelt among others, to try to figure out why these files are triggering the flag.

The sandbox output from the above tools didn't show anything malicious in the executables, even with my low level of assembly language knowledge.

I also used VirusTotal and Jotti's malware Scan to get the scanning results from scanners that i do not use, and the results were the same. Only one of the scanner sets (the same one that triggered my initial analysis) on both sites flagged the files.

I then loaded the files into a Virtual PC session, and launched tools like filemon/netmon/regmon and other similar tools and found nothing that indicate typical malicious behavior.

After doing these analysis, and it was truly a "DUH" moment, i re-examined the scanner log file.

Next to the flag of trojan, was an additional note of testfile. In the log, it was shown like (format changed somewhat below):

xxx.exe - a possible trojan--testfile

This led me to switch my train of thought. Assuming that these were indeed correct flags by this one single product, could these files be created to show that the product can detect the set of files, and that the product (at least its detection) works? Think of it as a scanner's own version of the Eicar test file.

I have not verified whether these are indeed files created for this purpose, but assuming that they are, what do this set of file truly represent? Probably in the hands of the marketing folks of that product, it could be used as a demonstration of how good the product is, no matter how one-sided it is.

Would these files be eventually detected by other antimalware products? Why would they do so? If they do not, would that just add ammunition to this company's salesforce?

In the hands of the uninformed, would the testing results be skewed unfairly to one product vs the others?

Don't we have enough confusion already?

Technorati tags: antivirus, testing

Open Season For Antimalware Testing?

2006-09-06T14:54:00.000-07:00

Not exactly following after the CR testing of the antispyware and antivirus products, but being brought into closer media attention from the CR aftermath, a number of other antimalware testing reports have been published recently.

These reports, published or co-published by "security experts", clouded the testing environment of a security space that is inherently complex and diverse to test fully. The consequence from these reports is a disservice to the antimalware product users, as this brings confusion and panick.

This space is highly specialized and requires years of dedicated discipline and knowledge before one is able to fully understand the challenges, let alone communicate these challenges to end users and the mass media.

With the major antimalware vendors labelling these reports as flawed and bogus, it creates a wrong impression that the antimalware vendors are bullying newcomers into the antimalware space, and/or that the vendors are attacking the results as these shows the weaknesses in the products. There's also an impression that "we know more than you do" that some bloggers, forum posters, and especially trolls, are using to create churn in a discussion that need to happen.

However, i believe that this is not the case. Experienced testing organizations like Virus Bulletin, ISCA/Checkmark, AV-Test, AV-Comparatives have been in the market to do the testing of antimalware products, and there are times that the software companies will still dispute the testing results. The communications that come out of these disputes will help to create a better test methodology and one that is going to be accepted by the software companies.

I've also seen non-responses to communication attempts by antimalware vendors to the publishers of the testing reports. If one don't even want to respond to queries on one's report, it really weakens the value of the report's output.

One action that i have not seen (it might be due to my lack of visibility) is the participation of the experienced testing labs on this issue, other than AV-Comparatives. If these other organizations got involved in the open discussions, it might help to reduce the fingerpointing from the parties involved.

Testings that are done by these various organizations need to have a way to be "reproed", so to speak. Well-documented methods and papers to test antivirus products are available, especially by the established testing labs and experienced independent consultants, but i do not see such documentation on some of the recent reports . Perhaps the usage of simulated malware is one of the reason for such non-disclosure, which flaws any testing of antimalware products from the start

If the testing methodologies are sound and correct, there shouldn't be as much argument about the reports as there are right now.

Thus, the challenges coming out of this open season of antimalware testing would be:

creating and defining an open, accepted testing methodology that its results can be reproduced in any other testing labs/organizations
the testing organziations need to be open to suggestions to improve their testing
to find a way to match the results from these testing organizations with mass media organizations like CR, magazines and other news media, perhaps by defining a list of criteria for "end-user perceived value" of an antimalware product such as
- speed of updates
- usability
- supportability
- coverage
- false positives
- compatibility with other products

With the Virus Bulletin and the AVAR conferences in the coming months, there lies great opportunities for these issues to be discussed and addressed if the publishers of the recent reports can attend and discuss with the AV researchers face-to-face, and most probably over an endless flow of alcohol. 8)

technorati: antispyware, AntiVirus, testing, Antimalware

Spyware testing (lackthereof?)

2006-08-29T14:55:00.000-07:00

Having looked at the numerous blog posts about Consumer Reports' September 2006 issue on the testing that they have conducted on anti-spyware applicatons, and having a chance to doing my own analysis (non-work related analysis) with the primarily (and apparently only) tool, Spycar, a suite of tools that was developed to "model some behaviours of spyware tools", i felt that perhaps i can also join in somewhat belatedly and share my jotted down personal comments on the Spycar suite viz-a-viz the CR report.

The Spycar suite itself is not malicious, that we know is true (at least right now in its current form. Therefore, the first point of discussion will be the basis of validity of an antimalware application, be it antivirus or antispyware application, flagging a non-malicious tool.

In most real-life situations and testings, having an application falsely flagged as malicious would be a bad experience for the end-user and the application company.

In the antimalware industry, the flagging of innocent applications would be termed as a false positive.
Since we have established that the Spycar suite is not malicious, and the simulated spyware actions are not malicious, how does a tester make full use of the results and behavioural blocking from antimalware products?

Should the tester give high scores for an antimalware application that does flag and block all the behaviours from the Spycar suite? Why should high scores be giving to blocking of non-harmful actions which are primarily registry settings, settings that can be changed through many other vectors as compared to the sole input from the suite?

The test coverage of the suite doesn't and cannot represent the overall quality of the antispyware application. Just because the application does not flag nor block the simulated actions (with reasons already mentioned above), the only thing that the tester can get out of this is that the application did not block the behaviour.

Does that mean that the application is inherently bad in detection and blocking? I dont think so. Furthermore, focusing on this single point, the tester will not know whether the non-blocking of the action is due to the program's specific design philosophy, its spyware categorization, its incompetence.

Which again brings my number one question in my mind when i was doing my own analysis is that how does the results that Spycar is looking for (behaviour blocking alone) represent a quality product?
With the large number of rogue and suspect antispyware products available, and adding some code to specifically flag and block the Spycar suite looks like a day's work at most, this increases the opportunity for an non-informed tester to use the expected behaviour from the rogue products and score them highly.

I can forsee the ads plastered all over the rogue applications websites "100% blocking for Spycar!".

Oftentimes, rogue applications would actually betray the "trust" of the end-users and maliciously flag valid security products for removal, thereby futher decreasing the overall security of the end-users' machines. Wouldn't this create an environment that further confuse the end-users?
The simulated infections from the Spycar suite do not necessary mean malicious intent, though it is common for spyware to do the simulated action.

For example, the blocking of the Internet Explorer Options screen is an action that most spyware would perform, but it's also an option that corporations and public terminals like library kiosks tend to use too. If this kind of behaviour is expected to be blocked and the end-user prompted, wouldn't that alarm the end-users unnecessarily?
Though the protection of the end-user's system is the primarily goal of any antimalware application, something that i've started to learn as a result of the months of constant discussions in my primary system, is that the best tool in the world wouldn't be useful (to anyone) if it takes up 100% of the CPU and 24 hours just to do a on-demand scan.

Thus, there might be additional test coverage and consideration for areas such as:
- performance of the tool
- user-friendlieness of the tool
- frequency of updates
- open-door policy for categorization of the spyware
Defense against malicious software in attacking itself would probably be very important too i believe. There's no use in having a antimalware application that can detect and block the behaviours simulated by Spycar if the application itself can be easily disabled before it can block the malicious actions. There's evidence in various viruses and malware that showed specific disabling of the more popular antimalware applications and defenses against it is critical.
The other important aspect of any antispyware application would be the effectiveness of the removal system. No antispyware is going to be useful if all it can do is to block and/or flag a spyware, but have problems removing the persistent ones!

You can also read up on more professionally written insights on this issue on various antimalware related blogs such as Eset's (by Randy), McAfee Avert Labs, Sunbelt's and Eric Howes' full commentary on this issue.

I just find it amazing that a antispyware test article (and the earlier antivirus test article) did not include a single real-life malicious sample. It's not that hard to find one on purpose these days!

With the above points in mind, i wonder how anyone can use the findings from the reviews to decide on what antimalware packages are the best to use.

And that's that for my first knowledge sharing post.

PS: this post was originally posted on the Offpoint blog, but i've decided to create a new blog to just focus on posts like these.

technorati: antispyware, AntiVirus, testing, Antimalware

hello world

2006-08-29T14:43:00.000-07:00

Hi. It's me.