Introduction

Responsible organizations agree that they need to protect their networks from virus attacks by installing an email security product. Yet, how does one choose the right solution out of the wide variety of virus scanning engines available? And is one anti-virus engine enough to protect the internal network from mass-mailing viruses, worms and other email-borne threats?

The tests detailed in this paper show that each virus scanner presents its own strengths and weaknesses. This means that no single anti-virus engine can fully protect against all possible threats. As a result, simultaneous use of more than one virus engine can achieve greater security than is technically possible when relying on only one anti-virus engine. The use of multiple virus engines also enables security administrators to be vendor independent when it comes to virus scanning, thereby able to use the best of breed virus engines available on the market.

Note: This paper does not cover desktop virus scanners. Its aim is to feature several popular virus-scanning engines and highlight the differences between each.

A Review of Current Anti-Virus Engine Tests

This paper examines the research currently available on leading anti-virus engines - namely, those developed by Trend Micro, Norton, Bit-Defender, McAfee and Norman - and studies their performance in three key areas:

• Overall detection rates of in the wild and zoo viruses;

• Their ability to scan through compressed and embedded files; and the coverage of non-virus malware

Detailed results for each set of tests are found in Appendices A, B and C respectively.

The results compiled in this paper are based on tests conducted by these anti-virus testing laboratories:

ICSA Labs - ICSA certification is regarded as the guarantee that a certain product is top notch and assures customers that the product has succeeded in a number of stringent tests.

West Coast Labs - The West Coast Checkmark has been developed as an independent testing and standards organization. Checkmark-certified products and services can be relied upon to an identified standard.

Virus Bulletin - The Virus Bulletin 100% award is given to those anti-virus products that detect all in the wild viruses through both on-demand and on-access scanning during testing.

AV-Test.org - This German organization of the University of Magdeburg consistently tests anti-virus software on behalf of companies and leading IT publications for client, server, UNIX and groupware products.

Virus TestCenter - The University of Hamburg Computer Science Department runs tests on anti-virus products and publishes the results in its Virus TestCenter, with an emphasis on the detection of zoo viruses.

For further information on each lab, please see Appendix D.

Test Results in a Nutshell

Considered together, the various test results show that no single anti-virus engine can fully protect against all possible threats (see Appendices A, B and C for full results).

For instance, Trend Micro does not scan ACE, B2 or TGZ compressed files, and it does not detect viruses compressed with the increasingly popular UPX. However, it excels in the MS Office files area, capturing all OLE objects embedded in such files in the AV-Test.org tests. Trend Micro's products also obtained good (but not full) results with non-virus malware.

While Norton AntiVirus achieves a good rate at detecting both ITW and zoo viruses, it fails to detect viruses compressed with packages such as UPX, Shrink, and ASPack. In the tests, it achieves an average detection rate of 75% of backdoors and Trojan files.

McAfee VirusScan yielded different results from different testing organizations regarding detection of ITW viruses on different platforms. According to the AV-Test.org tests of November 2001, VirusScan caught 99.5% of the in the wild (ITW) viruses. This product does not support compression formats RAR or ACE and does not detect viruses compressed with UPX and other similar products. However McAfee achieved good results in the non-virus malware section (ActiveX, backdoors and Trojans).

Norman's main strength seems to be in maintaining a high rate at detecting ITW and zoo viruses. However Norman is less powerful when the viruses are compressed with formats other than ZIP and ARJ, or using any self-extracting (SFX) archiving method such as WinZip.

BitDefender by SOFTWIN supports several compression formats like ACE, ARJ, RAR and ZIP. It also checks through files packed using popular packaged such as UPX, Neolite and ASPack. Yet, it missed one ITW file virus and caught 92% of all zoo viruses on test.

The Case for Using Multiple Engines

Given the inability of any individual anti-virus engine to provide full coverage against all email attacks, logic dictates that combining multiple engines will produce a more complete solution. In simple terms, if anti-virus products X and Y - each stronger in one area but weaker in another - are used together, their joined strength is likely to cover a wider range of security areas, and this way they can counteract each other's weak points.

Further analysis shows the validity of this theory. The tables below use data from the AV-Test.org tests of November 2001 to show the impact of using two or three virus scanning engines to increase protection.

Email security product A1 with Norman and BitDefender engines installed

This email product "A1" would cover 100% of ITW viruses, between 56.8% and 78.4% of the most popular compression methods, and 84% - 100% of samples from the "other malware" section.

Email security product A2 with McAfee, Norman and BitDefender engines installed

This product "A2" would cover 100% or ITW viruses, about 56.8% or more of the most popular compression methods, and 98.7% - 100% of samples from the "other Malware" section.

Another email security product - "B - uses the Norton virus-scanning engine. The table below shows the total coverage with this product:

A fourth email security product, "C" uses Trend's anti-virus engine, with the following results.

Comparing these four products, we notice that A2 has an advantage over the rest of the products, with A1 next on the performance list.

The table below gives a closer view of the compression area (where virus scanners tend to differ greatly in performance):

Here, one sees how BitDefender covers many more of the compression formats than the rest of the virus-scanning engines being used in this analysis. The email security product A2 would therefore provide a much more complete solution than a product making use of a single virus scanner.

As of the date of this writing, GFI MailSecurity for Exchange/SMTP was virtually the only product on the market to provide support for multiple virus engines, and the only one to offer the comprehensive protection of McAfee, Norman and BitDefender illustrated here.

Additional Considerations in GFI MailSecurity

While anti-virus protection is a critical component in protecting a network from email-related threats, virus protection alone cannot fully safeguard networks from email assaults. The fact that virus scanners only cover a portion of non-virus threats is well known. Therefore a fuller email security product should include features that protect against email-borne security threats apart from viruses, as well as multiple virus scanners.

Again, GFI MailSecurity for Exchange/SMTP provides a solution. In addition to the unusual simultaneous use of multiple virus engines discussed above, MailSecurity provides Email content & attachment checking - to quarantine dangerous emails; Exploit shield - to provide mail intrusion detection and defense; and an Email threats engine - to analyze and defuse HTML scripts, .exe files and more. This combination of features is unique in the industry, providing maximum protection against email-related network assaults. Other features of GFI MailSecurity include:

• Automatic removal of HTML scripts

• Automatic quarantining of Microsoft Word documents with macros

• Detects attachment extension hiding

• Rules-based configuration

• Apply rules to AD users or groups

• Approve/reject quarantined mail using the moderator client/email client/public folders

• Lexical analysis

• Seamless integration with Exchange Server 2000 through VS API

• Excellent value

GFI MailSecurity for Exchange/SMTP can be deployed at the gateway level, or at the information store level (based on the Exchange 2000 VS API). An evaluation version can be downloaded from: http://www.gfi.com/mailsecurity

About GFI

GFI has six offices in the US, UK, Germany, France, Australia and Malta, and has a worldwide network of distributors. GFI is the developer of GFI FAXmaker, MailSecurity, Mail essentials and GFI LANguard, and has supplied applications to clients such as Microsoft, Telstra, Time Warner Cable, Shell Oil Lubricants, NASA, DHL, Caterpillar, BMW, the US IRS, and the USAF. GFI is a Microsoft Gold Certified Partner and has won the Microsoft Fusion 2000 (GEM) Packaged Application Partner of the Year award.

Acknowledgments

Special thanks to Andreas Marx [amarx@gega-it.de] of AV-Test.org for his help and contributions to this paper.