An in-depth look at next-gen security software: Myths and marketing
The Age of Dinosaurs
There is a view of the current security market that is often recycled by the media these days. It assumes a split between 'first-gen(eration)' or 'traditional' (or even 'fossil' or 'dinosaur') malware detection technology – which is invariably claimed to rely on reactive signature detection – and (allegedly) superior technologies using 'next-gen(eration)' signature-less detection. This picture is much favored by some 'next-gen' companies in their marketing, but it doesn't reflect reality.
The Theory of Evolution
First of all, I'd take issue with that term 'first-generation'. A modern mainstream security suite can no more to be lumped in with early 'single layer' technologies – such as static signature scanners, change detection and vaccines – than Microsoft Word can be with ed or edlin.
They may have the same fundamental purpose as those long-gone applications – be it detection and/or blocking of malicious software, or the creation and processing of text – but they have a much wider range of functionality. A modern word processor incorporates elements that decades ago would have been considered purely the domains of desktop publishing, spreadsheets and databases.
The Origin of Species
A modern anti-malware-focused security suite isn't quite so wide-ranging in the programmatic elements it incorporates. Nevertheless, it includes layers of generic protection that go far beyond signatures (even generic signatures). They have evolved into very different generations of product, incorporating technologies that didn't exist when the first security products were launched. To talk about newcomers to the market as if they alone are 'the next generation' that goes beyond primitive signature-specific technology is misconceived and utterly misleading.
Signatures? What signatures?
Nowadays, even modern, commercial single-layer anti-malware scanners go far beyond looking for specific samples and simple static signatures. They augment detection of known, hash-specific families of malware with the inclusion of elements of whitelisting, behaviour analysis, behaviour blocking, and change-detection (for instance) that were once considered to be pure 'generic' technologies.
Not that I recommend in general that people should rely totally on a single-layer scanner such as those often offered for free by mainstream companies: they should be using other 'layers' of protection as well, either by using a commercial-grade security suite, or by replicating the multi-layered functionality of such a suite, while using components drawn from a variety of sources, including a single-layer anti-malware scanner.
However, the latter approach requires a level of understanding of threat and security technologies that most individuals don't have. Come to that, not all organisations have access to such a knowledgeable resource in-house, which leaves them potentially at the mercy of marketing masquerading as technical advice.
Back to basics
Although some next-gen products are so secretive about how their technology actually works that they make mainstream anti-malware products look like open source, it's clear that the distinctions between 'fossilized' and 'next-gen' products are often terminological rather than technological.
I don't consider that 'next-gen' products have gone further beyond these basic approaches to defeating malware, defined long ago by Fred Cohen (whose introduction and definition of the term 'computer virus' to all intents and purposes jump-started the anti-malware industry in 1984), than have 'traditional' solutions:
- Identifying and blocking malicious behaviour
- Detecting unexpected and inappropriate changes
- Detecting patterns that indicate the presence of known or unknown malware
The ways of implementing those approaches have, of course, become immeasurably more advanced, but that progression is not the exclusive property of recently-launched products. For example, what we generally see described as 'Indicators of Compromise' could also be described as (rather weak) signatures.
More than one vendor has failed to differentiate convincingly between mainstream anti-malware use of behaviour analysis and blocking, between its own use of (for instance) behavioural analysis/monitoring/blocking, traffic analysis (and so on) and the use of the same technologies by mainstream anti-malware. Instead, they've chosen to promote a deceptive view of 'fossil technology' and peppered their marketing with a hailstorm of technological buzzwords.
Welcome to the machine
Consider, for instance, the frequent lauding of 'behavior analysis' and 'pure' Machine learning (ML) as technologies that set next-gen apart from first-gen. In the real world, Machine learning isn't unique to one market sector.
Progress in areas like neural networking and parallel processing are as useful in mainstream security as in other areas of computing: for example, without some degree of automation in the sample classification process, we couldn't begin to cope with the daily avalanche of hundreds of thousands of threat samples that must be examined in order to generate accurate detection.
However, the use of terms like 'pure ML' in next-gen marketing is oratorical, not technological. It implies not only that ML alone somehow provides better detection than any other technology, but also that it is so effective that there is no need for human oversight.
In fact, while ML approaches have long been well-known and well-used in the mainstream anti-malware industry, they have their pros and cons like any other approach. Not least, in that the creators of malware are often as aware of ML as the security vendors who detect malware, and devote much effort to finding ways of evading it, as is the case with other anti-malware technologies.
On your best behaviour
Similarly, when next-gen vendors talk about behavioural analysis as their exclusive discovery, they're at best misinformed: the term behavioral analysis and the technologies taking that approach have both been used in mainstream anti-malware for decades. In fact, almost any detection method that goes beyond static signatures can be defined as behaviour analysis.
Natural and unnatural selection
Journalist Kevin Townsend asked me recently:
Is there any way that the industry can help the user compare and choose between 1st […] and 2nd generation […] for the detection of malware?
Leaving aside the totally misleading 1st versus 2nd-generation terminology, yes, of course there is. In fact, some of the companies self-promoted as '2nd-generation' and claiming that their technology is too advanced to test have nevertheless pushed an already open door even wider by their own attempts to compare the effectiveness of their own products and those of 'first-gen' vendors.
For example, at least one next-gen vendor has taken to using malware samples in its own public demonstrations: if different generations of product can't be compared in an independent test environment, how can such demonstrations be claimed to be accurate in a public relations exercise? Other misleading marketing from next-gen vendors includes claims that "1st-gen products don't detect 'file-less' malware in memory" (which we've done for decades).
One particularly inept example used a poorly constructed survey based on Freedom of Information requests to 'prove' 'traditional' anti-malware's 'abject failure' without attempting to distinguish between attacks and successful attacks.
Testing and Pseudo-testing
More commonly, VirusTotal (VT) is misused by misrepresenting its reports as if VT and similar services are suitable for use as 'multi-engine AV testing services', which is not the case. As VT puts it:
VirusTotal should not be used to generate comparative metrics between different antivirus products. Antivirus engines can be sophisticated tools that have additional detection features that may not function within the VirusTotal scanning environment. Because of this, VirusTotal scan results aren't intended to be used for the comparison of the effectiveness of antivirus products.
VT can be said to 'test' a file by exposing it to a batch of malware detection engines. But it doesn't use the full range of detection technologies incorporated into those products, so it doesn't accurately test or represent product effectiveness. One nextgen vendor talked up its own detection of a specific ransomware sample a month before the same sample was submitted to VirusTotal.
However, at least one mainstream/traditional vendor was detecting that hash a month before that next-gen detection was announced. You simply can't measure a product's effectiveness from VirusTotal reports, because VT is not a tester and its reports only reflect part of the functionality of the products it makes use of. Otherwise, there'd be no need for reputable mainstream testers like Virus Bulletin, SE Labs, AV-Comparatives and AV-Test, who go to enormous lengths to make their tests as accurate and representative as possible.
Towards cooperation
One of the more dramatic turnarounds in 2016 took place when VirusTotal changed its terms of engagement in order to make it harder for next-gen companies to benefit from access to samples submitted by "1stgen" companies to VirusTotal without contributing to VT themselves. To quote VirusTotal's blog:
…all scanning companies will now be required to integrate their detection scanner in the public VT interface, in order to be eligible to receive antivirus results as part of their VirusTotal API services. Additionally, new scanners joining the community will need to prove a certification and/ or independent reviews from security testers according to best practices of Anti-Malware Testing Standards Organization (AMTSO).
While many vendors in the next-gen space initially responded along the lines of "It's not fair", "The dinosaurs are ganging up on us", and "We don't use signatures so we don't need VT and we don't care", it seems that several big names were subsequently prepared to meet those requirements by joining AMTSO and thus opening themselves up to independent testing.
(By that I mean real testing, not pseudo-testing with VirusTotal.) Since next-gen vendors have tended in the past to protest that their own products cannot be tested, especially by the 'biased' testers represented in AMTSO, perhaps this suggests the possibility of an encouraging realisation that not all customers rely purely on marketing when they make purchasing decisions.
Share and share alike
Why have (some) next-gen vendors now decided that they do need to work with VirusTotal? Well, VT shares the samples it receives with vendors and provides an API that can be used to check files automatically against all the engines VT uses. This allows vendors not only to access a common pool of samples shared by mainstream vendors, but to check them against indeterminate samples and their own detections, thereby training their machine learning algorithms (where applicable).
And why not? That's not dissimilar to the way in which longer-established vendors use VirusTotal. The difference lies in the fact that under the updated terms of engagement the benefit is three-way. Vendors (of any generation) benefit from access to VirusTotal's resources and that huge sample pool. VirusTotal benefits as an aggregator of information as well as in its role as a provider of premium services.
And the rest of the world benefits from the existence of a free service that allows them to check individual suspect files with a wide range of products. Widening that range of products to include less-traditional technologies should improve the accuracy of that service, while the newer participants will, perhaps, be more scrupulous about not misusing VT reports for pseudo-testing and marketing when they themselves are exposed to that kind of manipulation.
Whole-product testing
The way that AMTSO-aligned testers have moved towards 'whole-product testing' in recent years is exactly the direction in which testers need to go in order to evaluate those less 'traditional' products fairly. (Or, at any rate, as fairly as they do mainstream products.) It can be argued, though, that testers can be conservative in their methodology. It's not so long ago that static testing was the order of the day (and to some extent still is among testers not aligned to AMTSO, which has discouraged it since the organization's inception).
AMTSO, despite all its faults, is greater (and more disinterested) than the sum of its parts because it includes a range of researchers both from vendors and from testing organisations, and marketing people aren't strongly represented. Thus, individual companies on either side of the divide are less able to exert undue influence on the organisation as a whole in pursuit of their own self-interest. If the next-gen companies can grit their teeth and engage with that culture, we'll all benefit.
AMTSO has suffered in the past from the presence of organisations whose agenda seemed to have been overly-focused on manipulation or worse, but a better balance of 'old and new' vendors and testers within the organisation stands a good chance of surviving any such shenanigans.
Into the Cenozoic
Several years ago I concluded an article for Virus Bulletin with these words:
But can we imagine a world without AV, since apparently the last rites are being read already? … Would the same companies currently dissing AV while piggybacking its research be able to match the expertise of the people currently working in anti-malware labs?
I think perhaps we have an answer to that. But if the self-styled next generation can come to terms with its own limitations, moderate its aggressive marketing, and learn the benefits of cooperation between companies with differing strengths and capabilities, we may yet all benefit from the détente.
This article is an adapted version of the corresponding section from ESET's 2017 trends paper, Security Held Ransom.
By David Harley, welivesecurity senior research fellow.
