Why We Should Love False Positives
Let's face it. False positives suck. On the other hand, while the subtitle of this post may sound funny, it isn't really that far from the truth. It’s true that many people experience a level of anger at false positives that could be used as a climax to a thriller movie. The reality, however, is that many of these results are not as bad as they seem, and can often actually be useful. This can be for a variety of reasons. Let's look at a few of them.
Why Perfection Isn't Perfect
In a perfect world, the best of Intrusion Detection System (IDS) signatures would be an exact match for the file or behavior that they are looking for. This is impractical to create for a couple reasons, and is often not even desirable in practice.
A perfect signature would need to be compared to literally millions of other files and/or sources to confirm it didn't match other files. This would add a tremendous cost, and more importantly time, to the development cycle before the signature is released on a new threat. In our industry, where seconds matter, the extra delay for such thorough testing before protection can be provided would be a serious drawback.
A much more important point is that we often do not want the signature to be an exact match. Malicious Threats that were introduced 20 years ago were usually unchanging, with predictable signatures that could be rather reliably matched. Unfortunately, the modern threat landscape is far more fluid. With the accelerating growth of polymorphic, metamorphic, kit-based, and other advanced threat forms, IDS's need to be able to match a similar pattern to a signature, or to match behavioral traits of the traffic. By casting a wider net, the signature can capture the threat as it evolves, minimizing the chance that the variant can compromise a system before a sample is identified.
The App Behind the Mask
Most security best practices do not work in favor of detection systems. A core part of running a vulnerability scan is identifying the type and version of any OS or Application/Service scanned. This can have varying results, depending on what hardening has been implemented. In short, the vulnerability scan often has to approximate this information, or surmise it from the format of a response. A good example of why this can be difficult is that my own web server, a hosted Linux server, advertises itself as IIS5.5 instead of the service actually running.
Some False Positives, Quite Simply, Are Not
I once had an upset customer call, insisting that we immediately remove a false positive from our vulnerability scans because he had to fix everything that showed on the report. He insisted that marking something as a false positive was unacceptable.
Here is the catch; it wasn't a false positive. The exploit could not be leveraged because the vulnerability required a folder in the document root that had a name of a minimum character length that began with the letter “a.” The security of his web server, and potentially the network it was on, was dependent on what folder name the web developers chose.
Some of my most common cases are the results of “conditional” false positives. So what's in a word? In this case, our livelihood. Many incidents, or vulnerabilities, are dependent on secondary factors that the system is unable to review. When a threat is not currently present, but can become a threat via a configuration change, or a change in a third party package, the duty of the system is to pass a potential threat on for further review.
What No One Wants
The key takeaway from this post should be that, whatever happens due to the issues listed above or otherwise, the one thing we never want to see is a false negative. Yes, a false positive requires time to look into, and can be annoying. I would rather have a thousand false positives that I have to adapt for when training a system than a single false negative.
As I often say, the most important piece of security infrastructure is the human sitting in front of the screen. Security solutions acknowledge this, and are designed specifically to augment the capability of the specialist by providing the information as efficiently as possible. If there is any doubt in the validity of an action, the role of the solution is to provide as much information as possible to assist in a review of the issue. When considering the alternative outcomes, we should appreciate that this happens.
