Intrusion Detection Techniques, Methods & Best Practices

No security strategy is perfect, but those that work via multiple layers are better than those that don’t. At many organizations, for instance, intrusion detection/intrusion prevention (IDS / IPS) solutions have been deployed for many years as a logical combination with one or more firewalls.

The idea is simple: if a firewall constitutes an entry point to the infrastructure, the IDS / IPS solutions use a variety of intrusion detection techniques to form a kind of secondary protection, designed to assess what’s happening beyond the firewall and either take direct action when problems crop up, or alert team members who should.

IDS / IPS, by the way, shouldn’t be confused with security information and event management (SIEM) solutions and user behavior analytics — UBA — solutions, about which I wrote recently. What’s the difference? Well, UBA solutions leverage sophisticated machine learning algorithms to try to approximate the analytical skills of human security experts, and they focus on user behavior. IDS / IPS as a rule do not use machine learning, and address technical events or activity in a more general sense.

Organizations worried about botnets and DDOS attacks often leverage IDS / IPS solutions to mitigate that threat. For instance, IDS / IPS capabilities can often identify rogue outbound traffic — like a malware-compromised endpoint that’s attempting to communicate with a command-and-control botnet server for instructions. This makes finding the endpoint and blocking the suspect traffic coming to/from it much easier. It also helps quarantine endpoints and cease malicious conduct even if they do fall prey to malware.

Let’s take a closer look at some of the key capabilities and techniques used by different types of IDS / IPS solutions.

Reviewing IDS / IPS best practices and solutions

IDS vs. IPS approaches

IDS and IPS are related, and often conflated, but they’re fairly different at a basic level. Intrusion detection is a form of passive network monitoring, in which traffic is examined at a packet level and results of the analysis are logged. Intrusion prevention, on the other hand, is a more proactive approach, in which problematic patterns lead to direct action by the solution itself to fend off a breach.

Host-based vs. network-based vs. application-based intrusion detection techniques

The distinction here primarily concerns the abstract element of the infrastructure that’s being covered.

Host-based intrusion detection techniques revolve around individual hosts — usually servers — by monitoring the hard drive and both inbound and outbound packets, and constantly comparing the results against a pre-created image of the host and the host’s expected packet flow. The idea is to look for malicious changes both in the logical contents of the host as well as the host’s activity. It often relies on a local client or agent of the IDS system to be installed on the host.

Application-based intrusion detection techniques widen the scope to an application in an abstract sense — meaning, everything in the infrastructure that’s involved in the way that application functions, but only that application. These solutions are used for applications that perform particularly crucial functions for the organization, because the potential consequences of a breach are high.

Network-based intrusion detection techniques expand the scope of coverage still further to all devices on a network or subnetwork (sometimes, multiple instances of solutions collaborate to accomplish this, due to the volume of traffic). Because they are the most general, they sometimes miss problems the other two might detect.

A variety of IDS / IPS detection methods and techniques

Now let’s consider some of the common ways IDS / IPS solutions actually work to accomplish these goals.

Anomaly-based intrusion detection techniques

Also called behavior-based, these solutions track activity within the specific scope (see above) looking for instances of malicious behavior — at least, as they define it, which is a difficult job, and sometimes leads to false positives. For instance, outbound URLs of Web activity might be considered, and sites involving certain domains or URL length/contents might automatically be blocked, even though it’s a human being trying to go there (not malware), and that user has a business-legitimate reason.

Signature-based intrusion detection techniques

This approach, also known as knowledge-based, involves looking for specific signatures — byte combinations — that when they occur, almost invariably imply bad news. Read: malware itself, or packets sent by malware in the attempt to create or leverage a security breach. These solutions generate fewer false positives than anomaly solutions because the search criteria is so specific, but they also only cover signatures that are already in the search database (which means truly novel attacks have good odds of success).

The future of IDS / IPS

Naturally, organizations should consider all these intrusion detection techniques in context — choosing a logical IDS / IPS approach that will pair well with their context, as well as interoperate with other elements of the total security infrastructure.

Going forward, we expect IDS / IPS solutions both to evolve in such a way as to integrate with more infrastructural solutions, as well as incorporate new strategies at a basic level.

For instance, through neural network/artificial intelligence capabilities, IDS/IPS anomaly-based solutions should be able to more accurately predict and recognize “normal” activity — which also means they’ll be able to spot malicious activity faster, and generate a much lower percentage of false positives – without the tremendous amount of continuous tuning effort that takes place today.