The Art of Triage: Types of Security Incidents
Understanding whether an event is an actual incident reminds me of that common expression, “I know it when I see it” made famous by US Supreme Court Justice Stewart. He was referring to obscenity rather than incident response, but a common misperception of “knowing it when you see it” can often plague the most well intentioned incident responders.
The uncomfortable truth is that you may not know it when you see it, because the latest attacker tools and techniques are increasingly stealthy, and can often hide in plain sight. The trick is to view your network and operations from the perspective of an attacker, looking for key indicators and areas of exposure before they’re exploited. And it all comes down to how artfully you can do incident triage.
Typically used within the medical community, effective triage saves lives by helping emergency medical personnel rapidly assess wound or illness severity and establish the right protocols, in the right order, to reduce trauma and sustain patient health and recovery. All in the midst of crisis, when every second counts.
In this chapter, we’ll give you the tools to craft your ability to triage information security incident types. You’ll learn how to identify the various types of security incidents by understanding how attacks unfold, and how to effectively respond before they get out of hand.
Security Incidents vs. Information Security Incidents
A quick note on the difference between a security incident and an information security incident… In this guide, the assumption is that we’re focused on the various types of information security incidents vs. your standard security incident, which might not involve digital information and could be completely contained within the physical world (e.g. physical assault). That said, there may be occasions that mix things up - types of information security incidents or attacks that do involve a physical component (e.g. laptop theft).
Different Types of Security Incidents Merit
Different Response Strategies
So what are you protecting against? The best way to determine the appropriate incident response in any given situation is to understand what types of attacks are likely to be used against your organization. For example, NIST has provided the following list of the different attack vectors:
An attack executed from removable media (e.g., flash drive, CD) or a peripheral device.
An attack executed via an email message or attachment (e.g. malware infection).
An attack that employs brute force methods to compromise, degrade, or destroy systems, networks, or services.
Any incident resulting from violation of an organization’s acceptable usage policies by an authorized user, excluding the above categories.
An attack executed from a website or a web-based application (e.g. drive-by download).
The loss or theft of a computing device or media used by the organization, such as a laptop or smartphone.
An attack that does not fit into any of the other categories.
Categorize Information Security Incident Types by Getting Inside the Mind of the Attacker
One of the biggest fallacies with traditional information security is the underlying assumption that you know which path an attacker will take through your network. For example, attackers rarely come through your front door, or in this context, your gateway firewall. But each attack does generally work through a certain pattern, or what Lockheed Martin has called the “cyber kill chain.”
The “cyber kill chain” is a sequence of stages required for an attacker to successfully infiltrate a network and exfiltrate data from it. Each stage demonstrates a specific goal along the attacker’s path. Designing your monitoring and response plan around the cyber kill chain model is an effective method because it focuses on how actual attacks happen.
1) Reconnaissance & Probing
- Find target
- Develop plan of attack based on opportunities for exploit
2) Delivery & Attack
- Place delivery mechanism online
- Use social engineering to induce target to access malware or other exploit
3) Exploitation & Installation
- Exploit vulnerabilities on target systems to acquire access
- Elevate user privileges and install persistence payload
4) System Compromise
- Ex-filtrate high-value data as quietly and quickly as possible
- Use compromised system to gain additional access, “steal” computing resources, and/or use in an attack against someone else
Which Security Events Do I Really Need to Worry About?
Which security events develop into the type of information security incident that requires my attention now? And… what do I do about it? To help categorize each incident type, align each one against the cyber kill chain to determine appropriate priority and incident response strategy. You can use this table as a start.
Port Scanning Activity* (pre‑incident)
Reconnaissance & Probing
Ignore most of these events UNLESS the source IP has a known bad reputation , and there are multiple events from this same IP in a small timeframe.
Bonus tip: AlienVault’s Open Threat Exchange® is a great way to check on an IP’s reputation score.
Delivery & Attack
Remediate any malware infections as quickly as possible before they progress. Scan the rest of your network for indicators of compromise associated with this outbreak (e.g. MD5 hashes).
Distributed Denial Of Service
Exploitation & Installation
Configure web servers to protect against HTTP and SYN flood requests. Coordinate with your ISP during an attack to block the source IPs.
Distributed Denial Of Service Diversion
Exploitation & Installation
Sometimes a DDoS is used to divert attention away from another more serious attack attempt. Increase monitoring & investigate all related activity, and work closely with your ISP or service provider.
Exploitation & Installation
Detect, monitor and investigate unauthorized access attempts – with priority on those that are mission-critical and/or contain sensitive data.
Identify the privileged user accounts for all domains, servers, apps, and critical devices. Ensure that monitoring is enabled for all systems, and for all system events, and also make sure it’s feeding your log monitoring infrastructure (your USM or SIEM tools).
Unauthorized Privilege Escalation
Exploitation & Installation
Configure your critical systems to record all privileged escalation events and set alarms for unauthorized privilege escalation attempts.
Backup all critical data and systems. Test, document, and update system recovery procedures. During a system compromise - capture evidence carefully, and document all recovery steps as well as all evidentiary data collected.
Advanced Persistent Threat Or Multistage Attack
Any one of the singular events that are listed here could actually be a part of the worst type of security incident imaginable… the dreaded APT. The important thing is to view each event through a larger context, one that incorporates the latest threat intelligence (see below for more on the need for threat intelligence).
Much of the incident responder’s job is spent eliminating irrelevant information and removing false positives. You’ll be constantly fine-tuning the radio of security monitoring to get to just the right signal.
Incident response is a discipline of continual improvement. As you see more and more events turn into incidents, you’ll discover new ways to categorize those incidents, as well as new ways to prevent them from ever happening in the first place.
* A Note About Port Scanning:
Even if you’re sure that an attacker is getting no useful information back from their scanning, if they seem to be doing a detailed and comprehensive scan of your external systems, it is reasonable to interpret this as intent to follow-up the recon with attack attempts later on. If the scanning originates from a legitimate organization’s networks, then contacting their security team (if they have one) or network management personnel is usually the best approach.
As a last resort, if no contact details are apparent, try the contact details listed in the WHOIS information for the domain. The email address [email protected] is often a contact email for this kind of communication, but may not be available for smaller or younger organizations. BTW, blocking the source address may be counterproductive, and merely cause the attacker to use a different source address.
** A Note About False alarms:
We’ve expressed the need to “concentrate on what you know” many times in this guide – much of the work that security monitoring discovers is mundane yet vital.
- Controls Failure: Firewall ports that shouldn’t be open to the world, categories of websites that should be blocked at the proxy, hosts that were compromised because they didn’t have endpoint security installed. Incident Response work is best thought of as “quality assurance” for the rest of your security efforts.
- Noise Reduction: If security analysis is about finding the ‘needle in a haystack,’ one of the best ways to make the job easier is to make a smaller haystack. Remove unnecessary traffic, unwanted services, outdated client software, and easily-patched vulnerabilities.
- Policy Violation: Ideally, you hope to be spending more of your time locating the things happening that put your network at risk, not cleaning up the results of that risk being exploited by a hostile party.
Combine Local & Global Threat Intelligence
for Effective Triage
We often think of incident response as being detailed, meticulous forensic work, looking closely at one system at a time. However, the great majority of security monitoring work can be addressed through seeing a larger more holistic picture of the state of, and activity on, your infrastructure.
Understanding where, which, and how your systems are communicating with other systems, and the changes being made to them, can reveal attacks that other security controls cannot.
Threat intelligence allows you to move away from a focus on vulnerabilities, exploits and patches, and focus on the things that are actively causing damage to your company’s data confidentiality, integrity, and availability.
The first step is to understand as much as possible about your current computing environment. Some people refer to this as environmental awareness or situational awareness or even contextual awareness. We like to think of it as local threat intelligence.
Once you combine rich information about your own network with the latest global threat intelligence (specifics on attacker tools, techniques, and trends), you’ll achieve effective triage. You’ll put your immediate focus on the types of security incidents that matter vs. wasting your time on false positives or irrelevant noise.
Unless your infrastructure is entirely static and unchanging, new vulnerabilities and exposures are being created all the time.
Good IT and Security management processes will do its best to minimize these, but the security analyst still needs to be aware of them to place other things into context.
Unexpected configuration changes to systems can reveal when a hostile party has control of the system through valid credentials and methods.
Many configuration options are related to certain compliance standards – alerting (or reporting) on these is a far better way to manage them than waiting for them to be discovered during your next audit.
You can’t do security just by looking for attacks and exploits – you have to look into what’s happening on your network and know the systems you have deployed.
The Bottom Line
By understanding what is happening on your network (environmental awareness) and connecting it to information about known sources of malicious activity (Global Threat Intelligence), it becomes simple to get large-scale reports on active threats within your infrastructure. For example, instead of searching through massive lists of alerts from various security controls to determine possible exploits and attacks, and attempting to prioritize them based on asset value, we look at environmental awareness data that can be connected to the indicators of compromise associated with threat actors.
Today, computing resources are cheap and plentiful – attacks can come from anywhere – especially from compromised systems on otherwise legitimate remote networks. Attackers fight a constant battle trying to make it difficult to locate the systems that control their malware, while still allowing their malware to reach these systems to receive execution instructions.
See our next chapter – Chapter Four - Incident Response Tools for how-to instructions on uncovering more information on attack sources.
Navigate Your Journey:
The Importance of Attribution
Let me tell you a tale from the front lines.
We were hunting down an active attack, endpoints had been compromised and they had migrated to using stolen credentials to access the network directly - without further use of the RAT (remote access tool) on the compromised system. As we watched our attackers access the network from multiple locations we began to build our profile of them - what hours were they active, could we determine a particular time zone they may be located in? How many people were acting together? Finally we realized something from the connection authentication information… The connections we saw from multiple remote locations were actually only from a single host - a cloud-provisioned host. As the host was brought online to stage attacks from it, it was being relocated to lower-load hardware clusters with an entirely different upstream connectivity. We noticed that the host had used many different IP addresses and physical connectivity sets (spanning three different countries), but it was still the same virtual machine instance the entire time.
We were well prepared to do identification of remote hosts on VPS -style co-location arrangements, but the global motility of hosts on cloud providers had temporarily thrown us for a loop - we realized that the game had once again changed right before our eyes.
In this case, we had an obvious advantage that we had pre-existing data points to correlate together that allowed us to uncover what was happening behind the curtain; but it would have been much better if we had the ability to easily identify this beforehand. And I realized that the network identification tools we have today are all built for a pre-cloud Internet - a world where IP addresses are tied to physical hosts in physical locations owned by identifiable registered organizations. Now anyone that remembers the Internet pre-1999 will remember the venerable Ident() protocol (rapidly made obsolete as it represented a security risk in the open untrusted Internet).
Yet couldn't a tokenized, anonymous version of this provide some measure of utility in a cloud-served public Internet? The ability to query Amazon's web services and know that all three EC2 instances currently attacking me are all operated by Tokenized Amazon Customer F8E993C?
Until an initiative to create something of this sort arises and gets implemented, I'll have to stick with more abstruse methods of remotely fingerprint cloud instances post facto. As cloud computing offers ever more copious amounts of utility computing: OS instances that can be launched, operated, and deleted in a matter of minutes, we on the defensive end of things need some way to keep up with the increasing complexity of attribution.
The increasing need for attribution techniques in incident response is not just some by-product of a Security Analyst wanting to play counter-intelligence agent. Attribution is vital for correlating and prioritizing the tidal wave of data we need to pour through to make informed response decisions. Being able to correlate two seemingly unrelated minor attack attempts on different parts of the infrastructure launched from two random hosts on the same multinational cloud computing provider can make a huge difference.
There is much work in progress on establishing reputation between cloud service providers and their customers - but the need to establish reputation information from cloud instances to the rest of the world is essential in the world of the Incident Responder.