Understanding whether an event is an actual incident reminds me of that common expression, “I know it when I see it” made famous by US Supreme Court Justice Stewart. He was referring to obscenity rather than incident response, but a common misperception of “knowing it when you see it” can often plague the most well intentioned incident responders.
The uncomfortable truth is that you may not know it when you see it, because the latest attacker tools and techniques are increasingly stealthy, and can often hide in plain sight. The trick is to view your network and operations from the perspective of an attacker, looking for key indicators and areas of exposure before they’re exploited. And it all comes down to how artfully you can do incident triage.
Typically used within the medical community, effective triage saves lives by helping emergency medical personnel rapidly assess wound or illness severity and establish the right protocols, in the right order, to reduce trauma and sustain patient health and recovery. All in the midst of crisis, when every second counts.
In this chapter, we’ll give you the tools to craft your ability to triage information security incident types. You’ll learn how to identify the various types of security incidents by understanding how attacks unfold, and how to effectively respond before they get out of hand.
A quick note on the difference between a security incident and an information security incident… In this guide, the assumption is that we’re focused on the various types of information security incidents vs. your standard security incident, which might not involve digital information and could be completely contained within the physical world (e.g. physical assault). That said, there may be occasions that mix things up - types of information security incidents or attacks that do involve a physical component (e.g. laptop theft).
So what are you protecting against? The best way to determine the appropriate incident response in any given situation is to understand what types of attacks are likely to be used against your organization. For example, NIST has provided the following list of the different attack vectors:
An attack executed from removable media (e.g., flash drive, CD) or a peripheral device.
An attack executed via an email message or attachment (e.g. malware infection).
An attack that employs brute force methods to compromise, degrade, or destroy systems, networks, or services.
Any incident resulting from violation of an organization’s acceptable usage policies by an authorized user, excluding the above categories.
An attack executed from a website or a web-based application (e.g. drive-by download).
The loss or theft of a computing device or media used by the organization, such as a laptop or smartphone.
An attack that does not fit into any of the other categories.
One of the biggest fallacies with traditional information security is the underlying assumption that you know which path an attacker will take through your network. For example, attackers rarely come through your front door, or in this context, your gateway firewall. But each attack does generally work through a certain pattern, or what Lockheed Martin has called the “cyber kill chain.”
The “cyber kill chain” is a sequence of stages required for an attacker to successfully infiltrate a network and exfiltrate data from it. Each stage demonstrates a specific goal along the attacker’s path. Designing your monitoring and response plan around the cyber kill chain model is an effective method because it focuses on how actual attacks happen.
1) Reconnaissance & Probing
2) Delivery & Attack
3) Exploitation & Installation
4) System Compromise
Which security events develop into the type of information security incident that requires my attention now? And… what do I do about it? To help categorize each incident type, align each one against the cyber kill chain to determine appropriate priority and incident response strategy. You can use this table as a start.
Port Scanning Activity* (pre‑incident)
Reconnaissance & Probing
Ignore most of these events UNLESS the source IP has a known bad reputation , and there are multiple events from this same IP in a small timeframe.
Bonus tip: AlienVault’s Open Threat Exchange® is a great way to check on an IP’s reputation score.
Delivery & Attack
Remediate any malware infections as quickly as possible before they progress. Scan the rest of your network for indicators of compromise associated with this outbreak (e.g. MD5 hashes).
Distributed Denial Of Service
Exploitation & Installation
Configure web servers to protect against HTTP and SYN flood requests. Coordinate with your ISP during an attack to block the source IPs.
Distributed Denial Of Service Diversion
Exploitation & Installation
Sometimes a DDoS is used to divert attention away from another more serious attack attempt. Increase monitoring & investigate all related activity, and work closely with your ISP or service provider.
Exploitation & Installation
Detect, monitor and investigate unauthorized access attempts – with priority on those that are mission-critical and/or contain sensitive data.
Identify the privileged user accounts for all domains, servers, apps, and critical devices. Ensure that monitoring is enabled for all systems, and for all system events, and also make sure it’s feeding your log monitoring infrastructure (your USM or SIEM tools).
Unauthorized Privilege Escalation
Exploitation & Installation
Configure your critical systems to record all privileged escalation events and set alarms for unauthorized privilege escalation attempts.
Backup all critical data and systems. Test, document, and update system recovery procedures. During a system compromise - capture evidence carefully, and document all recovery steps as well as all evidentiary data collected.
Advanced Persistent Threat Or Multistage Attack
Any one of the singular events that are listed here could actually be a part of the worst type of security incident imaginable… the dreaded APT. The important thing is to view each event through a larger context, one that incorporates the latest threat intelligence (see below for more on the need for threat intelligence).
Much of the incident responder’s job is spent eliminating irrelevant information and removing false positives. You’ll be constantly fine-tuning the radio of security monitoring to get to just the right signal.
Incident response is a discipline of continual improvement. As you see more and more events turn into incidents, you’ll discover new ways to categorize those incidents, as well as new ways to prevent them from ever happening in the first place.
Even if you’re sure that an attacker is getting no useful information back from their scanning, if they seem to be doing a detailed and comprehensive scan of your external systems, it is reasonable to interpret this as intent to follow-up the recon with attack attempts later on. If the scanning originates from a legitimate organization’s networks, then contacting their security team (if they have one) or network management personnel is usually the best approach.
As a last resort, if no contact details are apparent, try the contact details listed in the WHOIS information for the domain. The email address [email protected] is often a contact email for this kind of communication, but may not be available for smaller or younger organizations. BTW, blocking the source address may be counterproductive, and merely cause the attacker to use a different source address.
We’ve expressed the need to “concentrate on what you know” many times in this guide – much of the work that security monitoring discovers is mundane yet vital.
We often think of incident response as being detailed, meticulous forensic work, looking closely at one system at a time. However, the great majority of security monitoring work can be addressed through seeing a larger more holistic picture of the state of, and activity on, your infrastructure.
Understanding where, which, and how your systems are communicating with other systems, and the changes being made to them, can reveal attacks that other security controls cannot.
Threat intelligence allows you to move away from a focus on vulnerabilities, exploits and patches, and focus on the things that are actively causing damage to your company’s data confidentiality, integrity, and availability.
The first step is to understand as much as possible about your current computing environment. Some people refer to this as environmental awareness or situational awareness or even contextual awareness. We like to think of it as local threat intelligence.
Once you combine rich information about your own network with the latest global threat intelligence (specifics on attacker tools, techniques, and trends), you’ll achieve effective triage. You’ll put your immediate focus on the types of security incidents that matter vs. wasting your time on false positives or irrelevant noise.
Unless your infrastructure is entirely static and unchanging, new vulnerabilities and exposures are being created all the time.
Good IT and Security management processes will do its best to minimize these, but the security analyst still needs to be aware of them to place other things into context.
Unexpected configuration changes to systems can reveal when a hostile party has control of the system through valid credentials and methods.
Many configuration options are related to certain compliance standards – alerting (or reporting) on these is a far better way to manage them than waiting for them to be discovered during your next audit.
You can’t do security just by looking for attacks and exploits – you have to look into what’s happening on your network and know the systems you have deployed.
By understanding what is happening on your network (environmental awareness) and connecting it to information about known sources of malicious activity (Global Threat Intelligence), it becomes simple to get large-scale reports on active threats within your infrastructure. For example, instead of searching through massive lists of alerts from various security controls to determine possible exploits and attacks, and attempting to prioritize them based on asset value, we look at environmental awareness data that can be connected to the indicators of compromise associated with threat actors.
Today, computing resources are cheap and plentiful – attacks can come from anywhere – especially from compromised systems on otherwise legitimate remote networks. Attackers fight a constant battle trying to make it difficult to locate the systems that control their malware, while still allowing their malware to reach these systems to receive execution instructions.
See our next chapter – Chapter Four - Incident Response Tools for how-to instructions on uncovering more information on attack sources.
We were hunting down an active attack, endpoints had been compromised and they had migrated to using stolen credentials to access the network directly - without further use of the RAT (remote access tool) on the compromised system. As we watched our attackers access the network from multiple locations we began to build our profile of them - what hours were they active, could we determine a particular time zone they may be located in? How many people were acting together? Finally we realized something from the connection authentication information… The connections we saw from multiple remote locations were actually only from a single host - a cloud-provisioned host. As the host was brought online to stage attacks from it, it was being relocated to lower-load hardware clusters with an entirely different upstream connectivity. We noticed that the host had used many different IP addresses and physical connectivity sets (spanning three different countries), but it was still the same virtual machine instance the entire time.
We were well prepared to do identification of remote hosts on VPS -style co-location arrangements, but the global motility of hosts on cloud providers had temporarily thrown us for a loop - we realized that the game had once again changed right before our eyes.
In this case, we had an obvious advantage that we had pre-existing data points to correlate together that allowed us to uncover what was happening behind the curtain; but it would have been much better if we had the ability to easily identify this beforehand. And I realized that the network identification tools we have today are all built for a pre-cloud Internet - a world where IP addresses are tied to physical hosts in physical locations owned by identifiable registered organizations. Now anyone that remembers the Internet pre-1999 will remember the venerable Ident() protocol (rapidly made obsolete as it represented a security risk in the open untrusted Internet).
Yet couldn't a tokenized, anonymous version of this provide some measure of utility in a cloud-served public Internet? The ability to query Amazon's web services and know that all three EC2 instances currently attacking me are all operated by Tokenized Amazon Customer F8E993C?
Until an initiative to create something of this sort arises and gets implemented, I'll have to stick with more abstruse methods of remotely fingerprint cloud instances post facto. As cloud computing offers ever more copious amounts of utility computing: OS instances that can be launched, operated, and deleted in a matter of minutes, we on the defensive end of things need some way to keep up with the increasing complexity of attribution.
The increasing need for attribution techniques in incident response is not just some by-product of a Security Analyst wanting to play counter-intelligence agent. Attribution is vital for correlating and prioritizing the tidal wave of data we need to pour through to make informed response decisions. Being able to correlate two seemingly unrelated minor attack attempts on different parts of the infrastructure launched from two random hosts on the same multinational cloud computing provider can make a huge difference.
There is much work in progress on establishing reputation between cloud service providers and their customers - but the need to establish reputation information from cloud instances to the rest of the world is essential in the world of the Incident Responder.