Note: The product previously mentioned in this blog, AlienVault USM for AWS, is no longer being sold. Learn more here.
If you are starting a project to increase your visibility in AWS it won’t be long before you reach for your trusty old network-based IDS. However, just like the rest of us, you will soon start tearing at your hair trying to figure out how to deploy it in the new world of AWS. In this post we will explore why we always reach for our network monitoring tools, we will look at what role they played in our traditional security efforts, and we will analyze the approaches to AWS intrusion detection and look for alternative paths to the same end.
Why Do We Need Intrusion Detection in AWS?
The first question to really ask ourselves is “why ids?” Why have we started every monitoring project in the last 15-20 years with some form of network-based intrusion detection? Well, when we are honest with ourselves the answer is actually quite simple - it is easy and there is pretty good return for our effort. In traditional environments the network gives us a common chokepoint for us to monitor everything (well almost everything) in our environment. The fundamental truth of the matter is if it is important and it is in our data center it talks on the network. This is what we look for in security - a product that allows us to see something about everything. This is great! Especially since the pesky Ops guys keep on plugging in new boxes and the dev guys insist on upgrading the software installed - being able to tap the network gives us a nice consistent place to ensure we got some visibility into what is going on.
What is it good for?
Now that we have some visibility, the real question is “in a comprehensive monitoring program what is it good for?” A network based intrusion detection system is good at one thing - looking for known behavioral patterns in network traffic. Using the context of a breach (as defined out by the Cyber Kill Chain) lets look at how you could leverage IDS.
| Malicious Activity | IDS Utility | Use-case |
| Reconnaissance & Probing | High* | Detect known scanning tools |
| Delivery & Attack | Moderate* |
Detect exploit kits |
| Exploit & Installation | - | None |
|
System Compromise |
Moderate* |
Detect command and control protocols, |
*For unencrypted connections
When we analyze the uses of intrusion detection we can now start mapping it’s capabilities to the threats we are looking to detect. This allows us to independently evaluate how we detect each of the malicious activities listed above in the environment we are trying to protect. There is always a best tool for the problem at hand, and when our environment changes we should evaluate what tool we are using for what end goal to truly evaluate the cost/benefit.
Approaches to AWS Intrusion Detection (IDS)
Now that we have an appreciation for the utility of IDS and have reminded ourselves why we reach for it, lets take a look at how we can use network IDS in AWS. The core problem we face in AWS is that there is no way to get complete access to the low-level network traffic. One of the major benefits of the AWS environment is that the network infrastructure is largely abstracted away from us. While there are still many ways to customize and configure our VPCs and Subnets in EC2 we don’t have to deal with a large part of the operational management that exists in our traditional data centers. The trade-off with this improved automation is that we no longer have an easy way to drop a passive tap onto the network or create a SPAN port on a core router. Essentially, we no longer have traditional shortcut to get some visibility across our network. What options do we now have??
| Approach | Description | Advantages | Issues |
| Network Tap or SPAN | Use a network tap or SPAN to access all network traffic | Not possible in AWS. | |
| Forward Deployed IDS | Deploy a network-based IDS on every instance you deploy | IDS workload scales with your infrastructure | You have introduced a CPU-intensive process onto every single machine. |
| Traffic Replication | Deploy an agent on every instance to capture & replicate traffic for centralized analysis | The actual workload of network traffic analysis is not performed on your instance |
Traffic capture and replication is still CPU-intensive (particularly on Windows machines.) You significantly increase the internal network traffic in your environment (every inbound packet is duplicated in the transfer from the instance that captures the traffic to the instance that analyszes the traffic.) |
| Inbound IDS Tier | Add another tier to the application architecture where a load balance sends all inbound traffic to a tier of instances that performs the network analysis | IDS workload is now isolated to a horizontally scalable tier in your architecture | You have to maintain and manage another mission-critical elastic tier in your architecture. |
With these approaches laid out we can now sit back and appreciate the problem in its fullest.
Let …
it…
sink …
in….
Deploying IDS in AWS stinks!
There really isn’t an approach that doesn’t have significant impact on your architecture or introduce an additional workload into the instances you are running.
Alternatives to Network-based IDS in AWS
Now, when we run into a problem like this in AWS it is always good to ask ourselves “Is this because we are doing things in an outdated way? Are we applying an on-premise approach to a cloud environment?” Now the answer to that question is not completely yes or no, but lets go back to the use cases for intrusion that we outlined above - is there an alternative approach that is possible in AWS?
| Stage | Use case | AWS Alternative |
| Reconnaissance & Probing | Detecting known scanning tools | Capturing logs from public-facing series such as ELB access logs or the logs of your we server / proxy / load balancer. |
| Delivery & Attack |
Detecting exploit kits Detecting known exploit signatures |
Exploit kits are associated with end user behavior so not largely applicable to AWS. The attack surface for known exploit signatures is drastically reduced in your operational infrastructure, making it far more feasible to automate the collection of data from publicly exposed tiers. |
| Exploit & Installation | None | OS-level monitoring has never been easier in AWS with the prevalence of configuration management toolks like OpsWorks, Puppet and Chef. This allows for easy collection of system-level data. |
| System Compromise | Detecting command & control protocols and lateral pivots | Same as above. |
Beyond simply providing viable alternatives for the core IDS use cases, AWS also presents a new opportunity for us to improve the monitoring we have done in the past. When we consider the environments we are looking to monitor in AWS there is a new chokepoint (similar to the network layer in the past) that we can take advantage of - that is the management plane. Like never before we now have full visibility into every operation that is going on in our ‘data center’, we now have ways to easily hook into every server deployed and greatly simplified mechanisms for provisioning and configuring software. This is the core of the AWS API, it is the core of the promise of AWS and it is the best thing that has happened to security in a long time.
We can easily ensure that every server spun up has proper monitoring enabled and data flowing into our systems. We can easily analyze the complete history of every action taken with complete traceability back to the source by analyzing the AWS API audit log (CloudTrail). This gives us a new mechanism for detecting threats that we are only starting to really take advantage of. Using this new information we can now quickly answer questions and perform tasks that were only wishes in our past lives.
- Which employee has accessed my environment? Was it really them?
- Are all of my machines sending me their operational logs for analysis?
- I am about to spin up a new machine can I automatically provision it into my security monitoring system?
- Has the CPU spiked on any of my machines in the last hour?
- Who set up this server?
The benefit of a common management plane and the power of the provisioning tools (cloud-init, cloudformation) provided by AWS give us a new way to hit the easy button.
Look out for my next blog, where I'll dive into PCI compliance and AWS intrusion detection.
In the meantime, please check out USM Anywhere - this new product from AlienVault provides the ability to automate the monitoring discussed in this blog post - providing an easy way to embrace the new reality of security monitoring in the cloud.
