AWS File Integrity Monitoring and S3 Data Integrity

February 3, 2019 | AT&T Alien Labs

Get the latest security news in your inbox.

Subscribe via Email

No thanks. Close this now.

Today, 76% of organizations have adopted or are planning to adopt cloud services, including cloud storage. Amazon Web Services (AWS) is the public cloud market leader with 40% cloud market share. Its Simple Storage Service (Amazon S3) is one of its most popular services, used by nearly 195,000 unique domains.

S3 data integrity - what you need to know

Amazon S3 is probably one of the most popular services, especially among those companies that are already leveraging technologies from AWS.

By default, when you create an Amazon S3 bucket, the bucket will be private and only accessible by users or credentials belonging to that user account . To control who has access to data stored within the S3 bucket, users can apply an Access Control List (ACL) to the entire bucket, or different ACLs to specific objects stored within the bucket or the bucket itself.

Every S3 bucket has a unique name, for example, “: myinsecurebucket”. And, buckets are always private by default. Since the bucket is private, even if an attacker can guess the name of the bucket, it won’t be able to list or retrieve files from it:

$ aws s3 ls s3://myinsecurebucket

A client error (AccessDenied) occurred when calling the ListObjects operation: Access Denied

On the other hand, if the bucket is not properly configured, like if it is mistakenly configured as a public bucket, an attacker will be able to view and access the files in the bucket:

$ aws s3 ls s3://myinsecurebucket

2017-06-20 10:27:11 910 myfile

How often does this happen? Unfortunately, this is a fairly common error that can expose sensitive information, as evidenced in the following two examples:

  • A few weeks ago, a security researcher found an S3 bucket exposing highly sensitive US military data [no longer available] due to a US defense contractor making it publicly accessible.

  • While I was working on this blog post, another data breach was discovered that exposed the personal information of about 198 million American voters stored in an unsecured S3 bucket.

We have also found many instances of this same issue when searching for bug bounty issues and vulnerability disclosures. Here are some examples from HackerOne:

Affected CompanyLink






If you want to learn more about other issues regarding S3 permissions I recommend visiting the flAWS challenge.

This simple configuration issue in Amazon is just one example of the many kinds of misconfigurations that are easy to do and common when using AWS and other cloud infrastructure services like Microsoft Azure.

One the reasons why AlienVault created USM Anywhere was to help customers with threat detection and incident response in the cloud and on-premises environments. Once you deploy an USM Anywhere sensor in Amazon AWS, it will immediately start consuming CloudTrail data and provide you with visibility into AWS configuration issues and potential threats that you are likely completely unaware of today.

Using USM Anywhere to Monitor Your AWS S3 Buckets

When it comes to S3, USM Anywhere contains out of the box dashboards where you can quickly assess which S3 buckets/files are being accessed, by whom, and identify possible security or compliance issues.

In addition to that, we provide out of the box correlation rules that provide real-time alerts on many of the security issues that indicate a potential threat to your AWS services.

For example, if you create a public bucket or modify the ACL by mistake, making an S3 bucket public, USM Anywhere generates an alert:

By default, Amazon CloudTrail only logs certain type of events from the S3 service, such as bucket creation and deletion. To get more visibility into S3 buckets and files, including object creation, modification and access, you need to configure specific CloudTrail settings for each bucket that you want to monitor.

To do this, go to the CloudTrail configuration and under the “Trails” section, select the trail that USM Anywhere is monitoring.

Search for the “Data Events” section and click on the edit icon:

For each of the buckets you want to monitor, add the bucket name and whether you want to monitor Read/Write or all operations:

Once this option is setup, USM Anywhere can leverage that information to provide you with File Integrity Monitoring (FIM) capabilities for your AWS environment.

In addition, other out-of-the-box correlation rules can alert you on issues such as adding a file with an ACL that makes it publicly available:

Once configured, USM Anywhere is well-positioned to help you monitor your S3 buckets and to identify potential threats that can lead to a data leak or compromise. Use USM Anywhere to investigate issues and more quickly close the gaps to keep your data safe.

Bonus Tip: Create a HoneyFile

In case you are not yet convinced that USM Anywhere can be a valuable tool, I’m going to walk you through another cool use case. Think of this as a blog bonus that I highly encourage EVERYONE to do, regardless of whether or not you are using USM Anywhere.

The idea behind this exercise is to create a “HoneyFile” or a fake file that we know shouldn’t be accessed by anyone. We will leave this file in an S3 bucket, either public or private, depending on what activity you are trying to detect. My recommendation is to create files in a number of private buckets. Once we deploy these fake files, we will configure USM Anywhere to send us an e-mail and/or generate an alert when that file is accessed. This technique can help detecting attackers early in the kill chain.

This will create the data for you to look at. If you are also using USM Anywhere, you can create a rule to monitor the activity and generate an alarm.

The easiest way to create the rule is to access the file in the S3 bucket, and then search for the corresponding that event in the USM Anywhere interface. You can search for the filename, in our case “honey” and find a “GetObject” event that indicates the file has been accessed:

Click on “Create Rule” and setup the following conditions:

We will select the Data Source, the Event Name and the File Name fields. Now select the Action option, “Create an Alarm” and fill the Intent, Strategy, Method and Priority values. You can also select the fields that will be included as part of the metadata of the alarm:

Click save and from now on, if someone accesses that file, USM Anywhere will generate an alert like the following one:

anomalous user behavior

You could also select a different action such as Send a Notification->Email and receive an email every time someone accesses that file:

Cloud data leaks are a top concern of organizations that use cloud technology. USM Anywhere is built to natively support security monitoring for your AWS environment, providing threat detection and incident response through API-level integration with AWS CloudTrail as well as activity monitoring across your S3 and Elastic Load Balancer (ELB) resources.

Monitoring for threats in AWS is tricky. There aren’t many solutions on the market that give you the ability to effectively monitor for threats to your cloud infrastructure. USM Anywhere is one of those solutions. It delivers five essential security capabilities in a unified SaaS solution, giving you everything you need to keep your business secure in a single pane of glass. It combines asset discovery, vulnerability management, intrusion detection, SIEM, and behavioral monitoring in one affordable and easy-to-use solution. In the case of your AWS S3 instances, deploying USM Anywhere and implementing the above steps can mean the difference between detecting an insecure S3 bucket, and risking a data breach due to a simple misconfiguration..

AT&T Alien Labs

About the Author: AT&T Alien Labs
Read more posts from AT&T Alien Labs ›


Watch a Demo ›
Get Price Free Trial