AlienVault® USM Anywhere™

Collecting Amazon CloudWatch Logs

Role Availability Read-Only Analyst   Manager

Amazon CloudWatch Logs monitors applications and systems using log data, aggregating and storing application logs. CloudWatch Logs is useful because you can easily configure it to process additional metadata with the log files.

Important: If you choose to enable CloudWatch Logs in your Amazon Web Services (AWS) environment, you should make sure that you are not collecting more data than you need because this service incurs AWS costs based upon usage. See the Amazon CloudWatch pricing information to plan and configure your usage.

If not already done, install and configure the Amazon CloudWatch agent to collect logs from Amazon Elastic Compute Cloud (EC2) instances. See Amazon documentation for instructions.

USM Anywhere provides some CloudWatch log collection jobs out of the box, but they are disabled by default. You can enable them under Settings > Scheduler. When enabled, these jobs monitor certain log groups and collect logs from CloudWatch every five minutes. You must configure your CloudWatch agent to use these log group names and to keep the log types the same within a given log group.

USM Anywhere Log Collection Jobs and CloudWatch Log Groups
USM Anywhere Log Collection Job Name CloudWatch Log Group Name

Default File Path

Date Format

CloudWatch - Apache-Access-Logs Apache-Access-Logs /var/log/apache2/access.log %d/%b/%Y:%H:%M:%S
CloudWatch - Linux-Audit-Logs Linux-Audit-Logs /var/log/audit/audit.log Use the default
CloudWatch - Linux-Auth-Logs Linux-Auth-Logs /var/log/auth.log %b %d %H:%M:%S
CloudWatch - Osquery-Logs OSQuery-Logs /var/log/osquery/osqueryd.results.log Use the default

If you want to collect logs from other log groups, ensure that all streams in the same group are of the same type so that USM Anywhere can use a designated plugin to parse the collected raw log data. You can then set up a log collection job for each log group.

To create a new CloudWatch log collection job

  1. Go to Settings > Scheduler.
  2. In the left navigation list, click Log Collection.

    Note: You can use the Sensor filter at the top of the list to review the available log collection jobs on your AWS Sensor.

  3. Click Create Log Collection Job.

    Click Create Log Collection Job to add a scheduled log collection job

    Note: If you have recently deployed a new USM Anywhere Sensor, it can take 10 to 20 minutes for USM Anywhere to discover the various log sources. After it discovers the logs, you must manually enable the AWS log collection jobs you want before the system collects the log data.

  4. Enter the name and description for the job.

    The description is optional, but it is a best practice to provide this information so that others can easily understand what it does.

  5. In the Select App option, select Amazon Web Services.
  6. In the App Action option, select Monitor CloudWatch.

    Select the AWS sensor, the Amazon Web Service app, and the Monitor CloudWatch action

  7. Enter the Region Name, Group Name, and Stream Name information for your AWS account. Region name can be an asterisk ( * ) to monitor all regions for a given group.

  8. In Source Format, select either of the following log formats:

    • syslog: All messages transmitted to USM Anywhere are processed with the assumption that they are syslog formatted.

      When you choose syslog as the source format, the plugin selection is bypassed and USM Anywhere uses the auto-detect hints from the plugins to match the incoming messages to the correct plugin.

    • raw: Use for non-syslog formatted data.

      If you select this option, you must choose the plugin that USM Anywhere will use to parse all of the streams in the group. For example, to collect Amazon Virtual Private Cloud (VPC) flow logs, select the VPC Flow Logs plugin.

      Specify the region name, group name, and source format for collecting the CloudWatch logs

      Important: If a group contains streams of mixed log formats, USM Anywhere parses all of them with the plugin that you chose, which produces undesired results. In this case, you need to configure CloudWatch to separate the streams into different groups so that each contains only a single log type that can be mapped to the correct plugin.

  9. In the Schedule section, specify when USM Anywhere runs the job:

    1. Select the increment as Hour, Day, Week, Month, or Year.
    2. Set the interval options for the increment. The selected increment determines the available options.

      For example, on a weekly increment you can select the days of the week to run the job.

      Set the schedule for the job to run each week

      Or, on a monthly increment you can specify a date or a day of the week that occurs within the month.

      Set the schedule for the job to run each month

    3. Set the Start time.

      This is the time that the job starts at the specified interval. It uses the time zone configured for your USM Anywhere instance (default is UTC).

  10. Click Save.