AlienVault® USM Anywhere™

Collecting Amazon CloudWatch Logs

Role Availability Read-Only Analyst   Manager

Amazon CloudWatch Logs monitors applications and systems using log data, aggregating and storing application logs. Amazon CloudWatch Logs is useful because you can easily configure it to process additional metadata with the log files.

If not already done, install and configure the Amazon CloudWatch agent to collect logs from Amazon Elastic Compute Cloud (EC2) instances. See Amazon documentation for more information.

For non-syslog data, all of the streams in the same group must be of the same type so that USM Anywhere can use a designated plugin to parse the collected raw log data. You can then set up a log collection job for each group using the correct plugin for the log data in that group.

Important: Before you create a new Amazon CloudWatch log collection job, you must have enabled Amazon CloudWatch Logs in your AWS environment.

If you choose to enable Amazon CloudWatch Logs in your AWS environment, you should make sure that you are not collecting more data than you need because this service incurs AWS costs based upon usage. See the pricing information to plan and configure your usage.

To create a new Amazon CloudWatch log collection job

  1. Go to Settings > Scheduler.
  2. In the left navigation list, click Log Collection.

    Note: You can use the Sensor filter at the top of the list to review the available log collection jobs on your AWS sensorGoogle Cloud Platform (GCP) Sensor.

  3. Click Create Log Collection Job.

    Click Create Log Collection Job to add a scheduled log collection job

    Click Create Log Collection Job to add a scheduled log collection job

    Note: If you recently deployed a new USM Anywhere Sensor, it can take 10 to 20 minutes for USM Anywhere to discover the various log sources. After it discovers the logs, you must manually enable the AWS GCP log collection jobs you want before the system collects the log data.

  4. Enter the name and description for the job.

    The description is optional, but it is a best practice to provide this information so that others can easily understand what it does.

  5. In the Select App option, select Amazon Web Services.
  6. In the App Action option, select Monitor CloudWatch.

    Select the AWS sensor, the Amazon Web Service app, and the Monitor CloudWatch action

  7. Enter the Region Name, Group Name, and Stream Name information for your AWS account. Region name can be an asterisk ( * ) to monitor all regions for a given group.

  8. In Source Format, select either of the following log formats:

    • syslog: All messages transmitted to USM Anywhere are processed with the assumption that they are syslog formatted.

      When you choose syslog as the source format, the plugin selection is bypassed and USM Anywhere uses the auto-detect hints from the plugins to match the incoming messages to the correct plugin.

    • raw: Use for non-syslog formatted data.

      If you select this option, you must choose the plugin that USM Anywhere will use to parse all of the streams in the group. For example, to collect Amazon Virtual Private Cloud (VPC) flow logs, select the VPC Flow Logs plugin.

      Specify the region name, group name, and source format for collecting the CloudWatch logs

      Important: If a group contains streams of mixed log formats, USM Anywhere parses all of them with the plugin that you chose, which will produce undesired results. In this case, you need to configure CloudWatch to separate the streams into groups so that each contains only a single log type that can be mapped to the correct plugin.

  9. In the Schedule section, specify when USM Anywhere runs the job:

    1. Select the increment as Hour, Day, Week, Month, or Year.
    2. Set the interval options for the increment. The selected increment determines the available options.

      For example, on a weekly increment you can select the days of the week to run the job.

      Set the schedule for the job to run each week

      Or, on a monthly increment you can specify a date or a day of the week that occurs within the month.

      Set the schedule for the job to run each month

    3. Set the Start time.

      This is the time that the job starts at the specified interval. It uses the time zone configured for your USM Anywhere instance (default is UTC).

  10. Click Save.