Centralized logging is essential to network security and compliance reporting. So, how does log management evolve as you migrate services and workloads to public cloud infrastructure?
Not to sound dramatic, but log data is the lifeblood of your security posture. The data captured in the logs of your network devices, systems, and applications feeds into your Security Information and Event Management (SIEM) solution, telling you who, what, when, where, and how an attack happened—or, better yet—how an attack is happening so that you can respond immediately.
As you migrate services and workloads to a public cloud infrastructure like Amazon AWS or Microsoft Azure, it’s important to know what log data is available to you, how to access it, and how to analyze all your log data to get a complete view of your security and threat posture.
In this blog, we’ll look at how centralized log management changes in the public cloud as well as the native logging and monitoring services and components provided by AWS and Azure. We’ll also look at how to centralize all your log data—cloud and on-premises—in a cloud-ready unified security management solution.
Logging in the Cloud – A Shared Responsibility
When you run your systems and applications and store your data on- premises—whether in a physical or virtual network environment—centralized logging is a relatively straightforward process. When you own your physical infrastructure, you get complete, top-to-bottom visibility of your IT stack. In this environment, you can readily access, aggregate, and send log data from all your network devices, systems, and applications to your SIEM or unified security management platform for security analysis and storage.
By contrast, when cloud service providers like Amazon and Microsoft own the network infrastructure and make it available to you as a service—whether as Infrastructure as a Service (IaaS) or Platform as a Service (PaaS)—you no longer have the same control over the underlying hardware, computing, and networking resources that support your cloud workloads. Instead, the cloud service provider takes responsibility for maintaining and securing the cloud infrastructure (basically, everything from the hypervisor down to the physical layer), while leaving you responsible to secure the guest operating systems, services, and applications running on top of the cloud infrastructure. This separation of security concerns is known as the Shared Responsibility Model—a main pillar of cloud and hybrid cloud operations and security.
While the shared responsibility model does put some limitations on your visibility and control of your cloud IT stack, it’s not a total black box. Cloud service providers expose log data from their services and APIs through native log collection and monitoring services, so you have visibility of your cloud environment, resources, and activities. You can leverage these services to collect and send log data to your SIEM environment, where it can be correlated along with other data sources, to get a complete picture of the security posture of your hybrid cloud environment.
The cloud logging and monitoring services provided in AWS and Azure include (but are not limited to):
Amazon AWS CloudWatch
CloudWatch is a logging and monitoring service for your AWS resources and the applications you run on AWS. You can use CloudWatch to retrieve log data from your AWS instances (EC2), Elastic Load Balancing (ELB), S3 storage, as well as logs from the applications you run on your EC2 instances.
Amazon AWS CloudTrail
AWS CloudTrail is a log monitoring service that records all the user (human or machine) activity within your AWS environment. CloudTrail tells you who is accessing what in your AWS account (for example, IP requests to your AWS web application firewall) and allows you to track any changes made to your AWS resources. This information can help you to identify suspicious user behavior patterns in your AWS environment and to enrich your SIEM security analysis.
Microsoft Azure Monitor
Azure Monitor (formerly Azure Insights) is a new service through which monitoring data of all activities and actions on your Azure subscription, along with metrics and diagnostics from your deployed resources (e.g. Windows and Linux servers, Azure services and applications), can be obtained. This monitoring data can be queried using the Azure Monitor REST API, Command Line Interface (CLI), PowerShell cmdlets or via the .NET SDK or streamed to other locations in real time.
Microsoft Azure Diagnostics
Azure Diagnostics is a capability within Azure that makes it possible to collect rich diagnostic data, including security event logs, from a number of Azure services and push it into an Azure Storage instance, where it can then be sent to or queried by your SIEM for analysis. Azure Diagnostics can collect multiple types of metrics and data including boot information, event logs, Windows and IIS events, and custom application logs.
Security & Analysis in the Cloud – Do I Still Need a SIEM?
It’s important to keep in mind that, under a shared responsibility model, you are responsible for much of the security of the operating systems, middleware, and applications that you deploy on the cloud infrastructure. You can leverage services like CloudWatch and CloudTrail to facilitate log data collection, but they will only report out the log data and metrics and are not substitutes for a comprehensive security management solution.
Even with the native logging and monitoring services available in AWS and Azure, you still need a way to correlate the log data from across your environments–including your public cloud, virtual private clouds (VPCs) as well as your on-premises physical and virtual deployments–to look for threats and malicious patterns of behavior. This is a job for a SIEM.
However, traditional SIEM solutions present challenges that amplify in cloud environments. Legacy SIEM products were not built natively for the cloud, but rather adapted for the cloud. The problem is that traditional attack strategies, vulnerabilities, and security resources change as you move from on-premises to cloud deployments, requiring you to shift your approach to security management in ways that legacy products are simply not optimized to do.
Another problem with traditional SIEM products is that they require you to aggregate and correlate log data from multiple sources and disparate security tools, including asset discovery and inventory, vulnerability assessment, and IDS. This creates a lot of complexity and work for IT security professionals, who are already strapped for time and resources. Without deep understanding of emerging cloud threats, researching, writing, and updating correlation rules to detect those threats is a time-consuming task.
AlienVault breaks through these challenges with unified security management solutions that accelerate and simplify threat detection, incident response, and compliance management for your on-premises, cloud, and hybrid cloud environments. AlienVault USM Anywhere combines SIEM and log management with essential asset discovery and inventory, vulnerability management, intrusion detection, and behavioral monitoring capabilities—whether for physical, virtualized, cloud, or hybrid cloud environments.
In addition, the AlienVault Labs Security Research Team delivers continuous correlation rules and threat intelligence backed by the Open Threat Exchange (OTX) to the USM environment, so users can get the essential hybrid cloud security they need, even if they don’t have the in-house expertise to constantly research and write correlation rules around emerging cloud security threats.
