Anomaly Detection

Anomaly detection asks whether an observation is unusual for its context, rather than simply whether it exceeds a fixed limit. A sudden increase in failed authentication attempts may be suspicious, but the same count can have different meanings for a small administrative application and a large public service. Useful detection therefore considers the identity, resource, account, time period, and normal workload associated with an event.

Detection methods include statistical comparisons, machine learning models, and deviations from explicitly defined expected behavior. A baseline might describe an application's normal outbound traffic, an identity's usual API activity, or the daily pattern of access-denied errors. Seasonal models can distinguish an expected weekday peak from an unexpected overnight surge. Not all anomaly detection requires machine learning, and not every security finding represents an anomaly; some findings identify known malicious indicators or violations of explicit policies.

In the cloud, begin with reliable telemetry. Collect relevant audit events, network flow information, DNS activity, application logs, and resource metrics. Preserve identity, account, Region, resource, and timestamp information so investigators can correlate observations. For an organization, establish consistent collection and detection coverage across accounts and Regions, and monitor whether those controls remain enabled. Missing telemetry creates blind spots, and missing data must not automatically be interpreted as normal behavior.

Translate important security signals into metrics and actionable alerts. For example, derive a metric from access-denied audit events and monitor its rate against an expected range. A dashboard can show the observed metric alongside its baseline, related findings, and recent deployments. Combine anomaly-based alarms with fixed rules for events that are always significant, such as unauthorized changes to logging controls. A static threshold can be appropriate even when no historical baseline exists.

For AWS metric monitoring, Amazon CloudWatch can model expected metric behavior and evaluate alarms against an anomaly detection band. Amazon GuardDuty applies threat-detection techniques, including behavioral analysis, to supported data sources and produces security findings. Amazon Security Lake centralizes normalized security data for analysis by subscribers; it is not itself an anomaly-detection engine. AWS Security Hub aggregates supported findings and provides centralized security visibility, while Amazon Macie identifies sensitive data and S3 security concerns rather than serving as a general user-behavior detector.

Alert handling should consider severity, confidence, affected resource sensitivity, and corroborating evidence. An unusual API call from a new location deserves more attention when accompanied by privilege changes or unexpected data access. Route relevant findings to an incident workflow, assign ownership, and provide links to supporting evidence. Use approval or additional validation before disruptive automated actions such as disabling an identity, because legitimate operational changes can resemble attacks.

The main trade-off is sensitivity versus noise. Narrow expected ranges may generate false positives during releases, migrations, or traffic shifts; wide ranges can miss subtle attacks. New workloads need representative history, and baselines can drift as behavior changes. Exclude known exceptional periods from model training where supported, but avoid broad suppressions that hide future incidents. Validate detections with controlled tests, review missed incidents and unnecessary alerts, and track detection latency and investigation outcomes. Anomaly detection complements deterministic controls and threat intelligence; it does not replace them or guarantee that malicious activity will look unusual.