Regular Expressions

A regular expression represents a pattern rather than one fixed string. Literal characters match themselves, character classes such as [0-9] match a range, and quantifiers control repetition. For example, requestId=[A-Za-z0-9-]+ matches a request identifier containing letters, digits, or hyphens. Anchors such as ^ and $ constrain matches to the beginning or end of the evaluated text. Without anchors, many search functions accept a matching substring rather than requiring the entire message to match.

Cloud engineers use these patterns to investigate application failures, identify suspicious requests, extract latency values, and select logs for downstream processing. A useful pattern should reflect the actual log format. For example, status=5[0-9]{2} identifies a status field beginning with a three-digit server-error code, but could also match the beginning of a longer number unless an appropriate delimiter or boundary is required. Test representative successes, failures, missing fields, and malformed messages before relying on a pattern for alerting.

In Amazon CloudWatch Logs, distinguish filter patterns from CloudWatch Logs Insights queries. Filter patterns support searching log events and configuring metric filters and subscription filters. They include syntax for unstructured text, JSON, and space-delimited records; regular expressions are only one part of that language. Supported regex expressions are enclosed in percent signs, such as %ERROR|Timeout%. This regex dialect is restricted: parentheses and multibyte characters are not supported, and quotas limit regex use within filter patterns and log groups. A pattern copied from an application regex library might therefore be rejected.

CloudWatch Logs Insights uses a separate query language. For example, fields @timestamp, @message | filter @message like /ERROR|Timeout/ | sort @timestamp desc | limit 50 searches stored events for either term. Matching is case-sensitive by default; supported inline flags such as (?i) allow case-insensitive matching. The parse command can extract values using regular expressions: parse @message /latency=(?<latencyMs>[0-9]+)/ | stats avg(latencyMs) by bin(5m) extracts a numeric latency field and calculates five-minute averages. The parse command also supports glob patterns, which are not regular expressions.

Prefer structured fields when they already exist. A Logs Insights condition such as filter status >= 500 and status < 600 is clearer than searching serialized JSON with a regex, provided status is a numeric field. Similarly, a JSON metric-filter pattern such as { $.status >= 500 } uses structured comparison syntax, not regex syntax. Metric filters generate metrics from newly ingested matching events; they do not retroactively calculate metrics from historical logs. Subscription filters select incoming events for delivery without changing the source log group's retention policy.

Regex precision affects correctness and operational cost. Broad expressions can produce false positives, while overly strict expressions can silently miss changed log formats. Greedy repetition can capture more text than intended, and some backtracking engines can consume excessive CPU on pathological patterns; engine implementations differ. Restrict query time ranges and log groups to control scanned data. A regex filter should not be assumed to use a field index or reduce scan charges, and a result limit does not guarantee that only that many events are scanned.

Finally, distinguish regex from wildcard matching and surrounding configuration syntax. In regex, a dot generally matches any character and an asterisk repeats the preceding element; a standalone asterisk does not mean any string as it often does in a shell glob. Escaping may also occur at multiple layers when a pattern is embedded in JSON, a programming-language string, or infrastructure configuration. Validate the final pattern in the target service rather than assuming that success in a different regex tester guarantees compatibility.