Code Coverage

Code coverage is a measurement of code execution during testing. A coverage tool instruments application code or collects runtime execution data, then maps that data back to the source. The resulting report highlights exercised and unexercised code and calculates percentages for particular coverage metrics. Engineers use these results to identify missing tests, review changes, and enforce testing standards in continuous integration and continuous delivery pipelines.

Different metrics reveal different gaps. Line coverage measures executed source lines, statement coverage measures executed statements, and function coverage measures invoked functions. Branch coverage measures whether the alternative outcomes of decisions have been exercised. A test that executes an if statement might cover its line without testing both its true and false outcomes. Consequently, a high line coverage percentage can coexist with untested error handling or decision paths.

A typical cloud pipeline checks out a specific source revision, installs pinned dependencies, runs unit tests with coverage enabled, evaluates coverage requirements, and publishes reports. For example, a Python build can run pytest --cov=src --cov-report=xml:coverage.xml --cov-fail-under=85. With pytest-cov installed, this command measures the specified application source, writes an XML report, and returns a failing exit status when total coverage falls below 85 percent. Test failures also cause the test command to fail. The build configuration must preserve these failures rather than masking them with shell commands that always return success.

In an AWS implementation, AWS CodeBuild can execute the tests and collect supported coverage reports through the reports section of buildspec.yml. Test-result reports and code coverage reports serve different purposes: the former describe test outcomes, while the latter describe exercised code. Publishing either report does not automatically enforce a coverage threshold. Configure the test runner, coverage tool, or a separate validation command to fail the build when the coverage policy is violated. AWS CodePipeline can then prevent downstream deployment actions from proceeding when the associated build action fails.

Thresholds should reflect application risk and the existing codebase. A team might require an overall minimum while also requiring stronger coverage for changed code, avoiding a situation where a large, well-tested module hides an untested new feature. For a payment calculation, tests should exercise valid amounts, rounding boundaries, invalid input, and failure behavior rather than merely calling the calculation once. Review uncovered branches alongside requirements and defect history instead of treating an arbitrary percentage as the sole release criterion.

Coverage has important limitations. Executing a line does not demonstrate that a test checks its result, and 100 percent coverage does not prove correctness, security, or complete input coverage. Unit tests that mock a cloud SDK can thoroughly exercise application logic without validating IAM permissions, network access, or actual service behavior. Integration and end-to-end tests remain necessary for those concerns. Mutation testing can provide additional evidence about assertion effectiveness, although it increases execution cost.

Reliable measurement also requires consistent configuration. Include application files that tests never import, document justified exclusions such as generated code, and use compatible source mappings for transpiled applications. Parallel test workers should produce mergeable coverage data for the same source revision; simply averaging their percentages is not a valid substitute for merging execution records. Instrumentation adds runtime and storage overhead, so teams commonly run fast unit coverage checks on each change and schedule more expensive suites separately. Retain reports with revision metadata so engineers can investigate regressions and compare results meaningfully.