Start free
CLOUD CONCEPT

SHAP

SHAP (SHapley Additive exPlanations) is a framework for explaining individual model predictions by assigning each input feature a contribution relative to a reference prediction.

Cloud Concepts Hub →

What is SHAP

SHAP applies Shapley values from cooperative game theory to distribute the difference between a model output and a baseline among input features. Engineers use these attributions to investigate predictions, identify unexpected model behavior, and summarize feature influence across a dataset without confusing explanations with accuracy or causality.

SHAP explains a prediction by treating input features as participants in a cooperative game whose outcome is the model output. A feature receives credit according to its marginal contribution across different combinations of available features. For a particular record, the explanation expresses the output as a baseline value plus feature contributions, subject to the explainer's approximation error. The baseline is typically an expected model output over a selected background dataset, not necessarily zero or the prediction for an all-zero input.

Consider a loan-default classifier explained on the probability scale. Its reference prediction is 0.10, and one applicant receives a predicted default probability of 0.30. A hypothetical explanation might assign +0.15 to debt-to-income ratio, +0.08 to recent missed payments, and -0.03 to employment history. These contributions sum to the 0.20 difference from the baseline. Positive values increase the selected output; negative values decrease it. Always verify the output being explained: values expressed in log-odds cannot be read directly as percentage-point changes in probability, and multiclass explanations must identify the target class.

In a cloud workflow, engineers typically store a representative background sample, run an explanation job against a versioned model, and retain the resulting attributions with the model version and explainer configuration. Batch explanations are useful for validation datasets and periodic investigations. Online explanations can accompany selected predictions, but they may require many additional model evaluations and increase latency and cost. Use sampling, asynchronous processing, or model-specific explainers when explanations do not need to be returned for every request.

KernelSHAP is a model-agnostic approach that estimates contributions through sampled feature combinations and a weighted regression. TreeSHAP exploits tree-model structure and can compute exact Shapley values efficiently for supported formulations. The appropriate implementation depends on the model family, the definition of missing features, and operational constraints. Exact general-purpose Shapley computation becomes expensive as the number of features grows, so approximation settings and explanation stability deserve explicit testing.

A local explanation describes one prediction. Aggregating absolute SHAP values over many records produces a global feature-importance summary, but removes direction and can hide subgroup differences. Inspect signed values and individual records as well. For example, a strong attribution to a customer identifier may reveal leakage or an unintended proxy, while different attribution patterns across customer segments can motivate further evaluation. Neither finding alone proves discrimination or establishes a causal relationship.

The background dataset and treatment of feature dependence materially affect the result. Correlated inputs can share attribution differently under different explainers, and replacing missing features independently can create unrealistic combinations. Version the reference data, feature transformations, output scale, and explanation settings so comparisons remain meaningful. Attribution drift can reveal changing model behavior, but it does not establish that predictive accuracy has declined; labeled performance measurements are still required.

For model-evaluation questions, distinguish explaining outputs from measuring predictive quality, calibration, fairness, or causal effects. SHAP supports debugging and review but does not certify any of these properties. It also does not automatically explain a foundation model's generated answer: applying attribution to text requires defining the output score, feature units, and masking strategy, and the resulting explanation is not a record of the model's reasoning.

Key aspects of SHAP

SHAP provides reference-dependent, additive feature attributions for local and aggregate analysis, with important computational and interpretation constraints.

Additive prediction decomposition

A local explanation decomposes a selected output into a baseline and signed feature contributions. Check the output scale and approximation residual before interpreting the numbers.

Shapley-value allocation

Contributions reflect marginal effects across feature coalitions rather than a single feature-removal test. Exact Shapley values satisfy allocation properties such as efficiency, symmetry, and assigning zero contribution to a feature that adds no value.

Reference-dependent interpretation

The background dataset defines the comparison context and influences the baseline and attributions. Select representative reference records and version them alongside the explanation configuration.

Local and global views

Individual explanations show why a specific output differs from its baseline. Aggregated absolute attributions summarize influence across records, but require additional analysis to understand direction, interactions, and subgroup behavior.

Computation and approximation trade-offs

Model-agnostic explainers can require many prediction calls, while model-specific algorithms can reduce computation. Benchmark explanation latency, cost, and stability before enabling explanations in a production request path.

Noncausal interpretation

SHAP describes how a model uses inputs under a defined attribution procedure; it does not prove what would happen after a real-world intervention. Correlated features, leakage, and proxy variables require separate investigation.