Post-hoc explainability means applying explanation methods to an already-trained model to help people understand its behavior. It is particularly useful for models whose internal computations are difficult to interpret directly, such as deep neural networks and large ensembles. The explanation may describe one prediction, summarize behavior across a dataset, or show how predictions respond to changes in inputs.
The key distinction is between inherent transparency and explanations added afterward. A small decision tree can be transparent because a reviewer can follow its actual decision path. A simple linear model can be interpretable when its features, transformations, and coefficients are understandable. A complex neural network generally lacks that same direct transparency, but engineers can still investigate its outputs using post-hoc methods. An opaque model with an explanation attached has not become inherently transparent.
Local explanations address an individual prediction. For example, an engineer reviewing a loan-risk model might use SHAP values to estimate how income, outstanding debt, and payment history contributed to a particular score relative to a selected baseline. LIME instead fits an interpretable approximation around a prediction using perturbed inputs. Both approaches can help investigate model behavior, but their outputs depend on methodological choices and should not be interpreted as proof of causal relationships.
Global explanations summarize broader behavior. Engineers might aggregate feature attributions across a representative evaluation dataset or use partial dependence plots to investigate how average predictions change as a feature varies. A global summary can reveal that a model relies heavily on a particular input, but it can hide important differences between customers or operating conditions. Explaining one prediction also does not establish how the model behaves everywhere.
In a cloud deployment, explanation jobs can run against stored evaluation data, during batch inference, or alongside an online prediction endpoint. A practical workflow preserves the model version, preprocessing logic, input feature definitions, explanation configuration, and reference dataset. For example, a fraud-review application might display the features contributing most strongly to a flagged transaction, allowing an analyst to check whether the model is responding to plausible signals or to an accidental data artifact.
Explanation quality involves trade-offs. Model-agnostic methods can work across different model types, but may require many inference calls, increasing cost and latency. Model-specific techniques may be more efficient but depend on the model architecture. Correlated features can make attribution difficult to interpret, and unrealistic input perturbations can produce misleading results. Engineers should test explanation stability, use representative reference data, and check whether an explanation faithfully reflects the model behavior it claims to describe.
Explainability is not the same as accuracy, fairness, or compliance. A clear explanation can describe an incorrect or discriminatory prediction, and a high-performing model may remain difficult to explain. Likewise, a language model’s fluent account of why it answered a question is not reliable evidence of its internal computation. For certification questions, distinguish a model that is understandable by design from one whose behavior is interpreted afterward, and select an approach according to the audience, risk, and required level of evidence.