Catastrophic Forgetting

Catastrophic forgetting occurs when learning new information substantially degrades a model's ability to perform previously learned tasks. Neural networks often share parameters across many capabilities, so updates that optimize a new training objective can interfere with representations needed for earlier tasks. The problem is especially important in sequential training, continual learning, and foundation-model fine-tuning, where training data may no longer represent the breadth of the original model's experience.

For example, an engineer might fine-tune a general-purpose language model using only technical-support conversations. The resulting model could become better at troubleshooting a particular product while becoming worse at summarization, general question answering, or following instructions outside that domain. Similarly, an image classifier updated exclusively with examples of newly introduced products might lose accuracy on older product categories. Successful training on the latest dataset does not establish that earlier capabilities were preserved.

In a cloud training workflow, establish a baseline before updating the model. Maintain separate, held-out evaluation sets for the new task and important existing tasks, then compare both against the original checkpoint. Use task-specific metrics and explicit regression thresholds rather than relying only on an aggregate score. Evaluate checkpoints during training and select one that achieves acceptable adaptation without exceeding the permitted loss in retained capabilities. Preserve model artifacts, dataset versions, and training configurations so that results are reproducible and rollback is possible.

Replay, also called rehearsal, addresses forgetting by mixing representative examples from earlier tasks with new training examples. The sampling ratio matters: too little historical data may not protect old capabilities, while too much can slow adaptation and increase training cost. Historical examples must also satisfy privacy, retention, and licensing requirements. When original data is unavailable, carefully validated synthetic examples or knowledge distillation from a frozen earlier model can provide alternative preservation signals, although neither guarantees complete retention.

Other approaches constrain how the model changes. Elastic weight consolidation penalizes changes to parameters estimated to be important for earlier tasks. Freezing layers or using parameter-efficient fine-tuning methods such as adapters or low-rank adaptation limits which parameters are trained. Keeping base weights unchanged preserves the original checkpoint, but an active adapter can still alter behavior enough to degrade older tasks. Lower learning rates, fewer training steps, and early stopping based on both new-task and retention metrics can reduce harmful updates; they are not guarantees against forgetting.

Catastrophic forgetting is distinct from overfitting and underfitting. Overfitting means learning training-specific patterns that generalize poorly, while underfitting means failing to capture relevant patterns adequately. Forgetting concerns the loss of earlier capabilities after additional training, even when the new task generalizes well. These problems can coexist, so dropout or ordinary weight decay alone should not be treated as a complete forgetting-prevention strategy. If the requirement is simply to supply changing reference information, retrieval-augmented generation can avoid weight updates for that information, although retrieval quality and answer correctness still require evaluation.