An epoch represents one complete pass through the training dataset. During ordinary mini-batch training, the dataset is divided into batches, and the model processes each batch to calculate a loss and gradients. After all batches have been processed, one epoch is complete. Training for multiple epochs exposes the model to the dataset repeatedly, usually with the examples shuffled between passes. This concept concerns model training, not Unix timestamps or other computing uses of the word epoch.
Epochs, batches, and steps measure different things. Batch size specifies how many examples are processed together. A training step commonly means one optimizer update, although some frameworks distinguish batch-processing steps from optimizer steps. Without gradient accumulation, and retaining the final partial batch, a dataset of 10,000 examples with a batch size of 100 produces 100 optimizer updates per epoch. Training for five epochs produces 500 updates. Increasing the batch size to 200 reduces this to 50 updates per epoch, so the same epoch count no longer implies the same number of updates.
For a finite dataset, the number of batches per epoch is approximately the dataset size divided by the batch size, rounded up when the final partial batch is retained. Dropping that batch changes the calculation. With distributed data-parallel training, calculate dataset coverage using the global batch size across workers, not just the batch size on one accelerator. Gradient accumulation allows several microbatches to contribute to one optimizer update, further separating batch-processing steps from optimizer steps.
Cloud training jobs commonly expose an epoch count through an algorithm parameter or a training-script argument. The framework or training script must actually consume that setting; naming a parameter epochs does not automatically control an arbitrary training program. Engineers also configure validation, logging, checkpoints, and runtime limits. Some frameworks allow a maximum step count to override the epoch target, so verify which stopping condition controls the job. For streaming or indefinitely repeated datasets, an epoch may instead be defined by a configured number of steps rather than a natural dataset boundary.
Too few epochs can leave a model underfitted: it has not learned enough of the available signal. Too many can waste compute and cause overfitting, where training performance improves while validation performance deteriorates. Neither outcome is determined by epoch count alone; learning rate, model capacity, regularization, and data quality also matter. Compare training and validation metrics, and use early stopping when supported to end training after validation improvement stalls. Save or restore the checkpoint with the best validation metric rather than assuming the final epoch produces the best model.
For example, a classifier might be configured for a maximum of 20 epochs but reach its best validation loss at epoch eight. An early-stopping rule with suitable patience can avoid completing every remaining epoch. Keep an independent test set for final evaluation instead of repeatedly using it to select the epoch count. Under otherwise similar conditions, additional epochs increase accelerator usage and cost, although initialization, validation, and checkpoint overhead prevent exact proportionality.
During foundation-model fine-tuning, the epoch count controls repeated exposure to the customization dataset, not repetition of the original pretraining process. Excessive repetition of a small dataset can encourage memorization and reduce broader task performance. Evaluate representative held-out prompts and adjust epochs together with learning rate and effective batch size. More epochs do not repair poor labels, missing examples, or an unsuitable objective, and equal epoch counts do not make experiments equivalent when dataset sizes or batch configurations differ.
Amazon SageMaker
Amazon CloudWatch