These three concepts describe how data flows through a machine learning model during training. Together, they control how and how often the model updates its parameters.
Epoch — One full pass through the entire training dataset
Batch — A small subset of the dataset processed together at one time
Iteration — One parameter update (one batch processed)
Training data is divided into batches
Batches are processed one at a time
When all batches are processed, one epoch is complete
The cycle repeats for many epochs
An epoch is one complete pass through the entire training dataset.
Training dataset: 1,000 examples
Batch size: 100
Batches per epoch: 10
One epoch — All 10 batches processed — model has seen all 1,000 examples once
Epoch 1 — Accuracy: 52%
Epoch 5 — Accuracy: 74%
Epoch 10 — Accuracy: 85%
Epoch 20 — Accuracy: 92%
A single pass is rarely enough — the model needs many exposures to learn stable patterns.
Epoch 2: Training 61% | Validation 60%
The model has not had enough training time to learn meaningful patterns.
Epoch 99: Training 99% | Validation 72%
The model has memorized training examples rather than generalizing.
A batch is a small subset of training data processed together before the model updates its parameters.
Dataset: 10,000 examples
Batch size: 100
Batches per epoch: 100
Common sizes — 16, 32, 64, 128, 256 (powers of 2 for GPU efficiency)
Batch = entire dataset (e.g., 1,000 examples)
Very stable updates. May be too slow for large datasets.
Batch = small subset (e.g., 100 examples)
Most commonly used in practice — balances stability and efficiency
Batch = 1 example at a time
Very noisy but can help escape local minima. Updates parameters 1,000 times per epoch if dataset has 1,000 examples.
An iteration is one parameter update — one time the model processes a batch and adjusts its weights.
Iterations per epoch = Dataset size / Batch size
Example — 2,000 examples / 100 batch size = 20 iterations per epoch
Total iterations — 20 × 10 epochs = 200 total iterations
Dataset: 1,050 examples | Batch size: 100
Result: 10 full batches of 100 + 1 partial batch of 50
| Concept | Definition | Example |
|---|---|---|
| Epoch | One full pass through all training data | 10 epochs = data seen 10 times |
| Batch | Subset of data processed together | Batch size 100 from 1,000 = 10 batches |
| Iteration | One parameter update (one batch) | 10 batches per epoch = 10 iterations per epoch |
Iteration 100 — Loss: 2.5 (many errors)
Iteration 500 — Loss: 1.2 (improving)
Iteration 1000 — Loss: 0.4 (nearly converged)
Training loss decreasing
Validation loss also decreasing
Both losses converge to a low value
Training loss continues to decrease
Validation loss stops decreasing and rises
Growing gap indicates memorization
Epoch 5 — Validation loss: 0.50
Epoch 10 — Validation loss: 0.30 ← Best model saved here
Epoch 15 — Validation loss: 0.38 ← Getting worse
Stop training and use the Epoch 10 model
Dataset: 1,000 images | Batch size: 100 | Iterations per epoch: 10
50 epochs → 500 total iterations → Accuracy: 45% → 93%
Dataset: 5,000 emails | Batch size: 50 | Epochs: 8 | Total iterations: 800
Validation F1 score: 0.62 → 0.91
Dataset: 2,400 records | Batch size: 120 | Epochs: 15 | Total iterations: 300
MAE decreases from 15.2 to 3.8
Too few — Both training and validation accuracy low (underfitting)
Too many — Training high, validation drops (overfitting)
Best approach — Use early stopping with validation loss monitoring
Ask the AI if you need help understanding epochs, batches, iterations, how to choose batch sizes, early stopping, or how training progresses over time.