Overfitting occurs when the model learns training examples too closely, memorizing patterns instead of generalizing. The result: excellent training performance, poor performance on new inputs.
Training loss decreasing while validation loss rises. Model reproducing exact phrases from training data. Handles familiar phrasings well, fails on paraphrased versions. Responses converging on a small set of rote answers.
A related problem: fine-tuning overwrites pre-trained capabilities, making the model less capable at general reasoning or tasks outside the fine-tuning domain. Most severe with small datasets, high learning rates, many epochs, and full fine-tuning.
PEFT methods (LoRA, QLoRA) inherently protect against catastrophic forgetting because most weights are frozen — only the small adapter matrices are updated.
Validation set + early stopping — Hold out 10–20% of data. Monitor validation loss. Stop when it stops improving — do not train to convergence on training loss.
Lower learning rate — Fine-tuning learning rates should be 1e-5 to 5e-5. Smaller steps reduce the risk of overwriting pre-trained knowledge.
Fewer epochs — 1–3 epochs is often sufficient for behavioral SFT. More passes over the same data increases memorization risk.
Use PEFT (LoRA/QLoRA) — Frozen weights inherently prevent catastrophic forgetting. The standard recommendation for most fine-tuning projects.
Increase dataset diversity — More diverse examples with varied phrasings teach generalization rather than memorization. If validation loss rises, add data before reducing epochs.
Ask the AI assistant about overfitting, catastrophic forgetting, early stopping, or how to configure learning rates for fine-tuning.