Machine learning models learn patterns from data and are evaluated on unseen data to measure how well they generalize.
Training Data — Used to teach the model patterns
Validation Data — Used to improve and tune the model
Test Data — Used for final performance evaluation
Evaluating a model on the same data used for training can produce misleading results. Data splitting provides a realistic estimate of real-world performance.
Train and test on the same data.
Result: Artificially high accuracy.
Train on one dataset, test on unseen data.
Result: Realistic performance estimate.
Training data teaches the model by providing examples from which it can learn patterns and relationships.
Model receives input data
Model makes a prediction
Calculate error between prediction and actual answer
Update model parameters to reduce error
Repeat thousands or millions of times
Training loss measures prediction error during learning.
Beginning — Loss = 5.0
Later — Loss = 0.8
Lower loss indicates better learning.
Validation data is used during development to evaluate performance and guide model improvements. The model never learns from validation data.
Training Accuracy — 99%
Validation Accuracy — 70%
A large gap suggests the model is memorizing training data.
Validation performance can be monitored to stop training before overfitting begins.
Epoch 10 → 90%
Epoch 20 → 92%
Epoch 30 → 89%
Best model: saved at Epoch 20.
The test set is used only once training and tuning are complete. It provides the final estimate of model performance.
Test Accuracy — 95%
Represents expected real-world performance.
| Split | Training | Validation | Testing |
|---|---|---|---|
| 80/20 | 80% | - | 20% |
| 70/15/15 | 70% | 15% | 15% |
| 60/20/20 | 60% | 20% | 20% |
Generalization measures how well a model performs on new, unseen data.
Training: 92%
Validation: 90%
Testing: 89%
Similar values across all three datasets.
Training: 99%
Validation: 65%
Testing: 63%
Large gap suggests overfitting.
Fold 1 → Validate, Folds 2-5 → Train
Fold 2 → Validate, Folds 1,3,4,5 → Train
Repeat until all folds have been used.
Benefits — Better evaluation, reduced bias, more reliable estimates
| Dataset | Purpose | Updates Model? |
|---|---|---|
| Training | Learn patterns | Yes |
| Validation | Tune model | No |
| Testing | Final evaluation | No |
Result — Unrealistically high accuracy
A company has 50,000 customer records and wants to build a machine learning model. Explain how the dataset could be divided into training, validation, and test sets and describe the purpose of each dataset.
Ask the AI if you need help understanding training data, validation data, testing, overfitting, or model evaluation.