The training process is how a machine learning model learns from data. The model makes predictions, compares them to correct answers, and adjusts itself to improve.
Receive input data
Make a prediction
Compare the prediction to the correct label
Calculate the error using the loss function
Update internal parameters to reduce the error
Repeat for the next batch of examples
Neural Networks — Learn complex patterns from large datasets
Decision Trees — Learn rule-based splits from data
Linear Models — Learn weights for each feature
Ensemble Models — Combine multiple models together
Features
Label — Sale price
Sample 1: Model predicts $250,000
Sample 2: Model predicts $400,000
Predicted: $250,000 | Actual: $400,000
Predicted: $400,000 | Actual: $550,000
$250,000 − $400,000 = −$150,000 error
Goal — Minimize this error over many training steps
The loss function measures how wrong the model's predictions are. The goal of training is to minimize the loss.
Parameters are the internal values the model adjusts during training to improve its predictions.
High weight for "Size" — Model learned size is important for price
Low weight for "Street number" — Model learned street number is not important
Price = (Weight_Size × Size) + (Weight_Location × Location) + Bias
The model adjusts each weight and the bias during training.
An epoch is one full pass through the entire training dataset. Training usually requires many epochs.
Epoch 1 — Accuracy: 60%
Epoch 10 — Accuracy: 78%
Epoch 50 — Accuracy: 91%
Accuracy improves with each epoch, though gains slow as the model converges.
A batch is a small group of examples processed together before updating parameters.
Dataset: 10,000 examples
Batch size: 100
10,000 / 100 = 100 batches per epoch
The learning rate controls how large each parameter update is. It is one of the most important hyperparameters in training.
Parameters change too much each step.
Training may be unstable or fail to converge.
Parameters change too slowly.
Training takes very long to complete.
Calculate the current loss
Calculate the gradient — direction of steepest increase in loss
Move in the opposite direction of the gradient
Update parameters by learning rate × gradient
The goal is to reach the minimum of the loss function — where predictions are most accurate.
Model receives labeled examples. Learns to predict the label from features.
Example — Email → Spam or Not Spam
Model receives data without labels. Discovers patterns and structure on its own.
Example — Customer data with no predefined groups
Model receives rewards or penalties. Learns through trial and error.
Example — Game AI: +10 for winning, −5 for losing
Training accuracy: 95%
This might look good, but it could mean the model memorized training examples rather than learning real patterns.
Always check validation accuracy to confirm generalization
Training: 99% | Validation: 72%
Causes: Model too complex, dataset too small, too many epochs
Training: 58% | Validation: 56%
Causes: Model too simple, too few features, insufficient training
Epoch 5 — Validation accuracy: 88%
Epoch 10 — Validation accuracy: 91% ← Best
Epoch 15 — Validation accuracy: 89% ← Getting worse
Stop at Epoch 10 and save that version of the model
L1 Regularization — Encourages some weights to be exactly zero (feature selection)
L2 Regularization — Encourages weights to be small but non-zero
Dropout — Randomly removes neurons during training to prevent over-reliance on specific paths
7,000 labeled emails | 50 epochs
Training accuracy: 95% | Validation accuracy: 93%
Features: Size, bedrooms, location, year | 30 epochs
Training MAE: $18,000 | Validation MAE: $21,000
Epoch 1: 45% | Epoch 5: 71% | Epoch 10: 83% | Epoch 20: 91%
Validation accuracy ≈ Training accuracy
Model generalizes to new data
Has learned real patterns
Validation accuracy much lower than training
Model only works on training data
Has learned noise, not real patterns
Loss not decreasing — Learning rate may be too low, or data has problems
Loss oscillating — Learning rate may be too high
Training stalls early — Model may need more capacity or better features
Overfitting — Model needs regularization or more data
Ask the AI if you need help understanding the training cycle, loss functions, learning rate, gradient descent, overfitting, early stopping, or any other part of how machine learning models train.