Regression Metrics

• House price
• Temperature
• Salary
• Exam score
• Delivery time
• Sales revenue
• Stock price
• Energy usage

• Actual House Price: $400,000
• Prediction 1: $398,000 (much closer)
• Prediction 2: $250,000 (much farther)

Regression metrics measure how large the prediction errors are.

Error = Actual Value - Predicted Value

• Actual Value: 100
• Predicted Value: 90
• Error: 100 - 90 = 10

If the prediction is too high, the error may be negative. If too low, the error may be positive.

Absolute Error = |Actual Value - Predicted Value|

• Actual = 100, Predicted = 90: |100 - 90| = 10
• Actual = 100, Predicted = 110: |100 - 110| = 10

Both predictions are off by 10.

Squared Error = (Actual Value - Predicted Value)²

• Actual = 100, Predicted = 90
• Error = 10
• Squared Error = 10² = 100

MAE = Average of |Actual Value - Predicted Value|

• Actual Values: 100, 200, 300
• Predictions: 110, 190, 280
• Absolute Errors: 10, 10, 20
• MAE = (10 + 10 + 20) / 3 = 13.33

The model is off by about 13.33 units on average.

• MAE: $20,000
• Meaning: The model's predictions are off by about $20,000 on average.

MAE is useful when you want a simple average error size.

MSE = Average of (Actual Value - Predicted Value)²

• Actual Values: 100, 200, 300
• Predictions: 110, 190, 280
• Errors: -10, 10, 20
• Squared Errors: 100, 100, 400
• MSE = (100 + 100 + 400) / 3 = 200

• 2 minutes
• 3 minutes
• 30 minutes

The 30-minute error may be much more serious. MSE gives that large error more influence.

Note: MSE is harder to interpret because the units are squared.

RMSE = √MSE

• MSE = 200
• RMSE = √200 = 14.14

If predicting house prices, RMSE is measured in dollars. If predicting temperature, RMSE is measured in degrees.

• RMSE: $35,000
• Meaning: The model's typical prediction error is around $35,000, with larger errors affecting the score strongly.

RMSE is common in regression evaluation.

• R² = 0 — The model explains none of the variation
• R² = 1 — The model explains all of the variation
• Example: R² = 0.85 — Model explains about 85% of variation in target values

• High R² — Features like size, location, and bedrooms explain much of the price variation
• Low R² — The model may be missing important information

R-squared is useful for overall fit, but does not show average error size.

• The average prediction error
• Whether errors are small enough for real use
• Whether the model is overfitting
• Whether the model is fair across groups
• Whether important features are missing

A model can have a high R-squared and still make errors that are too large for the application.

• Actual Values: 100, 200, 300, 400
• Predictions: 90, 210, 310, 360
• Errors: 10, -10, -10, 40
• Absolute Errors: 10, 10, 10, 40
• Squared Errors: 100, 100, 100, 1600
• MAE = (10 + 10 + 10 + 40) / 4 = 17.5
• MSE = (100 + 100 + 100 + 1600) / 4 = 475
• RMSE = √475 = 21.79

RMSE is larger than MAE because the 40-point error is punished more strongly.

• You want an easy-to-understand error value
• All errors should count equally
• Large errors should not dominate too much
• You want the metric in the original unit
• You need a simple explanation for users

Example: A house price model has MAE of $15,000.

• Large errors should be punished more heavily
• You want the model to avoid big mistakes
• You are optimizing a mathematical loss function
• Squared errors make sense for the problem

Note: MSE is common during training, but less intuitive for explaining results.

• Large errors should matter more
• You want the metric in the original unit
• You want a commonly used regression metric
• You want to compare models based on typical error size

Example: Delivery time prediction may use RMSE because very large delays matter.

• You want to understand overall model fit
• You want to know how much variation the model explains
• You are comparing regression models
• You want a scale-independent score

Important: R-squared should usually be used with MAE or RMSE.

• Actual Price: $500,000
• Predicted Price: $470,000
• Error: $30,000
• MAE — Average dollar error
• RMSE — Dollar error that punishes large mistakes
• R² — How well features explain price variation

• Actual Temperature: 25°C
• Predicted Temperature: 23°C
• Error: 2°C
• MAE or RMSE: Shows how many degrees off the model is on average
• R²: Shows how much temperature variation the model explains

• Actual Delivery Time: 40 minutes
• Predicted Delivery Time: 55 minutes
• Error: 15 minutes
• RMSE may help: Large errors may frustrate users, so bigger mistakes should matter more

• Training RMSE: 8
• Test RMSE: 40
• Possible Problem: The model memorized training patterns.

• Training RMSE: 60
• Test RMSE: 62
• The model performs poorly everywhere.
• Possible causes: Model is too simple, or important features are missing.

• Using only one regression metric
• Ignoring large errors
• Misinterpreting MSE units
• Assuming high R² means low error
• Comparing metrics across different target units
• Looking only at training metrics
• Ignoring test metrics
• Ignoring outliers
• Ignoring whether the error size is acceptable in real life

• Regression metrics evaluate models that predict numbers.
• Prediction error is the difference between actual and predicted values.
• MAE measures average absolute error.
• MSE measures average squared error.
• RMSE is the square root of MSE.
• RMSE punishes large errors and uses the original unit.
• R-squared measures how much variation the model explains.
• Different regression metrics answer different questions.
• Regression metrics should be checked on unseen test data.