3 / 13

Training Data

What Is Training Data?

Training data is the collection of examples used to teach a machine learning model. Each example contains features (inputs) and labels (correct outputs).

Spam Detection Training Data

Features — Email text, Sender, Subject, Links

Label — Spam or Not Spam

The model learns patterns that distinguish spam from non-spam emails.

Why Training Data Matters

Good Training Data

Good Training Data → Better Model → Better Predictions

Poor Training Data

Poor Training Data → Weaker Model → Poor Predictions

A model can only learn what its training data teaches it.

What Training Data Contains

House Price Prediction

Features — House size, Bedrooms, Location, Year built

Label — Sale price

The model learns the relationship between features and the sale price.

Types of Training Data by Learning Type

Supervised Training Examples

Image recognition — Images labeled as cat, dog, or car

Spam detection — Emails labeled as spam or not spam

House prices — Listings with actual sale prices

Unsupervised Training

Customer data with no predefined groups — the model discovers clusters on its own

Reinforcement Training

Game AI receives a reward for winning and a penalty for losing — learns through trial and error

Types of Data in Training

Data Types

Text — Articles, reviews, emails, conversations
Images — Photos, medical scans, satellite images
Audio — Speech recordings, music
Video — Films, surveillance, driving footage
Tables — Structured data with rows and features
Sensors — Temperature, GPS, accelerometer data

Qualities of Good Training Data

Accurate

Correct features and correct labels.

Incorrect example — Email labeled as "not spam" that is actually spam

Complete

All important features should be present.

Problem — House price dataset missing location for half the records

Relevant

Relevant for exam score — Study hours, attendance, prior grades

Irrelevant — Student's favorite color, random ID

Representative

A voice assistant trained mostly on adult voices may struggle with children's voices.

Training data should reflect all groups and situations the model will encounter.

Sufficient Size

Too little — 10 dog photos per breed → poor generalization

Better — Thousands of photos per breed → much better performance

Problems with Training Data

Bias Causes

Data collected from limited sources
Historical decisions embedded in labels
Sampling from only certain populations

Overfitting Example

Training accuracy: 99%

Test accuracy: 70%

The model memorized training data rather than learning generalizable patterns.

Underfitting Example

Using only house size to predict price — ignoring location, bedrooms, and condition.

The model fails to capture the actual complexity of the relationship.

Why Not Train on All Data?

Bad Approach

Train on 100% of data → Test on the same 100%

The model appears to work perfectly, but this is misleading because it has seen all the answers.

Better Approach

Train on 70–80% → Test on 20–30%

The test set represents new unseen data. This measures real generalization.

Data Cleaning and Preprocessing

Cleaning Steps

Remove duplicates
Handle missing values
Fix incorrect values
Standardize formats

Preprocessing Examples

Scale numerical features to 0–1
Encode categories as numbers
Tokenize text into words
Resize images to fixed dimensions

Collect raw data

Clean and fix problems

Engineer useful features

Encode and scale features

Split into train/validation/test sets

Train the model on the training set

Training Data Split

Dataset	Typical Size	Purpose
Training	70–80%	Used to teach the model
Validation	10–15%	Used to tune and improve the model
Test	10–20%	Used to evaluate final performance

Common Problems in Training Data

Problems

Too few examples — Model may not generalize
Class imbalance — One category dominates
Missing values — Require imputation or removal
Mislabeled examples — Teach wrong patterns
Biased data — Unfair predictions for some groups
Outdated data — Patterns may no longer apply

Fraud Detection — Class Imbalance

Non-fraud: 99% / Fraud: 1%

A model predicting "not fraud" every time achieves 99% accuracy but detects 0% of fraud.

Oversample the minority class
Undersample the majority class
Use class-weighted loss functions

Ways to Improve Training Data

Improvements

Collect more examples, especially underrepresented cases
Review and fix incorrect labels
Add variety to cover more real-world scenarios
Apply data augmentation for images and audio
Remove or correct noisy and inconsistent examples
Balance class distribution

Real-World Example: Medical Diagnosis

Medical Diagnosis Training Data

Features — Patient age, Symptoms, Lab results, Medical history

Label — Diagnosis

Key requirements — Accurate expert labels, diverse demographics, sufficient examples for rare conditions

Real-World Example: Self-Driving Cars

Self-Driving Car Training Data

Inputs — Camera images, LIDAR readings, GPS, Weather

Labels — Object positions (cars, pedestrians, signs), Correct driving actions

Must include data from various road conditions, weather, and locations to generalize well.

Key Takeaways

Summary

Training data is the collection of labeled examples used to teach the model
Model quality depends directly on training data quality
Good training data is accurate, complete, relevant, representative, and large enough
Common problems include bias, class imbalance, missing values, and mislabeled examples
Data must be cleaned and preprocessed before training
Always split data so the test set represents truly unseen examples

Need Help?

Ask the AI if you need help understanding training data, how to identify good versus bad training data, class imbalance, or the importance of data splitting.