Gradient Descent

What Is Gradient Descent?

Gradient descent is an optimization method used to reduce loss. It helps a model find better weights by slowly changing them in the direction that lowers the loss.

01

Make prediction

02

Calculate loss

03

Find direction to reduce loss

04

Update weights

05

Repeat — predictions improve over time

Why Gradient Descent Matters

Gradient descent matters because machine learning models need a way to improve. At the beginning of training, the model's weights may be random and predictions may be poor.

Early Training

High loss

Predictions are poor or random

After Many Updates

Lower loss

The model is learning better weights

Optimization

Optimization means finding the best values for something. In machine learning, optimization usually means finding weights that minimize loss.

Optimization Question

Which weights give the lowest prediction error?

Gradient descent is one method for searching for those weights.

Loss Function Review

A loss function measures how wrong the model is. Gradient descent tries to reduce this loss.

Wrong Prediction

Prediction: Dog

Correct Answer: Cat

Loss: High

Correct Prediction

Prediction: Cat

Correct Answer: Cat

Loss: Low

Weights Review

Weights are learned numbers inside a model. They control how strongly inputs affect predictions.

Spam Detection Example

Feature: Suspicious links — Weight: High

Feature: Normal greeting — Weight: Low

Gradient descent changes these weights during training.

Gradient

A gradient tells the model how the loss changes when a weight changes.

Gradient Idea

Gradient — Direction and amount of change needed to reduce loss

Important — The gradient points toward increasing loss

Gradient descent — Moves in the opposite direction, toward decreasing loss

Descent

The word descent means moving downward. In gradient descent, the model moves downward on the loss surface toward lower loss.

Hill Analogy

You are standing on a hill. Your goal is to reach the lowest point.

You look at the slope and step downhill.

Gradient descent uses gradients to step toward lower loss the same way.

Loss Surface

A loss surface represents how loss changes for different weight values. Gradient descent searches this surface for lower loss.

01

Bad weights — High loss

02

Better weights — Lower loss

03

Best weights found — Lowest loss the model can reach

Learning Rate

The learning rate controls how big each gradient descent step is. It is one of the most important training settings.

Learning Rate

High learning rate — Large steps

Low learning rate — Small steps

Good learning rate — Helps loss decrease steadily

Learning Rate Too High

If the learning rate is too high, the model may take steps that are too large.

Problems
  • Loss may jump around
  • Model may miss good weights
  • Training may become unstable
  • Loss may fail to decrease

The model keeps jumping over the lowest point instead of settling near it.

Learning Rate Too Low

If the learning rate is too low, the model takes very small steps.

Problems
  • Training is slow
  • Loss decreases very slowly
  • Model may need many epochs
  • Training may get stuck before improving enough

A low learning rate can work, but it may take too long.

Good Learning Rate

A good learning rate is not too high and not too low. It lets the model move steadily toward lower loss.

Healthy Pattern
  • Loss decreases over time
  • Training remains stable
  • Validation performance improves

Choosing the learning rate often requires testing.

Gradient Descent Step by Step

01

Start with initial weights

02

Make predictions using forward propagation

03

Calculate loss

04

Calculate gradients

05

Update weights in the direction that reduces loss

06

Repeat many times

Weight Update Idea

A basic weight update subtracts the gradient multiplied by the learning rate.

Formula Idea

New weight = Old weight − Learning rate × Gradient

Old weight — Current value

Gradient — Direction loss increases

Learning rate — Step size

Subtracting the gradient moves the weight toward lower loss.

Simple Numeric Example

This example shows one gradient descent weight update.

Example

Old weight — 10

Learning rate — 0.1

Gradient — 4

Calculation — New weight = 10 − 0.1 × 4 = 10 − 0.4 = 9.6

The weight moved slightly in a direction that should reduce loss.

Gradient Descent and Backpropagation

Backpropagation and gradient descent work together during neural network training.

ProcessMain Job
BackpropagationCalculates gradients
Gradient DescentUses gradients to update weights

Gradient Descent in Neural Networks

Neural networks can have many weights. Gradient descent updates these weights gradually based on how they affected the loss.

01

Forward propagation — Makes prediction

02

Loss function — Measures error

03

Backpropagation — Calculates gradients

04

Gradient descent — Updates weights

Batch Gradient Descent

Batch gradient descent uses the entire training dataset to calculate one weight update.

Example

Training dataset — 10,000 examples

Batch gradient descent — Uses all 10,000 examples before updating weights

Advantage — Stable updates

Disadvantage — Can be slow for large datasets

Stochastic Gradient Descent

Stochastic Gradient Descent, or SGD, updates weights using one training example at a time.

Example

Training example 1 — Update weights

Training example 2 — Update weights

Training example 3 — Update weights

Advantage — Faster updates   Disadvantage — Updates can be noisy

Mini-Batch Gradient Descent

Mini-batch gradient descent updates weights using a small group of examples. It is commonly used in deep learning.

Example

Training dataset — 10,000 examples   Batch size — 32

The model processes 32 examples, calculates average loss, then updates weights.

Benefit — Balances speed and stability

Batch Size

Batch size is the number of examples used before one weight update. It affects training speed, memory use, and stability.

Batch Size Options

Batch size 1 — Very noisy updates

Batch size 32 — Common mini-batch size

Batch size 10,000 — Full batch update, stable but slow

Epochs and Gradient Descent

An epoch is one full pass through the training dataset. Gradient descent may update weights many times during one epoch.

Example

Dataset size — 1,000 examples

Batch size — 100

Batches per epoch — 10

Weight updates per epoch — 10

More epochs mean more chances for the model to improve.

Iterations

An iteration is one weight update step. If one batch produces one update, then one batch equals one iteration.

Key Terms

Epoch — One full pass through the dataset

Batch — Subset of the dataset used per update

Iteration — One weight update step

Gradient descent happens at each iteration.

Gradient Descent and Loss Curves

A loss curve shows how loss changes during training. It helps show whether gradient descent is working.

Healthy Pattern

Loss decreases steadily

Validation loss also improves

Bad Pattern

Loss jumps around or does not decrease

Training loss improves but validation loss gets worse

Local Minimum

A local minimum is a point where loss is lower than nearby points, but not necessarily the lowest possible loss overall.

Local Minimum

Lower than nearby points, but may not be the absolute lowest.

In deep learning, finding a useful low-loss point is often more important than finding a perfect minimum.

Global Minimum

A global minimum is the lowest possible point on the loss surface.

Global Minimum

Meaning — Lowest possible loss for the model and data

Simple problems — May be easier to find

Deep learning — Loss surfaces are complex; the goal is often good enough weights that generalize well

Gradient Descent and Overfitting

Gradient descent reduces training loss, but training too long can lead to overfitting.

Overfitting Pattern

Training loss — Keeps decreasing

Validation loss — Starts increasing

The model is improving on training data but getting worse on new data.

Possible fix — Early stopping

Gradient Descent and Underfitting

Underfitting can happen when gradient descent does not reduce loss enough.

Underfitting Pattern

Training loss — High   Validation loss — High

Possible Causes — Model is too simple, learning rate is too low, not enough epochs, poor features, too much regularization

Gradient Descent and Regularization

Regularization helps prevent overfitting and can affect how gradient descent updates weights.

Common Regularization Methods
  • L1 regularization
  • L2 regularization
  • Dropout

Goal: Encourage simpler patterns that generalize better.

Gradient Descent and Feature Scaling

Feature scaling can help gradient descent work better. If features have very different ranges, training may be slower or unstable.

Example

Feature 1 — Age ranges from 0 to 100

Feature 2 — Income ranges from 0 to 200,000

Problem — Different scales can make optimization harder

Solution — Scale features before training

Normalization and Standardization

Normalization and standardization are common preprocessing methods that can make gradient descent more efficient.

Normalization

Scales values to a range, often 0 to 1.

Standardization

Centers values around mean 0 and standard deviation 1.

Both methods can help gradient descent move more smoothly.

Gradient Descent for Regression

In regression, gradient descent reduces numerical prediction error.

Example

Task — Predict house price

Prediction — $420,000   Actual — $450,000

Loss — Based on price error

Goal — Update weights so future predictions are closer to actual prices

Gradient Descent for Classification

In classification, gradient descent reduces classification loss.

Example

Task — Classify image as cat, dog, or bird

Correct Class — Dog   Model Output — Dog = 0.20   Loss — High

Goal — Update weights so the model gives higher probability to Dog in similar examples

Gradient Descent for Language Models

Language models use gradient descent to improve token prediction.

Example

Input — "The cat sat on the"

Correct next token — "mat"

If the model gives low probability to "mat", loss is high.

Gradient descent — Updates weights so similar predictions improve later

Optimizers

An optimizer is the algorithm that updates model weights. Gradient descent is the basic idea behind many optimizers.

Common Optimizers
  • Gradient Descent
  • Stochastic Gradient Descent (SGD)
  • Adam
  • RMSprop

All optimizers use gradients to reduce loss.

Adam Optimizer

Adam is a popular optimizer in deep learning. It adapts learning behavior for different weights and often works well as a default choice.

Adam
  • Uses gradients from backpropagation
  • Adapts updates for different weights automatically
  • Often trains smoothly
  • Common default in deep learning

Adam is still based on gradient information.

RMSprop

RMSprop adjusts updates based on recent gradient sizes. It can help with unstable gradients and changing loss surfaces.

RMSprop

Uses recent gradient behavior to adjust updates.

Useful when — Training is unstable or gradients change a lot

Momentum

Momentum helps gradient descent keep moving in a useful direction by remembering past update directions.

Momentum Can Help
  • Smooth updates
  • Speed up training
  • Reduce zigzag movement
  • Move through shallow regions

Momentum is often combined with SGD.

Gradient Descent Analogy

A common analogy is rolling a ball down a hill. The height represents loss, and the ball's position represents the model's weights.

Hill Analogy

Height — Loss

Ball position — Weights

Slope — Gradient

Goal — Move downhill toward lower loss

Why Gradient Descent Does Not Always Work Perfectly

Gradient descent can struggle if the training setup is poor. Training a model requires monitoring and tuning.

Gradient Descent Can Struggle When
  • Learning rate is too high
  • Learning rate is too low
  • Data is noisy
  • Model is poorly designed
  • Features are not scaled
  • Gradients vanish
  • Gradients explode
  • Loss surface is difficult

Vanishing Gradients

Vanishing gradients happen when gradients become very small, causing learning to slow down.

01

Gradients become tiny

02

Weight updates become very small

03

Learning slows down — especially in earlier layers

ReLU, normalization, and better architectures can help reduce this issue.

Exploding Gradients

Exploding gradients happen when gradients become very large, causing unstable training.

01

Gradients become huge

02

Weight updates become huge

03

Training becomes unstable — loss may jump around

Gradient clipping can help control exploding gradients.

Gradient Clipping

Gradient clipping limits gradients when they become too large. This helps prevent unstable updates.

Gradient Clipping

If a gradient is above a chosen limit, reduce it to a safer size.

Useful for — Deep networks, sequence models, training with unstable gradients

Common Mistakes

Common Mistakes
  • Confusing gradient descent with backpropagation
  • Thinking gradient descent calculates gradients by itself
  • Using a learning rate that is too high
  • Using a learning rate that is too low
  • Ignoring feature scaling
  • Looking only at training loss
  • Training for too many epochs
  • Assuming lower training loss always means better real-world performance
  • Forgetting that validation loss matters
  • Thinking one optimizer is always best

Summary

Key Takeaways
  • Gradient descent is an optimization method for reducing loss.
  • Loss measures how wrong the model's prediction is.
  • Gradients show how weights affect loss.
  • Gradient descent updates weights in the direction that reduces loss.
  • Learning rate controls the size of each update.
  • Batch, stochastic, and mini-batch gradient descent use different amounts of data per update.
  • Backpropagation calculates gradients.
  • Gradient descent uses gradients to update weights.
  • Loss curves help show whether training is working.
  • Gradient descent is used in regression, classification, and language models.

Practice Prompt

A model has an old weight of 8, a learning rate of 0.05, and a gradient of 6.

Use the formula New weight = Old weight - Learning rate × Gradient to calculate the new weight. Then explain why the model subtracts the gradient instead of adding it.

Need Help?

Ask the AI if you need help understanding gradient descent, gradients, learning rate, loss curves, weight updates, batch size, epochs, stochastic gradient descent, mini-batch gradient descent, Adam, or the difference between backpropagation and gradient descent.