Forward propagation is the process of moving data through a neural network to produce an output. It is how a neural network makes a prediction.
Input data enters the network
Passes through the input layer
Moves through hidden layers
Reaches the output layer
A prediction is produced
The data moves forward through the network one layer at a time.
Forward propagation matters because every neural network prediction starts with it. Before a model can calculate error or update weights, it first needs to make a prediction.
Given this input, what does the model predict?
Forward propagation is one step in the full training cycle. It creates the prediction that will later be compared with the correct answer.
Model receives input data
Forward propagation produces a prediction
Loss function measures the error
Backpropagation sends error backward
Weights are updated
The process repeats
Input data is the information given to the neural network. It usually comes from features in the dataset.
Spam Detection — Email length, number of links, suspicious words, sender reputation
House Price Prediction — Square footage, bedrooms, bathrooms, location score
Image Classification — Pixel values
The input layer receives the original data. It passes input values into the rest of the network.
Square footage — 2000
Bedrooms — 3
Bathrooms — 2
Location score — 8
These values move forward into the next layer.
Weights are learned numbers that control how strongly inputs affect the next layer. During forward propagation, inputs are multiplied by weights.
Input — 3
Weight — 4
Input × Weight — 3 × 4 = 12
The weight controls how much influence the input has.
A bias is a learned number added to the weighted sum. It gives the model more flexibility.
Neuron value = weighted sum + bias
Weighted sum — 20
Bias — 5
Neuron value — 20 + 5 = 25
A weighted sum is the result of multiplying inputs by weights and adding them together.
Weighted Sum = input1 × weight1 + input2 × weight2 + input3 × weight3
Inputs — 2, 4, 6 Weights — 3, 1, 2
Calculation — 2×3 + 4×1 + 6×2 = 6 + 4 + 12 = 22
An activation function transforms the neuron value before sending it forward. Activation functions help neural networks learn non-linear patterns.
Without activation functions, neural networks behave more like simple linear models.
A neuron output is the result after the activation function is applied. This output can become input for the next layer.
Receive inputs
Multiply by weights
Add weighted values together
Add bias
Apply activation function → output
Hidden layers are layers between the input layer and output layer. Each hidden layer transforms the data.
Early Hidden Layers — Detect edges and colors
Middle Hidden Layers — Detect shapes and textures
Later Hidden Layers — Detect object parts
Purpose: Build complex patterns from simple inputs.
The output layer produces the final prediction. The type of output depends on the task.
Binary Classification — Spam or Not Spam
Multi-Class Classification — Cat, Dog, or Bird
Regression — Predicted house price
Text Generation — Next token or word
Input values enter the input layer
Inputs are multiplied by weights
Weighted values are added together
Bias is added
Activation function is applied
Output moves to the next layer
Steps repeat until the output layer
The final prediction is produced
This example shows one forward pass through one neuron.
Inputs — 2 and 3 Weights — 4 and 5 Bias — 1
Step 1 — 2 × 4 = 8, 3 × 5 = 15
Step 2 — 8 + 15 = 23
Step 3 — 23 + 1 = 24
Step 4 — Apply ReLU: ReLU(24) = 24
Output — 24
ReLU turns negative values into 0 and keeps positive values unchanged. It is commonly used in hidden layers.
Neuron value: −7
ReLU output: 0
Negative values are turned off.
Neuron value: 12
ReLU output: 12
Positive signals pass forward.
Sigmoid converts a number into a value between 0 and 1. This is useful for binary classification.
Sigmoid output — 0.87
The model estimates an 87% chance that the email is spam.
Threshold — 0.50 → Prediction: Spam
Softmax converts output scores into probabilities across multiple classes.
Cat: 4.0
Dog: 2.0
Bird: 1.0
Cat: 0.84
Dog: 0.11
Bird: 0.05
In classification, forward propagation produces class scores or probabilities.
Input — Email features
Forward Propagation — Moves features through hidden layers
Output — Spam probability = 0.91 → Prediction: Spam
Classification models often use sigmoid or softmax in the output layer.
In regression, forward propagation produces a numerical value.
Input — House features
Forward Propagation — Moves features through hidden layers
Output — 450000 → Prediction: $450,000
Regression models often use a linear output layer.
Image recognition models use forward propagation to move pixel data through the network.
Input — Image pixels
Hidden Layers — Learn edges, shapes, textures, and objects
Output — Class probabilities → Prediction: Dog
The model processes numerical pixel values, not images the same way humans see them.
Text models use forward propagation to process tokens or embeddings and predict language patterns.
Input — "The cat sat on the"
Model Output — Probabilities for possible next tokens
Predicted Next Token — "mat"
Large language models generate predictions one token at a time.
| Process | Direction | Purpose |
|---|---|---|
| Forward Propagation | Input to output | Produces a prediction |
| Backpropagation | Output error backward | Updates weights |
Inference means using a trained model to make predictions. During normal inference, the model usually only uses forward propagation.
New email enters trained spam model.
Forward propagation produces — Spam probability = 0.93
Prediction — Spam
No weight updates happen during normal inference.
During training, forward propagation is followed by loss calculation and backpropagation.
Input — Image of a cat
Forward propagation prediction — Dog
Correct answer — Cat
Loss — High
Backpropagation — Updates weights to reduce future mistakes
Neural networks often process data in batches. A batch is a group of examples processed together.
Batch size — 32 images
Forward propagation — The network produces predictions for all 32 images
Benefit — Batches make training more efficient
In a deep network, forward propagation moves through many layers. Each layer transforms the data before passing it onward.
Input layer
Hidden layer 1
Hidden layer 2
Hidden layer 3
Output layer — final prediction
Neural networks often use matrix multiplication to make forward propagation efficient.
Inputs × Weight matrix + Biases = Layer output
Computers can process many values at once, enabling large batches and deep networks.
This is one reason GPUs are useful for deep learning.
Weights and biases are model parameters. Forward propagation uses these parameters to make predictions.
Uses current weights and biases to make predictions.
Changes weights and biases based on error.
Before training, weights may be random. Forward propagation can still happen, but predictions may be poor.
Input — Email features
Prediction — Random or inaccurate
Reason — Weights have not learned useful patterns yet
After training, weights contain learned patterns. Forward propagation becomes more useful.
Input — Email with suspicious links and urgent wording
Prediction — Spam probability = 0.94
Reason — The model learned that these features often appear in spam emails
Forward propagation produces a prediction. The loss function compares that prediction to the correct answer.
Prediction: Dog
Correct Answer: Cat
Loss: High
Prediction: Cat
Correct Answer: Cat
Loss: Low
Loss tells the model how wrong the forward pass was.
Some output activations create probability-like scores.
Sigmoid — Binary classification. Example: 0.82 = 82% score for positive class.
Softmax — Multi-class classification. Example: Cat 0.70, Dog 0.20, Bird 0.10.
For binary classification, the output is often compared to a threshold.
Output — 0.72
Threshold 0.50 — Prediction: Positive
Threshold 0.80 — Same output would predict: Negative
Thresholds affect classification decisions.
Forward propagation can produce wrong predictions. This is normal during training.
Input — Image of a dog
Prediction — Cat
Correct Answer — Dog
The loss function measures the error, then training updates the model.
After training, the model only needs to move data forward to make predictions. It does not need to update weights.
Forward propagation
Loss calculation
Backpropagation
Weight update
Forward propagation only
Usually faster than training.
A neuron receives inputs 3 and 4. The weights are 2 and 5. The bias is 1. First calculate the weighted sum, then add the bias. If the neuron uses ReLU, what is the output?
Then explain how this example represents one small step in forward propagation.
Ask the AI if you need help understanding forward propagation, input layers, hidden layers, output layers, weights, biases, activation functions, loss, inference, or the difference between forward propagation and backpropagation.