In a neural network, neurons and weights are basic building blocks. A neuron receives input values and produces an output. A weight controls how important an input is.
Inputs — raw values enter the neuron
Multiplied by weights — each input is scaled by its weight
Added together — weighted values are summed
Activation function — transforms the sum
Neuron output — result passed to the next layer
Neural networks learn by adjusting weights during training.
An artificial neuron is a mathematical unit inside a neural network. It receives inputs, multiplies them by weights, adds a bias, applies an activation function, and sends an output forward.
Inputs are the values given to the neural network. They usually come from features in the dataset.
House price model — square footage, bedrooms, bathrooms, location score
Spam detection model — number of links, email length, suspicious word count, sender reputation
Image model — pixel values
Weights are learned numbers that control how strongly each input affects the neuron's output.
Square footage — Weight: Large — strongly affects predicted price
Random listing ID — Weight: Small — should not strongly affect predicted price
Each input is multiplied by its weight. This is how the model controls the influence of each input.
Input 1: 3 × Weight 1: 2 = 6
Input 2: 5 × Weight 2: 4 = 20
The neuron uses these weighted values to calculate its output.
After inputs are multiplied by weights, the neuron adds them together.
Formula — Weighted Sum = input1 × weight1 + input2 × weight2 + input3 × weight3
Inputs — 2, 4, 6 | Weights — 3, 1, 2
Calculation — 2×3 + 4×1 + 6×2 = 6 + 4 + 12 = 22
A bias is an extra learned number added to the weighted sum. Bias gives the model more flexibility.
Formula — Neuron Value = Weighted Sum + Bias
Example — Weighted Sum: 22 + Bias: 3 = Neuron Value: 25
Without bias, the model may have a harder time fitting patterns.
An activation function transforms the neuron's value before passing it forward. It helps the model learn non-linear patterns.
Without activation functions, a neural network would behave more like a simple linear model.
This example shows how one neuron turns inputs into an output.
Inputs: 2, 5, 1 — Weights: 4, 3, 6 — Bias: 2
Multiply inputs by weights — 2×4=8, 5×3=15, 1×6=6
Add weighted values — 8 + 15 + 6 = 29
Add bias — 29 + 2 = 31
Apply activation function — output depends on the function used
Weights matter because they control what the model pays attention to.
Suspicious links — High weight
Normal greeting — Low weight
Unknown sender — Medium weight
The model gives more influence to stronger spam signals.
A positive weight increases the neuron's output when the input increases.
Feature — number of suspicious links
Weight — positive
Meaning — more suspicious links may increase the chance of spam
A negative weight decreases the neuron's output when the input increases.
Feature — trusted sender score
Weight — negative
Meaning — a more trusted sender may decrease the chance of spam
A small weight means the input has less influence on the output.
Feature — email font size
Weight — small
Meaning — font size may not strongly affect spam prediction
During training, the model may reduce weights for weak or irrelevant features.
A large weight means the input has more influence on the output.
Feature — suspicious link count
Weight — large
Meaning — this feature strongly affects the spam prediction
Large weights can be useful, but weights that become too large may contribute to overfitting.
A model learns by adjusting weights. At first, weights may be random or poorly chosen. Training improves them over time.
Receive inputs
Multiply inputs by weights
Make a prediction
Compare prediction with correct answer
Calculate loss
Adjust weights
Repeat many times
If a prediction is wrong, the model changes weights slightly to reduce future mistakes.
A house price model predicts too low.
Possible update — increase weights for important features such as square footage or location
Goal — make future predictions closer to the correct answer
A loss function measures how wrong the model's prediction is.
Prediction: Spam
Correct answer: Not Spam
The prediction was wrong.
Prediction: Not Spam
Correct answer: Not Spam
The prediction was correct.
Backpropagation calculates how each weight contributed to the error and sends error information backward through the network.
Prediction — model makes a forward pass
Calculate loss — measure how wrong the prediction was
Send error backward — error flows back through layers
Adjust weights — update weights to reduce future error
Improve future predictions — repeat for next batch
Gradient descent is an optimization method used to update weights in the direction that reduces loss.
Calculate loss
Find how weights affect the loss
Change weights slightly
Repeat many times
The learning rate controls how big each weight update is.
Weights change quickly.
Risk: model may overshoot good weights.
Weights change slowly.
Risk: training may be very slow.
A good learning rate helps the model improve steadily.
A model improves by updating weights after mistakes.
Model predicts an email is not spam. Correct answer: Spam.
This process happens many times across many examples.
Inputs — Suspicious links: 3 | Word count: 5 | Trusted sender: 2
Weights — Link weight: 4 | Word weight: 2 | Trusted sender weight: -3 | Bias: 1
Multiply — 3×4=12, 5×2=10, 2×(-3)=-6
Weighted sum — 12 + 10 - 6 = 16
Add bias — 16 + 1 = 17 (high spam score)
Inputs — Square footage: 2.0 | Bedrooms: 3 | Age: 10
Weights — Sq ft: 80 | Bedrooms: 20 | Age: -2 | Bias: 50
Multiply — 2.0×80=160, 3×20=60, 10×(-2)=-20
Weighted sum — 160 + 60 - 20 = 200
Add bias — 200 + 50 = 250 (predicted price value)
A neural network usually has many neurons. Each neuron can learn a different pattern.
Neuron 1 — detects edges
Neuron 2 — detects curves
Neuron 3 — detects color patterns
Neuron 4 — detects textures
Together, they help the network understand complex images.
Hidden layer neurons transform the data and learn useful intermediate patterns.
Input — image pixels
Hidden neuron — detects vertical edges
Another hidden neuron — detects round shapes
Later hidden neurons — combine patterns to detect eyes, wheels, or faces
Output layer neurons produce the final prediction.
Cat — 0.70
Dog — 0.20
Bird — 0.10
Prediction — Cat (highest output probability)
Weights connect neurons between layers. Each connection has its own weight.
Input layer — 3 neurons
Hidden layer — 4 neurons
Connections — each input connects to each hidden neuron = 12 weights
Large networks can have millions or billions of weights.
More weights can give a model more flexibility, but more weights are not always better.
Weights can sometimes give clues about which inputs influence the model, but deep neural networks are hard to interpret because many layers interact.
Simple model — a strong weight may suggest a strong feature effect
Deep neural network — many weights interact across many layers
One weight does not always explain the full model behavior.
Weight initialization means choosing starting values for weights before training begins.
If all weights start the same, neurons may learn the same patterns.
Before training, weights may be random. After training, they contain learned patterns.
Weights are not useful yet.
Spam model may guess randomly.
Weights are adjusted using examples.
Model recognizes spam patterns better.
After training, weights contain learned information from the dataset.
These learned weights help the model predict new examples.
Overfitting can happen when weights adapt too closely to the training data.
Training accuracy — 99%
Test accuracy — 67%
The model may have learned training-specific details instead of general patterns.
Regularization, dropout, and early stopping can help reduce overfitting.
Regularization helps control weights to reduce overfitting.
L1 regularization — can push some weights toward zero
L2 regularization — shrinks large weights
Dropout — randomly turns off some neurons during training
Goal — learn simpler patterns that generalize better
During training, gradients help update weights. Sometimes gradients become too small or too large.
Updates become very tiny.
Learning slows down.
Earlier layers may stop improving.
Updates become too large.
Training becomes unstable.
More common in deep networks.
Deep learning uses many layers of neurons and many learned weights to build complex patterns.
Input — image pixels
Early layers — learn edges and colors
Middle layers — learn shapes and textures
Later layers — learn object parts
Output layer — predicts the class
Weights control how information flows through all of these layers.
A neuron receives three inputs: 2, 4, and 5. The weights are 3, 2, and -1. The bias is 6.
Calculate the weighted sum, add the bias, and explain how changing one weight could change the neuron's output.
Ask the AI if you need help understanding neurons, weights, weighted sums, bias, activation functions, backpropagation, gradient descent, or how models learn by adjusting weights.