juniorNeural Networks
What is an activation function?
Updated May 17, 2026
Short answer
An activation function introduces non-linearity into a neural network.
Deep explanation
Without activation functions, the network becomes a linear regression model regardless of depth. Common activations include ReLU, sigmoid, and tanh.
Real-world example
ReLU is used in CNNs for image classification.
Common mistakes
- Using sigmoid in deep networks causing vanishing gradients.
Follow-up questions
- Why is ReLU preferred?
- What is dying ReLU?