What is feature scaling for neural networks and why is it critical?
Updated May 16, 2026
Short answer
Feature scaling ensures neural networks train efficiently by keeping input values within a stable range.
Deep explanation
Neural networks rely on gradient-based optimization, which is highly sensitive to input magnitude. If features are not scaled, gradients can become unstable, leading to slow convergence or vanishing/exploding gradients. Standardization or normalization ensures faster and more stable learning.
Real-world example
Used in image recognition systems where pixel values are normalized before training CNNs.
Common mistakes
- Feeding raw unscaled numerical features into deep learning models.
Follow-up questions
- Why do deep learning models prefer normalized inputs?
- What happens if features are not scaled?