What is weight initialization and why is it important?
Updated May 17, 2026
Short answer
Weight initialization sets initial values of model parameters before training.
Deep explanation
Proper initialization prevents vanishing/exploding gradients. Methods include Xavier and He initialization depending on activation functions.
Real-world example
Used in deep CNN architectures like ResNet.
Common mistakes
- Initializing all weights to zero.
Follow-up questions
- What is Xavier initialization?
- What is He initialization?