juniorCost Function
What is Cross-Entropy Loss?
Updated May 15, 2026
Short answer
Cross-entropy measures difference between predicted probabilities and true labels.
Deep explanation
It is widely used in classification tasks. It penalizes confident wrong predictions heavily and is derived from information theory.
Real-world example
Used in image classification models like cat vs dog detection.
Common mistakes
- Forgetting to apply softmax before cross-entropy.
Follow-up questions
- Why is log used in cross-entropy?
- What happens if predicted probability is 0?