midProbability
What is KL divergence?
Updated May 17, 2026
Short answer
KL divergence measures difference between two probability distributions.
Deep explanation
KL(P||Q) quantifies how much information is lost when Q approximates P. It is widely used in machine learning loss functions.
Real-world example
Training variational autoencoders.
Common mistakes
- Assuming KL divergence is symmetric.
Follow-up questions
- Is KL symmetric?
- Where is KL used?