midProbability
What is entropy in probability theory?
Updated May 17, 2026
Short answer
Entropy measures uncertainty in a probability distribution.
Deep explanation
Defined as H(X) = -Σ p(x) log p(x). Higher entropy means more randomness. It is foundational in information theory and machine learning.
Real-world example
Measuring uncertainty in weather prediction.
Common mistakes
- Confusing entropy with variance.
Follow-up questions
- What is high entropy?
- Where is entropy used?