What is Jensen’s inequality and why is it important in probability?

Jensen’s inequality relates convex functions and expectations, stating that f(E[X]) ≤ E[f(X]) for convex f.

Jensen’s inequality is fundamental in probability and information theory. It states that applying a convex function after expectation produces a lower value than taking expectation after applying the function. This captures the idea that uncertainty increases nonlinear transformations. It is widely used in deriving bounds in machine learning, variational inference, and risk analysis.