seniorLinear Algebra
What is the relationship between Hessian matrix and curvature in optimization?
Updated May 16, 2026
Short answer
Hessian matrix measures second-order curvature of loss surface.
Deep explanation
The Hessian contains second partial derivatives, capturing how gradients change. Its eigenvalues determine whether a point is a minimum, maximum, or saddle point. Positive eigenvalues indicate convex curvature directions.
Real-world example
Used in Newton’s method for fast convergence.
Common mistakes
- Ignoring negative eigenvalues in non-convex optimization.
Follow-up questions
- Why is Hessian expensive to compute?