What is the relationship between Hessian matrix and cost function optimization?

The Hessian describes curvature of the cost function and guides second-order optimization.

The Hessian matrix contains second derivatives of the cost function with respect to model parameters. It reveals curvature information: eigenvalues indicate whether a region is convex, concave, or a saddle point. Positive definite Hessian implies local minima, while negative eigenvalues indicate saddle points. Modern optimizers approximate Hessian information to improve convergence speed beyond first-order methods like SGD.