What is second-order optimization compared to Gradient Descent?

Second-order methods use curvature (Hessian) information, while Gradient Descent uses only first-order gradients.

Gradient Descent updates parameters using first derivatives (gradient), whereas second-order methods like Newton's method use the Hessian matrix (second derivatives) to incorporate curvature information. This allows faster convergence near optima but is computationally expensive for high-dimensional problems like deep learning.