Why do deep learning cost functions have many saddle points instead of local minima?

High-dimensional parameter spaces make saddle points exponentially more likely than poor local minima.

In high-dimensional spaces, the probability that all eigenvalues of the Hessian are positive (local minimum) becomes extremely small. Instead, most critical points are saddle points where some directions curve upward and others downward. These points slow training because gradients can vanish in certain directions even when not at optimal minima.