What is sharpness-aware minimization (SAM) in cost optimization?

SAM optimizes for flat minima by minimizing worst-case loss in a neighborhood.

Sharpness-Aware Minimization modifies the cost function by considering perturbations in parameter space. Instead of minimizing loss at a point, it minimizes the maximum loss within a radius. This encourages flat minima which generalize better. It is a second-order-inspired method but avoids explicit Hessian computation.