How does batch normalization affect cost function optimization?

Batch normalization smooths the optimization landscape and stabilizes gradients.

Batch normalization normalizes activations within mini-batches, reducing internal covariate shift and improving gradient flow. It reshapes the cost surface by making it more isotropic, allowing larger learning rates and faster convergence. It also introduces stochasticity that acts as regularization, improving generalization.