What is implicit regularization in cost function optimization?

Implicit regularization refers to generalization effects not explicitly defined in the cost function.

Even without explicit penalties like L1 or L2, optimizers like SGD bias solutions toward simpler or flatter minima. This is due to optimization dynamics, noise, and parameterization structure. Deep networks often generalize well because the optimization process itself implicitly constrains solutions.