What is implicit temperature in stochastic gradient descent?

SGD behaves like a thermal system with an implicit temperature controlled by noise.

SGD dynamics can be modeled as a noisy physical system where the noise level acts like temperature. High temperature leads to exploration of the loss landscape, while low temperature leads to convergence. Batch size and learning rate jointly control this implicit temperature.