What is entropy-temperature duality in cost function optimization?

Temperature scaling in softmax acts as an entropy control mechanism over model predictions.

Temperature scaling modifies the sharpness of probability distributions, directly controlling entropy. Lower temperature reduces entropy (more deterministic predictions), while higher temperature increases entropy (more exploratory behavior). This creates a thermodynamic analogy where optimization behaves like energy minimization under entropy constraints.