What is the role of entropy in modern cost function design?

Entropy controls uncertainty, exploration, and calibration in probabilistic models.

Entropy appears in cost functions as a regularizer or constraint that prevents deterministic collapse. In reinforcement learning, entropy encourages exploration. In classification, it improves calibration. In generative models, it controls diversity of outputs. It acts as a thermodynamic control parameter shaping distribution sharpness.