What is Label Smoothing and how does it improve model generalization?

Label smoothing is a regularization technique that replaces hard labels with softened probability distributions to prevent overconfidence in neural networks.

In standard classification, labels are one-hot encoded, meaning the correct class has probability 1 and others 0. This can lead to overconfident models that do not generalize well.

Label smoothing modifies this by distributing a small probability mass to incorrect classes.

Formula:

Where:

Why it works:

Benefits: