How do transformer models internally represent uncertainty in next-token prediction?

Uncertainty is encoded as probability distributions over the vocabulary via softmax logits.

Transformers do not explicitly model uncertainty but implicitly represent it through logits produced at the output layer. The softmax distribution reflects epistemic uncertainty (lack of knowledge) and aleatoric uncertainty (data ambiguity). However, these probabilities are often miscalibrated, meaning high confidence does not always imply correctness. Temperature scaling, ensembles, and Bayesian approximations are used to improve calibration.