What is calibration under distribution shift?

It evaluates whether probability calibration remains valid when data distribution changes.

A model may be well-calibrated in training distribution but become miscalibrated under covariate or label shift. Evaluation under shift uses recalibration curves, ECE drift, and reliability diagrams across environments. This is critical in production systems where distribution shift is expected.