What is snapshot ensembling in deep learning?

Snapshot ensembling trains a single neural network and saves multiple models at different training stages.

Snapshot ensembling leverages cyclical learning rates to allow a single model to converge to multiple local minima during training. Each local minimum is saved as a separate ensemble member. At inference time, predictions from all snapshots are averaged. This reduces training cost while achieving ensemble-like performance.