What is self-supervised pretraining in vision models?

Self-supervised learning trains models using automatically generated labels from data itself.

Instead of manual labels, models learn by solving pretext tasks like predicting rotated images, reconstructing masked patches (MAE), or contrastive learning (SimCLR). This enables learning powerful representations from unlabeled datasets.