What causes training instability in deep unsupervised models?

Instability arises from poor loss landscapes, collapsing representations, and gradient issues.

Deep unsupervised models often suffer from unstable optimization because there are no explicit labels to anchor learning. Issues include vanishing gradients, mode collapse, and noisy self-generated targets. Techniques like normalization layers, momentum encoders, contrastive losses, and careful temperature scaling help stabilize training. Architectures like BYOL and SimCLR explicitly address these issues.