What is embedding collapse in unsupervised deep learning?

Embedding collapse occurs when all representations converge to a single point.

In self-supervised or contrastive learning, improper loss design or lack of negative samples can cause all embeddings to become identical, making representations useless. Techniques like normalization, contrastive loss, and variance regularization prevent collapse.