What is self-supervised learning in NLP and why is it effective?

Self-supervised learning trains models using automatically generated labels from raw text.

Instead of manual labeling, models learn from tasks like masked language modeling or next-token prediction. This allows scaling to massive datasets and enables learning general linguistic representations. It is the foundation of modern LLMs.