What is catastrophic forgetting in NLP models?

Catastrophic forgetting occurs when models lose previous knowledge after fine-tuning.

When neural networks are fine-tuned on new tasks, they overwrite learned weights, causing degradation in earlier tasks. Techniques like elastic weight consolidation mitigate this.