What is catastrophic forgetting in neural networks?

Catastrophic forgetting is when a model forgets old tasks after learning new ones.

It occurs in continual learning where new gradients overwrite previously learned weights.