What is catastrophic forgetting in Deep Q-Networks?

Catastrophic forgetting occurs when a neural network forgets previously learned behaviors due to new updates.

In DQN, as new experiences are learned, gradient updates may overwrite previously learned Q-values for older states. This is especially problematic in non-stationary or replay-buffer-limited training. Experience replay mitigates this by mixing old and new samples, preserving distributional stability.