How does Q-Learning handle catastrophic bootstrapping errors?

Catastrophic bootstrapping occurs when incorrect Q-estimates recursively propagate and destabilize learning.

Q-learning updates rely on bootstrapped targets, meaning current Q-values depend on future Q-estimates. If early predictions are incorrect, these errors are propagated forward and backward through updates, amplifying over time. This can lead to divergence or collapse of Q-values. Techniques like target networks, slower update rates, and clipped TD errors reduce this feedback amplification.