What is the bias-variance tradeoff in Q-Learning?

Q-Learning must balance bias from approximation and variance from sampling and exploration.

Bias arises from function approximation and bootstrapping errors, while variance comes from stochastic transitions and exploration noise. High bias leads to underestimation of Q-values, while high variance leads to unstable updates. Techniques like target networks reduce variance, while function approximation choices affect bias. The tradeoff is central in designing stable deep RL systems.