What is the role of the discount factor in long-horizon Q-Learning stability?

The discount factor controls how future rewards influence current Q-value updates and strongly affects stability in long-horizon tasks.

The discount factor γ determines how much future rewards contribute to the current Q-value estimate. High γ (close to 1) makes the agent consider long-term rewards, but it also increases variance and can destabilize learning due to deeper bootstrapping chains. Low γ makes learning more stable but myopic, focusing only on immediate rewards. In long-horizon problems, improper γ can cause vanishing reward signals or overly noisy Q-targets. Proper tuning is critical for convergence and stability.