How does Q-Learning handle multi-step decision dependencies?

Q-Learning handles multi-step dependencies through bootstrapped temporal difference updates but struggles with long chains.

Each Q-update propagates reward information one step backward, meaning multi-step dependencies are learned gradually over many iterations. However, this propagation can be slow and inefficient for long horizons. Multi-step TD methods (n-step returns) and eligibility traces improve learning speed by directly incorporating longer reward trajectories into updates.