How does Q-Learning handle delayed rewards?

Q-Learning handles delayed rewards through temporal difference updates that propagate reward backward over time.

Delayed rewards create a credit assignment problem. Q-learning solves this using bootstrapped updates where future rewards are gradually propagated backward through visited states. However, long delays slow convergence significantly and may require reward shaping or eligibility traces for efficiency.