What is the role of stochasticity in Q-Learning environments?

Stochastic environments introduce randomness in transitions and rewards, increasing learning uncertainty.

In stochastic environments, taking the same action in the same state can lead to different outcomes. Q-learning must therefore learn expected values rather than deterministic outcomes. This increases variance in updates and requires more samples for stable convergence.