How does Q-Learning interact with exploration randomness and deterministic policies?

Q-Learning learns deterministic optimal policies but relies on stochastic exploration during training.

During training, exploration strategies like epsilon-greedy introduce randomness to ensure state-action coverage. However, the learned optimal policy is deterministic, selecting the highest Q-value action. This separation allows efficient learning while ensuring convergence to a stable policy, assuming sufficient exploration.