What is the role of exploration decay strategy design in Q-Learning performance?

Exploration decay controls the transition from exploration to exploitation and strongly impacts final policy quality.

A poorly designed decay schedule can either freeze exploration too early, leading to suboptimal convergence, or maintain excessive randomness, preventing convergence. Common strategies include linear decay, exponential decay, and adaptive methods based on performance plateaus. The goal is to ensure sufficient state-space coverage early while stabilizing learning later.