What are the limitations of Q-learning in high-dimensional environments?

Q-learning suffers from scalability, instability, and sample inefficiency in high-dimensional spaces.

As dimensionality increases, Q-tables become infeasible and even deep Q-networks struggle due to sparse rewards, unstable gradients, and inefficient exploration. This is known as the curse of dimensionality in reinforcement learning.