How does curse of dimensionality impact reinforcement learning state spaces?

It makes exploration and value estimation intractable.

In reinforcement learning, state spaces grow exponentially with features. This leads to sparse visitation of states, making Q-value estimation unreliable. Function approximation (deep RL) is required to generalize across unseen states.