How does Q-Learning perform under high-dimensional observation spaces?

Q-Learning struggles in high-dimensional observation spaces due to sample inefficiency and representation challenges.

When observations are high-dimensional (e.g., images), the agent must learn both representation and value estimation. This increases sample complexity and makes exploration harder. Deep Q-Networks address this using convolutional architectures, but still suffer from instability and overfitting in complex environments.