How does Q-Learning behave under partial observability (POMDPs)?

Standard Q-Learning performs poorly under partial observability because it assumes full state visibility.

In POMDPs, the agent receives incomplete or noisy observations instead of true states. This violates Markov assumptions, causing Q-values to become inconsistent. Solutions include using recurrent neural networks (DRQN), belief state estimation, or stacking observation histories to approximate hidden state information.