How does Q-learning behave in non-stationary environments?

Q-learning struggles in non-stationary environments because the transition dynamics change over time.

Standard Q-learning assumes a stationary environment. In non-stationary settings, the reward distribution or transition probabilities change, causing outdated Q-values to become misleading. Techniques like continual learning, adaptive learning rates, or meta-learning are required to handle such settings.