How do you evaluate a Q-Learning agent beyond average reward?

Q-Learning agents should be evaluated using stability, sample efficiency, robustness, and policy quality—not just average reward.

Average episodic reward is insufficient because it hides instability, variance, and brittleness. Proper evaluation includes convergence speed, variance across seeds, robustness under environment perturbations, regret, and sample efficiency. In high-stakes systems, worst-case performance is often more important than mean performance. Evaluation should also include off-policy evaluation when deploying learned policies in real environments.