What is Prioritized Experience Replay in Deep Q-Learning?

Prioritized Experience Replay samples important transitions more frequently based on TD-error.

Instead of sampling uniformly from the replay buffer, transitions with higher temporal-difference (TD) error are sampled more often because they carry more learning signal. This improves sample efficiency and speeds up convergence. However, it introduces bias, which is corrected using importance sampling weights.