What is the role of replay buffer sampling distribution in learning bias?

Replay buffer sampling distribution determines which experiences influence learning and can introduce bias if not uniform.

Uniform sampling assumes all transitions are equally important, but this is often inefficient. Prioritized replay introduces bias toward high TD-error samples, improving learning speed but requiring correction via importance sampling. If uncorrected, the learned Q-function may become skewed toward overrepresented experiences.