What is evaluation contamination in LLM benchmarks?

It occurs when evaluation data appears in training corpora of LLMs.

LLMs trained on large web-scale datasets may inadvertently include benchmark datasets, leading to memorization rather than true generalization. This undermines benchmark validity and inflates reported performance.