AhaBulbStuff Everyone Should Know
AhaBulb: Stuff Everyone Should Know · ahabulb.com/ai-benchmark-contamination

What is benchmark contamination for a language model?

Start playing free

  1. A virus that corrupts the scores
  2. Users typing rude prompts
  3. Test questions slipping into the training data

🔒 The answer is for AhaBulb members. Members see the answer, why it's right and the official sources for every card of Stuff Every AI Expert Should Know, plus timed practice exams.

The first 50 cards of every test are free to play.

Play the free cards See membership

Like this card?
Play the game

References

  1. Brown et al., Language Models are Few-Shot Learners (arXiv 2005.14165) arxiv.org/pdf/2005.14165 Checked 2026-10-10.
  2. Yang et al., Rethinking Benchmark and Contamination for Language Models with Rephrased Samples (arXiv 2311.04850) arxiv.org/abs/2311.04850 Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceResearchers wrote GSM1k, fresh math problems like GSM8K. What happened to some models' scores?Artificial IntelligenceThe MMLU benchmark tests language models across how many subjects, from math to law?Artificial IntelligenceA 2025 paper by OpenAI researchers says hallucinations persist partly because most evaluations reward what?

That's stuff everyone should know.