AhaBulbStuff Everyone Should Know
AhaBulb: Stuff Everyone Should Know · ahabulb.com/ai-distilbert-numbers

DistilBERT made BERT 40 percent smaller. About how much of its language understanding did it keep?

Start playing free

  1. About 97 percent
  2. About 60 percent
  3. About 40 percent

🔒 The answer is for AhaBulb members. Members see the answer, why it's right and the official sources for every card of Stuff Every AI Expert Should Know, plus timed practice exams.

The first 50 cards of every test are free to play.

Play the free cards See membership

Like this card?
Play the game

References

  1. Sanh et al., DistilBERT, a distilled version of BERT (arXiv 1910.01108) arxiv.org/abs/1910.01108 Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceIn Hinton's knowledge distillation, how does the big teacher model produce soft targets for the student?Artificial IntelligenceHow did BERT's pretraining differ from a left-to-right GPT-style model?Artificial IntelligenceHow do AI, machine learning and deep learning fit together?

That's stuff everyone should know.