AhaBulbStuff Everyone Should Know
AhaBulb: Stuff Everyone Should Know · ahabulb.com/ai-distillation-soft-targets

In Hinton's knowledge distillation, how does the big teacher model produce soft targets for the student?

Start playing free

  1. By deleting half of its layers
  2. By giving only its single top answer
  3. By raising its softmax temperature

🔒 The answer is for AhaBulb members. Members see the answer, why it's right and the official sources for every card of Stuff Every AI Expert Should Know, plus timed practice exams.

The first 50 cards of every test are free to play.

Play the free cards See membership

Like this card?
Play the game

References

  1. Hinton, Vinyals and Dean, Distilling the Knowledge in a Neural Network (arXiv 1503.02531) arxiv.org/pdf/1503.02531 Checked 2026-10-11.
  2. Google Machine Learning Glossary: Generative AI (distillation) developers.google.com/machine-learning/glossary/generative Checked 2026-10-11.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceDistilBERT made BERT 40 percent smaller. About how much of its language understanding did it keep?Artificial IntelligenceHow do AI, machine learning and deep learning fit together?Artificial IntelligenceWhat did Alan Turing call his 1950 test of machine intelligence?

That's stuff everyone should know.