AhaBulbStuff Everyone Should Know
AhaBulb: Stuff Everyone Should Know · ahabulb.com/ai-pretraining-first-phase

What kind of learning is usually used in the first, pretraining phase of an LLM?

Start playing free

  1. Humans label every sentence first
  2. Unsupervised learning on raw text
  3. Reinforcement learning from human ratings

🔒 The answer is for AhaBulb members. Members see the answer, why it's right and the official sources for every card of Stuff Every AI Expert Should Know, plus timed practice exams.

The first 50 cards of every test are free to play.

Play the free cards See membership

Like this card?
Play the game

References

  1. Google ML Crash Course: LLMs and Transformers developers.google.com/machine-learning/crash-course/llm/transformers Checked 2026-10-10.
  2. Brown et al., Language Models are Few-Shot Learners (arXiv 2005.14165) arxiv.org/pdf/2005.14165 Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat does transfer learning reuse from a previously trained model?Artificial IntelligenceHow did BERT's pretraining differ from a left-to-right GPT-style model?Artificial IntelligenceHow do AI, machine learning and deep learning fit together?

That's stuff everyone should know.