Skip to the lesson

How AI worksLesson 1 of 6

Part 1 of 3, 0 done

13 min

How a chatbot picks its next word

A chatbot doesn't look up your answer: it writes a likely next piece of text, then another, over and over. Watch it happen, and see why the same question can get two different answers.

Ticket 01 · The guess

How many pieces to choose from?

Your phone's keyboard suggests the next word as you type. A chatbot runs on a language model that does something similar, on a much bigger scale: at every step it picks the next piece of text, called a token, from a fixed list. GPT-2, a model OpenAI published in 2019, had a list of how many tokens?

100,000
0200,000
Words to know

Spot something wrong?

Tell us what looks off. No name or email needed.

What kind of problem?

0 / 1000

We'll also send this page, the lesson, ticket 1, so we can find it. Nothing else. Privacy

·
Sources and every number (4)

Sources

  • Language Models are Unsupervised Multitask Learners (Radford, Wu, Child, Luan, Amodei and Sutskever; section 2.3, the vocabulary) · OpenAI, 2019
  • Understand and count tokens (Gemini developer documentation) · Google, 2026
  • Prompt design strategies: temperature, topK and topP (Gemini developer documentation) · Google, 2026
  • NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (July 2024), section 2.2 Confabulation · National Institute of Standards and Technology, 2024

Every number

  • 50,257openai.com ·2019 · The number of tokens on GPT-2's list (its vocabulary), from OpenAI's paper describing the model.
  • 100 tokensgoogle.com ·2026 · The token count in Google's rule of thumb for English text with its Gemini models.
  • 60–80 wordsgoogle.com ·2026 · About how many English words a hundred tokens make, by Google's rule of thumb for its Gemini models.
  • 70 wordsexample (example number: Worked out: the middle of Google's range.) · Worked out: the middle of Google's range.
  • 70 wordsexample (example number: Simplified example: a short reply, exactly one batch.) · Simplified example: a short reply, exactly one batch.
  • 100 tokensexample (example number: Worked out from the example: one batch of words is about a hundred tokens.) · Worked out from the example: one batch of words is about a hundred tokens.
  • 150 wordsexample (example number: Simplified example: one chatbot reply.) · Simplified example: one chatbot reply.
  • 214 tokensexample (example number: Worked out from the example: the reply's words divided by seventy, times a hundred.) · Worked out from the example: the reply's words divided by seventy, times a hundred.
  • 188 tokensexample (example number: Worked out from the example: the reply's words at eighty words for every hundred tokens.) · Worked out from the example: the reply's words at eighty words for every hundred tokens.
  • 250 tokensexample (example number: Worked out from the example: the reply's words at sixty words for every hundred tokens.) · Worked out from the example: the reply's words at sixty words for every hundred tokens.
  • 1example (example number: Simplified example: the lab's starting randomness.) · Simplified example: the lab's starting randomness.
  • 0example (example number: Simplified example: no randomness, so the top-scoring token always wins.) · Simplified example: no randomness, so the top-scoring token always wins.
  • 2example (example number: Simplified example: the lab's highest randomness.) · Simplified example: the lab's highest randomness.
  • 5 tokensexample (example number: Simplified example: all five of the lab's tokens may be picked.) · Simplified example: all five of the lab's tokens may be picked.
  • 3 tokensexample (example number: Simplified example: only the three highest-scoring tokens may be picked.) · Simplified example: only the three highest-scoring tokens may be picked.
  1. Lessons›
  2. The school›
  3. How AI works
  • ·Next: One neuron, one decision