Chapter 1What is a language model?
Autocomplete that read a library
A large language model is a program that has learned, from an enormous amount of text, to predict what comes next. That one skill turns out to cover answering, writing, summarizing and coding.
You type words; it writes words back. It doesn't look your answer up in a database the way a search engine does. It produces each reply fresh, one small piece at a time, from patterns it learned during training.12 That's why it can explain an expense policy in plain English or draft an email in your voice. It's also why it can sound sure of itself and still be wrong: it is generating plausible text, not checking facts.3
The model is the brain. The prompt is the briefing. The harness is the body and workshop. The tools are the hands.
Go deeper
"Large" refers to size. GPT-3, for example, had 175 billion internal settings, called parameters, tuned during training.1
Facts
- GPT-3 (2020) had 175 billion parameters.1
- Anthropic lists context windows of up to 1 million tokens for its current Claude models.4
- Models can state false things confidently. Researchers trace this partly to training and testing that reward guessing over saying "I don't know."3
Watch and read
Notes (5)
- 1Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, et al., “Language Models are Few-Shot Learners” (preprint, arXiv, May 28, 2020), https://doi.org/10.48550/arXiv.2005.14165.
- 2G. Sanderson (3Blue1Brown), “Large Language Models explained briefly,” 3Blue1Brown, made for the Computer History Museum, November 20, 2024, https://www.youtube.com/watch?v=LPZh9BOjkQs (accessed October 26, 2026).
- 3Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang, “Why Language Models Hallucinate” (preprint, arXiv, September 4, 2025), https://doi.org/10.48550/arXiv.2509.04664.
- 4Anthropic, “Models overview,” Claude Platform Docs, https://platform.claude.com/docs/en/models/overview (accessed October 26, 2026).
- 5FT Visual Storytelling Team and M. Murgia, “Generative AI exists because of the transformer,” Financial Times, September 12, 2023, https://ig.ft.com/generative-ai/ (accessed October 26, 2026).
Chapter 1 · page 1 of 2
Turn pages with the ← and → keys.