4 of 42

Chapter 1What is a language model?

Where "LLM" fits

You'll hear several bigger words around "LLM." Each one sits inside the one before it.

Where “LLM” fits

Illustration

Machines doing things we'd call intelligent. The name comes from a 1955 proposal for a summer research project at Dartmouth.¹ Much early AI ran on rules that people wrote by hand.²³
Example: A game player that looks ahead and scores each move by rules, like Shannon's 1950 plan for a chess program² or this site's Tic-Tac-Toe. ELIZA chatted in 1966 using hand-written keyword rules.³
Instead of writing every rule, people let the computer learn patterns from data.⁴ Arthur Samuel put the phrase in the title of his 1959 paper on a checkers program.⁵
Example: A spam filter that learns from the mail people mark as junk, or a bank's fraud alert that learns what unusual spending looks like.
Machine learning built from many layers of simple math units that learn their own features from data.⁶
Example: A photo app that recognizes faces, a ballpark, or the Gateway Arch in your pictures.⁶
Models that make new content, such as text, images, audio or video, resembling what they learned from.⁷
Example: An image generator, or a tool that drafts a first version of an email.
Generative models trained on huge amounts of text to predict the next piece of text.⁸ Today's are built on a design called the transformer.⁹
Example: The models behind Claude and ChatGPT.¹⁰¹¹

Not all AI is machine learning. Shannon's chess plan, ELIZA and the Tic-Tac-Toe player all follow rules people wrote, the way a program in BASIC does exactly what its lines say.

A simplified picture. In practice the borders between these words are fuzzier.

Notes (11)

1J. McCarthy, M. L. Minsky, N. Rochester, and C. E. Shannon, A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence (John McCarthy's archive, Stanford University, August 31, 1955), https://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html (accessed October 26, 2026).
2Claude E. Shannon, “Programming a Computer for Playing Chess,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 41, no. 314 (March 1950): 256–275, https://doi.org/10.1080/14786445008521796.
3Joseph Weizenbaum, “ELIZA—A Computer Program for the Study of Natural Language Communication Between Man and Machine,” Communications of the ACM 9, no. 1 (January 1966): 36–45, https://doi.org/10.1145/365153.365168.
4NIST, “machine learning (glossary term),” NIST Computer Security Resource Center, https://csrc.nist.gov/glossary/term/machine_learning (accessed October 26, 2026).
5A. L. Samuel, “Some Studies in Machine Learning Using the Game of Checkers,” IBM Journal of Research and Development 3, no. 3 (July 1959): 210–229, https://doi.org/10.1147/rd.33.0210.
6Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, “Deep learning,” Nature 521, no. 7553 (May 2015): 436–444, https://doi.org/10.1038/nature14539.
7National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1) (NIST, July 26, 2024), https://doi.org/10.6028/NIST.AI.600-1.
8Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, et al., “Language Models are Few-Shot Learners” (preprint, arXiv, May 28, 2020), https://doi.org/10.48550/arXiv.2005.14165.
9Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin, “Attention Is All You Need” (preprint, arXiv, June 12, 2017), https://doi.org/10.48550/arXiv.1706.03762.
10Anthropic, “Models overview,” Claude Platform Docs, https://platform.claude.com/docs/en/models/overview (accessed October 26, 2026).
11OpenAI, “ChatGPT: Optimizing language models for dialogue (now 'Introducing ChatGPT'),” November 30, 2022, https://openai.com/index/chatgpt/ (accessed October 26, 2026).

Chapter 1 · page 2 of 2

Turn pages with the ← and → keys.