Chapter 4From early models to today
Which part changed?
The products in that list will be old news soon. A question that stays useful is this: when something new arrives, which part of the system changed? Chapter 1 named four parts. The model is the trained program that predicts the reply. The prompt is everything the model reads for one reply. The harness is the software around the model, which builds the prompt, remembers and asks your permission. The tools are the actions the harness can take when the model asks.
Here is a worked example, using five entries from this chapter. GPT-3, 2020: a bigger model, trained on more text. The model changed. Instruction training, March 2022: feedback from people changed how the model answered. The model changed again, through its training. ChatGPT, November 2022: what the timeline records is a chat window anyone could use. A chat window is part of the harness, the software around the model. MCP, November 2024: a standard way to plug outside systems into an AI assistant. The tools changed, and the model could stay the same. Reasoning models, September 2024: the model works through a problem before it answers. The model changed.
Notice the pattern. Some “new AI” news is a new model. Much of it is new software or new tools around a model that already existed. Spot it today: when you read about a new AI release, ask which part changed, the model, the prompt, the harness or the tools.
Your turn
Try it. Two short tests of this chapter’s claims. First, be ELIZA for a minute, on paper. Write three sentences that start with “I am,” then apply the rule from Weizenbaum’s paper: “I am BLAH” becomes “How long have you been BLAH.” Notice that you never needed to know what BLAH meant. Second, take any recent AI headline or vendor announcement and write down which of the four parts changed. If you use an AI chat tool, ask it, “When did language models begin, and what came before ChatGPT?” Check its dates against the ones in this chapter. Notice whether it searched for anything, or simply wrote.
Think about it. Weizenbaum’s secretary knew ELIZA was a program and still wanted privacy with it. Where in your work or home life do people treat a fluent tool as if it understood them? Which decisions there would you keep for a person, as Weizenbaum argued some should be?
In 1966 Weizenbaum wrote that once a program’s inner workings are explained plainly, “its magic crumbles away.”1 That is this chapter’s answer, too. Today’s AI did not appear out of nowhere in 2022. It is the latest stretch of a road that began with Shannon counting letters in 1948, and each stretch added one ability: learning from data, reading a passage at once, following instructions, and using tools to act. ELIZA, the program that made people feel understood, stands beside that road: rules, not prediction. Along the way, models came to be sold in families, from small to frontier. Why build several sizes instead of one best model? That is the next chapter.
Check in
Try answering in your own words, then open each one to compare.read the answer to compare.
How did ELIZA come up with the reply “YOUR BOYFRIEND MADE YOU COME HERE”?
It matched a pattern in the person’s sentence, kept the person’s own words, turned the pronouns around and printed the result. A rule a person wrote did the work. It didn’t predict anything, learn anything or know what a boyfriend is.
Shannon once asked people to guess a passage of English one letter at a time (Chapter 3). What does that experiment share with GPT-3?
Both rest on guessing what comes next from what came before. Shannon used people to measure how predictable English is; GPT-3 is a program trained to make that guess, with 175 billion parameters.
A colleague says AI “came out of nowhere in 2022.” Which dates from this chapter would you offer?
Any of these: Shannon’s next-symbol idea in 1948; the transformer in June 2017; GPT-3 in May 2020; instruction training in March 2022. ChatGPT, in November 2022, put a chat window on work that had been building for decades.
A news story announces “a new open standard that lets AI assistants connect to your company’s files and apps.” Which of the four parts changed?
The tools, and the harness that plugs them in. The model can stay exactly the same, as with MCP in November 2024.
Your office is choosing between a help-desk bot built on hand-written rules and one built on a language model. Name one strength and one risk of each.
Rules: every reply can be read and traced, but the bot has nothing useful to say about wording nobody wrote a rule for. Language model: it handles new wording fluently, but no one can read off why it answered as it did, and it can sound sure while being wrong. With either, people may credit it with more understanding than it has, as ELIZA’s users did.
Notes (1)
- 1Joseph Weizenbaum, “ELIZA—A Computer Program for the Study of Natural Language Communication Between Man and Machine,” Communications of the ACM 9, no. 1 (January 1966): 36–45, https://doi.org/10.1145/365153.365168. Cited at page 36.
Chapter 4 · page 5 of 5
Turn pages with the ← and → keys.