Chapter 5A family of sizes
Matching the job to the size
Go back to the office manager’s three jobs and reason through each one. A worked example, written for this page.
- Tagging two thousand emails. The job is narrow and repetitive, and it comes in volume. Start at the bottom rung. Test a small model on fifty emails whose right tags you already know. If it gets them right, keep it; if not, move up one step and test again.
- The careful reply to an upset client. This is everyday writing, where tone matters and a person reads every word before it goes. A mid-size model is a sensible start; the manager edits the draft either way.
- The partnership agreement. The document is long, the reasoning takes many steps, and a missed clause could cost the client money. This is work for a large model, or a higher effort setting. Even then, someone who knows tax law checks the list against the agreement itself.
- A fourth job she might add: sorting files that hold clients’ tax ID numbers. The job is narrow, but the data is sensitive. A small model downloaded to the firm’s own computer could do it without the files leaving the building, if the firm’s rules allow it and a test shows it works.
Notice the questions behind each choice. How narrow is the job? How much is at stake if the answer is wrong? Does the model need facts it would have to remember, or are the facts in front of it? Must the data stay private? And will a person check the result? The biggest model is the right answer to some of those questions. It is not the answer to all of them.
Your turn
Try it. If an AI tool you use lets you pick a model or a “thinking” mode, give the same two jobs to its fastest option and its most capable one. First, write ten made-up email subject lines and ask each model to sort them into three kinds you choose. Second, ask each a factual question you can check, about your town, your field or your work. Compare the answers and how long you waited. Did the small model keep up on the sorting? Did it slip on the facts? On paper instead: list five jobs from your week you might hand to an AI tool, mark each small, middle or large, and write one reason beside each.
Think about it. Think of a task at your work or at home that you would gladly hand to a machine if your words never left your own computer. Is it narrow enough for a small model, or does it need facts the model would have to remember?
So AI companies sell a family of sizes because capability costs computing: each step up the ladder answers harder questions, more slowly and at a higher price, and the top step may come with extra safeguards. Most jobs need a rung well below the top, and a small model handed the right facts can do a narrow job well. But whatever its size, a model can work only with what it is given. What exactly does it read before it answers? That is the prompt, and it opens the next part of the book.
Check in
Try answering in your own words, then open each one to compare.read the answer to compare.
Why do AI companies sell several sizes of model instead of only their most capable one?
A bigger model is more capable, but every token it writes takes more computing, so it is slower and costs more to run. Most jobs don’t need the top model, so companies offer a ladder and let each job use the rung it needs.
Chapter 2 showed that a model writes one token at a time. Why does that make a long, step-by-step answer cost more than a short one, even from the same model?
Every token means running the model again, so more tokens mean more computing. A model that thinks at length before answering writes many more tokens, which is why an effort or “thinking” setting trades speed and cost for depth.
An HR manager has ten thousand survey comments to sort into five themes, and the comments contain employees’ names. Which kind of model would you try first, and how would you know it works?
A small model, since the job is narrow, repetitive and high-volume; a downloaded one running on the company’s own computer would also keep the names in-house, if company policy allows it. Test it first on a sample of comments whose themes you already know, and move up a rung only if it falls short.
A coach asks a small model on her laptop, with no internet connection, who won a regional championship twenty years ago. What is the risk, and what would help?
Small models store few facts and may give wrong or outdated ones with confidence. Better to hand the model the facts, such as a page of results pasted in, or to check the answer against a trusted record.
A manager says, “Just use the biggest model for everything. It’s the safe choice.” Weigh that advice.
It is safe only in a narrow sense. The biggest model may do best on long, hard work, but on simple, high-volume jobs it can be slower and costlier, and use more energy, for no better result. Size isn’t the only thing that matters, either: a smaller, well-trained model can beat a much larger one. A better rule is to test the job on a smaller rung first and move up only when the results fall short, keeping a person checking wherever mistakes would be costly.
End of Part II, The roots.Next: Chapter 6: The prompt →
Chapter 5 · page 4 of 4
Turn pages with the ← and → keys.