The whole loop, step by step

Start with the cast. The model is the trained program that predicts text: the brain. The prompt, also called the context, is everything the model reads for one reply: the briefing. The harness is the software around the model that builds the briefing, runs the loop and asks your permission: the body. The tools are actions the harness can take when the model asks, such as searching the web, opening a file or sending an email: the hands. And there is you. In this picture you only ever talk to the harness, never to the model directly. Figure 9.1 shows all five, with the steps numbered.

Try it: step through the whole loop, or choose a part to see what it does.

1. You ask. You type a request into an app: a chat window, a coding tool, an assistant inside your email. Whatever it looks like, the app is the harness.

2. The harness builds the briefing. Before the model sees anything, the harness assembles one long document. It holds instructions from the model’s maker and the app’s builder, a list of the tools the model may ask for, any saved notes, the conversation so far, earlier results and, last, your message.¹² Chapter 6 called this the stack. Most of it was written by someone other than you.

3. The model predicts a reply. The model reads the whole briefing and predicts what comes next. The reply takes one of two forms. It is either an answer for you, or a structured request for a tool, such as “read the file invoices-june.xlsx.” The model cannot carry the request out. Anthropic’s documentation sums it up in eight words: “The model never runs anything on its own.”³⁴

4. The harness checks, asks and acts. For a tool request, the harness takes over. It checks its rules: is this tool allowed, and is this step risky? If the rules call for it, it asks you first, as some harnesses do, for example, before deleting files.⁵ Then it runs the tool. For some tools, such as web search, the provider’s own servers do the running.³

5. The result goes back in. Whatever the tool returns is added to the briefing as more text: a page of search results or the contents of a file.³⁶ In this book’s examples, a refusal goes back the same way, as a note that you said no. The model reads the longer briefing and predicts again. It may ask for another tool, and another. Steps 3, 4 and 5 repeat until the model writes an answer instead of a request. Researchers described this pattern of reasoning, acting and observing in 2022 work called ReAct.⁷

6. The answer comes back. When the model’s reply is an answer, the harness shows it to you. If the app keeps memory, the harness may also save notes from the chat, to add to a later briefing.⁸ The model itself learns nothing from you along the way. Using a model doesn’t retrain it: when GPT-3’s developers gave it new tasks, they did it “purely via text interaction with the model,” without changing its parameters.⁹

Lay the lawyer’s case over Figure 9.1. Steps 1, 2, 3 and 6 happened, over and over. In the court’s account, nothing like steps 4 and 5 appears: no lookup, no result, nothing new for the model to read. The checks that worked happened outside the loop, in his own research: he looked for Varghese and could not find it, and later could not locate another of the cases.¹⁰

Three more things follow from the loop. A model that uses tools may take several turns for every one of yours. When it decides its own next steps until a task is done, people call the whole arrangement an agent.¹¹ Every tool result is text the model reads, so a web page or document can try to slip in orders of its own, a trick called prompt injection.¹² That is one reason step 4 matters; Chapter 8 tells how harnesses defend against it. And at every step your words move: into the briefing, to the company that runs the model, into a tool. Chapter 10 follows them.