Chapter 7The harness
One brain, many bodies
If the harness matters this much, the same model should score better or worse depending on the harness around it. Researchers have measured this. A benchmark is a fixed set of tasks used to compare systems. In a 2026 benchmark of research tasks, one model, Claude Opus 4.7, succeeded on 62.2% of the tasks in the Claude Code harness, 64.6% in Hermes Agent and 68.3% in a minimal harness called Mini-SWE-Agent.1 In leaderboard results reported by Zhang and colleagues, Claude Opus 4.5 scored 45.9% on a coding benchmark in one standard harness and 55.4% in Claude Code.2 And a 2026 study that ran seven harnesses with ten models underneath found that “no harness performs best across all ten backends,” meaning those models.3
Zhang and colleagues argue that swapping the harness can produce changes that “routinely dwarf the 2 to 4 percentage point shifts that papers report as meaningful model advances.”2 The model still matters, too. In the research benchmark, results dropped significantly with lower-tier models, and the authors concluded that “the intrinsic reasoning capabilities of the underlying model remain the primary bottleneck.”1 Both parts count. So when someone says “Model X scored 70%,” the first question to ask is: in which harness?
One way to picture it: the same pilot can fly a two-seat plane, an airliner or a fleet of drones. The aircraft decides which controls exist, and the tower decides what needs approval. The model is the pilot, the harness is the aircraft, its controls are the tools, and the tower’s approval is the harness asking your permission. The comparison stops at responsibility. A pilot trains for each aircraft and answers for the flight. A model goes into any harness unchanged, and in the Air Canada decision it was the company running the chatbot that answered for what it said.
Try it: send the same request through four kinds of harness and see what changes.
Your turn
Try it. Pick an AI tool you use at work or at home, and find out what its harness does. Look through its settings, and ask the tool itself: Can you search the web? Can you see my files, or only what I paste? Do you keep a memory of past chats? Will you ask me before you change or send anything? The model may not know its own harness, so check one answer yourself: open the memory setting, or share a file and see whether it can read it. On paper, make a table with one row for each of the six jobs in Figure 7.1, and fill it in for two tools you use, such as a chat app on your phone and an assistant inside your email. Where the rows differ, you have found why the same request gets different results.
Think about it. Think of an AI tool that your workplace, your school or a company you deal with has put in front of people. Who chose what it can see and do? If it told someone something wrong, who would answer for it?
So how much of what happens is the model, and how much is the software around it? The model writes the words. The harness decides what it reads, what it remembers, what it may do, when it asks you and when it stops. That is why the same model can feel like a chatbot in one app and a coworker in another. One of the harness’s jobs reaches beyond the harness itself: the tool router. How does a program that only writes text search the web or open a file? It asks, and the harness does the work. That is the next chapter.
Check in
Try answering in your own words, then open each one to compare.read the answer to compare.
What is a harness, and what are three of its jobs?
The software around a model. Any three of these: it builds the context, runs the loop, keeps memory, asks your permission, runs the tools the model asks for and decides when the job is done.
Chapter 6 showed that the model reads a whole stack of text, in which your message is only one layer. Which of the harness’s jobs puts that stack together?
The context builder. Before every reply it assembles the instructions, documents, saved memory, the conversation so far and your message into the context the model reads.
A coworker says, “The assistant in our office software and the chat app on my phone use the same model, so they’ll know the same things about our files.” What would you tell them?
The model may be the same, but the harnesses differ. Only a harness that can reach your files, and has permission to, can add them to what the model reads. Check what each tool can see.
A report says Model X scored 68% on a benchmark and Model Y scored 62%. Before you decide that X is the better model, what would you ask, and why?
In which harness was each one tested? In one 2026 benchmark, the same model scored 62.2% in one harness and 68.3% in another, a gap as large as this one.1 A fair comparison runs both models in the same harness.
A small bakery plans to put an AI chatbot on its website to answer questions about orders and refunds. Thinking of the Air Canada story, what would you check before it goes live, and why?
First, what the context builder gives it to read: the bakery’s current policies, so that its answers match the rest of the website. Then what it may do on its own, and when it asks a person first. In the Air Canada decision, the tribunal held the company responsible for what its chatbot told a customer.4
Notes (4)
- 1Jiayu Wang, Weijiang Lv, Bowen Fu, Jing Fu, Jiayi Song, Lingyu Zhang, Lanxuan Xue, et al., “Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle” (preprint, arXiv, June 5, 2026), https://doi.org/10.48550/arXiv.2606.07462. Cited at table 2.
- 2Yunbei Zhang, Janet Wang, Yingqiang Ge, Weijie Xu, Jihun Hamm, and Chandan K. Reddy, “Stop Comparing LLM Agents Without Disclosing the Harness” (preprint, arXiv, May 7, 2026), https://doi.org/10.48550/arXiv.2605.23950.
- 3Yilun Yao, Xinyu Tan, Chao-Hsuan Liu, Yaoming Li, Zhengyang Wang, Zongwei Lv, Wenhan Yu, et al., “Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows” (preprint, arXiv, October 15, 2026), https://doi.org/10.48550/arXiv.2605.27922.
- 4Civil Resolution Tribunal (Christopher C. Rivers, Tribunal Member), “Moffatt v. Air Canada, 2024 BCCRT 149: Reasons for Decision,” Small Claims Decisions, February 14, 2024, https://decisions.civilresolutionbc.ca/crt/crtd/en/item/525448/index.do (accessed October 27, 2026). Cited at para. 27.
Chapter 7 · page 4 of 4
Turn pages with the ← and → keys.