A worked example

Here is the whole method on six claims, of the kind AI tools give an accountant, an HR manager, a parent and others every day. These are examples written for this page, not things an AI tool said. The names, numbers and the court case are made up. For each claim, the question is the same: what kind of claim is this, and what would settle it?

• “Travel spending was $48,210 this quarter, up 12% from last quarter.” Redo the math. The spreadsheet is at hand, so recompute both numbers. A right total can sit beside a wrong percent.
• “Section 4.2 of the employee handbook gives new hires 15 days of paid time off.” Open the source. Turn to section 4.2 and read it. The section can exist and still say something else.
• “A 2019 decision, Smith v. Riverside Co., settled this question.” Open the source. Find the decision itself in the court’s records or a legal database, and read the part that matters. A case nobody can find doesn’t count.
• “The 40-page vendor contract has no termination clause.” Search the original. This is the hardest of the six. Search the contract for terminate, termination, cancel and notice, and read any section about ending the agreement.
• “The spring soccer season starts April 4.” Look it up elsewhere. The league’s own schedule is the authority. Dates are easy to state and easy to get wrong.
• “This applicant is the strongest fit for the HR role.” A person decides. No lookup settles it. Someone who knows the job weighs the applicants and can explain the choice.

Notice two things. If all six landed on one desk, they wouldn’t get equal care: the contract and the applicant matter most and go first, and the soccer date gets a quick look. And none of the six is checked by asking the tool, “Are you sure?” That only asks it to predict another plausible reply.

Your turn

Try it. Pick a subject you know well, from your work, your town or a hobby. Ask any AI chat tool you already use for a short summary of it with three sources. Then sort its answer into claims, as in Figure 12.1, and give each claim its check: redo any number, open each source and find the words, look up one date somewhere else. Count how many claims pass. No AI tool at hand? Do the same on paper with something you received recently, such as a report, a newsletter or a long email. Underline every factual claim and write in the margin the check it needs. Or test this chapter: pick one of its notes, find the source, and see whether it says what the chapter says it does.

Think about it. Think of the last piece of work you sent that leaned on something you hadn’t checked yourself, whether it came from an AI tool or a person. Which claim in it would have cost the most if it had been wrong? What would checking it have taken?

So how can you tell whether an AI tool’s answer is right? Not by how it sounds, because a model sounds the same when it is guessing. You tell by checking: read the answer as separate claims, give each the check that fits, and spend the most care where a mistake would cost the most. A checked answer is nearly ready to use. What is left is what you do before it goes to someone else: cutting it down, keeping private things out and saying how AI helped. That is the next chapter.