How to check (Chapter 14, page 2 of 2) · AI, from the Parts Up

Match the check to the claim: redo the math, open the source, search the original, look it up elsewhere, or ask a person who knows. Check more when more is at stake.

How would you check it?

Illustration

Interactive Pick a claim, then pick how you would check it.Try it on the web: aifromthepartsup.com/check-before-you-trust/2/

Different claims need different checks.

1. Pick a claim

2. How would you check it?

Now pick an answer.

“Travel spending was $48,210 this quarter, up 12% from last quarter.” Redo the math. You have the spreadsheet, so recompute it. A language model predicts the next token, digits included; a total that reads well can still be off.
“Section 4.2 of the employee handbook gives new hires 15 days of paid time off.” Open the source it names. It names its source, so open section 4.2 and read it. A real section number doesn’t make the sentence in it real.
“A 2019 decision, Smith v. Riverside Co., settled this question.” Open the source it names. Find the decision itself. Courts have sanctioned lawyers for filing decisions that never existed.¹
“The 40-page vendor contract has no termination clause.” Search the original document yourself. Claims that something is missing are the hardest to check. Search the contract yourself for “terminate” and “termination.”
“The spring soccer season starts April 4.” Look it up somewhere you trust. Check the league’s own schedule. Dates are easy to state and easy to get wrong.
“This applicant is the strongest fit for the HR role.” A person who knows decides. That is a judgment about a person, not a fact to look up. A person who knows the job decides (Chapter 16).

Examples written for this page, not things an AI tool said. The names, numbers and the court case are made up.

Read sideways. In a study of professional fact-checkers, Ph.D. historians and Stanford students judging websites, the fact-checkers did best: they “read laterally, leaving a site after a quick scan and opening up new browser tabs” to see what others said about it, and reached better-supported conclusions in a fraction of the time.² The same habit works on an AI tool’s answer: check it somewhere else.

Match the effort to the stakes. A dinner suggestion needs no checking; a number going to your board, a quote in a report or anything about a person’s health or job needs all of it. Chapter 12’s sensitivity levels are a good guide.

How this book did it. Every claim was graded before it shipped, and grade D, seen only in a snippet or summary, never ships. For the newest chapters, the full text of every source was obtained and read (the papers, the court order, the laws themselves), every quote was matched word for word, and each claim was checked three times: against an independent record of the source, in the source itself, and by a second reader who opened the source cold. See Appendix E.

Reading settings

How to check

How would you check it?

Check in

Cite this page

Using this book