Chapter 13Asking well
Ask, then ask again
Treat the first answer as a draft. Put what matters most where it will be used, hand over your sources, and keep the conversation going.
- Put the key instruction at the start or the end. In tests, models used information in the middle of a long input less reliably than information at the start or end.1
- Hand it your sources. When the facts are in front of it, it doesn’t have to recall them. In a 2020 study, a model that looked up relevant documents and read them along with the question gave more specific and factual answers than one without that step.2
- Ask it to ask you. “Before you start, ask me anything you need to know.” Its questions show you what it would otherwise guess.
- Say what to change. A follow-up such as “shorter, and lead with the deadline” costs a few seconds and keeps what already works.
How this book did it. Its chapters were drafted with AI help (see Appendix E) from instructions that named the reader, the research standard and the examples to use, then checked claim by claim and revised.
Go deeper: showing the steps
Showing a few worked examples that write out their steps, so the model writes out its own, improved reasoning on some math, commonsense and symbolic problems, but only in very large models, of about 100 billion parameters.3
Check in
Try answering in your own words, then open each one to compare.
What are the parts of a good request?
The job, who it’s for, background, an example, limits and the shape you want. Each part you leave out is something the model has to guess.
Where do the most important instructions go in a long request?
At the start or the end. Models use information buried in the middle of a long input less reliably.
A good request gets you a better answer, but not a checked one. Next: how to check it. On to Chapter 14: Check before you trust →
Notes (3)
- 1Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang, “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics 12 (February 23, 2024): 157–173, https://doi.org/10.1162/tacl_a_00638. Cited at abstract; p. 1.
- 2Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (preprint, arXiv, May 22, 2020), https://doi.org/10.48550/arXiv.2005.11401. Cited at abstract; p. 1.
- 3Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” in Advances in Neural Information Processing Systems 35 (2022), 24824–24837, https://doi.org/10.52202/068431-1800. Cited at §3.2; p. 4.
Chapter 13 · page 2 of 2
Turn pages with the ← and → keys.