steve sparks
18 of 42
ContentsPart III · Chapter 8

Chapter 8The prompt

What the model actually reads

A prompt is your message, but the model actually receives a whole stack of text: rules from the provider, instructions from the app, memories, documents, tool descriptions and earlier results. Your message is one small layer.

The model sees only what the software around it sends. Each time it responds, that software assembles one long document, the context, and the model continues it.1 Much of that document is written by someone other than you.2

What the model actually reads

Illustration

A stack of seven layers the model reads, top to bottom: system prompt, app instructions, tool descriptions, memory and documents, conversation so far, earlier tool results, and your message last.

  1. System promptwritten by the model's providerCore rules and persona.
  2. App instructionswritten by whoever built the appe.g., "You are a public library's help-desk assistant. Answer questions about hours, holds and library cards."
  3. Tool descriptionswritten by the harnessWhat the model can ask for, such as "search the catalog."
  4. Memory and documentswritten by the harnessSaved preferences, files, search results.
  5. Conversation so farwritten by the harnessSometimes summarized to save space.
  6. Earlier tool resultswritten by the harnessWhat came back from tools used earlier.
  7. Your messagewritten by you"Is the library open on Sunday?"

All of it goes to the model as one long document. The reply comes back below it.

Everything above your message was written by someone other than you. Order simplified; real products arrange layers differently.

The same desk from Chapter 2 holds all of these layers, so the harness has to choose what fits.

Go deeper

Anthropic, for example, publishes the system prompt it uses in its own Claude apps, and notes that those prompts don't apply to its API.2 OpenAI's published rules rank the layers by authority (platform rules above developer instructions above your message) and treat text returned by tools as information, not orders.3 A good prompt still matters. It's just one layer of many.

Facts

  • Anthropic defines context as the set of tokens the model receives when it generates a reply, and calls managing it "context engineering."1
  • Even a tool's status message is part of the prompt: in one benchmark, "Code executed successfully with no output" confused agents into retrying or assuming the tool was broken.4
  • Looking up relevant documents and adding them to the prompt (retrieval-augmented generation) makes answers more specific and factual.5

Read the rules yourself

  • Anthropic's published system prompts.2
  • OpenAI's Model Spec, the rules its models are trained to follow.3
  • Anthropic's constitution for Claude, describing the values it intends Claude to have.6
Notes (6)
  1. 1Anthropic, “Effective context engineering for AI agents,” Anthropic Engineering, September 29, 2025, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents (accessed October 26, 2026).
  2. 2Anthropic, “System prompts (release notes),” Claude Platform Docs, https://platform.claude.com/docs/en/release-notes/system-prompts/overview (accessed October 26, 2026).
  3. 3OpenAI, “OpenAI Model Spec (2025-10-27 version),” October 27, 2025, https://model-spec.openai.com/2025-10-27 (accessed October 26, 2026).
  4. 4JV Roig, “Towards a Standard, Enterprise-Relevant Agentic AI Benchmark: Lessons from 5.5 billion tokens' worth of agentic AI evaluations” (preprint, arXiv, November 11, 2025), https://doi.org/10.48550/arXiv.2511.08042.
  5. 5Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (preprint, arXiv, May 22, 2020), https://doi.org/10.48550/arXiv.2005.11401.
  6. 6Anthropic, “Claude's Constitution,” n.d., https://www.anthropic.com/constitution (accessed October 26, 2026).

Chapter 8 · page 1 of 1

Turn pages with the ← and → keys.

Cite this page

https://aifromthepartsup.com/the-prompt/1/

Using this book

© 2026 Steve Sparks. All rights reserved. You're welcome to quote short passages and cite any page, with a link back. To reuse diagrams or larger parts, ask through GitHub.

Quoted material and figures from other sources remain their owners'.