Skip to content

Make the agent remember with AGENTS.md

Full step  about 8 min

Because language models maintain no persistent internal state across independent sessions, launching a new conversation typically requires re-explaining directory structures, cleaning conventions, and project constraints. In this exercise, you observe this loss of context firsthand and resolve it by establishing an AGENTS.md configuration file.

Before you start: A project directory where key analytical decisions have already been made (Setup).

Background

When an interactive session terminates, the active conversational context is discarded (Context). The only way to preserve state across sessions is through persistent files.

An AGENTS.md file located in the project root is automatically ingested by most modern agent harnesses at session initialization, providing a standardized, portable mechanism for project memory (Memory).

Effective standing instructions must be concise and specific: documenting directory locations, active cleaning definitions, output paths, and strict negative constraints (such as never overwriting raw datasets). Because every line in AGENTS.md consumes context tokens on every turn, keep the file concise and focused.

General methodology

  1. Observe context loss: Launch a fresh agent session in the project directory and prompt it with a question that depends on prior decisions. Note how the agent either makes incorrect assumptions or asks for clarification on previously resolved conventions.
  2. Draft standing instructions: Instruct the agent to synthesize the established project conventions into AGENTS.md:
Create an AGENTS.md file in the project root (under 25 lines) documenting:
- Relative paths to raw data, cleaned datasets, and the data dictionary.
- Key data cleaning rules and cohort definitions established so far.
- Designated output directories for generated tables and figures.
- Explicit operational constraints (e.g., raw data is read-only; all numbers must report denominators).
Keep instructions concise and specific. Omit trivial details that can be derived directly from the code.
  1. Verify memory persistence: Launch another new session and re-issue the initial query from step 1. The agent should now execute the task adhering to the documented conventions without asking redundant questions.
  2. Refine instructions recursively: Direct the agent: "Review AGENTS.md and remove any vague generalities or redundant rules." Using the agent to audit its own instructions is a high-leverage maintenance practice (Steering).

Workshop example · Clinical: clinic network

Start from: the cleaned checkpoint (it has no AGENTS.md yet) (how to restore)
Produces: AGENTS.md

Prompt

Create AGENTS.md in this folder, under 25 lines, capturing: where the data and data dictionary live; the cleaning decisions in notes/cleaning-decisions.md; that original files are never modified and outputs go in out/; and that every reported number includes its row or patient count. Be specific. Leave out anything you could work out from the code.

Files used: notes/cleaning-decisions.md, data/clean/visits_clean.csv, scripts/clean_visits.py.

Step 1: see the problem. In a new session in this folder, before creating the file, ask:

Which clinics meet their HbA1c target? Use adult patients with diabetes and their most recent HbA1c.

You should see the agent do one of two things: read the raw clinic_visits.csv and ignore your cleaning, or stop to ask which file and rules to use. Either shows what is lost between sessions.

Step 3: test it. After creating AGENTS.md, start another new session and ask the same question. The agent should use data/clean/visits_clean.csv, report patient counts, and mention the rules without being told.

Reference answers on the cleaned data

Clinic Patients Share with most recent HbA1c < 8.0 Target
Hillcrest 479 57.6% 55%
Northside 296 65.5% 60%
Riverside 232 53.9% 60%

Compared with the raw first look (478, 295, 229 patients), the counts and shares moved slightly: the clinic spelling variants were merged and the mmol/mol values converted. Here the conclusions did not change, but the numbers did, which is why the cleaned data is the version to report.

A reference file is at scenarios/toy/checkpoints/cohort/AGENTS.md.

Verification criteria

  • Concise and specific: The file remains under 25 lines and provides concrete rules rather than vague generalities.
  • Security compliance: The file contains zero API keys, passwords, or confidential data (Secrets).
  • Effective steering: In a clean test session, the agent correctly cites and adheres to the rules defined in AGENTS.md.

Common pitfalls

Symptom Underlying cause Resolution
Agent ignores instructions File name or directory location does not match harness configuration conventions Verify the expected instruction file convention for your specific agent harness
Bloated, generic instructions Prompt failed to enforce brevity constraints Edit the file to retain only high-impact rules that prevent genuine operational mistakes
Contradictory rules Instructions reflect outdated draft decisions rather than finalized rules Reconcile AGENTS.md directly against notes/cleaning-decisions.md

Concepts behind this

  • Memory. Language models do not retain memory across sessions. Preserving knowledge requires externalizing guidelines, documentation, and tools into persistent files. Go deeper →
  • Context. Context represents the model's active working memory. Because every turn re-transmits the complete conversation, long sessions increase latency, cost, and attention drift. Go deeper →
  • Steering. Balance supervisory approval modes with task risk, intervene early when execution drifts, and codify recurring corrections into standing rules. Go deeper →

Builds on: Clean the data with a script, and show the flow