Coding agents without hallucinations: shorter instructions, grounding, and verification

Coding agents do not start “hallucinating” only because the underlying model is weak. We often create the conditions ourselves: too much stale context, conflicting rules, no clear source of truth, and no requirement to verify the result.

The practical consequence is uncomfortable: a long AGENTS.md or CLAUDE.md is not automatically an advantage. It can increase the cost of every step and add noise without improving task completion. A better pattern is a small set of durable instructions, context loaded on demand, grounding in real project evidence, and verification of the finished result.

A large context window is not reliable memory

A context window tells you how many tokens a model can technically accept. It does not mean the model will use every token equally well. The Lost in the Middle study found a characteristic U-shaped pattern: models used information more reliably near the beginning or end of the context and less reliably in the middle. The work evaluated older models and specific tasks, so it should not be turned into a universal percentage for today’s coding agents. The operational lesson still holds: more context does not automatically produce a more accurate result.

Long agent runs also accumulate plans, tool outputs, mistakes, corrections, and dead ends. When history is compacted, it becomes a lossy summary. Claude Code re-reads the project-root CLAUDE.md after /compact, but instructions that existed only in the conversation can disappear from the summary. Durable agreements therefore should not live only in chat.

AGENTS.md and CLAUDE.md are context, not a firewall

Codex builds an instruction chain from AGENTS.md files before it starts working, from the project root down to the current directory. Claude Code uses CLAUDE.md; if you want one shared set of rules, CLAUDE.md can import @AGENTS.md.

Both mechanisms are useful, but neither is hard enforcement. Anthropic explicitly describes CLAUDE.md as behavioral context rather than security configuration. A destructive command should be blocked through permissions, a sandbox, or a hook. The instruction file should primarily explain what the agent needs to know and how it should work.

A useful durable instruction looks like this:

## Change verification

- After changing invoice logic, run the invoices integration tests.
- Treat `src/tax/rates.ts` as the source of truth for VAT rates.
- If a fact cannot be verified in the repository or official docs, label it unverified.

A weaker version is a long list of generic prohibitions such as “do not make mistakes”, “do not invent facts”, and “always write high-quality code”. Those statements do not tell the agent where to find evidence or how to determine that the task is complete.

The AGENTS.md study: less certainty, higher cost

The Evaluating AGENTS.md preprint studied repository-level context files using SWE-bench tasks and repositories with developer-authored instructions. Its conclusion is not that teams should delete AGENTS.md. It is that these files need editorial discipline.

The authors found that context files did not generally improve task completion rates and increased inference cost by more than 20% on average. Developer-written instructions slightly helped in some settings, while automatically generated files often led agents to explore more and spend more tokens.

One detail matters: the 20% figure refers to average cost growth, not a universal 20% drop in quality.

I would therefore treat /init and similar generators as drafts for review, not finished configuration. An agent can usually rediscover obvious directory layouts, dependencies, and commands from the repository. Durable context is more valuable when it captures exceptions, hidden dependencies, unusual test steps, and the reasons behind decisions.

Grounding gives the agent a path to evidence

Grounding means the answer is not produced solely from the most likely continuation of text. The agent must consult a concrete source of truth.

In a coding workflow, that can mean:

• reading the current implementation and database schema,
• running a test or typecheck,
• opening the official documentation for the API in use,
• comparing the result with a log, screenshot, or real HTTP response,
• naming the file, test, or URL supporting an important claim.

The same principle applies beyond software development. An invoice agent should read accounting rules and order data rather than guess a tax rate. Lead triage should use CRM fields and explicit criteria. Support classification should return both a label and evidence from the customer message.

In n8n or Make, I would separate three stages: retrieve relevant data, let the model make a decision, then validate the output deterministically. If the model returns structured JSON, the next node can verify required fields, allowed values, dates, totals, or the existence of a cited record.

Eight rules for a more reliable coding agent

1. Separate general knowledge from project reality. Verify API versions, architecture, and company rules against current sources.
2. Keep durable instructions short. Store only information the agent cannot easily discover itself.
3. Write rules as concrete actions. “Run test X after this change” is better than “never break tests”.
4. Edit generated instructions. /init is a useful starting point, not ground truth.
5. Compact at a natural boundary. Before shortening history, save decisions, open questions, and evidence into the project or task memory.
6. Require grounding. The agent should read a file, database, log, or official documentation and say what it used.
7. Provide an escape route. If a fact cannot be verified, “unverified” is better than a confident guess.
8. Verify the finished result. Tests, linting, rendering, an HTTP check, or human approval should match the risk of the change.

What I would measure inside a company

Do not evaluate an agent by the quality of its explanation. For every workflow, track completed tasks, retries, tokens, cost, runtime, human interventions, and failure categories.

For coding agents, add targeted test results and whether the diff contains out-of-scope changes. For document extraction, measure against a reviewed sample. For CRM enrichment, verify the source and age of each fact. For support, monitor incorrect escalations rather than only the percentage of tickets that received a label.

Only an A/B test will tell you whether a new AGENTS.md helps. Take 30 real tasks, run them with the current instructions and a shorter edited version, then compare both success and cost. If the agent performs equally well with half the file, you removed context tax without sacrificing quality.

My conclusion

The most reliable agent is not the one receiving the most text. It is the one with short durable instructions, access to current facts, permission to acknowledge uncertainty, and an obligation to verify its result.

AGENTS.md and CLAUDE.md are valuable maps of a project’s non-obvious rules. When they become automatically generated repository encyclopedias, they add tokens and noise. For companies, maintaining agent context should therefore be treated as the same kind of operational discipline as tests, logging, and permissions.

Sources: Jirka Herník: Halucinuje vám AI? Mě ne!, OpenAI Docs: Custom instructions with AGENTS.md, Claude Code Docs: How Claude remembers your project, Gloaguen et al.: Evaluating AGENTS.md, and Liu et al.: Lost in the Middle.