Yar Kravtsov

AI Coding Agents Need a Source of Truth

· llm, ai

When an AI coding agent makes a bad plan, the usual fix is to “add more prompt.” Sometimes that helps a bit, but usually it hides the actual problem. The agent doesn’t need a bigger prompt, it needs something true to anchor against.

You know the fix I mean. More rules, more examples and more warnings. Tell it to be careful, tell it not to hallucinate, and tell it to respect every engineering acronym we have collected over the last twenty years.

This post covers what I’d do instead: a small brief, a human edit of that brief, sizing the task, and reviews against real artifacts.

A Big Prompt Is Not a Plan

Long prompts can look serious, with sections, rules, rubrics and stern little warnings, and they feel like process. But they don’t make the task true.

If the task is vague the agent fills the gaps by inventing files and requirements, and it adds tasks that sound useful but don’t match the goal. That is the expensive kind of wrong: plausible wrong.

Obviously bad output is easy to reject, but plausible wrong output wastes time because you have to compare it against the real task, find the drift and explain why the confident plan is nonsense. At that point the human is doing the grounding manually, and the workflow should have done that before the agent started planning.

Start With a Brief

Before the agent writes a plan, make it write a brief. I don’t mean a fake product spec and not a thirty-page ceremony, just a small file that says what is known:

# Brief

Goal:

- Add CSV import for contacts.

Expected behavior:

- User can upload a CSV file.
- The app validates required columns.
- Invalid rows are shown before import.
- Valid rows can be saved.

Non-goals:

- No Excel support.
- No background jobs.
- No new schema editor.

Open questions:

- Maximum file size?
- Should duplicate rows be skipped or rejected?

This is boring, and that is why it works.

Now the agent has something to compare against. A task is either supported by the brief or it isn’t. A feature is either in scope or it isn’t. An assumption is either written down (like the open questions about file size and duplicate rows) or it isn’t.

Without the brief, the agent works from vibes. With the brief, it has ground.

The Human Should Edit the Brief

The important part is not that the agent writes the brief. The important part is that a human reviews it before planning starts.

This catches cheap mistakes while they’re still cheap. Maybe the agent misunderstood the goal or quietly added a non-goal, or maybe it missed a constraint or an open question blocks the whole thing. Fixing that in the brief takes a minute. Fixing it after the agent has written code can take hours.

This pause feels slow only if you ignore the downstream cost, because bad direction gets more expensive at every step.

Size the Task Before Planning It

Not every task deserves the same workflow, so the agent should classify the task before it plans the work.

Some tasks are tiny (changing a label, fixing one error message, adding a missing flag) and these shouldn’t become a five-part architecture plan. Some tasks are medium, like adding a small feature, changing one flow or refactoring one package. These need a short checklist and a real verification step. And some tasks are large: they cross boundaries, touch multiple files and need review points.

If every request gets the same heavy process, the process becomes noise. You ask for one small fix and get a strategy document, and nobody in their right mind wants that.

Small tasks should stay small. This isn’t laziness, it’s matching process to risk.

Review Artifacts, Not Feelings

A critic step can help, but only if it checks something concrete. “Review this plan” is mush. This is better:

Review this task list against the brief.
Flag any task that:
- is not supported by the brief
- adds a non-goal
- depends on an unstated assumption
- has no verification step

Now the critic has a job. The same thing works after implementation, with the diff in place of the task list:

Check this diff against the brief.
Confirm that:
- the expected behavior is implemented
- non-goals were not added
- tests cover the important paths
- no unrelated files changed

That is much better than asking whether the code is “good.” Good according to what? The brief gives the answer.

Bigger Prompts Can Hide Weak Systems

Prompts matter, and clear instructions, examples and good defaults all help. But after a point, adding more prompt is like adding comments to confusing code. It may explain the mess, but it doesn’t fix the structure.

A sane agent workflow is usually boring:

  • Write a short brief.
  • Let the human edit it.
  • Classify the task size.
  • Generate a plan only when it’s needed.
  • Check the plan against the brief.
  • Check the diff against the plan and the brief.
  • Run the tests.

This is not magic, it’s normal engineering discipline around a tool that is very good at sounding confident, and that confidence is useful when it is grounded and expensive when it isn’t.

The Real Output Is the Workflow

The best agent setups don’t depend on one perfect prompt, they use small artifacts that keep each other honest. The brief grounds the plan and the plan guides the diff, then the tests check the diff and the critic checks whether each step stayed inside the brief.

That chain matters more than the model sounding smart. The agent did not fail because it lacked instructions. It failed because it lacked ground.

Give it a small truth first. Then let it work.