ai

Tooling your agents so they stop guessing your architecture

One instructions file, a folder of skills and a generated OKF knowledge bundle as the knowledge layer for coding agents, why a generated bundle goes stale in a way that causes confident wrong answers, and the parts I would not automate.

22 min read

Most complaints about coding agents come down to one thing. The agent wrote something that is locally sensible and globally wrong.

It added a write resolver to the read API. It wrote a database query without the index it needed. It hand rolled a table component when the design system already has one, with rules about how it must be used. It reauthenticated a user through a second code path instead of reusing the one the frontend already has.

None of those are model failures. They are missing context failures. The agent was not told that reads and writes are served by different applications, and there is nothing in the file it was editing that says so.

So we stopped treating this as a prompting problem and started treating it as a documentation problem with one unusual requirement: the primary reader is not a person.

That turned into three layers, and they answer three different questions. One file says what must never happen. A folder of skills says how to do a specific kind of task, in order. A generated knowledge bundle says what exists, so the agent stops guessing about structure. The rest of this is what each one is, and where each one bites back.

One file that everyone reads

The repository has a single instructions file at the root. It is the source of truth for agent behavior, and it says so in its own first line. One other agent specific file is a symlink to it, so there is exactly one place to edit. A third file exists for a different assistant format, and it cannot be a symlink, so instead it carries a warning at the top pointing at the real source and stating that it is maintained for compatibility only.

That sounds like a detail. It is the difference between one document and several that disagree.

The file is not a style guide. It carries only the rules that must not be broken, and it says so. Everything else lives somewhere else and the file tells you where.

It reads like this. Two rules from the frontend section, close to verbatim:

- **All API calls go through `@repo/query-kit`.** Every GraphQL query/mutation
  or write-API call from `apps/web` must be a `useQuery`/`useMutation` hook
  defined in `packages/query-kit/src/<domain>/` and exported first. Never inline
  `fetcher()`/`fetch()` in pages or components, including pre-existing ad hoc
  calls you touch.
- **Data tables** use `DataTable` + `useDataTable` + `DataTableAdvancedToolbar`
  from `@repo/ui`. Never hand-roll a `<table>`, filter, or sort dropdown.

Both are written as a rule plus the thing that gets violated, which is the shape that survives being skimmed. The file is markdown made of headings and bold lead sentences for the same reason. Nobody reads it top to bottom.

The rest of the rules are the ones where a reasonable engineer would make the other choice without being told:

  • Reads are GraphQL, writes go through the Java service. Never add a write resolver to the read API.
  • Every write has to invalidate the read cache explicitly, using the real type name as it appears in the schema rather than the model name, because a mismatch silently does nothing and serves stale data until it expires.
  • Merge direction is one way: feature into develop, develop into release. Never the other way round.
  • Branch integration happens through one tool. Nothing merges a branch except that tool.

That last one is the clearest example of why the reader being a machine matters. A human engineer knows not to force push to the release branch, because they have been in the meeting where it went wrong. An agent does not have that meeting, so the rule is written down in the imperative, with the failure mode attached.

I also write down the things that would be tempting to shortcut. There is a paragraph about not patching over uncertainty with a temporary decision, because a plausible shortcut that the agent does not fully understand is a bug being shipped. There is another about scale: paginate list queries, avoid unbounded scans, index what you filter on, and do not hardcode a limit that only works on today’s row counts.

Skills as the step by step layer

The main file is deliberately thin, because a document that covers everything gets read by nobody.

Underneath it is a folder of skills. Each one is a procedure for a specific kind of task. Adding a model to the schema. Adding a read query. Adding a write endpoint. Changing infrastructure. Diagnosing a connectivity problem. Checking production errors. Writing documentation. Producing a pull request description.

A skill is a markdown file with frontmatter naming it and saying when to use it:

---
name: spring-write-api
description: Implement Spring Boot write-side API work in apps/api-java.
  Use for POST, PUT, PATCH, DELETE endpoints, request validation, controllers,
  actions/services, JWT-authenticated flows, RabbitMQ queues, cache
  invalidation, and write-side tests.
---

That description is the whole trigger. It is what gets matched against the task, so it lists the situations rather than describing the skill, and it names the exclusions too: the read query skill’s description ends with “Do not use for write operations; writes belong in the write API”.

The value is mostly in the order. Adding a model to the schema touches the schema file, a migration, the generators, the read API and the tests. The skill lists them in the sequence that actually works:

pnpm --filter @repo/db db:migrate    # dev shell, creates and applies the migration
pnpm --filter @repo/db db:generate   # regenerate the client after a schema change
pnpm okf:generate                    # refresh the bundle
pnpm okf:validate

Followed by the gate: a migration must accompany every schema change, including one that renames a relation and touches no SQL, because the build fails otherwise. A bulk of the skill is that kind of operational detail, which is exactly the part that does not survive in someone’s memory.

Skills also carry the negative knowledge, which is the part I find most valuable and most often missing from documentation. The write endpoint skill has a mandatory section on cache invalidation, and the useful sentences in it are the failure modes:

- Call `cacheInvalidator.invalidate(typename, id)` on every write path. There is
  no automatic ORM-event listener, so raw and bulk writes and deletes are on you.
- `typename` must match the actual Pothos GraphQL type name, not the model name.
  A mismatch silently no-ops and leaves stale cache up to TTL.
- The call must stay synchronous inside the request that performs the write.
  Do not fire it async or move it into a queued handler.

Three rules, and none of them say what to do in the happy path. They say what breaks, why it breaks quietly, and what not to move. The typename one is the clearest: a wrong string does not error, it just serves stale data until the entry expires, which is the kind of bug that gets diagnosed as something else entirely. That sentence came from a bug.

Not every skill is a procedure. Some exist to make a tool usable at all. The one for checking production errors carries the connection details, because otherwise every session re-derives them:

BASE="https://errors.example.com/api/0"

for p in api docs graphql web; do
  curl -s -H "Authorization: Bearer $TOKEN" \
    "$BASE/projects/<org>/$p/issues/?query=is:unresolved&statsPeriod=14d"
done

Which projects to check, which window to use by default, and where the credential lives. That is unglamorous and it is the difference between an agent answering a question about production errors and an agent explaining that it cannot.

Others are pure policy. The self-check skill is a checklist to run before finishing, and most of it is telling the agent what not to do:

- Run `git status --short` and identify which changes are yours. Do not revert or
  overwrite unrelated user changes.
- Confirm no commit was created unless the user explicitly asked for one.
- Confirm no production or real database data was inserted, updated, deleted,
  truncated, or credential-modified without explicit permission.
- Confirm every `kubectl` command names a namespace explicitly.
- Confirm relevant tests, type checks, lint checks, or targeted smoke checks ran.
  If any were skipped, say why.

Read as a list of instructions to a person, that is a strange document. Every line is something a competent engineer does already. Read as instructions to an agent, it is a list of the exact ways an agent goes wrong: tidying up files it did not create, committing because the change was finished, treating a database as scratch space, running a command without a namespace and hitting the wrong cluster.

There are 46 of these at the moment, and the main file names the 11 that apply most often. Most of them are not ours. There is a lock file tracking where each one came from, and by that count the majority are imported: 26 from one vendor’s skill pack, 6 more from five other projects. Keeping them means a consistent procedure exists rather than each session improvising.

That mix is deliberate. A procedure for a third party library is not something we should be writing from scratch, and when the lock file records where it came from, refreshing it later is a command rather than an archaeology exercise.

The knowledge bundle

The third layer is OKF, the Open Knowledge Format. It is a generated bundle checked into the repository, not a folder of notes someone maintains by hand.

The bundle is a set of markdown files with frontmatter, linked to each other. One file per Prisma model, one per enum, one per package, one per infrastructure service, plus an index that ties them together. Nothing in it is a new fact. Every concept is derived from something that already exists: the schema files, the package manifests, the compose file.

The package concepts show the linking best. A package file records its directory, its scripts and its dependencies, and the workspace dependencies are wikilinks rather than names:

## Dependencies

- [[packages/types|@repo/types]]
- [[packages/typescript-config|@repo/typescript-config]]
- `@tanstack/react-query`
- `axios`

Internal packages become links you can follow, third party packages stay plain text, which is the right split. You can start at the index, walk to a package, follow it to the packages it depends on, and never open a manifest.

We wrote the generator ourselves. It is a single Node script that reads three sources: the Prisma schema directory, the root compose file, and the package manifests, which it finds by globbing the workspace. No Prisma client, no markdown library, no AST parser.

The parsing is the part worth showing, because it is where the judgement call lives:

const fieldMatch = trimmed.match(
  /^([A-Za-z0-9_]+)\s+([A-Za-z0-9_]+(?:\[\])?\??)(?:\s+(.*))?$/,
);
if (!fieldMatch) continue;

const [, name, rawType, attributes = ""] = fieldMatch;
const cleanType = rawType.replace(/\?|\[\]/g, "");
fields.push({
  name,
  type: rawType,
  required: !rawType.includes("?"),
  list: rawType.endsWith("[]"),
  relation: !scalarTypes.has(cleanType),
  attributes: attributes.trim(),
});

That is a schema field parsed with one regular expression. Name, type, optional marker, list marker, attributes, and a guess at whether it is a relation based on whether the type is in a hardcoded set of Prisma scalars.

The obvious objection is that this is hand rolled where a real parser exists. Prisma ships one, and it would give a typed structure instead of strings. We deliberately did not use it. The input is a file we own, in a format we do not control but which changes rarely, and the output is markdown that a human reads. The failure mode of a regex here is a field rendered wrong in a document, not a query executed wrong.

That trade would be a bad one in application code. In a documentation generator that runs on demand, the maintenance cost of owning a real parser dependency outweighs the parsing accuracy we would gain. The script is 700 lines and has never needed a dependency update.

The models are the useful half. Opening one gives you the fields, the types and the relations, so an agent working in the GraphQL layer does not have to read schema.prisma to find out what a Chat looks like. This is the generated concept for one model, trimmed:

---
title: "Chat"
type: "data-model"
source_path: "packages/database/prisma/models/chat.prisma"
table_name: "chats"
field_count: 22
relation_count: 8
---

# Chat

Part of [[index|Open Knowledge Format]].

## Fields

| Field | Type | Required | Relation | Attributes |
| --- | --- | --- | --- | --- |
| `id` | `BigInt` | yes | no | `@id @default(autoincrement())` |
| `status` | `String` | yes | no |  |
| `last_message_at` | `DateTime?` | no | no |  |
| `createdBy` | `User` | yes | yes | `@relation("ChatCreatedBy", ...)` |
| `messages` | `ChatMessage[]` | yes | yes |  |

## Indexes And Constraints

- `@@index([status])`
- `@@index([last_message_at])`
- `@@index([created_by, status], map: "created_by_status")`

## Related Models

- [[data/models/User|User]]
- [[data/models/ChatMessage|ChatMessage]]

Two things are worth noticing, because they are why this beats a hand written summary. The source path and table name are recorded, so the concept points back at the file it came from rather than replacing it. And the related models are links, so the bundle is navigable: from a model you can walk to its neighbours without a search.

It is generated because a hand maintained map of a codebase is wrong within a week, and a wrong map is worse than none.

That is not a hypothetical. A schema cleanup removed 52 models, and some of them were still described in the bundle afterwards. The description read exactly as confidently as it had before the models were deleted. Nothing flagged it. It turned up by accident, while someone was doing something else.

The trap is that generated files stop looking generated once they are checked in. They look like documentation, which means both people and agents treat them as fact. A model described in the bundle and absent from the schema is a contradiction with no obvious winner, and an agent will often resolve it the wrong way, because the bundle is the thing that reads like a reference.

So the rule is the same one that applies to the generated clients: regenerate OKF in the same change that alters what it describes, run the validate step, and treat a stale bundle as a bug rather than a chore.

pnpm okf:generate   # regenerate the bundle from the schema, manifests and compose file
pnpm okf:validate   # check the generated bundle is structurally sound

Two commands, no excuses. In practice the second runs in well under a second and reports the count:

Validated 186 OKF markdown files.

The validate script is the part that makes the pair worth having. It is a second hand written Node script, and it does not compare the bundle against the schema. It checks that the bundle holds together on its own terms: that the index and the log exist, that every file has a top level heading, that every file except the index has frontmatter, that the frontmatter declares a type, and that the index pins a format version. Structural, not semantic.

That is a deliberate limit rather than an oversight, and it is worth being clear about what it does not catch. A model deleted from the schema and still described in the bundle passes validation, because nothing in the bundle is checking the schema. That is exactly the stale entry described above, and it is caught by remembering to run the first command, not by the second.

So the honest version of the rule is: run generate, then validate, and understand that validate proves the bundle is well formed rather than correct. The correctness comes from having regenerated it, and the validation is there to catch the times generation produced something malformed.

Guardrails, not just instructions

Instructions get followed most of the time. The things that must hold every time are enforced by something that cannot be argued with.

A design system dependency with its own rules is plugged into the linter, but not yet switched on. The plugin is registered and none of its rules are enabled, because enabling them means clearing every existing violation first. The instruction file says exactly that, and adds that turning a rule on means fixing what it flags rather than adding an ignore. That sentence is there because the obvious move when a linter starts complaining is to silence it.

I think that is the right shape. A rule set that arrives with a hundred violations and gets ignored has taught nobody anything. Registering the plugin and leaving the rules off keeps the intent visible without pretending the code currently satisfies it.

The database is read only unless someone explicitly says otherwise, and destructive operations are called out. This is not about the agent being careless. It is that an agent with database access and a plausible reason to fix test data will fix test data.

The tool that merges branches is the only thing that merges branches, and an agent must report a failed integration as failed rather than inferring that it probably worked. That rule exists because inferring success from an ambiguous response is a very natural thing for a system that wants to be helpful, and it is unacceptable when the consequence lands in a shared branch.

What I would not automate

I would not generate the skills. A generated procedure is a summary of what the code looks like today, and the useful part of a skill is the reason behind the step, which is not in the code.

I would not try to keep one giant instructions file. It gets skimmed, then ignored, then it silently stops being the source of truth while still being the file everyone is told to read.

And I would not treat any of this as one time work. Every bug that came from a missing rule is a sentence that should now exist. The document grows from incidents, which is annoying and also the only way I have found to make it say things that are actually true about this codebase rather than generally true about code.

What it changed

The measurable part is the migration rule. Every schema change now lands with a migration beside it, including the ones that only rename a relation and touch no SQL, because the build fails otherwise. That rule is written in two places, the instructions file and the skill for adding a model, and it is the clearest case of documentation turning into a check.

The rest is harder to measure and I will not pretend otherwise. What I can say is that the wrong category of change stopped arriving in review: a helper that reimplements something the design system already exports, a model added without its generated output refreshed, a query built by hand where the shared hooks exist. Those were the recurring review comments, and the rules exist because of them.

The cost is a documentation surface that has to be maintained alongside the code, including a generated part that can quietly rot, and that part has already caught us once. I think that is a fair trade. An agent that does not know your architecture will confidently build a second one next to it, and cleaning that up costs more than writing the rule down once.