Most teams blame the model or write longer prompt files. When we linked three hundred autonomous coding runs in Fractal OS back to the exact git commits they produced, we found something much simpler: the repository itself had become illegible.
Seeing what the tokenizer sees
People read source files by line and indentation. Models read a flat stream of integer token IDs emitted by a byte-level BPE state machine11. Specifically o200k_base (200,000 vocabulary entries). Notice in the terminal above how the two emoji 💯🔥 (8 UTF-8 bytes) split across three tokens: 31446, 107, and 96606.. Because those boundaries don’t show up in an editor or a GitHub diff, teams routinely commit generated fixtures, inline SVGs, or sprawling lockfiles that quietly eat 30–40% of an agent’s context window on every single read.
You pay for those tokens twice: once on the inference bill, and a second time when useful architectural context gets pushed out of working memory. We wrote tc as a companion to wc(1) so both humans and CI pipelines can check the token weight of a directory in milliseconds:
What happens at 30,000 tokens
While building Fractal OS, we recorded every sub-agent session alongside the git commit SHA it produced22. Each <session> node in project.xml logs first-edit-turn, read-calls, edit-calls, and token usage so we can see where an agent spends its time before touching a file.. Instead of degrading smoothly as a codebase grows, agent performance hits a cliff once unindexed surface area in a subsystem crosses roughly 30,000 tokens:
| Subsystem state | Reads before 1st edit | Time to 1st edit | Cost / merged PR | PRs needing human rework |
|---|---|---|---|---|
Clean module boundaries (< 15k tokens) |
4 reads | 38s (Turn 6) | $1.40 | 4% |
Duplicate helpers & route drift (48k tokens) |
29 reads | 279s (Turn 21) | $6.75 | 38% |
After srclint + metadata-enforcer pass |
6 reads | 54s (Turn 8) | $1.85 | 5% |
Look at the last two columns together. When a subsystem drifts from 15,000 tokens of clean structure to 48,000 tokens of overlapping utilities and stale notes, the inference cost per pull request jumps nearly fivefold—but the real damage is the 38% human rework rate. That is the exact moment when an engineering organization feels like AI tools have made them slower rather than faster.
Checking prose like code
In a human-only workflow, stale documentation is a minor annoyance; a developer asks a teammate in Slack and moves on. In an agent workflow, every markdown file in the repository is treated as ground truth. If two docs disagree on how routes or CSS classes are structured, agents will alternate between both patterns on every commit.
We wrote metadata-enforcer to validate Markdown frontmatter and repository docs against declarative Python schemas in CI, failing the build the moment instructions drift:
A shared state for plans and runs
Keeping the repository legible solves half the problem; the other half is keeping track of the plan itself. Chat scrollback is a terrible place to run an engineering project. In Fractal OS, both human leads and autonomous sub-agents read and update a single XML document (project.xml) that ties every task to its verification criteria and execution history:
<task id="srclint-metrics" wbs="1.6" status="in-progress" assigned-agent="omarish-agy">
<outcome id="oc-legibility" status="met">
<criterion status="met">Deterministic commit-to-transcript linking</criterion>
</outcome>
<session id="run-srclint-d9a891dc" model="gemini-3.1-pro"
first-edit-turn="15" read-calls="228" edit-calls="157" />
</task>
When the plan, the transcript, and the codebase all stay small and structured enough to inspect directly, a very small team can run dozens of parallel changes without losing the thread.