NewPramaan now supports India data residency for litigation records.Talk to sales
PRAMAANLitigation record intelligence
Engineering
Engineering · AI ENGINEERING

AI is too eager. Don't ship what the team can't explain.

JUL 17 2026 · 8 MIN READ
TL;DR

Coding agents flipped the economics of our work. Writing software got cheap; understanding what was written got expensive. Our hours stopped going into tickets and started going into everything around them: reconstructing what the agent did, which files it touched, which guardrail it had just proved we needed. For a company that sells auditability, a system the team cannot explain is not leverage. It is debt with a quiet interest rate.

This is our current answer, dated and unfinished: a rulebook where every rule names the incident that created it and the check that enforces it, one trace ID across every tier, a written walkthrough shipped with every change, a memory that versions itself. None of it is a best practice, because the best practices of this lifecycle are still being written in public. Here is how we found our answer, and how fast it keeps moving.

AI is too eager. Don't ship what the team can't explain.

For fifty years the expensive part of software was writing it. The whole classic lifecycle arranged itself around that fact. Review was a tax on production. Testing was a tax on production. Documentation was the first thing cut when a deadline bit, because production was the point and everything else was overhead.

Then production got cheap. A coding agent drafts in an afternoon what a careful engineer drafted in a week, and it is eager in a way no engineer is: it never hesitates, never doubts, never wonders whether the team has caught up. The economics of every stage inverted. When generation is nearly free, the taxes become the work. Review is the work. Verification is the work. Understanding what exists, what changed, and why: that is the work.

Everyone building with agents is living some version of this shift, and nobody has a settled playbook for it. This post is ours, with a date on it. First the problem as it arrived at our door, then what the market is reaching for, then how we stumbled into our own answer and what we still have not figured out.

The problem we solved

The agents got fast before we got ready.

A ticket that used to occupy an engineer for two days started coming back in under an hour. The code was usually plausible and often good. And then the real time began: reconstructing what the agent had done and why, which files it had touched, which edge case it had not seen, which assumption it had quietly invented. Review stopped being a skim and became archaeology. We built the guardrail the last mistake had just proved we needed. We redid the piece it got wrong. Our hours moved from writing software to discussing software that already existed, and our process was still priced for the old economy.

Three scars from that period:

  • Five deploys in one week, every pipeline green, while five pieces of the runtime were dead. Green was a lie every time.
  • A feature passed every component test wired to an empty fixture, then broke against the real system on first contact. Fixtures lie in your favor.
  • A board automation began closing tickets the moment they looked done. The model had not written any of these bugs. Improving the model would have made the bugs faster, not safer.

The gap now has a name in our vocabulary: comprehension debt, the distance between what the system does and what the team can explain. It compounds like any debt. Every unexplained layer makes the next change harder to review, which makes the next rubber stamp more likely, which deepens the debt.

Most companies service this debt as slower velocity. We cannot. PRAMAAN builds software for litigation, where the product is the ability to explain: every claim sourced, every contradiction surfaced, every decision reconstructable. A system we cannot explain is not leverage for us. It is the one thing we sell, unbuilt.

The solution in the market

When we started comparing notes outside the building, the same shape kept appearing.

Martin Fowler and his Thoughtworks colleagues published the cleanest formulation: agent quality is model capability multiplied by harness quality. You rent the model. You build the harness: the rules, tools, contracts, and feedback loops around it. LangChain published an anatomy of the agent harness: filesystems, tools, memory, context management, planning and verification.

OpenAI went furthest. In February they published a field report from an internal team that shipped a product of about a million lines with zero human-written code, in roughly a tenth of the time hand-writing would have taken. Their scarcest resource, they wrote, was not the model. It was human time and attention. In April they open-sourced Symphony, a spec that turns an issue board into a control plane for agents, after hitting the ceiling every team hits: one engineer can supervise about three to five agent sessions before context switching eats the gains.

Around all of this, a convention hardened on its own: repositories began carrying instruction files written for agents, AGENTS.md, the way they once carried READMEs for humans. Coding CLIs, multi-agent orchestrators, home-grown tools, all crystallizing within months, in public, everyone borrowing from everyone.

The best practices of this lifecycle do not exist yet. They are being written right now, by the teams doing the work. That is the discussion this post belongs to, and it is why nothing below is presented as the answer. It is our current entry in a shared notebook.

How we stumbled into ours

Our harness did not begin with those articles. It began with arguments.

After each scar, the team stopped and argued the same three questions: what exactly failed, what would have caught it, and what do we write down so it cannot happen twice. The arguments became a time-boxed investigation, and the investigation became a written doctrine. The doctrine was first proposed as six files and collapsed to one after pushback, on a principle that now runs through everything we build: plans should mature at the speed of understanding, not faster.

Our tooling is unremarkable. Several coding CLIs, a multi-agent orchestrator, a few home-grown tools, and frontier models from multiple vendors, chosen on purpose so that no single model's blind spots become the system's blind spots. The difference is not the tools. It is what the arguments built around them. Four pieces of machinery carry most of the weight.

The rulebook. Every repository carries an instruction file for agents, and every rule in it follows the same contract: the rule, the incident that created it, and the mechanical check that goes red when it is violated. Prose without a named check does not protect. A gate must ship with a planted violation that proves the gate can fail, because no green counts until it has been red. The rulebook itself has a size budget enforced in CI, and a drift check deletes retired phrases, because an instruction file that grows without discipline becomes the next thing nobody understands.

The trace. We adopted OpenTelemetry across every tier over the last three months, with AWS X-Ray and CloudWatch carrying the general traffic and Pydantic Logfire carrying the agent and model spans. The rule that matters is not the stack. It is the split: auto-instrument the plumbing, hand-instrument the meaning. Manual spans come from a closed registry of approved names and attributes, and adding one is a review decision, not a drive-by. One trace ID crosses the whole system, and the first move on any bug is to get it. If the trace ID is missing, that is itself the bug. A cold start three tiers down once cost us a full day of blaming the frontend. The same class of bug now takes about thirty seconds.

The walkthrough. Every change ships with a short written account of what changed and why, checked by CI and published inside the team. Knowledge transfer stopped being a meeting and became a build artifact: more than five hundred walkthroughs in about two months, written by agents for the humans who own the consequences. The sharpest rule in this set is the comprehension gate. A change begins with a written statement of what exists, what will change, and what will not. If that statement is wrong, the PR is rejected even when the code compiles.

The memory. Decisions are frozen as records, superseded but never edited: sixty-five of them in early May, a hundred and sixty-eight today. We mined several hundred merged PRs into a catalog of learnings so each mistake is paid for once. Post-mortems are not allowed to end in prose; every preventive measure is filed as tracked work. And we built the arguments themselves into the machinery. Our review panels refuse to run unless the reviewing agents come from at least three different model vendors, because a panel that all thinks alike is agreement theater. A red-team pass once overturned a fix that had already landed. We kept that rule.

Then, this spring, the Symphony post made the rounds. Reading it was a strange experience. They had hit the same ceiling, made the same turn, and arrived at the same ratio: the model is rented, the harness is the engineering. Their orchestrator makes each agent produce proof of work: CI status, review feedback, a walkthrough of what it did. Three weeks later our own walkthrough gate went live. Independent invention, same shape, same weeks. For a young company, that convergence is the strongest signal available that the shape is real.

What we still have to figure out

The honest list, kept in the same document that defines the harness:

  • Stale doctrine is still collected by hand. No scheduled garbage collection finds the rules and docs that have drifted from reality. It is our largest known gap.
  • We have almost no behavior assertions on production: checks that the running system, not the code, still means what we think it means.
  • Some of the most important judgment still lives in one person's head, legible to humans and invisible to agents.
  • Escalation is too coarse. An agent that hits something the rulebook does not cover can stop and ask, but it cannot yet distinguish "I do not know" from "nobody should decide this yet."

We keep this list in the open because a harness that hides its gaps teaches the wrong lesson.

The slope

None of this is arrival. Ten weeks separated "debugging is manual archaeology" in our decision log from traces crossing every tier. A branch-protection rule once locked out our own founder because it assumed a team larger than one, and the fix carries a note naming the trigger that will revisit it. We are a small team, and the field we work in is inventing itself in public.

A company can be measured at a point or as a slope. The point is easy to show and easy to fake. We would rather be measured on the slope: how fast the scars become rules, the rules become checks, and the checks become the next round of questions. The lifecycle has changed. The playbook is being written. We are glad to be one of the teams writing in its margins, and we are reading everyone else who is.


References: Martin Fowler / Thoughtworks, "Harness engineering for coding agent users" and "Harness engineering: first thoughts"; LangChain, "The anatomy of an agent harness"; OpenAI, "Harness engineering: leveraging Codex in an agent-first world" (2026) and "Symphony: an open-source spec for Codex orchestration" (2026), with the Symphony spec and reference implementation.