AI-built

What are the real risks of shipping software that nobody fully read?

Not 'AI writes bad code'. Seven exposures that come from understanding failing to keep pace with production.

Short answer

The risk is not code quality — it is that nobody in the company can answer questions about a system the company depends on, and that the usual signals of that condition are absent. Seven specific exposures follow: silent failure, unverified documentation, unrepeatable answers, duplicated approaches, thin unhappy paths, unowned scheduled work, and a handover that cannot happen because there is nothing to hand over. Each has a specific cost and a specific check.

#The right framing

Arguments about whether coding agents write good code are unresolvable and, for an owner, beside the point. The quality varies with the task, the model, the prompt and the reviewer, and it is improving. It is not the exposure.

The actual exposure

A company depends on a system, and nobody in it can answer questions about that system. Not because the answers are hard, but because nobody ever had to produce them — the understanding that used to be a free by-product of writing the code was never created.

What makes this harder to notice than ordinary legacy: the people are still here, the code is often good, the system is new, and there is frequently a lot of documentation. Every signal that normally says "you have a problem" is absent.

#The seven exposures

  1. Silent failure

    Failure handling has to be imagined rather than described, so it is the part most often thin. A job that fails quietly, an integration that stopped last month, an export nobody notices is not arriving.

    Check: does every failure path produce something a human would see?

  2. Unverified documentation, believed

    Generated documentation is fluent, extensive and indistinguishable from verified documentation. People act on it. Where it is wrong, it is wrong confidently and in detail.

    Check: take five specific claims from it and test each against the code.

  3. Answers that cannot be repeated

    Asking an agent produces a good answer today and nothing tomorrow. Ask again next quarter and you get a different answer with no way to compare the two, and no record of which one was true when.

    Check: can anybody produce what the system looked like three months ago?

  4. Three solutions to one problem

    Different sessions solving the same thing differently, each locally sensible. The cost is not aesthetic — a change has to be made three times, and somebody will make it twice.

    Check: is there more than one way this system does caching, validation, retries or error reporting?

  5. Unowned scheduled work

    Jobs that run on a schedule and were added in one session. Nobody remembers them, nobody watches them, and nothing alerts when one stops.

    Check: list everything that runs on a schedule from the code, the console and the crontabs. Compare the three.

  6. Access assumptions nobody checked

    An endpoint that should have been internal. A webhook that does not verify its sender. An ownership check missing on a route that passes every test where one user exists.

    Check: go through the route list explicitly and mark each as intended-public or intended-private.

  7. A handover that cannot happen

    When a new team asks what the system contains, there is no answer to give them — not a stale one, none. The onboarding cost is therefore the full reconstruction cost, and it is a surprise.

    Check: could you hand this to another team next month, and what would you hand them?

#What each one actually costs

Exposure, and where the bill arrives
ExposureCostArrives when
Silent failureData loss, missed obligations, a partner who stopped receiving somethingMonths later, from outside
Unverified documentationA decision taken on a false premiseWhen the decision is executed
Unrepeatable answersEvery question is asked from zero, foreverContinuously, invisibly
Duplicated approachesFixes applied in two places out of threeAs a bug that was already fixed
Unowned scheduled workSomething stopped and nobody noticedWhen somebody outside complains
Access assumptionsExposure of data or of an operationAnywhere between never and this week
Impossible handoverMonths of reconstruction, paid at full rateThe moment somebody leaves

#How to tell whether this applies to you

Six questions. If three of them are uncomfortable, this page is about your system.

  • Could somebody in the company name every component and say what each does?
  • Has anybody verified the documentation that exists?
  • Could anybody say what changed structurally in the last two months, without reading pull requests?
  • Does anybody know everything that runs on a schedule?
  • If your most active developer were unavailable for a month, could the rest ship?
  • Is there anything in the system that nobody remembers asking for?

#What reduces it

None of this argues for slowing down. It argues for one habit and one check.

The habit: whenever an agent produces a substantial part of the system, have something derive an account of what now exists — separating what it read from what it inferred, requiring open questions to be stated rather than filled in — and keep that account somewhere the next one can be compared against. Done well this is close to free, because the agent that wrote the code can produce the account.

The check: the production checklist, worked through once before real traffic and again after each significant push.

#What goes wrong

Concluding that the answer is to write less code with agents.

Instead The speed is a real gain. The thing to change is that understanding is now produced deliberately rather than as a side effect.

Adding a code review process and calling it done.

Instead Review catches defects in a change. None of the seven exposures above is a defect in a change; they are properties of the whole.

Generating more documentation.

Instead Verify what exists before adding to it. Volume is not the problem; nobody having checked any of it is.

Waiting for something to break.

Instead Four of the seven are silent by construction. Waiting for a signal means waiting for somebody outside the company to provide it.

Treating this as a reason to distrust the team.

Instead Nobody did anything wrong. The rate at which software is produced changed; the mechanism that produced understanding did not scale with it.

#What this does not cover

What this does not do

  • Whether AI-written code is better or worse than hand-written code. Unresolvable, and not the exposure.
  • Security assessment. Two of the seven overlap with security hygiene; none of this is a security review.
  • Legal or regulatory implications of how software was produced.
  • Anything about intellectual property in generated code, which is a real question and a legal one.

Establish what is actually there.

One read-only instruction, run by the agent that built it, produces a checkable account with the evidence behind each claim and an explicit list of what could not be established.

Build your project map — free The production checklist