Short answer
#What is actually different
Understanding of a system used to be a free by-product of building it. Somebody read every line, argued about some of them, and carried a model of the whole in their head afterwards. That model was fragile and undocumented, but it existed.
When a large part of a system is written faster than anybody reads it, that by-product does not appear. The result is a specific and slightly disorienting condition:
- There is no stale documentation to fix. There is often a lot of generated documentation, and nobody has verified any of it.
- There is no departed expert to blame. The people are here; they did not read it either.
- The code is frequently fine. Which makes the situation harder to take seriously than it deserves.
- The system may be weeks old. Age has nothing to do with it.
#The method
-
Get an explicit inventory
Components, what each is for, what depends on what. Derived from the system rather than from recollection — which in this situation is unusually thin.
-
Demand the read/inferred distinction
An agent explaining a system produces observed and inferred statements in identical prose. Insist that each claim names the file and lines it came from, or is marked as inference.
This single requirement is what separates a useful account from a fluent one.
-
Test the generated documentation
Take five specific claims from any README or design note and check each against the code. Where they disagree is where somebody's mental model is already wrong.
-
Trace the money path end to end
Payment, order, signup — whichever is commercially critical. Follow it from entry point to database, including what happens when each step fails.
The unhappy path is the part that gets generated rather than designed, and it is where fast builds are weakest.
-
Check the outside edges
Every inbound webhook: does it verify who sent it? Every outbound call: retries, timeouts, and whether repeating it is safe. These are the places a plausible implementation and a correct one look most alike.
-
Find the duplicated approaches
Three implementations of the same idea is the characteristic signature. Each is usually fine; the cost is that a change has to be made three times and somebody will make it twice.
-
Write down what could not be established, and re-check after the next push
Given how fast the system moves, the comparison between two analyses is worth more here than almost anywhere else.
#Separating read from inferred
The one discipline that matters most, and the one nothing does by default.
Ask any capable agent to explain a system and you get a fluent, well-organised answer. Some of it is read directly from the code. Some is a reasonable inference from naming, framework convention and shape. Both arrive in the same voice, with the same confidence, in the same paragraph.
For a reader who cannot tell them apart, that is not a small problem: they will make a decision that depends on which half a sentence came from, with no way to find out.
WITHOUT the distinction
"Failed payments are retried three times with exponential
backoff before being marked as failed."
WITH the distinction
CLAIM Failed payments are retried three times.
PERSPECTIVE Observed
EVIDENCE app/Jobs/SettlePayment.php:22 ($tries = 3)
CLAIM Retries use exponential backoff.
PERSPECTIVE Inferred — the framework default is exponential
and no backoff is configured. Not read directly.
UNKNOWN What happens after the third failure? No handler
was found. HIGH
The first version is more readable and less useful. The second tells you that one statement is checkable, one is a guess, and one important question has no answer at all — none of which the first version lets you see.
#Patterns fast agent work leaves behind
None of these is a defect on its own. All of them are worth looking for, because they cluster.
- Three solutions to one problem. Different sessions solving the same thing differently, each locally sensible.
- Thorough happy paths, thin unhappy ones. What happens when the provider times out, the queue backs up, or the same request arrives twice.
- Configuration that exists and is never read, or is read from two places with different defaults.
- Extensive comments that describe an earlier version of the function they sit above.
- Tests that assert the implementation rather than the requirement, and therefore pass whatever the code does.
- Dependencies added for one use and never removed.
- Generated documentation that is fluent, extensive and unverified.
#Using the agent to explain its own work
This works well and is the fastest route to a picture. Three things to insist on, or you get prose rather than an account:
- A locator for every claim. File and line range, or explicitly marked as inference.
- An unknowns section that is not empty. An agent that reports nothing it was unsure about has filled the gaps and stopped distinguishing them.
- A coverage statement. What it actually looked at, and what it skipped. Without this you cannot tell a complete picture from a partial one.
These three requirements are exactly what 1ADK's Evidence Package format encodes, and there is nothing stopping a team from asking for the same thing themselves.
#What goes wrong
Trusting a fluent explanation because it is well written.
Instead Ask for the locator. Fluency is uncorrelated with accuracy and always has been.
Reviewing the code line by line.
Instead Derive the structure, then read the parts the gap list points at. Reading a large generated codebase linearly is slower and produces a worse picture.
Treating generated documentation as documentation.
Instead Treat it as a claim to be tested. Extensive unverified documentation is more dangerous than none, because it is believed.
Asking again next quarter instead of keeping the answer.
Instead An agent holds nothing between sessions. Two answers a month apart cannot be compared unless somebody kept the first.
Concluding that AI is the problem.
Instead The bottleneck moved. Software got faster to write; understanding did not get faster to create by hand. That is a process problem, not a tooling one.
#What this does not cover
What this does not do
- Code review. This is about understanding what exists, not about whether it is well written.
- Security testing. Checking that a webhook verifies its sender is on the list; a security assessment is a different exercise.
- Testing strategy. Whether the tests are adequate is a separate question with a separate method.
- Whether to consolidate the duplicated approaches. The picture shows you the three; the judgement is yours.