Technical due diligence on an AI-built codebase

The short version

• Volume is free now. Commit count, coverage percentages, documentation weight — everything the old checklist used as a proxy for a functioning team can be generated in an afternoon, and proves nothing.
• What still carries signal is review, understanding, and operation. Who pushed back on a change, whether an engineer can explain a file nobody wrote, whether anyone has ever restored a backup.
• The most predictive exercise is watching a small change made end to end. Not reading the code — watching the team change it.
• You are not buying what the product does today. You are buying the ability to change it safely, which is the one property no demo shows.

The checklist measures effort, and effort is free now

The traditional technical diligence pack asks how old the repository is, how many people contributed, how steady the commit cadence looks, what the test coverage number is, and whether the documentation exists. Every one of those questions is a proxy. What you actually wanted to know was whether a competent team spent real attention here, and the proxies worked because the artefacts were expensive to fake: nobody wrote four thousand commits, ninety percent coverage and a tidy README by accident.

Code generation broke the proxies. An agent produces all of those artefacts as a side effect of being asked to build the product, and a codebase written mostly by agents will often present better on paper than a decade-old human one: consistent formatting, a docstring on everything, high coverage, immaculate commit messages. None of that was ever the thing you cared about — it was evidence of the thing. The evidence has come loose from the thing it used to evidence.

To be clear about what this guide is not saying: generated code is not automatically bad code, and an AI-built product is not automatically a bad acquisition. Some of the best-run small products I see are built almost entirely with agents, carefully. The point is narrower and more useful — you can no longer read team quality off artefacts, so diligence has to test the underlying properties directly. The rest of this guide is how.

Commit history: what four thousand commits mean now

Start by discarding what the history used to tell you. Volume, cadence, contribution graphs, the impressive-looking burst of activity before the raise — an agent produces all of it in weeks, and a founder who knows you are coming can produce it deliberately. Treat the quantity of history as noise.

What still carries signal is the review trail — the parts of the history where a human exercised judgement against the machine's output:

• Rejections. Ask to see the last five changes a human refused, and why. A team that reviews its agent's work has these to hand. A history with no rejections and no reverts is a history in which nothing was ever questioned.
• Substantive review comments. Pull requests where a person pushed back on a design, not just approved. Self-merged pull requests with green ticks and no comments are the generated codebase's default state, and they tell you the review step was ceremonial.
• Authorship concentration. git log --format='%an' | sort | uniq -c takes ten seconds. If one account — often the agent's — wrote ninety-five percent of the commits, that is not itself a problem, but it means the review trail is now the only evidence of human judgement, so its absence becomes decisive.

Then pick three or four commits that look significant and ask, in the room: who reviewed this one, what would a wrong version of it have looked like, and what did you push back on? The answers are hard to fake in real time, which is exactly what makes the question useful.

The one question that predicts the rest

If you only have an hour with the engineering team, spend it on this. Pick a file that matters — the permissions check, the sync engine, the piece that would make the news if it broke — that no human on the team wrote. Put it on screen. Ask an engineer to walk you through it: why is it shaped the way it is, what breaks if this line changes, which test pins that behaviour.

Three outcomes are possible, and they sort acquisitions better than anything else in the pack:

1. A fluent explanation. The team has absorbed the generated code — read it, questioned it, made it theirs. This is the good outcome, and it genuinely happens; generation and understanding are not opposites.
2. Live reading. The engineer reads the file in front of you and reasons correctly about it. Capable person — but the map of the system is being drawn during your meeting, which means it did not exist before your meeting. The codebase knowledge is one search away, and so is everyone else's.
3. Deflection. "The agent handles that layer." You are being offered software that nobody understands, and every other number in the data room should be read in that light.

Follow it with a softer question that is just as revealing: what is the worst part of this codebase? A team that knows its system answers instantly, with specifics and a story. "It's all pretty clean" is the wrong answer from anyone who has actually operated software, because no operated software is all pretty clean.

Test suites that assert the implementation

Here is the trap specific to generated codebases: when the same model writes the code and the tests, the tests tend to encode the code's current behaviour — bugs included — as the expected outcome. The suite then verifies that the code does what the code does. Coverage climbs; safety does not. The coverage percentage is the most gameable figure in the data room, because executing a line asserts nothing about it.

You can spot this in ten minutes without being a specialist:

• Read the shape first. One test file per source file in perfect symmetry; tests named after functions rather than behaviours; assertions that a mock was called with particular arguments; internal modules mocked, not just the network and database edges; snapshot tests everywhere, asserting that the output is whatever the output was.
• Run the crude mutation test. Pick one business rule the product depends on. Deliberately break it in the source — flip a comparison, shift a boundary by one — and run the suite. If it stays green, the suite does not protect that rule, and the coverage number attached to it is decoration. Do this three times on three different rules. Tools like Stryker do it systematically; three hand-picked mutations tell you most of what you need in diligence.
• Ask what is deliberately untested, and why. A real engineering answer names trade-offs. Silence, or surprise at the question, means untested-by-accident rather than untested-by-decision.

Dependencies and licences arrive wholesale

An agent solving a ticket installs whatever solves the ticket. Repeat that a few hundred times and the dependency tree has grown without any single decision having been made about it. The exposure is checkable in an afternoon with free tooling, and in my experience it is the section most often skipped:

• Ask who chose them. Count the direct dependencies, pick five, and ask who decided each one belonged there. The honest answer is often "nobody" — and nobody is the finding, not the count.
• Scan the licences. Copyleft in shipped code changes what you are buying: AGPL in a hosted backend can oblige you to offer your server source to users of the service. That is a conversation with your lawyers, not a footnote — which is why you find it before completion, not after.
• Check health, not just presence. Known vulnerabilities via OSV and the package manager's own audit; abandonment by last-publish date; the wider picture on deps.dev. A committed lockfile with pinned versions, or a build that resolves differently every month.
• Ask about provenance. Generated code carries no trail of where a block came from — synthesised fresh, or reproduced from something licensed. Whether that distinction matters legally is an open question that has already reached the courts; what you can establish in diligence is whether anyone has ever looked, and write down the answer.
• Scan the full git history for secrets with gitleaks or trufflehog — a key committed early and "removed" later is still in the history, and may still be live. You are about to inherit every key ever committed.

Can anyone actually run it?

A product that works and cannot be operated is a demo with customers. The questions here are old ones, but generated codebases fail them in a particular way: the building was fast, so the operating muscle never had time to grow.

• The runbook test. Is there a document a stranger could follow to deploy, roll back, and restore from backup? If rollback is "revert the commit and redeploy", ask when that was last actually done.
• The restore question. "When did you last restore a backup, and how long did it take?" A backup that has never been restored is a hypothesis. This question has ended more than one of my reviews early.
• The 3am question. When it breaks at 3am, what pages whom, and what do they look at first? If the answer involves opening a chat window and asking the agent, write that down — it is an answer, and it tells you what the operational tooling is.
• The last incident, start to finish. Teams that have operated software have incident stories with timestamps and lessons. A product that has never had an incident has either never had users, or never noticed.
• The real bus factor. Often it is one founder plus a chat history. If the operational process is genuinely "ask the agent", then the agent's configuration, context files and prompts are operational artefacts — ask whether they are in the repository, and what happens when the tooling changes under them.

Best single move in this section: ask them to deploy to staging while you watch. Not to production, nothing dramatic — just the routine act, live. Fifteen minutes of watching a deploy outranks a page of architecture diagrams.

Where the personal data actually sits

On completion, the data liabilities transfer to you whole. Generated apps accumulate third parties the way they accumulate dependencies — an auth provider, a database, an analytics tool, error tracking, transactional email, an LLM API — each one holding some slice of personal data, often added by the agent mid-task with no decision recorded anywhere. So the assessment starts from the code, not from the paperwork:

• Enumerate the third parties from the code — search for API hosts and SDK imports — and compare the list against the privacy policy. In my experience the two routinely disagree, and the disagreement is a finding whichever direction it runs.
• Trace the LLM flows. Does personal data go into prompts — support tooling, "summarise this user" features, anything that pastes a record into a model call? To which provider, under which terms, retained for how long?
• Read a day of production logs. Generated logging tends to serialise whole request objects, emails and tokens included. What is in the logs is where the data sits, whatever the architecture diagram says.
• Check residency and retention as facts, not claims. Where the database region actually is versus what the policy says, and whether deletion actually deletes or just flags.

To be plain about the boundary: this section tells you where to look, and that is all it does. Whether what you find is compliant is a legal judgement for qualified advisers — the ICO's UK GDPR guidance is a sound orientation, and your lawyers are the ones who turn findings into a position.

"It works" is not what you are buying

The demo proves today's behaviour. You are buying tomorrow's: the bug fixed without breaking three other things, the feature added, the security review passed, the framework upgrade survived. The ability to change software safely lives in exactly three places — tests that pin behaviour, people who understand the system, and a release path that fails safely — and every section above is a way of measuring one of them. A generated codebase can hold all three. The point of this guide is that you have to check, because the artefacts that used to prove them no longer do.

"It works" also hides a security dimension. Veracode's 2025 study across 100+ models found AI-generated code chose the insecure implementation about 45% of the time, with no improvement in newer models (Veracode). So "no known vulnerabilities" from a team that has not run a real review means "nobody has looked", and it should be read as such.

Which leads to the single best exercise in the whole process. Ask the team to make one small, real change while you watch — rename a field, tighten a validation — end to end, until they would be comfortable shipping it. Watch where they start, what the tests catch, what they check before calling it safe, and how long the whole loop takes. That hour tells you more than the data room does, because it is the product you are actually buying: not the code, the ability to change it.

The questions to take into the room

Everything above, compressed to one page. Ask them roughly in this order — the later ones only mean something once you have the earlier answers.

History and review

• Show me the last five changes a human rejected, and why.
• This commit — who reviewed it, and what would a wrong version have looked like?
• What was your last revert?

Understanding

• Walk me through this file. What breaks if this line changes, and which test catches it?
• What is the worst part of this codebase?
• What does the system do that you wish it didn't?

Tests

• Break this rule in the source, in front of me. Does the suite go red?
• What is deliberately untested, and why?

Dependencies and licences

• Who chose these five dependencies?
• When did anyone last look at the licences you ship under?
• Has the full git history been scanned for secrets?

Operations

• When did you last restore a backup, and how long did it take?
• Tell me about your last incident, start to finish.
• Deploy to staging now, while I watch.

Data

• List every third party that holds your users' personal data — then we compare the list against the code.
• What personal data ends up inside LLM prompts, and with which provider?

The last hour

• Make this small change now, end to end, while I watch.

If you want help

Appaya runs this as a buyer-side review: the checks above, executed against the actual repository and infrastructure rather than the data room's account of them, written up with the evidence each finding is derived from — so you can check the reasoning instead of taking my word for it. If the codebase turns out to be in good shape, that is what the report says. It sits alongside your legal and financial diligence; it does not replace either.

Request a fit check How the audit works