AI Coding Agents Need a Review-Capacity Test, Not a Code-Volume Target

ROI & Cost Optimisation

21 October 2026 | By Ashley Marshall

Your coding agent may be writing more code while your team delivers no more software. The hidden cost is the review queue.

The output illusion starts before a pull request is merged

If your software team has started using AI coding agents, the first dashboard improvement will probably be activity. More code is written, more commits appear and more pull requests arrive. Those numbers are real, but none tells you whether customers received a useful feature sooner. A pull request can contain working code and still need extensive review, integration, security testing and deployment. Treating code volume as business output is like treating the number of invoices processed as cash collected.

The distinction is no longer theoretical. In their August 2026 working paper, Harvard researchers Fiona Chen and James Stratton analysed roughly 300 million work events across 718 firms. They found that adopting AI coding agents was associated with 30% more lines of code, 20% more commits and 23% more pull requests. Yet they found little evidence of a corresponding increase in completed software work at firm level. Their evidence covers activity through March 2026, so it is a useful warning about measurement, not a permanent verdict on every newer tool.

For a UK business leader, the useful question is not whether an assistant can produce code. It can. The question is whether the whole delivery system can absorb, validate and release that code without building a new queue. If the same senior engineers must inspect a rising volume of changes, faster drafting simply moves the constraint. Ask for a baseline that starts when a task is accepted and ends when the change is safely in production. Count user-visible outcomes, defect rates and the effort required to get there. Anything earlier is an intermediate measure, not proof of return on investment.

Review capacity is the constraint to price into the business case

The same Harvard study reported that average pull-request review time rose by 49% after AI coding agents were introduced. The share of pull requests requiring changes nearly doubled and reviewer comments rose by 35%. These are averages across the observed firms, not a prediction for your team. They do, however, point to a cost that licence comparisons routinely miss: scarce reviewer attention. A principal engineer spending an extra hour untangling a generated change may wipe out several hours saved during drafting.

What this means in practice is straightforward. Track the time a change spends waiting for review, the hands-on minutes spent reviewing it, how many revision rounds it needs and whether it later causes an incident. Separate agent-assisted and ordinary changes, then compare similar task types rather than averaging a tiny configuration edit with a payment-system migration. If the queue grows, lower the arrival rate or make changes smaller. Buying another coding seat is unlikely to help a reviewer bottleneck.

This is also a planning issue. Teams often let agents open several large pull requests overnight because creation is cheap. In the morning, one or two maintainers inherit a pile of unfamiliar code. Give reviewers a daily capacity limit and require each change to identify its owner, tests, affected systems and rollback route. A modest, well-explained patch can be faster to deliver than a spectacular agent-produced rewrite. If leadership wants a forecast, model both sides of the pipeline: expected coding hours saved and expected review, remediation and release hours added. The net difference is the part worth discussing with finance.

Use the NCSC spectrum to decide where autonomy belongs

The National Cyber Security Centre's 2026 guidance on the vibe-coding spectrum argues for calibrating the amount of AI involvement to the context and to today's reliability, rather than assuming that all AI-assisted software development is equivalent. That is a better procurement frame than an organisation-wide yes or no. A disposable internal prototype, an isolated data transformation and the authentication path of a customer-facing product do not deserve the same degree of autonomy.

Create three lanes. The first is low-risk exploration, where a developer may use an agent to draft experiments in a sandbox with synthetic data. The second is routine production work, where the agent can propose a small change but a named engineer reviews it and standard automated checks run. The third covers sensitive changes such as payments, permissions, personal-data handling and infrastructure access. Here, require explicit scope, a security-aware reviewer and evidence of tests before merge. A team can move work between lanes, but it should record why.

The point is not to turn every edit into a committee meeting. It is to put friction where the consequence of a mistake is high. For UK firms processing personal data, that decision should sit alongside existing security and data-protection duties. The NCSC's secure AI development guidance stresses secure design and deployment rather than treating AI output as inherently trustworthy. Your existing code review, secrets handling, access control and incident processes still apply when the first draft comes from a model. Record the lane decision in the ticket so the next reviewer can see the reason, not just the outcome. Revisit it when the scope expands or the code starts touching a more sensitive system.

Make a small pilot measure the whole delivery cycle

A meaningful pilot needs a comparison group or at least a defensible before-and-after baseline. Select one class of repeatable work, such as modest API integrations, internal reporting changes or well-scoped bug fixes. Record the median time from a ready ticket to a deployed change over the previous month. Then run the same workflow with an approved coding assistant or agent. Record author time, reviewer time, elapsed review time, revision rounds, deployment lead time and defects found within the following weeks. Keep task size, team composition and release rules as similar as possible.

Do not reward an agent for opening more pull requests. Define success as more accepted, stable outcomes per unit of total team effort. If the pilot saves drafting time but increases the time that senior staff spend reading code, call that out plainly. If it speeds small maintenance jobs yet slows architecture-heavy changes, make adoption conditional on task type. A tool does not need to win every contest to be useful. It needs a clear operating envelope and a result that holds after downstream work is counted.

The evidence itself argues for that caution. A September 2026 review of 20 principal sources found heterogeneous productivity results: one bounded JavaScript task was completed 55.8% faster with Copilot, while experienced maintainers in mature repositories took 19% longer when AI was permitted. Those are different settings, not contradictory instructions. They show why a generic claim that developers are a fixed percentage faster is a poor business case. What this means in practice is that your pilot should identify where your team gains, where verification costs rise and where the tool should stay out.

Put quality evidence beside every change

The review queue shrinks when an author, human or agent, supplies evidence in a format the reviewer can use. Require each pull request to state the user problem, scope of the change, affected dependencies, tests run, known limitations and rollback approach. For an AI-generated change, add the prompts or task brief where appropriate, but do not mistake a transcript for a design explanation. A reviewer needs to understand what the code does and what could fail, not the conversational route by which it appeared.

Automated tests help, but a passing suite is not a substitute for looking at the right risks. An agent can write a test that confirms its own incorrect assumption. Ask for negative cases around permissions, malformed inputs, data leakage and failure recovery. Use existing static analysis, dependency checks and secret scanning in the normal pipeline. For a high-impact change, the reviewer should be able to point to a business requirement and trace how the implementation satisfies it. These controls are useful for human-written code too; AI simply makes weak evidence more expensive because output arrives faster.

A common counterargument is that another AI model can review the first model's work. That can improve triage and catch routine errors, but it does not eliminate accountability. The Harvard researchers observed broad uptake of AI review tools, yet human review still dominated their dataset. Treat AI review as a filter that flags concerns and improves explanations, not as proof that a change is safe to release. Assign one human owner to the outcome, especially where customer data, money or operational continuity is involved. If ownership is ambiguous, the apparent labour saving is only an unpriced transfer of risk.

A buyer test for leaders funding the next coding tool

Before renewing or expanding an AI coding contract, ask the technology lead for a one-page account of the complete workflow. Which task types are eligible? How many changes did the team deploy during the pilot? What happened to review time, rework and escaped defects? Who may let an agent access repositories, credentials and production-like data? Which engineer signs off a sensitive change? If those questions have no answers, the organisation is buying generation capacity without deciding how to use it.

Set a threshold that makes commercial sense. For example, the team might require a meaningful fall in median ready-to-production time with no deterioration in defect and incident measures. The exact target is your management choice, not a statistic borrowed from a vendor case study. Review the result after a full release cycle, since a three-day demonstration cannot reveal maintenance cost. Include the subscription, compute usage, training, review and remediation effort in the calculation. A cheap licence can be expensive if it creates work elsewhere.

None of this is an argument against AI coding. It is an argument for deploying it as part of a production system. The Harvard paper's findings could change as agents improve, and the NCSC explicitly recognises that today's risk balance may shift. But a future improvement does not make today's review capacity infinite. Start with a bounded lane, insist on small explainable changes, compare total delivery outcomes and expand only where the evidence is positive. That is how a UK business turns faster code generation into actual software value rather than another impressive activity chart.

Frequently Asked Questions

Do AI coding agents make developers faster?

They often speed code generation, but the net effect depends on review, testing and release work. Measure the complete task cycle for your team.

Why can more pull requests fail to improve delivery?

Each request still needs review, revision, integration and release. If reviewer capacity is fixed, extra arrivals create a queue.

What is the best metric for an AI coding pilot?

Use stable, accepted changes deployed per total team hour, supported by ready-to-production time, review effort and defect measures.

Should AI-generated code always receive human review?

Production changes should have a named accountable owner and risk-appropriate review. Higher-risk changes need stronger evidence and a security-aware reviewer.

Can an AI tool review another AI tool’s code?

Yes, it can flag routine concerns and improve triage, but it should not replace an accountable release decision or independent quality checks.

How long should a coding-agent pilot run?

Run it through at least one complete release and early maintenance cycle so you can observe review, deployment and defect outcomes.

Which tasks are good starting points?

Choose bounded, repeatable, lower-risk tasks with clear acceptance tests, such as small internal integrations or well-scoped maintenance fixes.

Does the Harvard study prove agents are not worthwhile?

No. It reports average bottlenecks in an observed dataset ending in March 2026. Your task mix, controls and newer tools may produce a different result.