Every row in the answer checks out. That still does not tell you which rows the AI failed to return.
Checking returned rows does not prove nothing was missed
An AI assistant can return a tidy list of customers, contracts or stock items, with a correct explanation beside every entry, and still give you the wrong business answer. The failure is not necessarily a fabricated number. It can be an eligible record that never reaches the list. Checking the rows you received proves something useful, but it does not prove the assistant found everything it should have found.
The GDP.xlsx research paper, dated 13 November 2026, makes this distinction concrete. Its authors assembled 70 professional requests across 13 domains, using 58 Excel workbooks and 12 CSV files. The files contained about 2.18 million populated cells. These were comprehension tasks over complete files, not simply requests to write a formula or produce a polished workbook.
In its public tenancy example, an agent missed three tenancies that qualified through break clauses recorded on another tab. One relevant tenancy was hidden by an AutoFilter. The returned entries could look convincing while the missing entries changed the answer. This is the kind of omission that a spot check of the output cannot reveal.
For a UK business owner, the practical question is therefore not just whether the answer contains mistakes. It is whether the process can demonstrate coverage of the relevant population. If you ask for every contract requiring action before year end, the word every creates a different acceptance test from asking for a few examples.
Separate those two tests in the brief. First, each returned record must qualify. Second, each qualifying record must be returned. They need different evidence, and a reassuring explanation from the same assistant is not independent proof of either.
Correct returned rows do not establish that every qualifying row was found.
Define the population before asking for the answer
Start with a short written definition of what belongs in the search. For a contract review, specify the reporting date, relevant legal entities, active and closed contracts, renewal and break conditions, and treatment of amendments. Say whether the task covers all underlying records or only records visible through the current filters. Otherwise, two different answers may both follow plausible interpretations of your instruction.
Then make an inventory of the workbook. List its sheets, tables, record identifiers, date fields, comments and any meaningful colour or formatting conventions. Identify which sources are authoritative and which are summaries. A dashboard is not automatically the full population. An archive tab is not automatically irrelevant. A badly named worksheet may still contain the only version of an important condition.
The GDP.xlsx authors report that 41 of their 70 tasks required cross-sheet linkage and 21 depended on structure or formatting. Those figures describe this deliberately challenging benchmark, not the prevalence of these problems across British businesses. They do, however, explain why reading one attractive summary tab is a weak general method.
What this means in practice is simple: request a coverage statement before accepting the final list. It should identify the sheets inspected, the rules applied and any parts of the workbook the tool could not interpret. Ask the assistant to distinguish a genuinely empty result from a missing or unreadable source.
Keep this proportionate. You do not need a custom platform to begin. An Excel table, Power Query query or established reporting export may already provide a reliable population. Use those existing controls first. The AI can help interpret or explain exceptions without being the only mechanism deciding which records exist.
Agree what every means before judging whether the answer is complete.
Reconcile identifiers, not just reassuring totals
A control total is helpful, but an equal total can conceal different mistakes. An omitted record and a duplicate record may cancel each other numerically. Two amounts may offset. A workbook can therefore reconcile at headline level while the action list is still wrong. For consequential work, compare the identities of the records as well as their count and value.
Imagine an illustrative test with 120 active service contracts. An independently checked rule says nine need a renewal decision this quarter. The assistant returns eight. Do not simply ask it whether it is confident. Compare its returned contract IDs against the independently derived candidate set. The missing identifier tells the reviewer where to look. These numbers are a proposed test example, not a reported client result.
Ask for a small evidence table containing the stable record ID, source sheet, relevant cell or row, qualifying condition and proposed action. Keep rejected candidates separately, with the reason for rejection. That makes it possible to check both omissions and inappropriate inclusions without rebuilding the whole exercise from a narrative paragraph.
For repeatable tabular rules, Excel filters, Power Query or an existing database report may produce the reference set more reliably than an open-ended chat. AI can explain ambiguous wording or help investigate unmatched records. It should not replace a deterministic check merely because the conversational interface feels easier.
When an independent reference set is unavailable, say so. A second model can offer another perspective, but both models may miss the same tab or inherit the same incomplete export. Treat agreement as additional evidence, not a guarantee. Where the financial or customer impact is material, retain a competent manual coverage review until you have a cheaper check that genuinely works.
Matching totals are useful; matching the actual record identifiers is stronger.
Test omissions deliberately, then price the review
The strongest counterargument is that this sounds like asking people to redo the work. If a person must inspect every sheet and rebuild every result, where is the saving? That is a fair objection. Some workflows will not deliver a worthwhile benefit yet. The answer is to measure the complete accepted process, including the search for missing records, rather than counting only the seconds until an AI response appears.
The GDP.xlsx paper reports a top strict pass@1 score of 38.3% across 20 configurations, with a median of 19.3%. Strict passing means satisfying every rubric criterion, not getting that percentage of cells correct. Tasks were selected from workflows where models had previously failed professionals. Provider errors and timeouts were excluded from scoring. These qualifications matter: the benchmark is not a universal spreadsheet failure rate or an end-to-end reliability promise.
Use a small representative test pack with known answers. Include a qualifying item on a second tab, an active filter, an amendment that changes eligibility, a duplicate identifier and a record that almost qualifies but should be excluded. Preserve the original files and expected sets. Test the actual tool configuration your staff use, not a supplier demonstration with a simpler workbook.
Record preparation time, run time, reviewer time, missed qualifying records, false inclusions and unresolved questions. Compare that with the current process on equivalent files. Decide in advance which omissions make the result unacceptable. A research shortlist and a payment instruction should not share the same release rule.
Recent analysis by Dean Lee also emphasises the cost of checking coverage. The useful business outcome is an accepted answer that saves work, not an impressive first draft that quietly transfers the difficult checking back to your team.
Measure the time to an accepted answer, including the work needed to find omissions.
Make completeness a release condition, not a prompt promise
For the next reporting cycle, choose one recurring list that matters: renewals needing action, overdue supplier evidence, contracts with expiring terms or customers awaiting a promised response. Give it a named business owner. Write down the population, qualifying rule, source files and evidence needed before anyone acts on the answer. Start with one bounded workflow rather than an instruction to analyse everything.
Keep the assistant in an advisory role during the trial. It can assemble candidate records and explain exceptions, but it should not automatically send notices, change customer status or initiate payments from an unverified list. Define an unresolved state for incomplete source coverage. A useful system can say that it cannot establish completeness and return the work for review.
If the workbook contains personal data, accuracy is also a data protection concern. The ICO accuracy guidance says organisations should take reasonable steps to ensure personal data is not incorrect or misleading and make its source and status clear. The ICO notes that this guidance is under review following the Data (Use and Access) Act. That is not a new legal requirement to use this particular checklist; it is relevant context for preventing misleading operational records.
What this means in practice is that a customer should not be labelled as having no outstanding issue merely because their row was missed. Preserve the difference between no qualifying record found and a complete search establishing none exists. Keep a correction route and evidence of what was actually checked.
After a few representative cycles, review whether independent controls make checking affordable. If they do, expand cautiously. If they do not, keep the AI for lower-consequence assistance. The goal is not to distrust every answer. It is to know which business claims your evidence can support.
A system that cannot establish completeness should say so before anyone acts.
Key Takeaways
- Define the full record population, reporting date and qualification rules before the AI runs.
- Require stable record identifiers and source references, not only a narrative answer.
- Compare returned and expected identifier sets independently; totals alone can hide offsetting mistakes.
- Test hidden rows, secondary tabs, amendments, duplicates and near-miss exclusions on known-answer files.
- Include completeness review time in ROI and block consequential actions when coverage remains unresolved.
