AI Test Harnesses Should Come Before Agent Tool Access

Tools & Technical Tutorials

2 September 2026 | By Ashley Marshall

Most agent risk appears after the demo, when a system can touch real tools. A test harness turns that risk into evidence before production access is granted.

Start with the harness, not the agent

The most useful technical control for a business AI agent is often not another approval meeting. It is a small, repeatable test harness that proves what the agent can do before it touches live tools. A harness is the controlled environment where prompts, model settings, retrieval sources, tool permissions, approval rules and logging are tested together. It should run the same realistic tasks every time a model, prompt, connector or policy changes. For UK teams, this matters because the risky part of agentic AI is rarely a single answer on a screen. The risk appears when an agent can read customer data, call an API, update a ticket, trigger an email, query finance records or act across several systems without a person checking every step.

The NCSC has warned that organisations need to consider how agentic systems behave when they do not function as expected, and to plan how they are deployed, constrained, observed and responded to. Its August guidance says controls should be proportionate to the level of autonomy, and that higher autonomy increases the possible impact if the system accesses information it should not or acts outside scope. That is exactly the problem a harness is designed to expose. It lets a team move from confidence by demonstration to confidence by evidence.

What this means in practice is simple: before granting an agent production tool access, build ten to twenty representative tasks and run them in a sandbox. Include normal jobs, edge cases, hostile instructions, missing context, ambiguous requests, permission boundaries and rollback scenarios. Capture the prompt, model, tool calls, outputs, human approvals and final decision. A pass is not that the agent sounded plausible. A pass is that it completed the task within the written boundary, produced the right evidence, escalated at the right point and left an audit trail a manager could inspect. See the NCSC guidance on managing the cyber risk of agentic AI.

Treat tool access as a testable security boundary

Many organisations still treat AI tool access as a configuration choice: connect the CRM, connect the document store, connect the ticketing system and rely on the model to behave. That is backwards. Every connected tool is a security boundary and every permission needs a test. The harness should therefore start with an access map. List each tool, the actions available, the data classes exposed, the write operations allowed, the approval requirement and the log event expected. If an agent can search SharePoint, create a Zendesk response, update a HubSpot property or call an internal workflow, those capabilities should be visible in the harness before anyone argues about productivity.

The NCSC advice is especially useful here because it says organisations should understand the model, inference service and harnesses they are using, while recognising that built-in safeguards are not holistic. Model-level safety controls may be bypassed, may not be adequate for higher-risk environments and should be combined with technical and operational controls. A harness gives that combination a home. It tests prompts, model behaviour, tool permissions, sandbox restrictions, monitoring and emergency stop procedures as one operating system rather than as separate policy documents.

For a UK business, the first practical test set should include four categories. Permission tests prove the agent cannot access records outside the intended role. Action tests prove write operations require approval or are blocked when they should be. Data tests prove personal information, special category data and commercially sensitive material are not copied into places where they do not belong. Recovery tests prove a human can see what happened and reverse or contain the outcome. The counterargument is that this slows deployment. In reality, a harness usually speeds the useful work because it replaces vague concern with specific pass and fail evidence. Teams stop debating whether agents are safe in the abstract and start fixing the exact boundary that failed.

Use assurance techniques across the lifecycle

A good agent harness is also an AI assurance artefact. GOV.UK describes AI assurance as measuring, evaluating and communicating whether a system meets criteria such as regulation, standards, ethical guidelines and organisational values. Its portfolio of AI assurance techniques lists approaches including impact assessment, bias audit, compliance audit, certification, conformity assessment, performance testing, formal verification and ongoing testing. The important lesson for business leaders is that assurance is not one document at procurement time. It runs across scoping, data preparation, development, deployment, live operation and retirement.

That lifecycle view should shape the harness. At scoping, the harness records the intended purpose, users, data classes, business owner and unacceptable actions. During development, it checks task completion, refusal behaviour, permission boundaries and retrieval quality. At deployment, it tests the exact production configuration, including model version, connectors, tool scopes and approval gates. During live operation, it replays representative tasks after model upgrades, prompt changes, permission changes or supplier updates. At retirement, it proves the agent has lost access and that logs, outputs and residual data are handled properly.

Two figures from the UK assurance roadmap explain why this matters commercially. DSIT reported that the UK AI assurance market had over 524 companies and an approximate value of GBP1.01 billion gross value added in 2024, with potential to reach over GBP18.8 billion by 2035 if barriers to widespread AI adoption are addressed. That growth is a signal, not a decoration. Buyers, boards and regulated customers will increasingly expect evidence that AI systems are tested rather than merely described. A harness gives internal teams something concrete to show: test cases, results, owners, exceptions and change history. See GOV.UK guidance on the portfolio of AI assurance techniques and the trusted third-party AI assurance roadmap.

Build privacy and redress into the tests

Agentic AI is not just a cyber security question. It is also a data protection design question. The ICO has said that organisations remain responsible for data protection compliance when they develop, deploy or integrate agentic AI. Its Tech Futures report identifies risks including unclear controller and processor responsibilities, broader automated decision-making, purposes set too widely, processing personal information beyond what is necessary, unintended use of special category data, complexity affecting transparency and information rights, and new cyber security threats. Those risks are too specific to be handled by a generic statement that staff should use AI responsibly.

The harness should turn those privacy risks into test cases. If the agent receives a broad instruction, does it narrow the task or collect more personal information than needed? If a user asks it to infer health, political, financial or employment information from ordinary records, does it refuse or escalate? If a customer later asks why a decision was made, can the business reconstruct the data sources, tool calls and human approvals that led to the outcome? If a supplier model changes, can the team prove which workflows might be affected? These are not theoretical tests. They are the evidence that makes transparency, accountability and redress possible.

What this means in practice is that legal, security and operations teams need to help write the harness, but engineering should keep it executable. A data protection impact assessment can identify the risks, while the harness proves whether the implemented controls behave as promised. For example, if the DPIA says the agent may read only active customer support tickets, the harness should include closed tickets, restricted tickets, staff notes and unrelated account data to check the boundary. If the policy says significant decisions need human review, the harness should verify that a decision cannot complete without a named approval event. See the ICO report on agentic AI.

Make the pass criteria boring and measurable

A harness fails when it relies on subjective review alone. The pass criteria should be boring enough that a product owner, security lead and operations manager can agree what happened. Start with task completion, but do not stop there. Record whether the agent selected the right tool, stayed within the approved data scope, avoided unnecessary personal data, produced a source-backed output, escalated exceptions, respected spending or rate limits, and wrote an audit log with the information a human would need later. If the workflow involves customers, include response quality, harmful advice checks, disclosure requirements and complaint routes.

For technical teams, this does not require a huge platform on day one. A first version can be a repository of test prompts, fixtures, mocked tools, expected outcomes and a small runner that records model responses and tool-call traces. As the system matures, connect the harness to observability tools such as OpenTelemetry traces, SIEM alerts, model evaluation frameworks, policy engines and ticketing systems. The point is not to buy the biggest AI governance product. The point is to create a repeatable evidence loop that survives staff changes, model upgrades and supplier changes.

The strongest counterargument is that real-world work is too varied for fixed tests. That is true, but it misses the role of a harness. The harness is not a promise that every future task will be safe. It is a minimum release gate that catches predictable failure modes and documents the assumptions behind a launch decision. You can still use live monitoring, user feedback and incident response after release. In fact, the harness should feed those systems. When a live incident happens, convert it into a regression test so the same failure is not accepted twice. Over time, that makes agent deployment less dependent on individual judgement and more dependent on organisational learning.

Make harness results part of the release gate

The final step is governance by release gate, not governance by slide deck. No production agent should receive new tool access until the harness has run against the exact configuration being released. That includes model version, prompt version, retrieval index, tool scopes, user role, approval settings, rate limits, logging and rollback path. The release record should show who approved the change, which tests passed, which exceptions were accepted, what monitoring is in place and when the next review happens. If the agent fails a critical test, the release stops. If it fails a lower-risk test, the exception needs an owner and an expiry date.

This approach fits the way UK boards and senior leaders increasingly need to talk about AI. They do not need to understand every prompt trick, but they do need to know whether the business can prove control over live systems. A harness gives them a practical answer. It also gives procurement better questions for suppliers: can we test your agent before production, can we inspect tool-call logs, can we pin model versions, can we export evaluation results, can we run our own red-team cases, and can we disable access quickly if behaviour changes?

The businesses that get this right will still move quickly. They will just move with a stronger operating rhythm. New use case proposed, risk tier assigned, harness extended, sandbox run completed, evidence reviewed, production access granted or refused. That rhythm is more credible than trusting a demo, and more workable than freezing every agent project until perfect regulation arrives. The practical standard is not zero risk. It is documented, tested and reviewable control over the risks the organisation is choosing to accept.

Frequently Asked Questions

What is an AI agent test harness?

It is a controlled test environment that runs repeatable tasks against an AI agent configuration, including prompts, tools, permissions, data fixtures, approval rules and logs.

Is this different from ordinary prompt testing?

Yes. Prompt testing checks language behaviour. A harness checks the whole workflow, including tool calls, data boundaries, escalation, monitoring and recovery.

How many tests should a first harness include?

A useful first version can start with ten to twenty representative tasks covering normal work, edge cases, hostile instructions, permission failures and rollback.

Who should own the harness?

Engineering should keep it executable, but security, operations, legal and the business owner should help define the risks and pass criteria.

Do we need a paid AI governance platform?

Not at the start. Many teams can begin with test prompts, fixtures, mocked tools and trace logs, then integrate specialist tools as risk and scale increase.

When should the harness be rerun?

Rerun it before production access, after model or prompt changes, when connectors or permissions change, after supplier updates and after relevant incidents.

What counts as a failed test?

A test fails if the agent uses the wrong tool, exceeds permission scope, processes unnecessary personal data, misses required approval or leaves an inadequate audit trail.

Does a harness remove the need for human oversight?

No. It makes oversight more useful by giving humans evidence about what the agent did, what it could do and where escalation is required.