Prompt Injection Test Suites for UK Business AI Agents

Tools & Technical Tutorials

23 June 2026 | By Ashley Marshall

An AI agent that can act inside a business needs more than a good system prompt. It needs a launch test suite that tries to break the workflow before production does.

Prompt injection testing has become a launch gate

A UK business that is preparing to launch an AI agent needs a different security question from the one used for a chatbot pilot. The question is no longer only whether the model answers politely or refuses obviously bad requests. The question is whether the agent can be manipulated into reading the wrong source, trusting hostile content, calling the wrong tool, leaking information, changing a record, sending a message or escalating a workflow without authority. Prompt injection testing should therefore be a production launch gate, not a late demo exercise.

The UK National Cyber Security Centre has been direct about the agentic risk. In guidance published last month, the NCSC describes agentic AI as systems that can access data sources, remember context, make decisions, use tools and take actions without continuous human intervention. It warns that agents inherit LLM risks such as jailbreaking and prompt injection, while their extra autonomy and complexity can increase the attack surface and make behaviour harder to predict, test and govern. That is the exact reason a generic penetration test is not enough for an agent with business permissions.

A test suite should be built around the launch scope. If the agent reads email, test malicious email. If it summarises uploaded PDFs, test hidden instructions in PDFs. If it browses websites, test hostile web pages, metadata and product reviews. If it can update CRM records, approve refunds, draft responses or create tickets, test whether injected instructions can influence those actions. The point is to measure the agent in the environment it will actually use, with the tools, permissions, source material and approval gates it will have in production.

This is also a governance issue. The UK government's AI Cyber Security Code of Practice sets out thirteen principles across secure design, secure development, secure deployment, secure maintenance and secure end of life. It includes documenting data, models and prompts, conducting appropriate testing and evaluation, and monitoring system behaviour. For a business AI agent, prompt injection tests are where those principles become evidence. Related: MCP tool access reviews for business agents.

Sources: NCSC agentic AI guidance and GOV.UK AI Cyber Security Code of Practice.

The threat is indirect, multi-source and business-specific

The leading misconception is that prompt injection is only a user typing a hostile instruction into a chat box. That is the easiest pattern to understand, but it is not the pattern most likely to matter in production. The serious business risk is indirect prompt injection: the model consumes untrusted content from a document, web page, email, ticket, spreadsheet, calendar invite, transcript, knowledge base article or uploaded image, and then treats malicious text inside that content as an instruction. Once the agent has tools, the attack is no longer just about the answer. It is about the action that follows.

OWASP's 2025 LLM risk guidance keeps prompt injection as LLM01 and explicitly separates direct and indirect injection. It also notes that techniques such as retrieval augmented generation and fine-tuning do not fully mitigate the vulnerability. That matters because many UK businesses are launching agents specifically around retrieval, internal knowledge and workflow automation. The system is designed to pull in external or semi-trusted context. That useful context can also become the delivery mechanism for hostile instructions.

The NCSC's adversarial AI guidance, published on 29 April 2026, gives a useful taxonomy for test design. It lists direct prompt injection, indirect prompt injection, attacks split across multiple modalities, privacy compromise, integrity attacks, knowledge base poisoning, defence evasion, execution, exfiltration and other agent security issues. A practical test suite should turn that taxonomy into scenarios. For example: a supplier PDF tells the agent to ignore the approval policy; a web page hides a request to send customer data to a third party; a helpdesk ticket asks the agent to close itself; a CV includes instructions to rank the applicant highly; a product review tells a buying agent to favour a specific vendor.

Business specificity matters because a retail returns agent, a legal intake assistant, a sales research agent and a finance reconciliation agent all have different failure modes. A generic list of jailbreak prompts may catch obvious problems, but it will miss workflow risks. The useful test suite starts with threat modelling: which sources are untrusted, which tools can cause harm, which actions need approval, which data must never leave the workflow, and which outputs become operational records?

Sources: OWASP LLM01 prompt injection and NCSC adversarial AI guidance.

Recent benchmarks show why launch tests must be realistic

Recent benchmark work shows why prompt injection test suites need to look like real work, not theatre. A simple test that says ignore previous instructions is useful as a smoke test, but it gives a false sense of coverage if the agent will operate across browser content, files, tools and multiple stakeholder interests. Attackers do not need to break the model in the abstract. They only need to get the deployed agent to take a harmful path while the user believes the task is progressing normally.

CSO Online reported on 12 June 2026 that StakeBench researchers ran 3,168 adversarial executions across NanoBrowser and BrowserUse using 264 benchmark cases. In that reporting, indirect prompt injection attacks hidden in ordinary web content such as product reviews and metadata achieved attack success rates from 41.67% to 68.16%, while direct prompt injection exceeded 79% across all tested configurations. The important lesson is not the exact number for your stack. It is that realistic web-agent tasks produced repeated failure patterns, including attacks that could harm third parties without obviously disrupting the user's delegated task.

Visual and multimodal agents add another layer. VPI-Bench, published for ICLR 2026 and last modified on 11 April 2026, introduced 306 visual prompt injection test cases across five widely used platforms. The authors reported that current computer-use agents and browser-use agents could be deceived at rates of up to 51% and 100% on certain platforms, and that existing defences offered only limited improvement. For businesses considering desktop automation, browser control or screenshot-based agents, this is a warning that test suites must include what the agent sees, not just what the user types.

A realistic launch suite should therefore include direct prompts, indirect sources, multi-turn set-ups, file uploads, retrieved knowledge, tool outputs, browser pages, images where relevant, permission boundaries and human approval gates. It should score more than pass or fail. Track whether the agent completed the user's task, advanced the attacker's objective, leaked data, called a forbidden tool, created unstable behaviour, asked for approval at the right point, or left enough logs for investigation. That gives security, operations and leadership a launch decision based on evidence rather than reassurance.

Sources: CSO Online on StakeBench and VPI-Bench on OpenReview.

A useful suite tests controls, not only prompts

The second misconception is that prompt injection testing is mainly about finding the perfect defensive system prompt. Better instructions help, but they are not a security boundary. A production test suite should test the whole control design: source trust, tool permissions, retrieval filters, output handling, user confirmation, transaction limits, logging, alerting, rollback and incident response. If a malicious instruction reaches the model, the business should still have layers that stop a bad output becoming a bad action.

The GOV.UK implementation guide for the AI Cyber Security Code is useful here because Principle 9 says released models, applications and systems should be tested as part of a security assessment process. It gives a chatbot example that includes testing against the OWASP Top 10 for LLM applications, focusing on indirect injection risks from PDF uploads identified in the threat model. That is the right pattern: tests come from the actual threat model, not from a generic security checklist copied at the end.

For a business AI agent, build the suite in layers. First, model behaviour tests: can the agent identify hostile instructions, maintain its role and refuse unsafe requests? Second, retrieval tests: can poisoned or irrelevant content influence the final answer? Third, tool tests: can an injected instruction cause the agent to call a tool outside policy, use the wrong parameters or skip confirmation? Fourth, data tests: can the agent expose secrets, personal data, pricing, credentials or commercially sensitive information? Fifth, workflow tests: can the agent bypass approvals, alter status fields, create records or send messages without the required human step? Sixth, observability tests: can reviewers reconstruct what happened from logs?

This design also makes the suite maintainable. Each scenario should have an owner, risk rating, data fixture, expected behaviour, detection rule and retest trigger. Retest when prompts change, retrieval sources change, the model changes, a tool permission changes, a new connector is added, a vendor releases a material update, or the workflow moves from pilot to wider deployment. The result is a living safety net for agentic automation, not a one-off report that is stale before launch week ends.

Source: GOV.UK implementation guide for the AI Cyber Security Code.

UK businesses need launch criteria, not just red team notes

Red team notes are useful, but they are not enough for a production decision. A business needs launch criteria that leadership can understand and that engineering can enforce. That means defining which failures block launch, which failures are acceptable with mitigation, which failures need a risk owner, and which features must be disabled until the control improves. Without that discipline, prompt injection testing becomes a collection of interesting examples rather than a decision system.

A practical launch gate should include minimum thresholds. No scenario should allow unauthorised access to sensitive data. No scenario should allow the agent to send, delete, approve, refund, publish or update a material record without the required policy step. Any tool call that changes business state should require explicit approval unless the workflow has a documented low-risk automation rule. The agent should refuse or quarantine hostile source instructions, preserve task context, and record enough evidence for review. Where the agent cannot reliably distinguish hostile content, the safest control may be to reduce agency: read-only mode, draft-only mode, smaller tool scope or mandatory human approval.

The ICO's agentic AI work is relevant because many agents process personal data and may support automated decision-making. The ICO says it is hosting industry workshops, updating guidance on automated decision-making and profiling in light of the Data (Use and Access) Act, working through the Digital Regulation Cooperation Forum, and continuing international work through the G7 Data Protection Authorities Emerging Technologies Working Group. For UK businesses, the message is simple: agentic AI is moving into a regulatory conversation about information rights, accountability and meaningful control.

Launch criteria should therefore combine security and data protection. Classify the data the agent can access, define whether outputs affect individuals, record whether a human review is meaningful, and confirm retention and audit arrangements. A prompt injection test that produces a personal data leak is not only a model reliability issue. It may be a breach scenario. A test that tricks an agent into making or shaping a significant decision may require data protection review as well as security remediation.

Source: ICO tech futures report on agentic AI. Related: AI inference audit trails for board governance.

The counterargument is right, but incomplete

The strongest counterargument is that prompt injection may never be fully solved, so test suites can create a false sense of security. That concern is valid. A test suite cannot prove that an AI agent is immune to manipulation. It cannot cover every phrasing, every document, every website, every future model update or every attacker strategy. It can also be gamed if teams only optimise for the known tests while leaving the broader architecture over-privileged. Treating a passing suite as proof of safety would be a serious mistake.

But that does not make testing optional. It changes what the suite is for. The goal is not to certify that prompt injection has disappeared. The goal is to expose likely failure modes before customers, staff, suppliers or attackers find them; to verify that compensating controls work; to define the residual risk leadership is accepting; and to create a repeatable evidence base for future changes. In other words, a prompt injection suite is closer to a fire drill than a guarantee that fire cannot happen.

The suite should also help the business decide when not to use an agent. If the workflow cannot tolerate a manipulated output, if the agent needs broad write access, if source material is highly untrusted, if approval would be meaningless in practice, or if logs cannot reconstruct an incident, the right answer may be a narrower automation. Use retrieval without tool actions. Use draft mode instead of send mode. Use deterministic rules for the high-risk step. Use an agent only after a human has narrowed the task and selected the source bundle.

For UK businesses, the practical standard is proportionate control. Start with bounded pilots, low-risk tasks and clear permissions. Build a test library from real documents, websites, emails, tickets and tool calls. Keep the failures. Turn each failure into a control improvement or an explicit risk decision. Retest after changes. Over time, that suite becomes one of the most valuable assets in the AI operating model because it captures the difference between a working demo and an agent ready for production.

Sources: NCSC agentic AI guidance and OWASP LLM01 prompt injection.

Frequently Asked Questions

What is a prompt injection test suite?

It is a structured set of scenarios that tests whether an AI agent can be manipulated by direct user prompts or hostile content inside documents, websites, emails, tickets, images or tool outputs. For business agents, it should test the whole workflow, including tools, approvals, data access and logs.

Why is prompt injection more serious for AI agents than chatbots?

Agents can often read business systems, remember context, call tools and take actions. A manipulated chatbot may produce a bad answer. A manipulated agent may update a record, send a message, reveal sensitive information or trigger a workflow.

Should UK SMEs run these tests before every AI launch?

Yes, in proportion to risk. A low-risk internal drafting assistant may need a lightweight suite. A customer-facing, data-rich or tool-using agent needs deeper testing before production because the possible harm is higher.

Can a test suite prove an agent is safe from prompt injection?

No. Prompt injection cannot be fully eliminated by a fixed test list. The value of the suite is to expose likely failures, verify controls, document residual risk and create a repeatable process for retesting after changes.

What sources should be included in indirect prompt injection tests?

Include every source the agent can consume: PDFs, web pages, emails, support tickets, CRM notes, spreadsheets, calendar invites, transcripts, knowledge base pages, product reviews, metadata and images or screenshots where relevant.

Who should own prompt injection testing?

Ownership should be shared. Product and operations define business harm, security defines threat scenarios, engineering instruments the tests, data protection reviews personal data risk, and the process owner accepts or rejects launch risk.

What should block launch?

Launch should be blocked if tests show unauthorised data access, sensitive data leakage, forbidden tool use, approval bypass, unlogged material action, inability to reconstruct incidents or agent behaviour that could create customer, staff, financial or legal harm.

How often should prompt injection suites be rerun?

Rerun the suite before launch, after model changes, after prompt or retrieval changes, when new tools or connectors are added, when permissions change, after security incidents and before widening a pilot into wider production.