What should staff do if an email or document tells our AI assistant to ignore its rules?

2 November 2026

Stop the action, not just the suspicious sentence

If an email or document tells your AI assistant to ignore its rules, stop any connected action, keep the original material and report it to the person responsible for the workflow. Do not follow the instruction, test it against live customer records or assume a reassuring AI reply means nothing happened. A document is information to read, not permission to change how your business operates.

An assistant that only drafts a summary has a different risk profile from one that can send messages, retrieve confidential files or update accounts. Establish what it could actually do during the affected session. If it can act externally, an authorised administrator should suspend the relevant automation or connection while the incident is checked. Staff should use the agreed stop mechanism rather than improvise changes they cannot reverse.

Keep ordinary work moving through a manual fallback where possible. For example, read the genuine supplier message yourself and prepare a reply without involving the affected assistant. Do not forward the suspicious attachment around the whole company as a warning. Share its location through your approved incident channel, with access limited to the people investigating.

Why this is a business security issue

This is commonly called indirect prompt injection: instructions arrive inside material the assistant has been asked to process. A supplier email, downloaded PDF, web page or customer attachment can contain text that tries to redirect the assistant. It might request a different answer, a new recipient, a confidential file or a change to the assistant's normal approval process.

The UK National Cyber Security Centre gives a concrete recruitment example: a CV contains hidden instructions telling an AI system to approve the candidate for interview. That is an illustration of the attack mechanism, not a reported client incident or a claim that every recruitment tool is vulnerable. The important point is that the candidate controls the document, but should not control your selection rules. See the NCSC explanation of prompt injection.

The same NCSC guidance warns that language models do not enforce a reliable security boundary between instructions and data. Telling the model to ignore malicious text may help, but it is not equivalent to removing the assistant's ability to disclose files or send an email. A convincing answer is not evidence that a security boundary held.

What staff should record and who should investigate

Record when the material was received, where it is stored, which assistant processed it and what the assistant appeared to do. Preserve the original email or file, relevant conversation and any visible error or action notice. Avoid copying personal data into an unapproved ticketing tool. Your incident owner should decide what evidence is necessary and where to keep it securely.

Report suspected exposure as well as confirmed harm. Staff do not need to prove that an attack succeeded before raising a concern. A useful report says: the assistant read this source, it was connected to these systems, and this unexpected action appeared. It does not need a technical diagnosis or an accusation against the apparent sender, whose account might itself be compromised.

The administrator or supplier should then review actual application logs, tool calls, recipients, downloads and changed records. Asking the assistant whether it obeyed the malicious instruction is not a substitute. Check the specific account and time window, including queued actions that might execute later. If a connection is suspended, record who owns the decision to restore it and what checks are required first.

Can a paid business AI account prevent this?

No subscription label makes arbitrary documents safe. Microsoft Copilot, ChatGPT and a custom assistant can be useful alternatives for different jobs, but compare the permissions and controls of the actual workflow rather than treating a brand as a security guarantee. A business account may offer administrative controls; it does not automatically mean every connection has been configured appropriately.

Ask the supplier to demonstrate that untrusted content cannot directly authorise a new recipient, change payment details or retrieve another customer's files. Where consequential actions are possible, approval should happen through a separate control that checks the action itself. A confirmation message generated by the same assistant is not necessarily independent approval.

Prefer narrowly scoped access, approved destinations and read-only processing where it meets the need. Keep secrets out of the material the model can retrieve. Separate reading a supplier document from changing a supplier's bank details. These controls constrain the consequences even when the model misunderstands content. Existing email security, phishing reporting and access management still matter; buying another AI product should not be the first response to an unclear incident.

What UK data protection duties apply?

A malicious sentence is not automatically a reportable personal data breach. However, an assistant disclosing client information, altering personal records without authority or causing a significant loss of availability may create one. The person responsible for data protection should assess the facts promptly, not wait for a supplier to label the event. The ICO personal data breach guide explains that breaches concern confidentiality, integrity and availability, not only stolen files.

The ICO gives another concrete UK regulatory example: theft of a customer database that could enable identity fraud would need notification because of the likely consequences for individuals. That is guidance illustrating the threshold, not a claim about a particular AI incident. Assess the people affected, the sensitivity of the information and the likelihood and severity of harm.

A notifiable breach must be reported to the ICO without undue delay and, where feasible, within 72 hours of becoming aware. Notification is not required where the breach is unlikely to risk people's rights and freedoms, but the decision must be documented. High-risk breaches also require informing affected individuals without undue delay. Keep records of personal data breaches even when notification is unnecessary, and obtain specialist advice where the facts or obligations are unclear.

What should you require before restarting?

Do not restart just because someone deleted the suspicious paragraph. The underlying source could reappear, and a different document could exploit the same access. Identify whether the assistant merely produced a wrong summary, attempted a blocked action or completed an unauthorised one. Correct affected records and communications through an approved process, preserving the evidence needed to understand what happened.

Use a safe test environment with invented records to reproduce the failure and check the repair. The NCSC secure AI development guidance covers four lifecycle areas: secure design, development, deployment, and operation and maintenance. Its practical implication here is that incident handling and monitoring belong in the service, not only in the launch checklist.

Require evidence that access restrictions, recipient checks and approval controls work outside the model's own judgement. Check normal business tasks too, so a supposed repair does not quietly break the workflow. A named owner should accept the remaining risk, record the restart decision and watch the first resumed runs. If the supplier cannot explain the permissions or provide usable logs, keep consequential actions disabled.

When this approach is not right for you

Not every suspicious phrase needs a company-wide shutdown. A staff member reading a security article that quotes an attack is not the same as an assistant silently changing customer records. Contain the affected workflow in proportion to what it can access and do. Avoid accusing a genuine customer because their document includes instructions intended for a human reader.

Equally, an autonomous assistant is not suitable where a single mistaken action would cause unacceptable harm and your business cannot independently restrict that action. Use ordinary rules-based automation, a manual review queue or a draft-only assistant instead. Less autonomy can still remove useful work without giving an external document influence over payments or confidential disclosures.

Your policy should fit on a short instruction staff can remember: stop connected actions, preserve the source and report through the agreed route. Managers then own investigation and recovery. If you want help deciding whether a proposed assistant has sensible boundaries, bring its actual permissions and one representative workflow to a conversation with the implementation team. Start with the risk you need to control, not a promise that an AI system can never be fooled.

Is This Right For You?

This guidance is for UK businesses whose assistants read third-party material or connect to business systems. Use proportionate containment for a draft-only tool; require independent controls before allowing confidential disclosures or consequential actions.

Frequently Asked Questions

Should staff delete the email immediately?

Preserve it through your approved reporting process unless your security team instructs otherwise. Deleting the only copy can remove evidence needed to establish what the assistant read and did.

Is this the same as ordinary phishing?

They overlap, but prompt injection targets the assistant's handling of content. A document can attempt to redirect the assistant without persuading a person to click a link.

Can we solve this with a stronger system prompt?

A stronger prompt may reduce some failures, but it is not a reliable security boundary. Limit permissions and enforce consequential approvals outside the model.

Does read-only access remove every risk?

No. Read-only access prevents some changes but can still expose confidential information through generated answers or connected output channels. Restrict retrieval and destinations too.

Must every attempted prompt injection be reported to the ICO?

No. Assess whether a personal data breach occurred and the risk to individuals. Record relevant decisions; notify qualifying breaches within the applicable deadline.

Who should tell customers if information was exposed?

The designated incident and data protection leads should decide the required communication, timing and content. Staff should escalate promptly rather than independently send speculative messages.