AI Fraud Controls Should Come Before Customer Agent Rollout

AI Trust & Governance

21 September 2026 | By Ashley Marshall

AI fraud is becoming a workflow design problem. If a customer agent can change records, trigger payments or trust synthetic evidence, the fraud controls need to exist before the launch demo looks impressive.

AI fraud is moving faster than normal approval controls

AI fraud is no longer a specialist threat that sits neatly inside cyber security or financial crime teams. It now belongs in the operating design for any UK business planning to deploy customer-facing agents, onboarding automation, finance assistants or delegated payment workflows. The reason is simple: the same capabilities that make an AI system useful for service and sales also make fraud attempts cheaper to create, easier to personalise and faster to vary.

The FCA's July 2026 Mills Review identified four major AI-related shifts in retail financial services, including the amplification of fraud and cyber risks. It also reported that 20% of consumers, equivalent to 11 million UK adults, said they would be likely to use AI capable of acting autonomously within pre-set goals. That matters beyond banking. If customers become comfortable delegating decisions to agents, criminals will build workflows that imitate customers, staff, advisers and suppliers inside the same channels.

For business leaders, the useful conclusion is not that every AI project should slow down. It is that fraud evidence should move earlier in the release process. A customer agent should not go live because its answers look helpful in a demo. It should go live when the business can show how it detects impersonation, blocks suspicious escalation, records the decision path and hands risky cases to a human with enough context to act. This is not abstract governance. It is the practical difference between a useful automation layer and a new route into the business.

Fraud risk now sits across platform, process and evidence

The strongest signal from recent UK material is that AI fraud is becoming a shared responsibility problem. The Home Office published Fraud in the Digital Age on 14 July 2026, looking at barriers to investigating and prosecuting fraud against businesses and individuals. TLT's September AI Brief summarised one of the key policy directions: more pressure on online services and platforms to prevent fraud, including discussion of stronger prevention obligations for regulated user-to-user services.

That direction should make business buyers think differently about AI tooling. The common mistake is to treat fraud control as a vendor feature: does the platform have moderation, identity checks, anomaly detection or a trust and safety team? Those are useful, but they are not enough. A business still has to decide what evidence it needs when an AI-assisted interaction results in a payment, contract change, account reset, refund, procurement request or regulated recommendation.

What this means in practice is that the control design has to cover the whole chain. The platform should capture technical signals. The workflow should preserve the business context. The CRM or case system should keep a usable record. The escalation path should say who can override the automation and what they must check before doing so. If those pieces live in separate tools with no shared identifier, post-incident investigation becomes a reconstruction exercise. That is where losses grow, because the organisation cannot quickly tell whether it saw one unusual case or the edge of a wider campaign.

Customer agents need permission boundaries before they need more channels

The NCSC's Frontier AI guidance is blunt about the direction of travel. AI makes it easier and faster for criminals to attack organisations that lack basic cyber protections. It also highlights agentic AI as a class of tools that can plan, decide and act on a user's behalf, which means organisations need to understand what access those agents have to systems and data.

This is directly relevant to customer experience projects. Many AI agent proposals are framed around convenience: let the assistant check orders, amend subscriptions, raise tickets, update contact records, prepare renewal options, or move a user through onboarding. Each of those tasks can be reasonable in isolation. The risk appears when the agent can combine them under social pressure, false identity, compromised account access or misleading uploaded evidence.

A practical permission model should start with fraud scenarios, not just job descriptions. Can the agent change an email address and issue a refund in the same session? Can it accept new bank details from a supplier and also approve an invoice for payment? Can it retrieve personal data after a caller passes weak identity checks? Can it create a quote, apply a discount and send a payment link without a second review? These are design questions, not edge cases. The right answer is often a stepped authority model: low-risk information tasks stay automated, medium-risk changes require extra verification, and high-risk financial or identity actions require a human approval record before execution.

Synthetic media makes verification a workflow issue

Deepfake risk is usually discussed as an audio or video problem, but for most businesses the bigger issue is workflow trust. A realistic voice note, video clip, document image or chat message does not need to fool everyone. It only needs to arrive inside a process where staff are tired, the request looks familiar and the system has no second route for verification. That is why AI fraud controls should be designed around the moment a decision is made, not around the format of the content.

TLT's September brief noted that the FCA is increasing pressure on major technology companies to do more against investment fraud, with AI enabling scams to be created, distributed and adapted at greater speed and scale. It also cited FCA figures that 2,329 warnings were issued about unauthorised or potentially fraudulent firms in 2025, that 89% of the most-viewed social media posts promoting cryptocurrency trading breached financial promotion rules, and that action was taken against 74 finfluencers last year. Those numbers point to a wider lesson: fraudsters do not rely on one perfect fake. They test volume, channels and timing.

For a business deploying AI, the immediate control is not to buy every detection product on the market. It is to map where synthetic evidence could change an outcome. Customer identity, supplier bank changes, recruitment screening, insurance claims, refund approvals and executive instructions are all obvious candidates. Each one should have a second-channel confirmation rule, a record of the evidence reviewed, and a defined point where automation stops. The counterargument is that this adds friction. It does, but only where the risk warrants it. The target is not a slower customer journey. It is a journey where high-consequence requests cannot be completed by a convincing fake alone.

The evidence pack should be designed before rollout

The most useful question before launching a customer agent is not simply whether the model is accurate. It is whether the business could explain a disputed decision three months later. That explanation needs more than chat transcripts. It needs the prompt or policy version in use, the tools the agent could access, the user verification state, the retrieved records shown to the model, any risk scores or rule triggers, the handover notes and the final human or automated action.

The FCA's Mills Review says AI adoption in financial services introduces growing risks including sophisticated AI-enabled fraud and identity abuse, algorithmic bias and opaque decision-making. Even if your business is outside financial services, the operating lesson travels well. Opaque decision-making is not only a regulatory concern. It is a management problem. If an agent approves, rejects, escalates or changes something and the business cannot reconstruct why, then it cannot improve the workflow after near misses. It also cannot reassure customers, insurers, auditors or partners when something goes wrong.

What this means in practice is that every higher-risk AI workflow should have an evidence pack template. Keep it boring and operational: owner, system name, data sources, permitted actions, blocked actions, fraud scenarios tested, logs retained, escalation rules, review cadence and incident contact. Add sample cases showing what a normal interaction looks like and what a suspicious interaction triggers. That pack should sit with the release record, not in a separate policy folder nobody reads. When the workflow changes, the evidence pack changes with it.

Start with three controls that change behaviour

The most effective first move is to build a fraud abuse case library for each AI workflow. Do not make it theoretical. Write ten examples that would actually hurt the business: a customer using a synthetic document to pass onboarding, a supplier changing bank details through a support channel, a compromised mailbox asking for a refund, a sales agent being pushed to misstate terms, or a chatbot being used to gather enough personal data for identity abuse. Test the workflow against those cases before launch and every time the model, prompt or tool access changes.

The second control is a transaction boundary. Decide which actions the AI can complete, which it can prepare, and which it can only route to a person. A useful customer assistant may safely answer questions, gather documents and draft a case summary. It may be unwise to let the same assistant change verified identity details, issue payment links and close complaints without review. The boundary should be visible in product requirements, not hidden in a risk register.

The third control is fraud telemetry. Track rejected identity attempts, unusual tool sequences, repeated refund requests, account changes followed by money movement, manual overrides and model refusals. Review those signals weekly at first. This is where the misconception usually shows up: leaders assume fraud prevention is a launch checklist. It is not. It is an operating rhythm. AI changes the speed of both useful work and hostile probing, so the business needs feedback loops that are fast enough to notice when yesterday's safe workflow has become today's weak point.

Frequently Asked Questions

Why should AI fraud controls come before customer agent rollout?

Because customer agents often sit close to identity, account changes, refunds, onboarding and support decisions. If those controls are added after launch, the business may already have created a faster route for impersonation or payment fraud.

Is this only a concern for financial services firms?

No. The FCA material is strongest for financial services, but the operating lesson applies to any business where an AI system can influence identity checks, payments, supplier records, contracts, claims or customer decisions.

What is the minimum control set before launch?

Start with permission boundaries, second-channel verification for high-risk actions, clear audit logs, fraud abuse test cases, human escalation rules and named ownership for weekly telemetry review.

Does deepfake detection solve the problem?

Detection can help, but it is not enough. The workflow also needs to decide what happens when evidence is uncertain, when a request is high consequence, or when multiple small changes combine into a risky pattern.

How should a business test an AI agent for fraud abuse?

Write realistic abuse cases, run them through the workflow, record which controls triggered, and fix any route where the agent can complete a high-risk action without adequate verification or human review.

What evidence should be kept for disputed AI decisions?

Keep the interaction record, model or prompt version, data sources used, tools available, verification state, risk triggers, handover notes and final action. The aim is to reconstruct why the decision happened.

Will these controls make customer service slower?

Only in the places where friction is justified. Low-risk information requests can stay fast, while identity, payment and account-change actions get stronger checks because the consequence is higher.

Who should own AI fraud controls?

Ownership should be shared across the workflow owner, security, compliance and operations. A single risk team cannot control fraud if the agent's permissions and escalation paths are designed elsewhere.