What happens when AI gets it wrong in a business context?

23 July 2026

Who is responsible when AI gives the wrong answer?

The business is responsible. That is the blunt answer UK leaders need to hear. If your chatbot promises a refund, your sales assistant misquotes a price, your AI screening tool rejects candidates unfairly, or your internal agent sends the wrong instruction to a supplier, the customer, regulator, employee, or court will look at your organisation first.

The clearest public example is the Air Canada chatbot case. The airline's chatbot told a passenger he could claim a bereavement fare retrospectively. Air Canada later said the bot was wrong and argued the chatbot was a separate legal entity responsible for its own actions. The tribunal rejected that argument. The company was ordered to pay C$650.88 in fare difference, plus interest and fees. The BBC reported the total as $812.02, about £642.64 at the time. The important lesson is not the amount. It is the principle: if you put AI in front of customers, you own what it says. Source: BBC Travel and The Guardian.

In a UK business, that responsibility can sit across several areas. Customer-facing mistakes become complaint handling, refunds, contract disputes, or reputational damage. Data mistakes become UK GDPR and Data Protection Act 2018 issues. Employment mistakes can become discrimination, fairness, and process challenges. Financial mistakes can become audit, insurance, and professional negligence questions.

The defence of "the AI did it" is weak because AI is part of your chosen process. You selected the tool, configured it, connected data, approved the use case, and decided whether humans checked the output. If you would not accept "the spreadsheet did it" as a complete excuse for a wrong invoice, you should not expect "the model did it" to work for AI.

What kinds of AI mistakes actually matter?

Not every AI mistake is a business incident. A rough first draft of an internal email is not the same as a chatbot giving binding refund advice. The risk comes from consequence, not from the presence of AI itself.

The common serious mistakes fall into five buckets. First, factual errors: the AI invents a policy, misreads a document, gives the wrong delivery date, or makes up a technical answer. Second, decision errors: the AI approves, rejects, prioritises, or flags something incorrectly. Third, data errors: it exposes personal data, uses information for the wrong purpose, or combines data in a way the business cannot justify. Fourth, action errors: an AI agent sends a message, creates an invoice, updates a CRM record, or triggers a workflow without enough control. Fifth, tone and judgement errors: the answer is legally defensible but commercially stupid, insensitive, or off-brand.

Here is the practical difference. A hallucinated paragraph in a marketing draft costs 10 minutes. A hallucinated product guarantee on a website can cost refunds, chargebacks, complaints, and angry screenshots. A wrong internal summary might be embarrassing. A wrong HR screening recommendation can create discrimination risk. A wrong finance automation can send money to the wrong place or hide a payment issue until month end.

UK adoption is rising, so these incidents are becoming more likely. The Office for National Statistics reported that 23% of businesses were using some form of AI technology in late September 2025, up from 9% when the question was introduced in September 2023. That is a rapid change in two years. Source: ONS Business Insights and Conditions Survey.

The main test is simple: could a reasonable person rely on the output to make a decision, spend money, share data, accept advice, or treat someone differently? If yes, a wrong answer needs controls. If no, it is usually a productivity issue, not a governance crisis.

What should you do in the first 24 hours?

The first 24 hours determine whether an AI error stays small or becomes a proper incident. The worst response is to quietly edit the prompt, delete logs, and hope nobody notices. That destroys evidence and makes the business look unserious if the issue later becomes a customer complaint, insurer query, or regulator question.

Start with containment. Switch off the affected workflow, restrict the AI feature to staff-only mode, or require human approval before any output leaves the business. Do not keep the system live while you investigate if it can repeat the same mistake.

Then preserve evidence. Keep the original prompt, AI output, source documents, time stamps, user ID, model or tool name, affected records, screenshots, and any downstream actions. If the AI sent a customer message, keep the exact version the customer saw. If it changed a record, preserve the before and after state. This is not bureaucracy. It is how you prove what happened.

Next, assess harm. Ask four direct questions: who relied on the output, what decision or action followed, did personal data play a role, and can the same error affect more people? If personal data is involved, involve your data protection lead quickly. The ICO's AI guidance covers accountability, governance, transparency, lawfulness, fairness, accuracy, and statistical accuracy, and its AI risk toolkit is specifically designed to help organisations reduce risks to people's rights and freedoms. Source: ICO guidance on AI and data protection.

Then correct the affected people. That might mean refunding a customer, correcting advice, apologising, re-running a decision with human review, or telling staff not to rely on a previous AI-generated summary. Be plain. Say what was wrong, what the correct position is, and what you are doing to prevent recurrence.

For a small UK business, a modest AI incident can easily consume £1,000 to £5,000 in staff time, refunds, technical support, legal review, and client management. A serious regulated or high-value incident can cost far more. The cheapest incident is the one where you act quickly, keep evidence, and fix the failed control.

What are the legal and regulatory risks in the UK?

The legal risk depends on the use case. AI is not regulated by one single UK AI Act in the same way the EU has introduced a dedicated AI Act. UK businesses still face existing obligations through UK GDPR, the Data Protection Act 2018, consumer law, employment law, equality law, contract law, sector regulation, and professional standards.

For most SMEs, the biggest immediate regulatory issue is data protection. If an AI system processes personal data, you need a lawful basis, purpose limitation, data minimisation, security, fairness, transparency, and accountability. If the AI output affects a person in a meaningful way, you also need to think about explainability, human review, bias, and whether the person can challenge the outcome.

The numbers are not theoretical. The ICO says the higher maximum fine under UK GDPR and the Data Protection Act 2018 is £17.5 million or 4% of worldwide annual turnover, whichever is higher for an undertaking. Most small businesses will not face anything close to that, but the maximum matters because it shows how seriously data protection failures can be treated. Source: ICO fining guidance.

Consumer law is the second common issue. If your AI gives a customer misleading information about a price, refund, contract term, service limit, guarantee, or eligibility rule, the business may have to honour the representation or compensate the customer. A website chatbot is not a casual conversation in the pub. It is part of your commercial operation.

Employment and equality law matter when AI ranks CVs, scores interviews, monitors staff, predicts performance, or recommends disciplinary action. A biased model does not become acceptable because the bias came from historic data. If the process disadvantages a protected group, the business has to explain and justify the process.

The honest answer is this: AI errors rarely create a new category of liability. They expose old responsibilities through a faster, less visible, harder-to-audit process. That is why governance is not optional once AI starts making or influencing decisions.

How much can an AI mistake cost?

The direct cost can be tiny. The total cost can be painful. A chatbot mistake might start as a £30 refund and become two days of management time, a public review, support backlog, and emergency technical work.

For a practical UK SME, use these rough ranges. A low-level content or admin error usually costs £50 to £500 in rework. A customer-facing misstatement usually costs £250 to £3,000 once refunds, goodwill credits, support time, and management time are included. A workflow error affecting multiple customers can cost £3,000 to £25,000, especially if you need technical investigation, legal advice, and proactive communication. A regulated data, finance, employment, or safety-related error can exceed £25,000 quickly, even before any fine or claim.

The hidden cost is confidence. Once staff see an AI tool produce rubbish, they either stop using it or over-check everything it does. Both outcomes damage the business case. Once customers see a public AI error, they do not think about model limitations. They think the company is careless.

There is also opportunity cost. A founder, operations manager, or service lead pulled into incident response is not selling, delivering, hiring, or improving the business. At a conservative internal cost of £50 per hour, a 20-hour incident is £1,000 before external support. Add a solicitor at £200 to £400 per hour, a developer at £75 to £150 per hour, and goodwill credits, and the invoice becomes real very quickly.

This is why cheap AI can become expensive. A £20 per month tool connected to the wrong workflow without permissions, logging, and review can create a five-figure mess. The price of the subscription tells you almost nothing about the risk of the process.

How do you reduce the chance of AI getting it wrong?

You do not reduce AI risk by telling staff to "be careful". You reduce it by designing the workflow so a wrong answer is caught before it matters.

Start by classifying the use case. Low-risk use includes brainstorming, summarising internal notes, drafting non-binding content, and generating ideas. Medium-risk use includes customer support drafts, CRM updates, sales research, management reporting, and operational recommendations. High-risk use includes employment decisions, financial approvals, legal advice, medical or safety advice, regulated complaints, pricing commitments, and anything involving sensitive personal data.

For low-risk use, training and review may be enough. For medium-risk use, add approved knowledge sources, clear prompts, staff sign-off, version history, and spot checks. For high-risk use, require human approval, test sets, audit logs, access controls, documented risk assessment, fallback procedures, and a named accountable owner.

The best control is often boring: do not let the AI act directly. Let it draft, summarise, flag, or recommend, but keep final authority with a trained person. If you do use an AI agent that can send emails, update systems, or trigger tasks, give it narrow permissions and reversible actions. It should not be able to discount a product, terminate an account, reject an applicant, or share customer data without a control point.

Use retrieval and source-grounding where possible. If a customer bot answers from your policy documents, make it cite the policy section internally so staff can inspect why it answered that way. Keep a change log when policies change. Test the top 50 customer questions, edge cases, and hostile prompts before launch, then test again monthly.

Finally, decide in advance what counts as an incident. A single poor wording choice is not the same as a repeated wrong refund answer. Your team needs thresholds, ownership, and a short recovery checklist before the first problem happens.

When this does NOT apply

This level of governance does not apply to every casual AI use. If someone uses ChatGPT, Claude, Gemini, or Microsoft Copilot to tidy an internal paragraph, generate headline ideas, or summarise notes that they already understand, you do not need an incident committee.

It also may not apply where the output is clearly labelled as a draft, never leaves the business, and cannot trigger a decision. A marketing assistant asking AI for ten blog title ideas is not creating the same exposure as a customer service bot answering refund questions from live customers.

Where businesses get into trouble is the middle ground. They start with harmless productivity use, then quietly connect AI to real customer data, live systems, quote generation, HR screening, or support replies. The tool still feels experimental, but the consequences are no longer experimental.

If your business is pre-revenue, has no repeatable process, or cannot describe what a correct answer looks like, do not automate the workflow yet. Fix the process first. AI makes weak processes faster and less visible. It does not make them reliable.

If you already have a mature internal AI, data, security, and compliance team, you may not need an external consultant for this. Large organisations with internal governance functions should use their existing risk, legal, procurement, and security processes. The principle is the same, but the ownership sits inside the business.

Is This Right For You?

This applies if AI is influencing customer messages, staff decisions, sales quotes, HR screening, finance checks, service advice, operational workflows, or any process where a wrong answer could cost money or trust.

It does not apply in the same way if you are using AI only for low-risk brainstorming, internal note drafting, or harmless first drafts that a competent person rewrites before anyone relies on them. In those cases the risk is lower, but the habit should be the same: AI output is never treated as authority just because it sounds fluent.

If your business cannot tolerate mistakes in a particular workflow, AI may still be useful, but only behind human approval, narrow permissions, audit logging, and clear escalation routes. If you cannot fund those controls, do not put AI into that workflow yet.

Frequently Asked Questions

Can a business blame the AI vendor if the system gives a wrong answer?

Sometimes you may have a claim against a vendor, but that does not remove your responsibility to the customer, employee, or regulator. Your contract may help you recover costs later. It will not usually stop the immediate complaint landing with your business.

Do we need to tell customers when AI made a mistake?

If the customer relied on the wrong output, yes, you should usually tell them plainly and correct the position. If personal data or legal rights are affected, you may need a more formal response and should involve your data protection or legal adviser.

Is an AI hallucination a data breach?

Not automatically. A hallucination is a false or unsupported output. It becomes a data protection issue if personal data is processed unlawfully, exposed, used inaccurately in a decision, or handled in a way that breaches UK GDPR obligations.

Should customer-facing AI always have human review?

For low-risk FAQs, full human review of every answer may be impractical. For refunds, complaints, eligibility, pricing, contracts, regulated advice, or anything involving personal data, human approval or strong guardrails are sensible and often necessary.

How often should we test an AI system after launch?

Test before launch, after every major policy or data change, and at least monthly for customer-facing or operational systems. High-risk systems need ongoing monitoring, logged exceptions, and periodic human review of real outputs.

What records should we keep when AI affects a decision?

Keep the prompt or input, the output, source documents used, system version, time stamp, user, action taken, human reviewer, and final decision. Without records, you cannot investigate the error or prove the process was fair.

Can AI errors invalidate a contract or quote?

They can create a dispute. The result depends on the facts, the customer journey, your terms, and consumer or contract law. Practically, many businesses choose to honour small errors because fighting them costs more than fixing them.

What is the safest first AI use case for a small UK business?

Start with internal drafting, summarisation, research support, or process documentation where a human reviews the output before use. Avoid live customer promises, HR decisions, finance approvals, and sensitive personal data until you have proper controls.