AI Accessibility Needs a Real Customer Journey Test

Tools & Technical Tutorials

26 October 2026 | By Ashley Marshall

A chatbot can sound helpful while the customer remains stuck. The accessibility test is whether they can finish the task and recover when the AI gets it wrong.

Accessibility is a customer outcome, not a compliance badge

An AI assistant can answer a customer in two seconds and still make the service harder to use. It may respond with a dense paragraph to someone using a screen reader, forget what a customer has already explained, or insist on a document upload that their device cannot complete. A dashboard recording a successful chatbot response will miss every one of those failures. The useful question is whether a person can finish the job independently, understand the result and recover when the system gets it wrong.

The UK government’s September 2026 digital inclusion pilot makes the distinction clear. It found that access to a device or connection was only part of the story: confidence, accessible design, suitable assistive technology and trusted support all shaped whether disabled adults could actually participate. The report also cites around 15% of people unable to complete foundational digital tasks and a further 6% completely offline. Those figures are not an AI failure rate. They are a warning against treating a digital channel as universally available.

For a UK business, start with the whole journey. Can somebody ask the AI for help, understand its answer, act on it and reach a human without repeating themselves? Can they do it by keyboard, voice input, screen reader and mobile browser? If the answer depends on a particular device or a patient friend, the journey is not complete. Measure task completion and recovery, not just response speed.

This does not mean every AI project needs a months-long research programme. It means the first release should define a small number of important customer tasks, identify where exclusion would matter most and test those tasks with the people who use them. The cost of doing that early is usually less than rebuilding a journey after complaints arrive.

Test the hand-off, not only the chat window

Most teams review the visible chatbot interface. The real obstacle often appears one step later. A customer receives a useful answer but the linked form loses keyboard focus. A generated summary leaves out a detail needed for a reasonable adjustment. A verification step times out while someone switches to an assistive tool. The AI is only one component of the service, so an accessible interface alone cannot prove an accessible outcome.

Map the journey from the first question through to confirmation. Include at least one mistaken answer, one failed upload, one interrupted session and one request to speak to a person. For each step, record what information is retained, what the customer is told and how they can challenge the outcome. Give the human adviser enough context to help, but do not make the customer’s only option an opaque AI-generated summary. Let the person review or correct it before it becomes the record.

The government’s 2026 inclusion monitoring report offers a useful comparison from digital identity services, even though it is not a survey of AI assistants. Among 44 certified live providers, 80% collected data on incomplete journeys and 66% of those collected reasons for drop-out. It also found that 49% of providers using biometrics offered an alternative for people who did not wish to provide biometric information. The transferable lesson is to log why a journey fails and offer another route, rather than calling every completed automated step a success.

In practice, give the service owner a weekly sample of abandoned or escalated journeys, broken down by task and channel where lawful and proportionate. Review those examples with customer support and accessibility specialists. A rising hand-off rate may be a sign that customers are being protected from a bad answer, not evidence that the AI should be made more persistent.

Use automation to find defects, then ask people what matters

Automated accessibility checks are valuable because they are quick, repeatable and capable of catching obvious regressions. Use tools such as axe-core or Lighthouse in a development pipeline to flag missing labels, weak structure and some contrast issues. Test keyboard navigation manually as well. For a conversational feature, include test prompts that produce lists, links, error messages, tables and long answers, then check how those outputs behave with assistive technology.

That is the beginning of assurance, not the end. AbilityNet’s September analysis of synthetic accessibility testing describes how AI-generated users might examine more journeys earlier, while warning that simulated findings need validation by real people. Its concrete example, Collectivally, explicitly warns that AI-generated reports can contain inaccuracies. A model can imitate a screen reader interaction pattern; it cannot establish how an individual interprets a confusing instruction under stress or whether a workaround feels safe enough to use.

Choose a small, paid panel of users with varied access needs for the highest-impact journeys. Avoid asking one disabled person to speak for everyone. Give them a real task, not a demonstration of the feature, and watch where the journey consumes extra effort. Ask what they expected, what they trusted and when they would have stopped. Keep the testing environment respectful and make feedback accessible in more than one format.

The counterargument is that user testing takes time and money. It does. So does launching a feature that excludes customers, damages trust and then needs emergency repair. Use automated checks on every change and targeted human testing at meaningful release points. This division makes the limited human time more valuable: participants spend less of it reporting defects a tool could have found and more of it revealing whether the service works as a service.

Put reasonable adjustments into the service design

Accessibility is not merely a technical standard. In UK services, the Equality Act 2010 and the duty to make reasonable adjustments matter when a disabled customer faces a substantial disadvantage. The 2026 draft services code of practice discusses these obligations and practical adjustments. It is a draft code, not a new blanket rule that every private AI chatbot must meet a particular WCAG version. A business should take advice on its own duties, but it should not wait for a legal dispute before providing a usable route.

Build the alternative path before launching the AI path. That might be a telephone number reachable without completing the bot conversation, a text route for someone who cannot make a call, a longer timeout, or a human review of a decision. Publish those choices in plain English at the moment the customer needs them. Do not bury the route in a footer or instruct the AI to offer it only after three failed attempts.

If the AI asks for information about disability or health to personalise support, treat that as sensitive service design. Ask only what is needed, explain how it is used, decide how long it is retained and make sure an adviser can honour the adjustment without exposing more detail than necessary. The same principle applies to transcripts: useful context can reduce repetition, but a speculative AI label must not become a permanent description of a person.

What this means in practice is a release checklist owned jointly by product, support and accessibility leads. List the tasks the AI may handle, the points where it must hand over, the channels available and the person accountable for fixing a failed adjustment. A clear owner is more useful than a generic promise that the model has been trained to be inclusive.

Measure exclusion without pretending the data is perfect

A service owner needs evidence that accessibility is improving, but a single completion percentage is misleading. Someone might finish a task only after restarting twice, changing devices or asking another person for help. Start with several measures: completion, abandonment, repeat contact, time to human help, correction requests and the share of users who can complete the task on their chosen channel. Add qualitative findings from user testing alongside the numbers.

The government inclusion monitoring report illustrates both the value and limits of this approach. It reports that 75% of surveyed certified digital identity providers said they met WCAG 2.0 AA or higher, while 11% did not know whether they met it. It also says 77% offered at least one accessibility tool. These are provider-reported indicators from a small, specific market, not proof that three quarters of UK digital services are accessible. The fact that some providers could not answer is itself a reminder to establish a baseline before claiming success.

Where you ask customers about disability or access needs, collect only information you have a defensible reason to use. Small cohorts may risk identification, and a customer should not have to disclose a diagnosis to get help. Combine opt-in research, anonymised operational measures and direct feedback. Check complaints and support notes for patterns, but do not let an AI sentiment score replace a careful reading of what happened.

Set a monthly review with authority to change the service. One useful test is to compare the AI-assisted journey with a non-AI route on the same task. If the AI saves handling time but drives more repeat contacts or leaves some customers unable to finish, the efficiency claim needs revision. Accessible design is not a decorative extra cost: it is part of whether the promised productivity gain is real.

Make the next release a controlled accessibility experiment

For the next AI feature, choose one task that customers genuinely need to complete, such as changing an appointment or resolving a billing query. Write a plain definition of success: the customer understands what happened, completes the task on their preferred channel and can reach a person when the automated route fails. Define what the system must not do, including inventing policy, forcing a single identification method or hiding the human route.

Then run three layers of testing. First, automated technical checks on each build and a keyboard review of every major state. Second, scenario tests that deliberately include ambiguous requests, interruptions, long answers and failed hand-offs. Third, paid sessions with users who have different access needs. Record a small set of defects with severity, owner and due date. Do not declare the feature ready because the average chatbot answer sounds good.

Keep the first release narrow enough to learn from. Compare task outcomes with the existing route for several weeks. Review a sample of failed conversations and ask support staff whether the AI summary helped or hindered the hand-off. If defects cluster around a specific step, fix that step before expanding the model’s authority. If the data is too thin to judge, say so. A controlled pilot is a decision tool, not marketing evidence.

There is a genuine upside here. Well-designed AI can make information easier to find, allow people to use natural language and support different ways of interacting. The 2026 digital inclusion pilot stresses that accessibility, confidence and trusted support work together. Put those conditions around AI and it can widen access. Ignore them and the same technology can place a polished interface in front of an inaccessible process. The business choice is whether to measure the real customer journey.

Frequently Asked Questions

Can an automated accessibility scanner prove an AI chatbot is accessible?

No. A scanner can identify some technical defects, but it cannot show whether someone understands an answer, completes the linked task or recovers from an incorrect response.

Do we need to test the entire website before launching one AI feature?

Test the complete high-impact journeys the feature affects, including linked forms, identity checks and hand-offs. Prioritise scope by consequence, then expand coverage.

Can synthetic AI users replace disabled participants in testing?

No. Synthetic tests may help find patterns early, but real users are needed to validate usability, trust and the effort a journey demands.

What is a sensible first accessibility metric?

Track successful task completion together with abandonment, repeat contact and time to reach a human. Review real examples so the numbers retain context.

Should customers have to disclose a disability to get human support?

No. Make alternative support routes visible to everyone. Collect sensitive information only where needed for an adjustment and explain its use.

Does WCAG compliance settle the question?

No. WCAG is an important technical baseline, but a journey can meet technical checks while remaining confusing or unusable in practice.

What if human testing is too expensive for every release?

Automate repeatable technical checks on every change and reserve paid user testing for important journeys and meaningful release points. Use the findings to improve the test suite.

Who should own the result?

Give a named product or service owner responsibility for task outcomes, with support and accessibility specialists involved in release decisions and defect review.