AI Accessibility Tests Belong in Procurement, Not After Launch

AI Trust & Governance

13 October 2026 | By Ashley Marshall

The AI demo worked beautifully. Then a customer tried to use a screen reader. That is a buying failure, not a late design snag.

The demo is not the customer journey

A polished AI demonstration usually shows a confident presenter asking a short question and receiving a fluent answer. It rarely shows a customer navigating with a screen reader, somebody enlarging text to 400%, or an employee trying to correct a voice input error. The difference matters because AI is being placed at the point where people ask for help, make decisions and resolve problems. If that point is inaccessible, the business has not merely bought an awkward interface. It has created a barrier in a core service.

In August, UK accessibility charity AbilityNet warned that buyers may discover these barriers only after contracts and integrations are in place. Its examples include chatbots that are difficult to navigate with a screen reader and AI answers without the structure some people need. That is an expensive time to discover that a supplier's attractive demo covered only one mode of interaction. Treat the demonstration as a starting sample, not evidence that the whole journey works.

This is not a theoretical issue reserved for government websites. A retailer can make returns harder when an assistant replaces a clear form. A professional services firm can exclude an employee if its internal knowledge assistant cannot be used with a keyboard. A bank can leave a customer unable to understand or challenge an automated explanation. The business question is whether the person can complete the task, not whether the model produces a plausible paragraph.

In practice, take one important journey and ask the supplier to show it end to end with keyboard-only navigation, a screen reader, enlarged text and an error that needs correcting. Include a hand-off to a human. Note each point where a user must switch channel or start again. This modest exercise is much more informative than asking whether the tool is 'accessible' and accepting a yes or no answer.

AI raises the bar on evidence

AI can help teams create captions, summarise long documents and identify potential accessibility defects. That benefit is real. Yet it is easy to confuse an AI feature that helps accessibility work with an AI service that is itself accessible. The first is a capability; the second is a property of the entire experience, including input, output, feedback, recovery and support. A supplier may do the former while failing the latter.

The 2026 State of Digital Accessibility report from Level Access draws on more than 2,500 digital experience professionals across North America and Europe. It says four in five respondents use AI for accessibility work. Its more important finding for a buyer is that mature programmes are three times as likely to say AI helps them work proactively. That is an association reported by a vendor survey, not a UK-wide performance guarantee. It points towards governance, training and established processes as the conditions under which automation helps.

There is also a useful counterweight to a purely technical checklist. In an August project with the Consumer Technology Association, disability-led organisation Disability Belongs documented lived experience of AI in transport, household life, healthcare and employment. Those are task contexts, not benchmark scores. A model can look strong on a general test and still fail when a person has to navigate an interface, understand a consequence or correct a mistake under pressure.

What this means in practice: ask for evidence from the intended task and user group. Request the testing method, the devices and assistive technologies used, the date of the test, known limitations and the plan to address them. An old conformance statement for a static interface should not be treated as proof that a changing AI workflow is safe for all users. Insist on evidence you can review, not just a confident supplier declaration.

Put accessibility into the buying criteria

The strongest negotiating moment is before a supplier is selected. At that point a business can require a demonstration of the complete journey, compare alternatives and price remediation. After signature, the same requirement can become a dispute about scope. Procurement teams therefore need to turn accessibility from a general principle into a small set of observable acceptance criteria. This is not a call for a long questionnaire that nobody reads. It is a call for evidence at the point of choice.

Start with the tasks the AI system will perform: answer a customer question, summarise a document, suggest a decision or complete an internal workflow. For each task, identify who might be excluded and how that person could continue. Ask the supplier to provide a current accessibility report, known defects, support arrangements and a route for reporting new barriers. Ask who owns a fix if the cause sits in a model, a third-party component or the buyer's own integration. Then make the answer part of the evaluation, rather than a footnote to security review.

UK Government's service manual on WCAG 2.2 says government services must achieve level AA and should use both automated and manual tests. It covers keyboard operation, text alternatives, understandable errors and robust status messages. Those principles are useful to private firms too, but a public-sector requirement must not be carelessly presented as a universal legal threshold for every UK company. For private services, Equality Act duties and the specifics of the service still matter. Get legal advice for a particular regulated case.

A practical scorecard can use three evidence levels: supplier claim only, independent or reproducible test evidence, and observed success in a representative task with disabled users. Mark unresolved barriers against the business-critical journeys they affect. If two suppliers both meet a functional requirement, the one that can show how people actually complete the task offers a lower delivery risk than the one with a polished promise. That is a buying decision, not a charitable extra.

Test the changing service, not a frozen screen

Traditional software can change frequently, but AI adds another moving part: the same interface may produce different responses after a model update, prompt change, data-source change or tool integration. A response might stop using headings, bury an important qualification or fail to explain what the user can do next. A new agent capability might remove a human checkpoint. The visual page could still look unchanged while the effective service becomes harder to use.

AbilityNet's procurement analysis makes this point plainly: buyers must establish who is responsible when a model or feature change creates a barrier. The answer cannot simply be that the supplier will 'monitor quality'. The contract and operating model should define material-change triggers. Examples include a new model family, a changed response format, a redesigned chat control, a new channel, or automation of a decision previously handled by staff. Each trigger should lead to a proportionate retest of the affected journey.

For a UK business, begin with a short set of regression tasks. Can a keyboard user reach the input, review the reply, follow a link and escape a modal? Does a screen reader announce when a new answer is ready without reading the whole page again? Can the user ask for clarification, correct a mistaken answer and reach a person? Does the service retain context during that hand-off? Check content as well as controls: headings, lists, plain language and a clearly separated source or caveat can change whether an answer is usable.

A counterargument is that such testing will slow every model upgrade. It need not. Tier the response to the risk: a copy tweak may need a focused check, while a new autonomous action in a customer journey needs representative testing and sign-off. Keep a dated record of what changed, what was tested, who accepted the remaining limitations and when the next review is due. That produces release discipline without turning every improvement into a six-month procurement project.

Give the human route real authority

'A human can always help' sounds reassuring until the person is trapped in a chat window that will not hand over, a phone line that requires the same inaccessible interaction, or an agent who has no record of what happened. Accessibility is an operating requirement, not merely an interface requirement. The organisation must decide how a user reports a barrier, how quickly someone responds and what the human can actually do. A fallback that repeats the AI's answer is not a fallback.

The Disability Belongs research is a useful reminder that AI use is situated in real life. Transport, healthcare and employment questions can carry consequences well beyond a neat digital transaction. A business should therefore map the points where a person needs to understand a decision, challenge an error or choose another channel. In a low-stakes internal drafting tool the answer may be a simple help route. In a customer service or eligibility journey, the escalation should be clear, staffed and able to remedy the issue.

What this means in practice: add an accessibility incident category to the ordinary support workflow. Record the task the user was trying to complete, the assistive technology or interaction mode if they choose to share it, and the point of failure. Do not require a person to disclose a diagnosis to get help. Assign an owner across service, supplier and accessibility teams. A monthly review should look for repeated failures, not merely count tickets closed.

There is a commercial case here. An inaccessible assistant can increase contacts, rework and complaints while its dashboard celebrates a high automation rate. Measure successful task completion and effective hand-offs alongside containment. If the tool answers quickly but creates repeat contacts for a particular group, the apparent saving is misleading. Design the human route before launch and test it with the same seriousness as the AI response.

A workable first 30 days

The next step is not a grand accessibility transformation programme. Choose one AI-enabled journey that matters to customers or colleagues. Write down the task, its business owner, the supplier and the people most likely to encounter a barrier. Request the supplier's current accessibility evidence and known limitations. Run a small task-based test with keyboard navigation, screen reader support, enlarged text and an error-recovery scenario. Where possible, include disabled participants and pay for their time. The aim is to learn how the service behaves, not to produce a flattering score.

In week two, sort each finding into a clear category: can be fixed before launch, needs a documented alternative route, requires supplier commitment, or means the product is unsuitable for that journey. Ask the supplier for ownership and dates. In week three, put the material-change and retest triggers into the contract or service review. In week four, repeat the high-risk tasks after fixes and record a named decision. A buying team can do this on an existing pilot without reopening every policy in the organisation.

Use the GOV.UK WCAG 2.2 overview as a practical vocabulary for digital interactions, not as a claim that one automated scan proves compliance. Its own guidance calls for both automated and manual testing. Use the Level Access finding as a reminder that AI tends to amplify the quality of an existing programme. If ownership, user research and release controls are weak, buying more AI testing software alone will not solve the underlying problem.

The decision at the end should be explicit. Proceed if people can complete the critical task and remaining limitations are owned. Restrict the use case if the tool works only for some channels. Pause if an essential task has no accessible route or the supplier cannot supply evidence. That is not anti-AI. It is a better definition of 'ready to deploy' and a far cheaper choice than discovering the problem after users do.

Frequently Asked Questions

Does every UK business have to meet WCAG 2.2 AA?

No universal rule applies to every private-sector service in the same way. UK government services have a stated WCAG 2.2 AA requirement. Private organisations should assess Equality Act duties, sector rules and the nature of the service with appropriate legal advice.

Can an automated accessibility scanner approve an AI chatbot?

No. A scanner can flag some code and content issues, but it cannot establish that a person can complete a changing conversation, recover from an error or reach effective human support.

What should we ask an AI supplier for first?

Ask for a current accessibility report, test method and date, known limitations, evidence from the intended task, an incident route and commitments to retest after material changes.

Do we need disabled people in user testing?

For important journeys, yes where practical. Assistive-technology checks by specialists are valuable, but lived task experience can reveal problems that technical checks miss. Pay participants for their time.

What if the supplier will not share its full test report?

Ask for a useful redacted summary, a live task demonstration and a contractual statement of known limitations. Treat an unsupported assertion as weaker evidence in the supplier evaluation.

How often should an AI service be retested?

Set change-based triggers rather than relying only on a calendar. Retest affected journeys after material model, interface, workflow or hand-off changes, with periodic review for drift.

Is a human fallback enough to make an inaccessible AI tool acceptable?

Not automatically. Users must be able to reach the alternative without repeating an inaccessible step, and staff must have the authority and information needed to resolve the task.

What metric should leaders watch?

Track successful task completion and effective hand-offs across interaction modes, alongside complaints and repeat contacts. A high AI containment rate alone can hide exclusion.