How should we test an AI workflow after its supplier changes the model?

Treat a supplier's model change as a change to your business process, not a free upgrade you can assume is safe. Re-run representative work, check permissions and failure handling, and require a named person to accept the results before restoring unattended actions. A model that scores better on a public benchmark can still misread your product codes, miss an exception or produce an answer your downstream system cannot process.

The issue is the whole chain. Your workflow combines instructions, retrieved documents, a model, connectors and business rules. Changing one component can alter the way the others behave. A more helpful assistant might start filling gaps you previously expected it to flag. A more cautious model might refuse legitimate work and silently increase the manual workload. Neither change is necessarily visible in a polished demonstration.

The UK's National Cyber Security Centre guidance on secure operation and maintenance explicitly says that changes to data, models or prompts can change system behaviour. It recommends treating major updates like new versions and supporting evaluation through measures such as preview access and versioned APIs. That is a specific UK security recommendation, not a claim that every supplier offers those controls.

Your buying question is therefore straightforward: can this supplier demonstrate that the changed service still meets our acceptance criteria, and can we stop it safely if it does not? If the answer is no, keep the workflow assistive rather than autonomous.

Ask what is changing, when it takes effect and which parts of your service are affected. Request the model identifier where available, changed capabilities, known limitations, planned retirement dates and whether the old version can remain available during evaluation. Distinguish a model replacement from a new prompt, retrieval configuration, connector or permission change. Bundled changes make failures harder to trace.

Ask for results relevant to your use case, not just the supplier's headline benchmark. A customer enquiry assistant needs evidence about factual answers, escalation and inappropriate commitments. An invoice assistant needs evidence about amounts, tax treatment, duplicate handling and approval boundaries. The supplier should explain what they tested, the failures they found and which risks remain yours to manage.

The UK government's AI Cyber Security Code of Practice names testing and evaluation in Principle 9, monitoring in Principle 12, and human responsibility in Principle 4. Provision 3.1.1 calls for addressing security risks introduced by updated settings or configuration. These are requirements within a voluntary code, not a new statutory certification scheme or proof that a compliant supplier cannot fail.

Agree responsibilities commercially. Who pays for acceptance testing? Who investigates a regression? How much notice is promised? What happens if the previous model is withdrawn? Do not assume your support agreement includes unlimited remedial work. Record the answers before relying on a critical workflow, and ask for a plain-English explanation if the supplier cannot expose its underlying model.

Use a fixed set of realistic cases with an expected business outcome. Include routine work, awkward exceptions and situations where the correct behaviour is to stop. Use synthetic or appropriately minimised records in a controlled environment. Copying complete client files into an unapproved testing service creates a new problem rather than solving the original one.

For an illustrative small-business enquiry workflow, a starting pack might contain 30 routine requests, 10 ambiguous requests and 10 cases involving restricted information or prohibited actions. These 50 cases are a planning example, not a recognised assurance standard. Select cases to cover your actual risks; a safety-critical or regulated process needs specialist assurance well beyond a small sample.

Write the expected outcome before running the new model. For a request outside an agreed quote, success might mean flagging the scope question and routing it to the account owner without promising a price. For an unknown product code, success means asking for clarification rather than inventing a match. Include missing attachments, conflicting instructions, unusual date formats and documents that contain malicious instructions.

Compare the old and new configurations using the same cases where the old version remains available. Judge whether the outcome is acceptable, not whether the wording is identical. Repeat important cases because generative outputs can vary between runs. Keep a few fresh cases outside the usual pack so repeated tuning does not disguise a narrow test fit.

Do not let the same model be the only judge of its own work. Automated checks can catch missing fields or incorrect totals, but an appropriately qualified person must assess ambiguous commitments and consequential errors. Save the actual result and the reviewer's reason, not simply a green tick.

Separate quality from safety. A slightly clumsy internal summary may be acceptable. Revealing another client's information, changing bank details or sending an unapproved contractual commitment is not rescued by excellent performance on everything else. Define critical failures that block release independently of the overall score.

Test access boundaries directly. Give a test user access to one client workspace and confirm the assistant cannot retrieve another client's records. Place instructions in a sample document asking it to ignore its rules and send a file elsewhere. The expected result is refusal or safe escalation, with no unauthorised action. Ask the technical owner to check the connector and authorisation controls, not just the words returned by the model.

Check structured outputs and downstream behaviour too. Can the receiving system reject malformed data? Does a retry create a duplicate task, invoice or email? Does a timeout leave a clear exception for a person, or does work disappear? A correct-looking answer is not sufficient if the surrounding integration takes the wrong action.

Measure the economics as well. In a clearly hypothetical example, 100 items taking an extra three minutes each to check add five hours of work. At an assumed internal cost of £25 per hour, that is £125 of additional handling. This is arithmetic, not a market rate, supplier quote or promised saving. Use your own volume and loaded staff cost before deciding that a model upgrade is commercially beneficial.

Name the person who accepts the release and the evidence they require. They should be able to say what passed, what remains uncertain and why the remaining risk is acceptable for this particular use.

Some managed services update automatically and offer no model pinning. That does not mean you must stop using them altogether. It does mean your controls need to assume that behaviour can change without a separate deployment by your team. Keep consequential actions behind independent approval and run repeatable checks after supplier notices or unexpected behaviour.

A rollback is useful only if the previous configuration is still supported and safe. Do not restore a vulnerable model simply to recover familiar wording. If rollback is unavailable, your fallback might be read-only operation, drafting without sending, a manual queue or temporary suspension. Make sure someone can actually operate that fallback during normal working hours and absences.

Release a changed workflow to a limited audience or low-risk workload before wider use. Watch actual completion, exceptions, checking time and complaints. A staged release reduces exposure but is not permission to experiment on people with material decisions or confidential information. Test in a controlled environment first and retain appropriate oversight.

Microsoft Copilot, a custom OpenClaw workflow and a conventional rule-based automation create different control options. A managed assistant can reduce infrastructure work, while a custom workflow may give you more control over configuration and release gates. Neither automatically provides model pinning, guaranteed rollback or safe permissions. Verify the particular product, plan and implementation rather than buying the label.

If your supplier cannot explain changes, provide useful evidence or support a safe fallback, that is a purchasing and dependency risk. Discuss it before extending the workflow into higher-value activity. Our related guide on AI automations that quietly stop working covers the separate issue of detecting operational failure after launch.

A formal release pack for every minor wording change is excessive if a person uses an assistant only to brainstorm public marketing ideas and checks everything before use. A short review of the changed behaviour and existing data rules may be proportionate. The point is to match assurance to consequences, not to turn every experiment into a procurement programme.

At the other end, this article is not sufficient assurance for clinical decisions, safety instructions, lending, employment decisions or other high-impact activities. A small test set cannot establish legal compliance, fairness across populations or fitness for a regulated purpose. Involve the relevant professional and data protection expertise before allowing the system to influence those outcomes.

Nor should you buy elaborate AI testing to protect a process that does not need AI. If work follows fixed rules and structured inputs, ordinary validation or a conventional automation can be easier to inspect and maintain. Paying for model flexibility you do not use adds a dependency without a clear benefit.

For a small business, the sensible first step is to ask the supplier for the change details, choose a representative acceptance pack and identify the person who can stop the workflow. If those basics are missing, pause expansion rather than purchasing more features. If you need help deciding whether an existing workflow has become too dependent on its supplier, seek a review of that specific dependency and its fallback, not an open-ended transformation project.