AI Translation Benchmarks Need a Language-Specific Acceptance Test

Model Intelligence & News

1 November 2026 | By Ashley Marshall

The translation sounds convincing. That does not mean the warning, refund condition or delivery promise survived.

The new translation research is about systems, not a winning model

A translation can read beautifully and still give a customer the wrong instruction. That is why the useful question for a UK business is not which model sounds most natural. It is which complete translation process preserves the meaning, terminology and operational details that your customers need.

Intento's State of Translation Automation report, published on 15 October 2026 and updated on 27 October, makes that distinction explicit. It evaluates 229 model-and-workflow configurations built around 57 models and translation engines from 37 vendors. Its test covers English into 100 target languages. Those numbers describe the scope of this supplier's research, not a promise that any one configuration is suitable for your business.

The important design choice is what Intento counts as a solution. An engine with a glossary and the same engine without one are different configurations. Prompts, reference material, formatting rules and subsequent checking all affect the result. Comparing model names while ignoring those surrounding controls can therefore answer the wrong purchasing question. You are buying an operating process, even if the invoice calls it an AI subscription.

There are limits worth keeping visible. The benchmark uses four English source texts, totalling approximately 3,000 words. It is not a representative sample of every product catalogue, customer complaint or technical manual. English-to-French results do not establish French-to-English performance, and an aggregate score cannot establish the suitability of a translated safety warning.

What this means in practice is straightforward: use the research to construct a shortlist, then test your actual material. Keep the source text, glossary, instructions and review method fixed while comparing candidates. Record the full configuration so a later model or prompt change can be assessed against the same evidence. The surrounding workflow is part of the product you are evaluating, not an optional finishing touch.

Source: Intento translation automation report.

Treat fluency, terminology and operational accuracy as separate tests

The distinction between readable language and usable business content is not academic. A sentence can be grammatically excellent while changing a delivery deadline, weakening a cancellation condition or substituting an ordinary word for a product-specific technical term. Someone reading only the translation may never notice. An English-speaking manager admiring the presentation is particularly poorly placed to sign it off.

Intento separates general translation quality from additional requirements such as terminology, tone and formatting. Its report says the low-cost quality-estimation metrics used for shortlisting do not check whether terminology, tone-of-voice or gender-neutrality requirements are met. That is an important warning against interpreting a single attractive score as a complete acceptance decision.

Build your own checklist around the transaction. For a product page, check the product identifier, measurements, compatibility limits, warranty wording and purchasing action. For a support response, check the customer's problem, the proposed remedy, any exclusions and the next step. For an internal procedure, check who must do what, in which order and under which conditions. These are proposed checks, not claims about measured error rates.

Separate checks that software can perform reliably from judgements requiring language expertise. Compare numbers and identifiers with the original. Validate that links still point to the intended destination. Check that a required warning has not vanished. Then ask a qualified reviewer to assess meaning, register and terminology in context. A second model can help flag issues, but an automated agreement is not the same as independent professional approval.

In practice, an illustrative pilot might include 30 short items in one language, deliberately mixing ordinary copy with difficult cases. That is a manageable starting exercise, not a statistically sufficient assurance standard. Include negation, ambiguous dates and similar product names. Define what counts as a blocking error before looking at the results. Otherwise the team may quietly lower its standard to match the tool it already likes.

Source: Intento requirements and assessment method.

Do not make every translation carry the same approval burden

There is a reasonable objection to adding review: if every sentence must pass through a professional linguist, where is the saving? The answer is not to pretend that review is free. It is to distinguish content by consequence and measure the complete delivery process, including the work that remains after AI has produced a draft.

A rough translation of a public industry article for an employee's personal understanding is different from a translated contract clause sent to a customer. A low-risk draft of internal event information is different from machinery instructions. The acceptable workflow should reflect that difference. Restrict sensitive or high-consequence material to people with the relevant language and subject expertise, rather than assuming that one company's general translation policy covers everything.

Smartling's October 2026 comparison of translation tools describes options combining machine translation, language models, terminology resources and human review. It also reports that Secret Escapes reduced translation time by 25% across all languages using its AI Human Translation approach. This is a supplier-reported case study, not an independent estimate of what your company will save. Its useful lesson is that AI and human review need not be mutually exclusive.

Start by agreeing three practical routes: internal understanding, routine customer-facing content and specialist material. For each, define who may request translation, who may approve it and where the approved version is stored. These routes are a suggested operating model, not statutory categories. A business with only occasional translation work may need nothing more elaborate than a clear checklist and an accountable external language professional.

Measure elapsed time to approved publication, reviewer minutes, rework and consequential errors. Do not report the time taken to generate the first draft as the time taken to finish the job. If the tool saves ten minutes of drafting but creates twenty minutes of terminology repair, the apparent productivity gain has disappeared. If it gives your reviewer a consistently useful starting point, that is a genuine result worth expanding carefully.

Source: Smartling translation tools comparison, October 2026.

Buy the missing capability before building another translation platform

The language technology market is already crowded. Nimdzi's 2026 Language Technology Radar report, last updated on 1 September, describes more than 1,100 products developed by 756 companies. That does not mean all those products are appropriate or affordable for a small UK company. It does mean a custom translation agent should not be your automatic first move.

Nimdzi distinguishes translation management systems, terminology tools, machine translation and quality management rather than treating them as interchangeable. Those distinctions are useful when identifying the actual gap. You might have adequate translation quality but no shared glossary. You might have good linguists but a painful process for collecting revised source text. Neither problem necessarily requires changing the underlying model.

Begin with what is already available in the tools and services you use. Compare approved business translation services, your existing language supplier and suitable maintained terminology or localisation tools. Check supported languages, file formats, reviewer access and export options. Do not assume that an attractive demonstration includes the features your team actually needs, or that the free consumer product has the same contractual protections as a business service.

Give the supplier a small sample of your material and the acceptance checklist. Ask to see how approved terminology is enforced, what happens when source content changes and how a reviewer rejects a segment without losing the audit trail. Ask whether you can export the glossary and approved translations in usable formats. These are more revealing questions than asking whether the product uses the latest model.

Custom integration becomes reasonable when an existing option cannot meet a concrete requirement, such as handling a specialist document format or synchronising approved text with a particular publishing system. Even then, keep the scope narrow. A reusable export or a controlled handoff may solve the problem without creating another platform to maintain. The goal is fewer mistakes and faster useful communication, not a larger collection of AI components.

Source: Nimdzi Language Technology Radar, September 2026.

Approve languages individually and keep the source of truth visible

An approval for one language should not become a blanket permission for every market. Intento's benchmark explicitly acknowledges assessment limits: for four of its 100 target languages it cannot assess quality, and for another fourteen it can select a preferred solution without precisely assessing its quality. Those are limitations of that study, not a claim that those languages cannot be translated well. They show why uncertainty belongs in the decision record.

For each language you actually need, nominate a reviewer, a terminology owner and an escalation route. Record which content types the tested workflow may handle and which remain excluded. Keep the approved English source linked to the translated version, together with the review date and the tool configuration used. When the original changes, flag the translation for review rather than leaving customers with an attractive but outdated page.

Personal data requires a separate check. A support complaint or employment document does not stop being personal data because someone wants it translated. Article 5 of the UK GDPR sets principles including data minimisation and accuracy. In practical terms, remove unnecessary identifying details, use an approved processing route and check that the translation has not distorted relevant facts. Those principles do not certify a provider or settle every contractual and international-transfer question. Involve the person responsible for data protection when the material or processing warrants it.

The sensible next step is a narrow release decision, not a multilingual launch announcement. Choose one language and one low-consequence customer journey. Have a competent reviewer test both the text and its real presentation: page layout, dates, forms, links and the response customers receive afterwards. Agree a correction route and the person authorised to pause publication.

Expand only when that evidence supports expansion. You can accept AI translation as a useful capability without accepting every output as finished work. The commercial advantage comes from making more accurate communication affordable, not from removing the person who can tell when it has gone wrong.

Source: UK GDPR Article 5.

Read the benchmark assessment limitations.

Frequently Asked Questions

Does the highest-scoring translation model suit every language?

No. Results depend on the language direction, source material and surrounding workflow. Use a benchmark to shortlist candidates, then test the language and content type you actually need.

Can a second AI model replace a human translator?

A second model can flag possible errors and support review. It does not by itself establish that specialist terminology, legal meaning or safety instructions are correct. Match review expertise to the consequences of a mistake.

Do we need a translation management system?

Not necessarily. Occasional translation may need only an approved service, a shared glossary, version tracking and a competent reviewer. Consider a dedicated system when volume and repeated updates make those tasks hard to manage.

What should our first translation pilot contain?

Choose one language and one low-consequence customer journey. Include normal examples and difficult cases involving negation, numbers, dates and product names. Set blocking-error rules before reviewing outputs.

Can we paste customer complaints into a free translation tool?

Do not assume a consumer service is an approved route for personal data. Remove unnecessary identifiers and check your organisation’s requirements, provider terms and processing arrangements before submitting material.

How do we measure whether AI translation saves money?

Compare the complete cost and elapsed time to an approved result. Include tool charges, reviewer time, terminology repair, rework and handling consequential errors. First-draft speed alone is not a saving.

What happens when the English source changes?

Link translations to their approved source version and mark affected translations for review when that source changes. Give someone ownership of the update so an old translation does not silently remain authoritative.