Translator HubAnnotation Guidelines Translation

How Tomedes translates annotation guidelines so every language team labels the same way

Published onNovember 17, 2026

An annotation guideline is the instruction document that tells annotators what each label means and how to apply it. When annotators work in different languages, each team needs the same rules in a language it reads natively, or the labels start to drift. Inconsistent labels train inconsistent models.

I believe translating annotation guidelines belongs to annotation quality control, and it deserves the same review as the annotation itself. A guideline that means slightly different things in two languages produces the inconsistency a quality process exists to prevent.

In July 2026, Tomedes translated one assessment with a built-in annotation key into 20 locales, including Arabic for Egypt and Saudi Arabia, Hindi, Tamil, Portuguese for Brazil and Portugal, Japanese, and Korean. This post covers what gets translated, how the review works, and when the step is not needed.

Key takeaways

  • An annotation guideline tells annotators what each label means, and when annotators work in several languages, each team needs the same rules in a language it reads natively.
  • Tomedes translates and adapts annotation guidelines, labeling keys, and examples for each target language.
  • In July 2026, Tomedes translated one assessment with a built-in annotation key into 20 locales, at about 1,000 words per locale.
  • A translated guideline has to produce the same labeling decisions in every language, and a literal translation can fail that test.
  • Each task goes to a native speaker of the exact locale, and a second reviewer checks it against the written guideline.

What is an annotation guideline, and why does it need translating?

An annotation guideline is the instruction document that tells annotators what each label means and how to apply it, usually with examples of correct and incorrect labels. It needs translating when annotators work in different languages, because every team has to apply the same rules in a language it reads natively. A labeling key is the list of labels and their definitions inside the guideline.

Data annotation adds labels a machine learning model can learn from, such as a transcript for a recording or a tag on a phrase. AWS groups data labeling into computer vision, natural language processing, and audio processing, and Tomedes works on the two language-led types, speech and text.

As far as I understand, many guideline problems in multilingual projects start with a rule that reads clearly in the source language and ambiguously in another.

What does Tomedes translate in an annotation project?

Tomedes translates the annotation guidelines, the labeling key, and the examples for each project language, so every team follows the same rules. The annotation itself is done by native speakers of the exact locale. Tomedes works in 270+ languages, so each team can read the guideline in its own language.

Item                   What it contains                                                Why it is translated
GuidelinesThe rules for applying each labelSo annotators in every language follow the same rules
Labeling keyThe list of labels and what each one meansSo each label keeps one meaning across languages
ExamplesSample items with the correct labelSo annotators see the rule applied in their own language

I think the examples need the most care, because an example that works in the source language may make no sense in another. A glossary of label names helps keep fixed terms fixed, and our guide to building a multilingual glossary covers how to build one.

Does a translated guideline have to match the original word for word?

No. The test is whether annotators in each language make the same labeling decision on the same item, and a literal translation can fail that test. A guideline is a set of decisions written as sentences, so the translation has to carry the decision as well as the words.

Take a guideline that tells annotators to mark hesitations, unintelligible speech, and non-speech sounds, the label types Tomedes lists for verbatim transcription. The sounds people make when they hesitate, and the way each language writes them, differ by language, so the examples have to be rewritten for each locale. In my view, that adaptation is part of the translation job.

How does a 20-locale guideline project run?

In July 2026 a new customer sent Tomedes an assessment with a built-in annotation key to be translated into 20 locales, each of about 1,000 words. Tomedes assigned dedicated linguists and project managers, because the translations would be used in testing and had to follow the same standard as the original assessment. The deadline was a few days after the order.

The locales included two versions of Arabic and two of Portuguese, so one language sometimes needed more than one locale. A typical multilingual data annotation project starts with scope, then a pilot batch that tests the guidelines, then production, review, and delivery.

My read is that two locales of one language is a good reminder that the locale, not the language, is the unit of work.

Who works on a guideline translation?

Native-speaker linguists of the exact locale do the work, a second reviewer checks it, and one dedicated project manager coordinates the batches and reports progress. In the 20-locale project, Tomedes assigned dedicated linguists and project managers because the translations would be used in testing.

Tomedes confirms native-speaker availability before a project starts, which matters when a model needs speakers of languages such as Tagalog, Amharic, or Yoruba. My preference is to confirm who will review each locale before the guideline is final, because a reviewer who reads the guideline early can flag unclear rules.

How do you check that every language says the same thing?

Match each task to a native speaker of the exact locale, add a second reviewer, and measure both against the written guideline. A pilot batch before full production shows where a guideline reads differently across languages. Tomedes is ISO 17100:2015 certified for its language services, and the standard includes an independent review step.

I would add one check of my own, which is to compare how annotators in different languages label the same pilot items and ask why they differ. Disagreement on a pilot item often points to a guideline that reads differently across languages.

Article 10 of the EU AI Act lists annotation and labelling among the data-preparation steps covered by data governance for high-risk AI systems. In my opinion, a translated guideline for each language is part of the documentation a reviewer would want to see, and this is not legal advice.

When is guideline translation not needed?

It is not needed when every annotator reads the source-language guideline natively, because a translation then adds cost and a second version to maintain. It is needed when annotators work in different languages, or when the guideline is long, technical, or full of examples that depend on the language.

I could be wrong on this, but a short guideline with few examples may be easier to keep in one language with a glossary of terms, and a long one is where translation pays off.

When every language team starts from the same rules, the labels that come back are easier to compare.

Planning annotation in several languages? Send the guidelines and a sample task when you request a quote from Tomedes.

FAQs

Q: What is an annotation guideline?
A: 
An annotation guideline is the instruction document that tells annotators what each label means and how to apply it, usually with examples of correct and incorrect labels. A labeling key is the list of labels and their definitions inside it. Clear guidelines are how a team of annotators reaches consistent labels across thousands of items.

Q: Can annotation guidelines be translated?
A: 
Yes, and for multilingual teams they should be, because annotators need the rules in a language they read natively. Tomedes translates and adapts annotation guidelines, labeling keys, and examples for each target language so labels stay consistent across teams. Examples usually need adapting, and a literal translation can miss that.

Q: How do you keep annotation labels consistent across languages?
A: 
Give every language team the same written guideline in its own language, match each task to a native speaker of the exact locale, and add a second reviewer who checks completed work against the guideline. A pilot batch before full production shows where a guideline reads differently across languages.

Written by Rachelle Garcia
Connect on LinkedIn

Rachelle leads product and AI at Tomedes, where she runs the experiments that turn internal data into better translation experiences. She writes about what actually happens when you build AI products such as MachineTranslation.com — the numbers, the surprises, and the parts that don't go to plan.

Share:

STAY INFORMED

Subscribe to receive all the latest updates from Tomedes.

Post your Comment

I want to receive a notification of new postings under this topic

Do It Yourself

I want a free quote now and I'm ready to order my translations.

Do It For Me

I'd like Tomedes to provide a customized quote based on my specific needs.

Want to be part of our team?