20.06.2023

Multilingual Skills Assessment Translation series, part 4: Maximising fairness, reliability, and validity in multilingual tests after translation

Published in:  cApStAn updates, Survey translation and localisation, Surveys and Opinion Polls, Talent Management, Test translation and localisation, Tests and Assessments, Translation profession, Translation technology

This is the 4th issue in our “Multilingual Skills Assessment Translation series”. The primary objective of this series is to provide a comprehensive illustration of the complexities inherent in test translation. By doing so, we aim to underscore the immense value derived from establishing a robust linguistic quality assurance design.

This episode takes a closer look at the components of linguistic quality assurance that take place after the actual translation, including translation verification and the final optical check, particularly important in computer-delivered tests.

Let us take again the example of a (fictional) test to measure consultants’ competencies in helping companies in damage control situations (the “Chaos Management Skills Assessment”, CMSA). This is the second assessment cycle, and part of the items can be recycled from the previous cycle. The test is administered in 12 different languages. In the first cycle, the test was administered in 6 of those languages. It was agreed to have the new assessment items translated by professionals.

In the first 3 episodes we looked at steps that take place before translation:

  • In episode 1, we organized the retrieval of trend items and our translation technologist created and leveraged translation memories to that effect.
  • In episode 2, we performed a translatability assessment to identify issues which could be problematic further down the line, and drafted translation and adaptation notes.
  • In the 3rd episode our translation technologist prepared the translation files in a computer-assisted translation tool, or CAT tool, and provided support to translators.

We are now at the stage where we have a translated version based on a robust linguistic assurance flow and good tools. While all those steps added value, they are not enough to ensure that the target language versions of the test are fair, reliable, and valid. No test translation process is complete without linguistic quality control and equivalence checks.

A two-tier linguistic quality control design was set in place, including a translation verification and a final optical check.

What is “translation verification”?

  • The scope of verification is broader than a linguistic revision. Proofreading skills are but a modest subset of the skills deployed by a verifier, who needs to strike the right balance between faithfulness to the source and fluency in the target version.
  • Trained verifiers focussed on linguistic equivalence, which is a proxy for functional equivalence. They performed a thorough, segment-by-segment equivalence check between the target version and the source version of the test items and used cApStAn’s set of 14 verifier intervention categories to report each case where equivalence is threatened.
  • The categories are used so that verifiers report on the issues in a standardized way. They implement the suggested correction directly into the CAT tool and describe the rationale of the intervention in English in a dedicated comment field.
  • This procedure allows us to generate reports on the type of issue identified in each language version, but also in each item across all language versions.
  • Part of the verifier’s brief was to report on compliance with each item-by-item translation note, so we also had actionable statistics on that aspect.

What does final optical check mean?

  • Once the verified translation was ready, the XLIFF files were imported back into the test delivery platform, and a final optical check was performed.
  • The verifier looked at each screen to check what the candidate would see, and checked that all imports appeared correctly, that no label was truncated and that the translated version was ready for production.

………………………………………………………………………………………………………………………………………………………………………………………………

Check out the On-demand webinar on Translation Verification for Your Assessments & Surveys: What, Why & How? by Marielle Lerner and Grace DeLee, Localization Specialists at cApStAn LQC headquarters in Brussels, Belgium, and Musab Hayatli, Managing Director of cApStAn Inc, USA

Meanwhile, if you’d like to learn about how we can help you with your test localization projects, contact us via this form and we’ll get back to you as soon as we can.