What are “intervention categories” and how can revisers use them to improve translation quality?
by Pisana Ferrari – cApStAn Ambassador to the Global Village
How do you define “translation quality”? How can it be measured? In a recent live webinar cApStAn’s CEO Steve Dept explained how “intervention categories” used in a translation verification process can help report on translation quality in a standardized way and generate meaningful statistics. Translation quality in multilingual surveys and tests is crucial as meaning shifts, which are language-driven, and perception shifts, which are culture-driven, can affect the psychometric properties of items and thus compromise comparability of data. A robust linguistic quality control procedure is essential in order to ensure that multiple language versions of a document meet the highest standards of linguistic, cultural and functional equivalence. cApStAn’s methodology is applied in many high-stakes projects, including international large scale assessments, surveys, polls and talent management assessments.
Translation quality evaluation is about moving away from subjectivity
Comments to the effect that a translation “doesn’t sound right”, “is a little awkward”, or “you don’t really say that in my language” reveal how much leeway there can be in describing translation quality. Translation quality evaluation (TQE) is about moving away from subjectivity and having a framework that allows the evaluation to be replicated, so that it no longer depends on the individual evaluator. In TQE for tests and surveys, the aim is not to evaluate translators, but to ensure that tests and questionnaires are comparable across cultures and languages. Intervention categories help translation verifiers frame their diagnoses and formulate objective comments. It channels them towards factual reporting.
A framework that provides meaningful feedback for test authors
A checklist of 14 verifier intervention categories was first developed by cApStAn for the OECD’s Programme for International Student Assessment (PISA) 2006 Field Trial. These include: missing or added information, adaptation issues, mistranslations, inconsistencies, register, grammar, syntax, words left in source language, format, layout. Verifiers were trained to check the source and target versions segment by segment against the 14 categories, aided by scroll-down menus, and to diagnose potential issues, report deviations, propose/implement corrective action and document any interventions in a central monitoring tool. The categories were linked to formulas embedded in the worksheets to generate metrics. In the PISA 2006 Field Trial there were 5,380 cApStAn verifier comments, covering 42 national versions in 36 languages for 38 countries (See Table 1 below). cApStAn has been in charge of linguistic quality control for PISA since its inception; in PISA 2021, this represents over 86 countries and over 100 national versions.
How to use standardized intervention categories to improve your tests and surveys
Check the webinar recording for more details about how you can use intervention categories to improve the translation quality and reliability of your tests and surveys at this link.
Would you like your translated/adapted assessments or questionnaires verified by cApStAn? An independent quality check is always desirable in order to ensure that the translated test or questionnaire is fair and comparable in all languages. Request a quote here.