Developments in Translation Verification Procedures in Three Multilingual Assessments: A Plea for an Integrated Translation and Adaptation Monitoring Tool

Dept, S., Ferrari, A. and Wäyrynen, L. (2010).

In Harkness, J. A. et al. (eds.), Survey Methods in Multinational, Multiregional and Multicultural Contexts. John Wiley & Sons, Hoboken, pp 157-173.

This paper examines test adaptation and verification practices in three international studies: OECD/PISA (2006); IEA/TIMSS (2007); and UNESCO/LAMP (Pilot test in 2006-2007). All three are multilingual, cross-cultural projects with sophisticated methodologies for adapting the tests and documenting the adaptation process. It is suggested that documenting should be seen as an integral part of the adaptation process. In these surveys, efforts were made to alert test developers to potential translation difficulties. Detailed adaptation guidelines were developed to draw the attention of translators to survey-specific translation and adaptation requirements. Attempts were made to achieve a higher level of standardization in documenting the linguistic quality and equivalence of national versions. Verification reports were linked to statistical analyses such as e.g. Differential Item Functioning (DIF) analyses.

The comparative overview is followed by observations from the practitioner’s point of view. In each test adaptation process, those features that seem to work particularly well are highlighted. Empirical evidence and illustrative examples are used to substantiate findings. Advantages and drawbacks are described in operational terms: what practices keep test adaptation focused on the best possible balance between equivalence and fluency? How much information can be collected at different stages of the test adaptation and verification processes without adverse effects? How can this information usefully be interpreted and exploited?

The conclusion is a plea for a monitoring tool to be used consecutively by all players involved in test adaptation: test developers, translators, reviewers, reconcilers, national expert panels, verifiers, referees, test adjudicators. This tool is designed to record all observations concerning the adaptation of a test in a different language and should provide a wealth of information directly relevant to interpreting test results.

Write to us for more details about the publication!