11.05.2021

The Wondrous Adaptation Cycle of a PISA2018 Questionnaire Item (and Subsequent Evolutions)

The Wondrous Adaptation Cycle of a PISA2018 Questionnaire Item (and Subsequent Evolutions)

Speaker: Shinoh Lee, Survey Localisation Manager at cApStAn LQC

Contributor: Elica Krajçeva, Survey Localisation Manager at cApStAn LQC

Recorded on 09.03.2021 | Duration: 20 minutes

Sophisticated translation designs have become the norm in high-stakes tests and are essential in order to collect comparable data across participating countries. For example, in the OECD’s Programme for International Student Assessment (PISA), which assesses knowledge and skills in 15-year-old students, questionnaires go through a complex, well-documented adaptation process. cApStAn prides itself of having been in charge of the language component of PISA since it was first established in 2000; for PISA 2021, this represents 86 countries and over 100 national versions.

Join cApStAn’s localization expert Shinoh Lee as she describes the life-cycle of a PISA questionnaire item, from the development of the source version, to pre-translation analysis and post-translation linguistic control, to survey launch. Read about

    • How a pre-translation “translatability assessment” of source items, by trained linguists, allows for early detection and resolution of potential translation and adaptation hurdles;
      • Why it is necessary to discuss country- and language-specific adaptations or structural and content-related adaptations with the questionnaire authors and National Centres;
        • Why a double translation and reconciliation design has been adopted for PISA;
          • How post-translation linguistic quality verification, conducted by expert verifiers, with the aid of intervention categories, helps ensure linguistic quality and consistency.

          The documentation of each step of a Linguistic Quality Assurance process such as the one above is essential to trace back issues that have been identified, and see how they were resolved. This , for example, is very useful in case of Differential Item Functioning (DIF).


          About Speaker and Contributor

Fill out the form to receive the webinar recording:

Shinoh Lee, Localisation and Knowledge Manager

Shinoh Lee is a survey localisation manager and knowledge manager at cApStAn. After having finished her studies in English Language and Literature in Seoul, she moved to Belgium where she obtained a MA in Germanic Philology from the Katholieke Universiteit Leuven (KUL) with major in Linguistics. The positions she previously held include language instructor (Korean and English), verifier for the Korean survey materials and operations director. She speaks Korean, English, Dutch, French and has some elementary knowledge of Turkish, Chinese and Japanese.

Elica cApStAn

Elica Krajceva, Senior PM & Questionnaire Adaptation Specialist

Elica received her education in Macedonian and Serbo-Croatian, and holds an MA degree in Cultural Anthropology. She has lived in Misrata, Brussels, Bucharest and Prague and worked as a translator, a teacher of English and a project manager. She also speaks French and has an elementary knowledge of Romanian, Greek, Czech and other Slavic languages. Since 2000, Elica has participated in numerous cApStAn linguistic quality assurance operations and attended training courses in questionnaire adaptation. She is the lead linguistic quality assurance manager for questionnaires in large-scale surveys such as OECD/TALIS, ESS, ESENER-2, OECD/PIAAC and OECD/PISA.