PISA follows a three-year cycle and assesses knowledge and skills in 15-year-old students worldwide. The results of this large-scale assessment are used to inform education policies, so cross-linguistic, cross-national and cross-cultural equivalence is not only an objective – it is a requirement. Since the inception of PISA, cApStAn has been commissioned to set up and implement linguistic quality control procedures that minimize meaning shifts—which are usually language-based—and perception shifts, which may be culture-based. Metrics are applied to report on compliance with PISA Translation and Adaptation Guidelines.
All cApStAn verifiers attend face-to-face PISA verification workshops. These specially trained linguists then verify—segment by segment—national versions produced by the participating countries, and document each intervention they suggest. They use cApStAn’s framework of 14 verifier intervention categories to register their edits. National Centres review the verification feedback and decide whether to accept interventions, reject them (with a justification) or propose an alternative solution. The final check by cApStAn seeks to establish and document whether each issue requiring follow-up has been addressed in a satisfactory and consistent manner.
The degree of sophistication in translation, adaptation and verification designs in PISA has led to frequent best practice citations: the item parameters are stable, and PISA results are deemed highly reliable and truly comparable across countries. This is, of course, due to factors such as standardized test administration procedures and robust conceptual frameworks, but also to a higher awareness of the need for linguistic quality assurance and equivalence checks. The amount of PISA data that needs to be discarded due to translation flaws, cultural bias or differential item functioning is extremely low, and we are proud to contribute to this endeavour.