The PISA 2021 Core D contract awarded to cApStAn
by Steve Dept – cApStAn partner
The OECD Programme for International Student Assessment, or PISA (see the short PISA video here) is a system-level assessment of knowledge and skills in 15-year old students across the world. In other words, PISA implements (and analyses) internationally standardised assessments of student learning outcomes towards the end of compulsory education. The first edition of PISA was in 2000, and at the time cApStAn linguists were commissioned to verify 32 national versions of the PISA assessment instruments. PISA is a triennial survey, and we have been involved in linguistic quality assurance (LQA) and linguistic quality control) of PISA materials in all 7 rounds of this large-scale scientific endeavour. In PISA 2015, over half a million students were assessed, representing 28 million 15-year-olds in 72 countries/economies.
For the current survey cycle, PISA 2018, the OECD had broken down the project in a number of different Cores, so that linguistic quality control was envisaged as a separate contract. Last year, the OECD issued a similar call for tenders for the implementation of PISA 2021. While we don’t know how many organisations submitted a proposal for <PISA 2021 Core D —Linguistic Quality Control>, we do know that several language service providers attended the Q&A sessions organised by the OECD. Knowing that we would be competing with reputable colleagues, we put a lot of effort in preparing our proposal, highlighting the technical innovations that we have brought to PISA and other projects over the years. We are rather proud of our track record for maximising cross-national and cross-language comparability and for documenting the entire translation, adaptation and verification history of each test item in each language. With over 90 verified national versions in PISA 2018, our team of expert reviewers verified about 21 million words—and this is just for PISA 2018!
So we are genuinely delighted to confirm today that cApStAn’s proposal prevailed and that we shall again be responsible for LQC in the 8th Cycle of PISA: the PISA 2021 Core D contract between the OECD and cApStAn has just been signed. Meanwhile, we have already initiated our signature translatability assessment on newly developed Creative Thinking assessment items. The main domain will be Mathematical literacy in PISA 2021, but Reading literacy and Scientific literacy will be assessed, too, and Creative Thinking is an innovative domain.
In close cooperation with the other PISA contractors and under the leadership of Dr Irwin Kirsch, from ETS, the PISA 2021 Core A contractor, cApStAn will be in charge of upholding the stringent PISA quality standards for translation and adaptation of assessment instruments. It is expected that about 90 different country teams will translate or adapt the cognitive tests and contextual questionnaires. cApStAn will assist country teams in using an open source translation editor. The bulk of this work will take place in 2019 and 2020.