Five Elite Linguists from Niger: a workshop for a UNESCO project provides a cross-cultural perspective on test localization

by Steve Dept – cApStAn partner

Fourteen years ago, in 2005, the UNESCO Institute of Statistics (UIS) were preparing the implementation of their Literacy Assessment and Monitoring Programme (LAMP). The programme aimed to measure a spectrum of literacy levels in a number of developing countries. The tested populations were adults and young adults (15 or older) not attending school. The Reading Literacy framework was adapted from the International Adult Literacy Survey (IALS), supplemented with an assessment of component skills, and this was familiar territory for cApStAn.

When we were contacted to organise an independent translation verification of national versions of instruments, we knew that this would be a difficult but rewarding project. We did have trained Arabic, Mongolian and Swahili verifiers—so we felt confident that our colleagues in Gaza, Nairobi and Ulaanbaatar would be up to the task—but we had not performed any linguistic quality control work in national languages of Niger other than French: Fulfulde, Hausa, Kanuri, Songhay Zarma and Tamashek. At the time, it was quite rare to have profiles of African linguists on the Internet and, besides, I wanted to contact people who (i) had studied linguistics abroad; (ii) had returned to their country to use their knowledge in the field; and (iii) were not into politics.

We used contacts in our university network to eventually reach the Linguistics and National Languages Department (LILAN) of the Institute for Human Science Research Department (IRSH) of Abdou Moumouni University in Niamey. The person I eventually talked to after numerous exploratory phone calls, a native speaker of Songhay Zarma, had a PhD in linguistics from a French university and was actively involved in creating, evaluating and improving educational materials in the national languages other than French. His expertise was impressive, beyond what I could hope for. When I explained how cApStAn worked, he was very enthusiastic, and agreed to bring together of top linguists for each of the other four languages and agreed to be the Songhay Zarma verifier and the team coordinator. Each would have to be proficient in English, of course, because the LAMP instruments were designed in English.

So, I prepared draft contracts, training materials and guidelines but, for good measure, I decided to translate everything in French so that the linguists would have reference material in the language of their academic training. I booked my flight and flew to Niamey to train the linguists to verify linguistic quality, construct relevance in Niger and equivalence across languages in terms of level of difficulty. The reception I received was unforgettable: four out of five linguists had a PhD from a Western University (the fifth, our Tuareg colleague studied in Niger and became an expert in Bible translation into Tamashek) and were involved in applied linguistics projects in the country. They knew far more than I did about the cultural relevance of literacy items and about the linguistic features that affect psychometric properties of test items in their language. In fact, I had found the very best linguists in the country: while I was there, drinking sweet green tea and eating grilled goat meat with my fellow linguists, the LAMP National Centre in Niamey contacted them to perform the translation. They all refused, because they were already under contract with cApStAn: this does say a lot about their ethical standards. The conflict of interest was as clear to them as it was to me, and I can vouch that they did not even consider working on both translation and verification.

So, our training workshop lasted 2.5 days and, of course, we became friends. This is probably the most interactive and the most rewarding training workshop I have given in all those years, to the extent that it led us to adapt our methodology to accommodate specificities of the languages. I was humbled: the roles of trainer and trainee seemed interchangeable, but in fact we all learnt a lot.

Let me close with an anecdote, to illustrate the level of feedback we received. The Reading Components test was designed to measure, among other skills, the respondents’ competencies in sentence processing. It included a set of short statements, which the respondents had to label as true or false. One of the sentences was “All plants need light to grow.” Our Tuareg colleague pointed out that “in the desert, there is a lot of light all year and nothing grows at all. Perhaps it would be less obvious for Tuareg respondents to identify this statement as true than for somebody living in a place where sunlight is at times scarce?” Since the test item was intended to assess sentence processing skills rather than scientific literacy, he proposed to adapt the statement to “All plants need water to grow.”

This is the perfect example to illustrate that adaptation is an intentional deviation from the master version, which becomes necessary when a straightforward translation might put the respondent at an advantage or a disadvantage.