Multilingual Surveys: Five cost-effective steps to improve your questionnaire before translation begins
by Steve Dept, cApStAn CEO
In multilingual surveys, there is a strong trend towards performing more “upstream” quality assurance work to reduce the need for “downstream” corrective action. The late Professor Janet A.Harkness (1948-2012) relentlessly insisted on how important it is to craft questionnaire items carefully before they serve as a basis for translation and adaptation into multiple languages. Her holistic approach contributed to raise awareness of questionnaire localisation issues in item writers, investigators and language professionals.(1)
If your organization conducts surveys in multiple languages and countries, investing in upstream linguistic quality assurance will yield a high return on investment in terms of cross-national, cross-linguistic and cross-cultural comparability.
The five steps to take pre-translation for your multilingual surveys
In this article we shall take a closer look at five cost-effective steps that can take place before the translation/adaptation process begins.
1. Parsing and segmentation
An essential operation in preparing a master questionnaire for translation is parsing and segmenting the text. Parsing is about deciding what is translatable and what is not, and protecting the latter. Segmentation is about splitting the text into smaller segments that computer-assisted translation tools can manage effectively. The optimal result of this process is that only the translatable parts of the text are editable for the linguist whereas the untranslatable parts are either hidden or locked.
2. Defining project specific attributes
It is interesting to define project-specific attributes, which can later be checked in the target versions by means of automated checks. This includes determining a maximum number of digits for keys and captions in graphic materials, defining <forced adaptations>, identifying the need for dynamic text, or determining what metadata needs to travel with the files, and in what form.
3. Glossaries and automated checks
Before translation begins, terms and expressions are extracted from the master documents. Once the glossary is constructed according to certain rules, it is possible to set up automated checks on consistent adherence to the glossary. At cApStAn we use a combination of term extraction software and human expertise to compile a list of terms and expressions from the Master documents. Glossaries can be created for a particular client or even for a specific project for a client.
4. Style guides
Creating Style guides includes creation of and adherence to sets of typographic conventions, forms of address, notation of numbers, time, mathematical units, currencies, footnotes or endnotes, quotes, indents, etc. At cApStAn we either create a new style guide as per the requirement, or adapt our client’s existing style guide for a given project and a set of target languages.
5. Translation and Adaptation notes
If concepts, alternatives, objectives, possible ambiguities or intended meaning of certain terms or expressions in the master questionnaire are explained clearly and concisely, the impact on quality is immense. That is why question by question translation, and adaptation notes, are an essential component of a robust translation/adaptation design.
Translatability Assessment for multilingual surveys
Translation quality in multilingual surveys is crucial as meaning shifts, which are language-driven, and perception shifts, which are culture-driven, can affect the psychometric properties of items and thus compromise comparability. cApStan’s “Translatability Assessment” process will identify any potential issues upfront and report them in a centralised monitoring tool. Occasionally, alternative text is proposed — but without loss of meaning. A translatability assessment will ensure that it is possible for multiple language versions of a document to meet the highest standards of linguistic, cultural and functional equivalence. Our translatability assessment methodology is being applied in small and large multilingual projects, in both the private and the public sector.
Read more about cApStAn’s translatability assessment process in this article
What steps should you take after translation of your multilingual surveys
Once a questionnaire is translated, an independent quality check of the translation is always desirable in order to ensure that the translated questionnaire is fair and comparable in all languages
1) Janet A. Harkness Janet A Harkness was a cross-cultural survey methodologist with expertise in instrument design, adaptation, and implementation. She had a multi-disciplinary academic background and training -comparative medieval studies (MA); linguistics, cultural anthropology (PhD); and extensive experience in survey methods and cross-national survey research, having being affiliated over 15 years with the International Social Survey Programme, a member of the European Social Survey central coordinating team from its inception till 2008, and a consultant or PI on other large cross- national projects, both basic research and survey projects. She has held the Donald and Shirley Clifton Chair in Survey Science at the University of Nebraska-Lincoln in 2005 and became Director of the Survey Research and Methodology Program at the UNL Gallup Research Center in 2006. She has published, presented, and taught on numerous aspects of cross-cultural survey research and cross-cultural communication.
Photo credit: Shutterstock
Do you want more information about us? Check out the following resources:
Latest blog posts on tests and assessments
- How to best approach the translation and adaptation of tests and surveys for gendered languages
- Translated Instruments: Coverage at the cost of Equity?
- The Four Key Moments of a Test or Survey Localization Project
- The translation and adaptation journey of a TIMMS grade 8 science item
- What are “intervention categories” and how can revisers use them to improve translation quality?
- Do SMEs and Linguists mix like Water and Oil?
- Do Trained Interviewers Follow Their Script? Reports from Zedland
- Cognitive Pre-Testing in More Than One Language
- How can you create culturally fair content for tests and surveys?