Survey Translation in 2025: is expertise still needed with today’s toolkit?
We’re thrilled to share that cApStAn’s founder Steve Dept was invited to speak at Google’s internal Survey Conference last week, where he shared insights on the latest trends and best practices in survey translations.
Abstract from the session:
The blueprint for high-quality questionnaire translations using sophisticated workflows was developed at a time when data collection instruments were pencil-and-paper questionnaires. The TRAP-D design (Translation, Review, Adjudication, pre-testing and Documentation, see e.g. Harkness et al. 2003, 2004) is an often-cited methodology for survey translation and adaptation, and it has been widely recognised as best practice by the scientific community.
Meanwhile, all questionnaires are digital and delivered through a variety of devices and the tools have evolved.
The combination of mature translation technology, such as computer-aided translation tools (CAT tools), and new advances in machine learning, machine translation (MT) and large language models (LLMs) can bring efficiency gains but require an expertise that is not always available to survey methodologists.
Just like back translation used to give questionnaire authors the illusion of control over translation quality, the use of MT or LLMs for back translations provide the illusion of oversight.
Today’s tools are indeed advanced but then accurate translations are not very good at solving culture-driven perception shifts. Survey methodologists and field practitioners argue whether it is preferable to tweak the tools to accommodate best practice in survey translation or to reform established best practice so that new advances in technology can be leveraged effectively.
If the tools are used professionally, it has become possible to design workflows in which experts add value at specific points in the survey localisation management process. This requires discernment in using AI purposefully, traceability and, where possible, pre-testing of translated version of the survey questionnaires.
The objective remains cross-linguistic comparability, a tall order that technology doesn’t solve on its own.