Diversity, Equity, Inclusion and Bias Reduction: Analysis and Adjustment of a Demographics Question Bank used in International Surveys
Awareness of diversity, equity, inclusion and bias reduction (DEI-BR) in surveys is growing, becoming increasingly indispensable as ensures the reliability of any data collection instrument. cApStAn is a language service provider with a 20+ year experience in partnering with organizations who administer surveys in multiple languages and cultures. Our initial contribution to DEI-BR started with our “Translatability Assessment” process (TA), which aims to identify and resolve potential translation and adaptation issues in questionnaires well before the translation process begins. These include sensitive content, relevance for all segments of the target audience, portability of constructs, translatability, ambiguity, gender mainstreaming, racialized stereotypes, and potential cultural bias. Over time our TA process was progressively adapted to include DEI-BR evaluation by linguists, with input from subject matter experts.
Project Scope
Our client (anonymous due to confidentiality agreement) wanted to create a question bank from which researchers could pull demographic questions to add to their surveys for later analysis. cApStAn was commissioned to specifically look into the gender options and propose the appropriate and inclusive options that should be made available in an international survey. The Initial template the client had included 3 different types of gender questions (non-binary, binary, open-response), 2 race/ethnicity questions (using US census model), 2 educational background questions (OECD and country-specific), and 2 disability status questions (general, specific diagnosis).
Challenges
Some of the question categories in the client’s original template were not culturally appropriate or had potential for legal repercussions, e.g., version of gender question, asking about race as opposed to nationality and the specific categories therein.
cApStAn’s proposed solution:
Step 1: For each country (UK, Japan, India, Switzerland, Singapore), we reviewed the most recent national census and, where applicable, the materials from a national office of statistics equivalent or any large-scale existing national surveys or assessments (usually universities).
Step 2: We documented our findings with sources and suggested ways the questions could be adapted to collect the accurate information from the survey respondents without being culturally insensitive or legally invalid in the target countries.
Step 3: We proposed to map nationally adapted versions to the pre-existing US questions with references so researchers could be assured they were collecting the correct demographic information but translate and field only the adapted versions where the wording/structure was appropriate to the country in question.
Conclusions
- Diversity, Equity and Inclusion adjustments in surveys are a real challenge, and the more people you aim to include the bigger the challenge. The challenge is exponentially bigger when adding multilingual and multicultural dimensions.
- Have a process in place to mitigate possible problems.
- Include representatives of the sample that is being targeted in initial preparations of developing the survey questionnaire to ensure cultural appropriateness and thus fairness and equity.
Select some sample items or sample questions and request a free DEI analysis at hermes@capstan.be or contact us via the form below for more details.