2022 ATP Innovations in Testing Conference
The ATP Innovations in Testing 2022 Conference is almost here and we’re excited to be there! We are looking forward to presenting at the conference with some of our industry colleagues on the following topics:
You Can Do Better Than Comply with the Standards: A Collaborative Cross-check of your DEI-BR Processes
Speakers: Christine Mills, Kelly Von Lunen from Ascend Learning; Musab Hayatli, Steve Dept from cApStAn
March 22, 10:00 am – 10:50 am [Virtual session]
The testing and credentialing culture has evolved rapidly in the past two years; there is a need to adjust priorities. Fairness is a key validity issue. Testing sponsors are revisiting the spectrum of techniques and processes that yield valid outcomes and fair experiences for all examinees. Diversity, Equity, Inclusion, and Bias Reduction (DEI-BR) is not just a box that needs to be ticked off. One can strive to go beyond compliance with standards, take a fresh look at one’s processes, and draw up a more ambitious DEI-BR roadmap.
In this session, attendees will hear about Ascend Learning’s initiative to set up a collaborative cross-check of their existing DEI-BR processes with cApStAn, an external organization.
This project’s scope has been defined as completing a proactive process review of training for staff and subject matter experts, document tracking, subject matter expert recruitment, and organizational DEI guidelines. cApStAn first looked at the process through the lens of cultural competence.
The extent to which organizations address DEI in education and credentialing bears directly on the cultural competence students will implement as they transition to be professionals. As credentialing or testing organizations, we have an obligation to do what we can in our products and assessments to foster culturally competent professionals who can address the needs of all clients they will encounter.
Fairness in Multilingual Assessments – Leaning on the Standards, and Taking Necessary Trade-Offs in Stride
Speakers: Steve Dept cApStAn, Joy L. Mattews-López JML Measurement & Testing Services, LLC
March 22, 4:40 pm – 5:30 pm EDT [In-person session]
The more one learns about the complexity of adapting tests in multiple languages; the more one wonders whether there are techniques and processes to develop and administer several language versions of a high-stakes test fairly to all test takers.
Fairness is a validity issue; it touches all aspects of test development, delivery, and maintenance.
Each assessment component must be fair: test design, constructs, items/content, policies that guide test administration, and scoring/reporting.
In this presentation, Joy Matthews-López and Steve Dept will first elaborate on interpreting existing standards and guidelines when setting up a test adaptation design. Then they will refer to use cases to contextualise informed decision-making in test adaptation workflows.
The important questions that test owners and item writers need to ask themselves when developing a program can be reframed in terms of fairness:
For example:
What is the intended use of assessment outcomes (resulting scores), and are these equally interpretable for source and target populations?
Does anything in the content advantage or disadvantage examinees in the source or (one of) the target language(s)?
The presenters will refer to case studies from certification exams and large-scale international assessments to illustrate good practice (upstream quality assurance work and corrective action).
Navigating Diversity, Equity and Inclusion in Multilingual and Multicultural Contexts
Speakers: Steve Dept, cApStAn
March 23, 8:30 am – 9:20 am EDT [In-person session]
Fairness, validity, and reliability are objectives that every testing organisation represented at this conference strives to address.
Cultures and societies are dynamic. So is awareness of social justice issues (#BlackLivesMatter, #MeToo, broader acceptance of non-binary gender identity). Theoretical and practical approaches to fairness in testing need to evolve. Diversity, Equity, and Inclusion (DEI) has become a crucial component in this regard.
Many conference delegates have had to deal with challenges related to DEI in the fast-changing testing landscape of the US and Canada. And some may have had a glimpse of what is at stake when the candidates or respondents come from very diverse backgrounds and when the test program needs to be administered in multiple languages and regions.
In this Peas in a Pod discussion, Steve Dept, who has been dealing with DEI issues on five continents, will present examples of content at risk of being at odds with cultural sensitivity in certain contexts and discuss adaptation strategies for controversial elements.
Participants will be invited to share challenges they are aware of or have faced when exporting assessment content to different languages and cultures.