Revitalising multiple choice questions for the digital age
by Pisana Ferrari – cApStAn Ambassador to the Global Village
Multiple choice questions (MQCs) have been the traditional way of testing for generations, and have not changed substantially since they were first introduced, over a century ago, by Frederick Kelly*. His “Kansas Silent Reading Test” was possibly the first (timed) reading test that could be given to groups of students all at once, without them having to write a single sentence, and which could be graded very easily.
Professor Geoffrey Crisp, of the University of New South Wales, interviewed in the article, says the old-fashioned MCQ fails to test a learner’s problem-solving or critical thinking skills. Multiple choice questions tend to reward a good memory rather than measuring depth of understanding. But this need no longer be the case. Crisp is a long-time advocate of adding interactive elements to MCQs, such as, for example, embedding a link in the MCQ to a digital tool or to a resource that students must use to solve the problem. Whether this is a simulation, a video or audio file, a 3D image that can be manipulated, or Excel spreadsheets with macros, students need to understand the concepts to be able to use the tool in order to answer the question. Crisp says that, of course, students can still “guess”, but the reply is not based on recall alone. He says that this kind of interactivity and flexibility “must be viewed as a crucial part of revitalising the form, function, and suitability of the MCQ in the digital age”.
New item formats, such as the ones described by Crisp, represent new challenges in multilingual testing, and it is important to involve specialists, including linguists, at test development stage to identify possible translation and adaptation hurdles before piloting and before the translation process begins. Examples of computer-based testing include the International Computer and Information Literacy Study (ICILS), sponsored by the International Association for the Evaluation of Educational Achievement (IEA). At cApStAn we have a 20+ year experience in linguistic quality assurance and translation of data collection instruments, from needs analysis, to tailor-made execution, to reliable output. Flagship projects include large-scale international assessments such as OECD/PISA, Programme for International Student Assessment, and IEA/TIMSS, Trends in International Mathematics and Science Study.
*Frederick Kelly, in his doctoral thesis in 1914 at the Kansas State Teacher’s College, highlighted the fact that different teachers give different judgments of student work and, in order to eliminate the subjectivity of the evaluations, he put forward the idea of using standard tests with predetermined answers.
Source: