Accessibility of a Digital Library for Educational Assessments
cApStAn’s Toledo Solution for On-the-Fly Translation and Quality Estimation
About the client
The client is an international community of national institutions committed to promoting the culture of educational assessment as an essential factor to improve the quality of education worldwide, against the backdrop of transitioning to digital assessments. In 2023 the client founded a consortium made up of five of its national member institutions to build a collaborative platform for sharing educational assessment resources across countries, languages, and subjects. This digital library allows each member to upload their own materials in their own language and also to access, browse and borrow materials uploaded by other members.
Project scope
Since library users are based in different countries, they will upload and share assessment content in languages that other national members may not understand. To ensure the digital library’s effectiveness in a global, diverse context, on-the-fly translation is required. English is used as the common or pivotal language among all members, which means materials uploaded in other national languages must be translated to English so that they are accessible and understandable for all the other members.
cApStAn provides a translation service through its solution Toledo (a web API) that allows the client’s platform to integrate instant machine translation of the resources uploaded by their users to the digital library. Together with the translation of each package submitted, Toledo provides an automatic estimation of the quality of that translation, so that the library users are aware of how reliable the translation is and how convenient or necessary a revision of the translation is (called “post-editing” in translators’ jargon).
*Based on the QE score and confidence label, a post-editing step may be added for quality control
Toledo also provides a vendor-neutral translation project for the purposes of post-editing. Library users can hire a post-editor (or outsource that to cApStAn) and give them access to that translation project to let them improve the quality of the translations. A post-edited version of the English can be used as the basis for translations into other national languages for members who find that resource interesting. Post-editor’s changes can also be used to train Toledo and adapt future translations.
Challenges
A conventional translation project involves some preparation tasks whereby a translation engineer or the project administrator examines every source document and takes certain decisions about how the job must be processed. In the case of Toledo, those decisions are taken upstream based on the preliminary analysis of customer requirements and sample materials, rather than for each set of documents users will send. That means the success of each translation request from the platform relies on the submitted materials’ compliance with the initial requirements in terms of format, content, etc.
Conclusions and outcomes
The client conducted a pilot using Toledo as the translation solution for materials uploaded by five country members. The results met expectations and the pilot was considered a success. At the moment, users can see their materials in English in about 1 minute after upload.
Want to try out this process on your materials?
If you are an organization that requires integrated translation of materials in specialized formats with a view to further exchange and processing, you may select some sample items or sample questions, and contact us for more details at hermes@capstan.be.