If Quality Matters, Define It | Translating Europe Forum (TEF)

Article by Steve Dept – Founder, cApStAn

If Quality Matters, Define It

Each year, under the banner of Multilingualism, translation and language-based AI services, the European Commission’s Directorate-General for Translation organizes a high-level conference known as the Translating Europe Forum (TEF). Last year’s theme was “Words Matter”, this year’s conference theme was “Quality Matters”. When we founded cApStAn 25 years ago, we defined it as a linguistic quality control agency, so it made perfect sense for us to attend and learn how different players, stakeholders, academics, language experts and practitioners define quality, how they measure it, what is missing, and how technology boosts or hampers quality (and quality evaluation).

Impressive Venue

On November 5-6-7, I attended in person, in the remarkable Charlemagne Building of the Commission in Brussels. It was a brilliant event, worth every minute, in terms of both content and encounters.

I made it through the security check, then from the moment I shared a first coffee with an interpreter, a professor and a translation technologist, I knew that this event would be intellectually rewarding and humanly enriching. There were 700 in-person attendees and 2000 online participants from over 100 countries.

Clear Language as an Emerging Priority

The genial hosts Anna Holmén and Miha Žličar from DG Translation whetted our appetite with brevity and flair and, indeed, the opening speech, the first keynote and breakout sessions were dedicated to clear language. It is a courageous and necessary move to set the stage by combining clarity and quality. Plain language does not need to be imprecise or primitive. It can be accurate and inclusive. Let me share a bar chart that worked as an eye-opener form any of those who attended the same breakout session as I did: the blue bars represent the German public’s language proficiency at each CEFR level, the brown bars represent the proficiency level needed to read and understand corporate communication. This is not a proficiency gap; it is a great rift! Language professionals must focus on getting the message across rather than showcasing the sophistication of their vocabulary.

Creativity and Translator’s Satisfaction as a Component of Quality

After lunch, we all scrambled back to the Alcide De Gaspieri Room for Ana Guerberof’s keynote. I already knew the broad and narrow definitions of translation quality by Koby et al (2014) with which she opened her presentation. But then she incorporated creativity as a component of translation quality (I agree, the focus was on literary translation, but it isn’t it essential to incorporate that human differentiator in at least some quality metrics?), and she associated this element with job satisfaction. And of course, Ana was transparent about the research design and shared data, results and insights that I found particularly exciting.

Panels on Translation Quality

The panel that followed brought a diverse spectrum of definitions, approaches and perceptions of quality in translation, and what it takes to set the stage for quality before translation begins. The thunderous applause at the end was both an expression of gratitude to the experts and of support to Ukraine, as the moderator Oleksandr Bondarenko, member of the Language Industry Expert Group (LIND) thanked the audience for the support given to Ukraine by the European institutions and people.

Another panel for which I am grateful focused on terminology, aptly called the hidden driver of quality. Of course, I immediately forgot that I was tired when Maria Rzewuska-Waligóra (DG Translation), Jourik Ciesielski (a translation technology expert from the language industry) and Cristina Valentini (Head of terminology unit at the WIPO) demonstrated the breadth, scope and usefulness of terminology beyond just building glossaries.

Ozzy Osbourne Revisited by a Translator

The Commission had organised a networking reception at the end of this long day, and I believe no-one would have wanted to leave. As we all know, most translators have two passions (one of them is, of course, translation). For Carlos la Orden Tovar, the other passion is rock ‘n roll: who would have thought that the live music at a reception at the Commission would include songs from Ozzy Osbourne and Black Sabbath?

As you can see, we had a whale of a time (the selfie is with Christoper Kurz, who is the driving force behind ISO 5060, and Diego Cresceri, from Creative Words)

Standards and Labels

The opening keynote on Day 2 was given by Bruno Herrmann, whose holistic approach hinges on value creation and effectiveness. Bruno has frameworks, models and quotes to drive the point home. Bruno’s enthusiasm is communicative and his recommendations is actionable.

This was a good stepping stone for the much-awaited panel session on quality standards. Professor Alan Melby (whose work I admire), Eva-Maria Tillmann (Head of Quality Management at oneword, and active in the DIN Norms task force) and Dr. Christopher Kurz (Head of Translation Management at Enercon, and the driving force behind ISO 5060) provided an overview of recent developments in translation-related standards. Ingemar Strandvik from EU-DGT moderated the exchanges, using the analogy of a menu in a restaurant and how we evaluate quality when we eat out. Alan Melby used the analogy of food labels and consumer protection to promote the Labels Project: “professionally verified translations” would be labelled as such, and all “unverified translations” would be labelled, too, regardless whether they are produced by people or by machines.

TEF Talks

The afternoon TEF Talks were completely different in tone: Dr Änne Troester is a dubbing scriptwriter who does not use AI and shared her passion. She also felt that highly creative work was not threatened by generative AI in the sense that automatically generated dubbing scripts do not succeed in conveying emotion and will not do so anytime soon.

Marina Pantcheva (Director of Linguistic AI Services at RWS) presentation was awe-inspiring, with a good balance between technical foundations and illustrative examples. I have never heard or seen anyone explain with such clarity what bias is and how it emerges in LLMs.

Enterprise-Grade Localisation 101

The afternoon panel discussion was a welcome illustration of an effective collaboration between the localization industry and academia: “Enterprise-Grade Localisation 101” is a 1-month pilot in which students get an opportunity to work under real conditions, with materials provided experts from the industry such as Balász Kis (one of the founders of MemoQ, and a member of the Language Industry Expert Group) and Diego Cresceri (founder of Creative Words and member of the same expert group).

Multi-layered Evaluation of AI-generated Content

The next high-level discussion was titled “All in One: How to Evaluate the Quality of Ai-Generated Content”. Agustín Da Fieno Delucchi (Director of Globalisation, AI and Data Science, Microsoft), Kateřina Gašová (Global Quality Solutions Strategist, Argos Multilingual) and Pavel Soukeník (Head of Global Solutions, Acolad) presented a multi-layered framework that captures all (or almost all) dimensions and categories of quality in multimodal content.

The Next Generation

Before lunch, the four young winners of the European Masters in Translation (EMT) Present Your Thesis award were welcomed and celebrated under the benevolent patronage of Christos Ellinides, and we share his confidence that the new generation is on track to bring the profession to the next level. Christos Ellinides’ closing remarks were aligned with European values and with the buoyancy of the profession rather than with the doomsayers and fearmongering populists.

The entire organization was seamless, the vibe was outstanding, the variety of perspectives was enriching, and I was delighted to end the day with a pan-European perspective by attending a thought-provoking guided tour of the House of European History.