The impact of text presentation on translator performance
Patrick Simianer | Lilt, Inc.
Joern Wuebker | Lilt, Inc.
Geza Kovacs | Lilt, Inc.
Spence Green | Lilt, Inc.
Widely used computer-aided translation (CAT) tools divide documents into segments, such as sentences, and arrange
them side-by-side in a spreadsheet-like view. We present the first controlled evaluation of these design choices on translator
performance, measuring speed and accuracy in three experimental text-processing tasks. We find significant evidence that
sentence-by-sentence presentation enables faster text reproduction and within-sentence error identification compared to
unsegmented text, and that a top-and-bottom arrangement of source and target sentences enables faster text reproduction compared
to a side-by-side arrangement. For revision, on the other hand, we find that presenting unsegmented text results in the highest
accuracy and time efficiency. Our findings have direct implications for best practices in designing CAT tools.
Keywords: computer-aided translation, human-computer interaction, user interface, text presentation, productivity
Article outline
- 1.Introduction
- 2.Background
- 2.1Text and document visualisation
- 2.2Text and document visualisation in CAT tools
- 2.3Understanding translator performance
- 3.Preliminary feedback from potential users
- 3.1Method
- 3.2Ideation
- 3.3Concept testing
- 3.3.1Praise and opportunities
- 3.3.2Criticism and limitations
- 4.Experimental methods
- 4.1Tasks
- 4.1.1Text reproduction (copy)
- 4.1.2Error identification (scan)
- 4.1.3Revision (revise)
- 4.2Materials
- 4.2.1Texts
- 4.2.2User Interfaces (UIs)
- 4.3Participants
- 4.4Procedure
- 4.5Data analysis
- 4.1Tasks
- 5.Experimental results
- 5.1Text reproduction (copy)
- 5.1.1Speed
- 5.1.2Accuracy
- 5.2Error identification (scan)
- 5.2.1Speed
- 5.2.2Accuracy
- 5.3Revision (revise)
- 5.3.1Speed
- 5.3.2Accuracy
- 5.4UI preference
- 5.1Text reproduction (copy)
- 6.Discussion and design implications
- 6.1Segmentation
- 6.2Orientation
- 6.3Limitations
- 6.4Future work
- 7.Conclusions
- Acknowledgements
- Notes
-
References
Published online: 23 November 2021
https://doi.org/10.1075/target.20006.lau
https://doi.org/10.1075/target.20006.lau
References
Arif, Ahmed Sabbir, and Wolfgang Stuerzlinger
Baayen, R. Harald, and Petar Milin
Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker
Bojar, Ondřej, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck et al.
2017 “Findings
of the 2017 Conference on Machine Translation (WMT17).” In WMT 2017:
Second Conference on Machine Translation: Proceedings, edited by Ondřej Bojar, Christian Buck, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck et al., 169–214. Stroudsburg: Association for Computational Linguistics. 

Bojar, Ondřej, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn et al.
2018 “Findings
of the 2018 Conference on Machine Translation (WMT18).” In WMT 2018:
Third Conference on Machine Translation: Proceedings of the Conference, edited
by Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck et al., 272–307. Stroudsburg: Association for Computational Linguistics. 

Bowker, Lynne
Campbell, Stuart
Carl, Michael
2012 “Translog-II:
A Program for Recording User Activity Data for Empirical Reading and Writing
Research.” In Proceedings of the Eighth International Conference on
Language Resources and Evaluation (LREC ’12), edited by Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno et al., 4108–4112. Istanbul: European Language Resources Association.
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tinsley, and Andy Way
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Rico Sennrich, Vilelmini Sosoni, Panayota Georgakopoulou, Pintu Lohar et al.
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Rico Sennrich, Andy Way, and Panayota Georgakopoulou
Coppers, Sven, Jan van den Bergh, Kris Luyten, Karin Coninx, Iulianna van der Lek–Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste
do Carmo, Félix, and Joss Moorkens
Dragsted, Barbara
Ehrensberger-Dow, Maureen, Andrea Hunziker Heeb, Gary Massey, Ursula Meidert, Silke Neumann, and Heidrun Becker
2016 “An
International Survey of the Ergonomics of Professional
Translation.” In Ergonomic Approaches to Professional Practices and
Translator Training, edited by Élisabeth Lavault-Olléon, special
issue of Journal of the Institute of Languages and Cultures of Europe, America, Africa, Asia and
Australia 271. 

Engelbart, Douglas C., and William K. English
Ericsson, K. Anders, and Herbert A. Simon
Federico, Marcello, Alessandro Cattelan, and Marco Trombetti
Francis, Wendy S., and Silvia P. Sáenz
Gelman, Andrew, and Jennifer Hill
Green, Spence, Jason Chuang, Jeffrey Heer, and Christopher D. Manning
2014 “Predictive
Translation Memory: A Mixed-Initiative System for Human Language
Translation.” In UIST ’14: Proceedings of the 27th Annual ACM
Symposium on User Interface Software and Technology, edited by Hrvoje Benko, Mira Dontcheva, and Daniel Wigdor, 177–187. New York: Association for Computing Machinery. 

Hassan, Hany, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang et al.
Hornbæk, Kasper, and Erik Frøkjær
House, Juliane
Jakobsen, Arnt Lykke
Junczys-Dowmunt, Marcin
2019 “Microsoft
Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine
Translation.” In Proceedings of the Fourth Conference on Machine
Translation (WMT19), vol. 21, edited
by Loïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow et al., 225–233. Stroudsburg: Association for Computational Linguistics. 

Karimova, Sariya, Patrick Simianer, and Stefan Riezler
Kay, Martin
Krings, Hans P.
1994 Texte Reparieren: Empirische
Untersuchungen Zum Prozeß Der Nachredaktion von Maschinenübersetzungen. Habilitation
thesis. University of Hildesheim.
Langlais, Philippe, Guy Lapalme, and Sébastien Sauvé
Läubli, Samuel, Rico Sennrich, and Martin Volk
2018 “Has
Machine Translation Achieved Human Parity? A Case for Document-Level
Evaluation.” In Proceedings of the 2018 Conference on Empirical
Methods in Natural Language Processing, edited by Ellen Riloff, Davide Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, 4791–4796. Stroudsburg: Association for Computational Linguistics. 

LeBlanc, Matthieu
Macklovitch, Elliott
2006 “TransType2:
The Last Word.” In Proceedings of the Fifth International Conference
on Language Resources and Evaluation (LREC 2006), edited by Nicoletta Calzolari, Khalid Choukri, Aldo Gangemi, Bente Maegaard, Joseph Mariani, Jan Odijk, and Daniel Tapias, 167–172. European Language Resources Association.
Green, Spence, Jeffrey Heer, and Christopher D. Manning
Miniukovich, Aliaksei, Antonella de Angeli, Simone Sulpizio, and Paola Venuti
Moorkens, Joss, and Sharon O’Brien
Müller, Mathias, Annette Rios, Elena Voita, and Rico Sennrich
2018 “A
Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine
Translation.” In WMT 2018: Third Conference on Machine Translation:
Proceedings of the Conference, edited by Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck et al., 61–72. Stroudsburg: Association for Computational Linguistics. 

O’Brien, Sharon
2009 “Eye
Tracking in Translation Process Research: Methodological Challenges and
Solutions.” In Methodology, Technology and Innovation in Translation
Process Research: A Tribute to Arnt Lykke Jakobsen, edited by Inger M. Mees, Fabio Alves, and Susanne Göpferich, 251–266. Frederiksberg: Samfundslitteratur.
O’Brien, Sharon, Maureen Ehrensberger-Dow, Megan Connolly, and Marcel Hasler
Rello, Luz, Martin Pielot, and Mari-Carmen Marcos
2016 “Make
It Big!: The Effect of Font Size and Line Spacing on Online
Readability.” In Proceedings: The 32th Annual CHI Conference on Human
Factors in Computing Science, edited by Jofish Kaye, Allison Druin, Cliff Lampe, Dan Morris, Juan Pablo Hourcade, 3637–3648. New York: Association for Computing Machinery.
Roberts, Teresa L.
1980 Evaluation of Computer Text
Editors. PhD diss. Stanford University.
Roberts, Teresa L., and Thomas P. Moran
Ruiz, Carmen, Natalia Paredes, Pedro Macizo, and Maria Teresa Bajo
Saffer, Dan
2005 The
Role of Metaphor in Interaction Design. Master’s thesis. Carnegie Mellon University.
Schneider, Dominik, Marcos Zampieri, and Josef van Genabith
Sennrich, Rico
2017 “How
Grammatical Is Character-Level Neural Machine Translation? Assessing MT Quality with Contrastive Translation
Pairs.” In 15th Conference of the European Chapter of the Association
for Computational Linguistics: Proceedings of the
Conference, vol. 21, edited
by Mirella Lapata, Phil Blunsom, and Alexander Koller, 376–382. Stroudsburg: Association for Computational Linguistics. 

Shih, Claire Yi-yi
Shneiderman, Ben
Yu, Chen-Hsiang, and Robert C. Miller
2010 “Enhancing
Web Page Readability for Non-Native Readers.” In CHI ’10: Proceedings
of the SIGCHI Conference on Human Factors in Computing Systems, edited by Elizabeth Mynatt, Geraldine Fitzpatrick, Scott Hudson, Keith Edwards, and Tom Rodden, 2523–2532. New York: Association for Computing Machinery. 

Zaretskaya, Anna
2015 D2.1:
User Requirement Analysis. Technical Report: EXPERT: EXPloiting Empirical appRoaches to Translation,
the European Union’s Seventh Framework Programme (FP7). http://expert-itn.eu/sites/default/files/outputs/expert_d2.1_20150210.pdf