Article published In:
TargetVol. 34:2 (2022) ► pp.309–342
The impact of text presentation on translator performance
Widely used computer-aided translation (CAT) tools divide documents into segments, such as sentences, and arrange
them side-by-side in a spreadsheet-like view. We present the first controlled evaluation of these design choices on translator
performance, measuring speed and accuracy in three experimental text-processing tasks. We find significant evidence that
sentence-by-sentence presentation enables faster text reproduction and within-sentence error identification compared to
unsegmented text, and that a top-and-bottom arrangement of source and target sentences enables faster text reproduction compared
to a side-by-side arrangement. For revision, on the other hand, we find that presenting unsegmented text results in the highest
accuracy and time efficiency. Our findings have direct implications for best practices in designing CAT tools.
Article outline
- 1.Introduction
- 2.Background
- 2.1Text and document visualisation
- 2.2Text and document visualisation in CAT tools
- 2.3Understanding translator performance
- 3.Preliminary feedback from potential users
- 3.1Method
- 3.2Ideation
- 3.3Concept testing
- 3.3.1Praise and opportunities
- 3.3.2Criticism and limitations
- 4.Experimental methods
- 4.1Tasks
- 4.1.1Text reproduction (copy)
- 4.1.2Error identification (scan)
- 4.1.3Revision (revise)
- 4.2Materials
- 4.2.1Texts
- 4.2.2User Interfaces (UIs)
- 4.3Participants
- 4.4Procedure
- 4.5Data analysis
- 5.Experimental results
- 5.1Text reproduction (copy)
- 5.2Error identification (scan)
- 5.3Revision (revise)
- 5.4UI preference
- 6.Discussion and design implications
- 6.1Segmentation
- 6.2Orientation
- 6.3Limitations
- 6.4Future work
- 7.Conclusions
- Acknowledgements
- Notes
-
References
References (54)
References
Arif, Ahmed Sabbir, and Wolfgang Stuerzlinger. 2009. “Analysis
of Text Entry Performance Metrics.” In 2009 IEEE Toronto
International Conference Science and Technology for Humanity (TIC–STH), 26–27 September 2009, Toronto
Canada, 100–105. Piscataway, NJ: IEEE. 

Baayen, R. Harald, and Petar Milin. 2010. “Analyzing
Reaction Times.” International Journal of Psychological
Research 3 (2): 12–28. 

Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting
Linear Mixed-Effects Models Using lme4.” Journal of Statistical
Software 67 (1): 1–48. 

Bojar, Ondřej, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck et al. 2017. “Findings
of the 2017 Conference on Machine Translation (WMT17).” In WMT 2017:
Second Conference on Machine Translation: Proceedings, edited by Ondřej Bojar, Christian Buck, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck et al., 169–214. Stroudsburg: Association for Computational Linguistics. 

Bojar, Ondřej, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn et al. 2018. “Findings
of the 2018 Conference on Machine Translation (WMT18).” In WMT 2018:
Third Conference on Machine Translation: Proceedings of the Conference, edited
by Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck et al., 272–307. Stroudsburg: Association for Computational Linguistics. 

Bowker, Lynne. 2005. “Productivity
vs Quality? A Pilot Study on the Impact of Translation Memory Systems.” Localisation
Focus 4 (1): 13–20.
Carl, Michael. 2010. “A
Computational Framework for a Cognitive Model of Human Translation
Processes.” In Proceedings of Translating and the Computer
32. London: Aslib.
Carl, Michael. 2012. “Translog-II:
A Program for Recording User Activity Data for Empirical Reading and Writing
Research.” In Proceedings of the Eighth International Conference on
Language Resources and Evaluation (LREC ’12), edited by Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno et al., 4108–4112. Istanbul: European Language Resources Association.
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tinsley, and Andy Way. 2017. “Is
Neural Machine Translation the New State of the Art?” The Prague Bulletin of Mathematical
Linguistics 1081: 109–120. 

Castilho, Sheila, Joss Moorkens, Federico Gaspari, Rico Sennrich, Vilelmini Sosoni, Panayota Georgakopoulou, Pintu Lohar et al. 2017. “A
Comparative Quality Evaluation of PBSMT and NMT Using Professional
Translators.” In Proceedings of Machine Translation Summit
XVI, vol. 11, edited by Sadao Kurohashi and Pascale Fung, 116–131. Nagoya: Asia-Pacific Association for Machine Translation.
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Rico Sennrich, Andy Way, and Panayota Georgakopoulou. 2018. “Evaluating
MT for Massive Open Online Courses.” In Human Evaluation of
Statistical and Neural Machine Translation, edited by Andy Way and Mikel L. Forcada, special issue
of Machine
Translation 32 (3): 255–278. 

Coppers, Sven, Jan van den Bergh, Kris Luyten, Karin Coninx, Iulianna van der Lek–Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste. 2018. “Intellingo:
An Intelligible Translation Environment.” In Proceedings of the 2018
CHI Conference on Human Factors in Computing Systems, edited by Regan Mandryk, Mark Hancock, Mark Perry, and Anna Cox. Montreal: Association for Computing Machinery. 

do Carmo, Félix, and Joss Moorkens. 2020. “Differentiating
Editing, Post-Editing and Revision.” In Translation Revision and
Post-Editing: Industry Practices and Cognitive Processes, edited by Maarit Koponen, Brian Mossop, Isabelle S. Robert, and Giovanna Scocchera. Abingdon: Routledge. 

Ehrensberger-Dow, Maureen, Andrea Hunziker Heeb, Gary Massey, Ursula Meidert, Silke Neumann, and Heidrun Becker. 2016. “An
International Survey of the Ergonomics of Professional
Translation.” In Ergonomic Approaches to Professional Practices and
Translator Training, edited by Élisabeth Lavault-Olléon, special
issue of Journal of the Institute of Languages and Cultures of Europe, America, Africa, Asia and
Australia 271. 

Engelbart, Douglas C., and William K. English. 1968. “A
Research Center for Augmenting Human Intellect.” In AFIPS 1968 Fall
Joint Conference, Part
1, 395–410. Washington: Thompson Book Company. 

Ericsson, K. Anders, and Herbert A. Simon. 1984. Protocol
Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press.
Federico, Marcello, Alessandro Cattelan, and Marco Trombetti. 2012. “Measuring
User Productivity in Machine Translation Enhanced Computer Assisted
Translation.” In Proceedings of the 10th Conference of the
Association for Machine Translation in the Americas. San Diego: Association for Machine Translation in the Americas.
Francis, Wendy S., and Silvia P. Sáenz. 2007. “Repetition
Priming Endurance in Picture Naming and Translation: Contributions of Component
Processes.” Memory &
Cognition 35 (3): 481–493. 

Gelman, Andrew, and Jennifer Hill. 2007. Data
Analysis Using Regression and Multilevel/Hierarchical Models. New York: Cambridge University Press.
Green, Spence, Jason Chuang, Jeffrey Heer, and Christopher D. Manning. 2014. “Predictive
Translation Memory: A Mixed-Initiative System for Human Language
Translation.” In UIST ’14: Proceedings of the 27th Annual ACM
Symposium on User Interface Software and Technology, edited by Hrvoje Benko, Mira Dontcheva, and Daniel Wigdor, 177–187. New York: Association for Computing Machinery. 

Hassan, Hany, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang et al. 2018. “Achieving
Human Parity on Automatic Chinese to English News Translation.” arXiv
Preprint 1803.05567. 

Hornbæk, Kasper, and Erik Frøkjær. 2001. “Reading
of Electronic Documents: The Usability of Linear, Fisheye, and Overview+detail
Interfaces.” In CHI ’01: Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, edited by Julie Jacko and Andrew Sears, 293–300. New York: Association for Computing Machinery. 

House, Juliane. 2013. “Quality
in Translation Studies.” In The Routledge Handbook of
Translation, edited by Carmen Millán and Francesca Bartrina, 534–547. Abingdon: Routledge.
Junczys-Dowmunt, Marcin. 2019. “Microsoft
Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine
Translation.” In Proceedings of the Fourth Conference on Machine
Translation (WMT19), vol. 21, edited
by Loïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow et al., 225–233. Stroudsburg: Association for Computational Linguistics. 

Karimova, Sariya, Patrick Simianer, and Stefan Riezler. 2018. “A
User-Study on Online Adaptation of Neural Machine Translation to Human Post-Edits.” Machine
Translation 32 (4): 309–324. 

Kay, Martin. 1980. “The
Proper Place of Men and Machines in Language Translation.” Research Report
CSL-80-11. Palo Alto, CA: Xerox Palo Alto Research Center.
Krings, Hans P. 1994. Texte Reparieren: Empirische
Untersuchungen Zum Prozeß Der Nachredaktion von Maschinenübersetzungen. Habilitation
thesis. University of Hildesheim.
Krings, Hans P. 2001. Repairing Texts: Empirical
Investigations of Machine Translation Post-Editing Processes. Translated by Geoffrey S. Koby, Gregory M. Shreve, Katja Mischerikow, and Sarah Litzer. Kent: Kent State University Press.
Langlais, Philippe, Guy Lapalme, and Sébastien Sauvé. 2001. “User
Interface Aspects of a Translation Typing System.” In Advances in
Artificial Intelligence: 14th Biennial Conference of the Canadian Society for Computational Studies of Intelligence, AI
2001, edited by Eleni Stroulia and Stan Matwin, 246–256. Berlin: Springer. 

Läubli, Samuel, Rico Sennrich, and Martin Volk. 2018. “Has
Machine Translation Achieved Human Parity? A Case for Document-Level
Evaluation.” In Proceedings of the 2018 Conference on Empirical
Methods in Natural Language Processing, edited by Ellen Riloff, Davide Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, 4791–4796. Stroudsburg: Association for Computational Linguistics. 

LeBlanc, Matthieu. 2013. “Translators
on Translation Memory (TM): Results of an Ethnographic Study in Three Translation Services and
Agencies.” Translation &
Interpreting 5 (2): 1–13. 

Macklovitch, Elliott. 2006. “TransType2:
The Last Word.” In Proceedings of the Fifth International Conference
on Language Resources and Evaluation (LREC 2006), edited by Nicoletta Calzolari, Khalid Choukri, Aldo Gangemi, Bente Maegaard, Joseph Mariani, Jan Odijk, and Daniel Tapias, 167–172. European Language Resources Association.
Green, Spence, Jeffrey Heer, and Christopher D. Manning. 2013. “The
Efficacy of Human Post-Editing for Language Translation.” In CHI
2013: Changing Perspectives, Conference Proceedings, edited by Wendy E. Mackay, Stephen Brewster, and Susanne Bødker. New York: Association for Computing Machinery. 

Miniukovich, Aliaksei, Antonella de Angeli, Simone Sulpizio, and Paola Venuti. 2017. “Design
Guidelines for Web Readability.” In DIS ’17: Proceedings of the 2017
Conference on Designing Interactive Systems, edited by Oli Mival, Michael Smyth, and Peter Dalsgaard, 285–296. New York: Association for Computing Machinery. 

Moorkens, Joss, and Sharon O’Brien. 2017. “Assessing
User Interface Needs of Post-Editors of Machine
Translation.” In Human Issues in Translation
Technology, edited by Dorothy Kenny, 109–130. Abingdon: Routledge.
Müller, Mathias, Annette Rios, Elena Voita, and Rico Sennrich. 2018. “A
Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine
Translation.” In WMT 2018: Third Conference on Machine Translation:
Proceedings of the Conference, edited by Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck et al., 61–72. Stroudsburg: Association for Computational Linguistics. 

Norman, Donald. 1988. The
Psychology of Everyday Things. New York: Basic Books.
O’Brien, Sharon. 2009. “Eye
Tracking in Translation Process Research: Methodological Challenges and
Solutions.” In Methodology, Technology and Innovation in Translation
Process Research: A Tribute to Arnt Lykke Jakobsen, edited by Inger M. Mees, Fabio Alves, and Susanne Göpferich, 251–266. Frederiksberg: Samfundslitteratur.
O’Brien, Sharon, Maureen Ehrensberger-Dow, Megan Connolly, and Marcel Hasler. 2017. “Irritating
CAT Tool Features That Matter to
Translators.” HERMES 561: 145–162.
Rello, Luz, Martin Pielot, and Mari-Carmen Marcos. 2016. “Make
It Big!: The Effect of Font Size and Line Spacing on Online
Readability.” In Proceedings: The 32th Annual CHI Conference on Human
Factors in Computing Science, edited by Jofish Kaye, Allison Druin, Cliff Lampe, Dan Morris, Juan Pablo Hourcade, 3637–3648. New York: Association for Computing Machinery.
Roberts, Teresa L. 1980. Evaluation of Computer Text
Editors. PhD diss. Stanford University.
Roberts, Teresa L., and Thomas P. Moran. 1982. “Evaluation
of Computer Text Editors.” In CHI ’82: Proceedings of the 1982
Conference on Human Factors in Computing Systems, edited by Jean A. Nichols and Michael L. Schneider, 136–141. New York: Association for Computing Machinery. 

Ruiz, Carmen, Natalia Paredes, Pedro Macizo, and Maria Teresa Bajo. 2008. “Activation
of Lexical and Syntactic Target Language Properties in Translation.” Acta
Psychologica 128 (3): 490–500. 

Saffer, Dan. 2005. The
Role of Metaphor in Interaction Design. Master’s thesis. Carnegie Mellon University.
Sennrich, Rico. 2017. “How
Grammatical Is Character-Level Neural Machine Translation? Assessing MT Quality with Contrastive Translation
Pairs.” In 15th Conference of the European Chapter of the Association
for Computational Linguistics: Proceedings of the
Conference, vol. 21, edited
by Mirella Lapata, Phil Blunsom, and Alexander Koller, 376–382. Stroudsburg: Association for Computational Linguistics. 

Shneiderman, Ben. 1983. “Direct
Manipulation: A Step Beyond Programming
Languages.” Computer 8 (16): 57–69. 

Yu, Chen-Hsiang, and Robert C. Miller. 2010. “Enhancing
Web Page Readability for Non-Native Readers.” In CHI ’10: Proceedings
of the SIGCHI Conference on Human Factors in Computing Systems, edited by Elizabeth Mynatt, Geraldine Fitzpatrick, Scott Hudson, Keith Edwards, and Tom Rodden, 2523–2532. New York: Association for Computing Machinery. 

Zaretskaya, Anna. 2015. D2.1:
User Requirement Analysis. Technical Report: EXPERT: EXPloiting Empirical appRoaches to Translation,
the European Union’s Seventh Framework Programme (FP7). [URL]
Cited by (2)
Cited by two other publications
Briva-Iglesias, Vicent, Sharon O’Brien & Benjamin R. Cowan
Zhou, Wei, Bei Gao & Zhao Kaifa
2022.
[Retracted] Innovation and Exploration of Computer‐Aided New Media Translation Course Teaching Mode under the Ecological Environment.
Journal of Environmental and Public Health 2022:1

This list is based on CrossRef data as of 27 september 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.