PHD, Kent State University, 2024, College of Arts and Sciences / Department of Modern and Classical Language Studies

This dissertation investigates the cognitive processes and performance in human-ChatGPT interaction during machine translation post-editing (MTPE), with a focus on how student translators and expert translators interact with GPT-4 generated translations. The research examines how different post-editing guidelines (light vs. full post-editing) and search conditions (termbase vs. internet search) affect the translation process and quality. The study employs a mixed-methods approach, combining quantitative analysis of keystroke logging and eye-tracking data with qualitative assessment of translation quality through manual error annotation. The experimental design simulates a professional translation environment using SDL Trados Studio, involving 46 participants (30 student translators and 16 expert translators) post-editing English-to-Chinese translations generated by GPT-4. To ensure ecological validity, a new research tool, Trados-to-Translog-II, was implemented to integrate keystroke data collected within Trados and eye-tracking data with the CRITT Translation Process Research Database (TPR-DB). The findings reveal that syntactic complexity metrics significantly correlate with translation quality across GPT-4, student translators, and expert translators. GPT-4's output shows greater susceptibility to errors as syntactic complexity increases compared to human translators, with expert translators demonstrating the highest resilience to complex structures. The complexity of GPT-4 output has a more pronounced influence on student translators' performance compared to experts, highlighting the enduring value of expertise in handling difficult translation tasks. The study also finds that students' post-editing performance decreases when conducting tasks with internet search compared to working with a specific termbase. Analysis of students' post-editing interventions reveals significant variations across different tasks, suggesting the need for enhanced training i (open full item for complete abstract)

Committee: Michael Carl (Advisor); Isabel Lacruz (Committee Member); Françoise Massardier-Kenney (Committee Member); Masaru Yamada (Committee Member); Ryan Miller (Other) Subjects: Linguistics

PHD, Kent State University, 2022, College of Arts and Sciences / Department of Modern and Classical Language Studies

Post-editing of machine translation (MT) is a workflow that is being used for an increasing number of text types and domains (Koponen, 2016; Hu, 2020; Zouhar et al., 2021),but the sections of text that post-editors need to fix have become harder to detect due to the increased human-like fluency that neural machine translation (NMT) affords (Comparin & Mendes, 2017; Yamada, 2019). This dissertation seeks to address this problem by developing a word-level machine translation quality estimation (MTQE) system to highlight words in raw MT output that need editing in order to aid post-editors. Subsequently, this MTQE system is tested in a large-scale post-editing experiment to determine if it increases productivity and decreases cognitive effort and error rate. This MTQE system is based on two automatically generated features: word translation entropy, generated from the output of multiple MT systems (a feature that has never been used in MTQE), and word class (based on part-of-speech tags). For the post-editing experiment, a within-subjects design assigns raw MT output to participants under three different conditions. Two experimental conditions consist of MT output that has been enhanced with highlighting surrounding the stretches of text that likely need to be edited. In the first experimental condition, this highlighting is supplied automatically by the MTQE system, and in the second experimental condition, this highlighting is supplied by an experienced translator, indicating what text needs editing. The control condition constitutes MT output without highlighting. Participants post-edit three experimental texts in Trados Studio while time-stamped keystroke logs are gathered (which are later integrated into the CRITT Translation Process Research Database (TPR-DB)), and various measures of temporal, technical, cognitive, perceived effort, and group editing activity are used to assess the efficacy and usefulness of highlighting potential errors in the post-editing user (open full item for complete abstract)

Committee: Michael Carl (Advisor); Lucas Nunes Vieira (Committee Member); Isabel Lacruz (Committee Member); Erik Angelone (Committee Member) Subjects: Artificial Intelligence; Language; Linguistics

PHD, Kent State University, 2018, College of Arts and Sciences / Department of Modern and Classical Language Studies

This study tests the time course of language activation in reading for translation. Reading for translation has been modeled vertically (two monolingual systems activated serially), horizontally (both monolingual systems automatically activated in parallel), and as some blend of these perspectives. Schaeffer, Dragsted, Hvelplund, Balling, and Carl (2016b) provided evidence supporting early horizontal processing in reading for translation. Translators displayed longer first fixations when a word (for example, Spanish "grande") had been translated in more than one way ("big" and "large" in English). This has a parallel in monolingual studies, where all meanings of polysemous words can automatically be considered or accessed at an early stage (Onifer & Swinney, 1981). In the context of reading for translation, these findings suggest the study of words that are polysemous in the source language (for example, Spanish "dedo"), but where each meaning has a distinct translation in the target language ("finger" and "toe" in English). The horizontal model predicts automatic early activation of all translations of such words in the target language. On the other hand, the vertical model argues against early activation of target language words. One experiment, using eye tracking methodology, and one observational study, based on the extensive CRITT database of recordings of different translation tasks in several language pairs, were carried out to investigate the influence of dual language activation on reading for translation. To study the effects of cross-linguistic polysemy, participants sight translated sentences with ambiguous and non-ambiguous control words in neutral context. The observational analysis completed the experimental data by focusing on an English-Spanish subset of the CRITT database to assess how word translation entropy, a measure of variability of word translations in actual translation output, influenced the length of first fixation durations on a (open full item for complete abstract)

Committee: Isabel Lacruz (Advisor); Erik Angelone (Committee Member); Gregory Shreve (Committee Member); Ryan Miller (Committee Member); William Merriman (Committee Member) Subjects: Foreign Language; Modern Language; Psychology

PHD, Kent State University, 2017, College of Arts and Sciences / Department of Modern and Classical Language Studies

This dissertation describes the evolution of Cognitive Translation Studies (CTS) from the perspective of the Philosophy of Science and examines how to differentiate and choose when faced with two or more competing constructs in an interdisciplinary research enterprise. The work of Larry Laudan is applied to develop a set of criteria to assess conceptual performance. These criteria are applied in two brief case studies analyzing and comparing milestone constructs in the field: translation expertise and translation competence, and translation expertise and situated translation and interpreting expertise (STIE). This study contributes new perspectives to describe the evolution of Cognitive Translation Studies, stressing that scientific progress is interactive and plural, and proposing a shared, comparative conceptual performance model to help CTS scholars navigate that plurality.

Committee: Gregory M. Shreve (Advisor) Subjects: Behaviorial Sciences; Epistemology; Philosophy of Science

PHD, Kent State University, 2014, College of Arts and Sciences / Department of Modern and Classical Language Studies

Drawing on empirical research methods and design from cognitive psychology and translation studies, this dissertation focuses on cognitive effort during the translation process when translation memory is used. More specifically, two questions are addressed by means of an experimental study. The first question is whether the use of translation memory affects the cognitive effort of the translator during the process of translating segmented texts compared to translation without the use of a TM. The second research question addressed in this study is whether translators perceive translation memory proposals as useful to the translation task. Both of these questions are experimentally investigated in an attempt to illuminate the effects resulting from the use of translation memory. This study first provides an overview of translation technology, and outlines key concepts, such as translation memory, post-editing, working memory, and cognitive effort. These concepts are explored within the context of professional translation and the existing literature is reviewed. Next, a novel, Web-based data collection method is proposed to elicit translation process data from Spanish-to-English translators with four to seven years of professional experience. Following this description, the results are presented in light of the two overarching research questions. Moreover, the results are examined in light of Angelone's (2010) notion of triadic metacognitive bundles, consisting of problem recognition, solution proposal, and solution evaluation behaviors. The dissertation concludes by suggesting implications for translation pedagogy, research design, and translation tool design. Finally, the economics ramifications are highlighted, and potential avenues for future research are proposed.

Committee: Keiran Dunne (Advisor); Gregory Shreve (Committee Member); Erik Angelone (Committee Member); Jonathan Maletic (Committee Member); William Merriman (Committee Member) Subjects: Foreign Language; Language; Linguistics