Linguistics Machine Translation
by
François Yvon
  • LAST MODIFIED: 13 January 2014
  • DOI: 10.1093/obo/9780199772810-0170

Introduction

Machine translation (MT) is an interdisciplinary scientific field that brings together linguists, lexicologists, computer scientists, and translation practitioners in the pursuit of a common goal: to design and develop electronic resources and computer software capable of automatically translating a document in a source language (SL) into an equivalent text in a target language (TL). By extension, machine translation technologies also include tools aimed at helping human translators to perform their work more efficiently using computer-assisted translation (CAT) technology. Machine Translation started in the late 1950s with attempts to automatically translate Russian into English. Realization of the extreme difficulty of the task led the MT community to concentrate its efforts on more focused and realistic problems, starting the field of natural language processing (NLP) studies. MT was thus broken down into three main sub-issues: analyzing the SL into a more abstract representation, transferring this representation into an equivalent target representation, and, finally, generating a proper surface realization in TL. Capitalizing on the progress in applied NLP and artificial intelligence, MT made slow progress over the next thirty years, using mostly symbolic models of language processing to accomplish the analysis, transfer, and generation processes. In spite of several remarkable achievements, these models were challenged in the 1980s by corpus-based methodologies, which rely on the analysis of large bodies of manually translated bitexts to generate translations of new documents. In particular, the statistical approaches in machine translation introduced in the early 1990s, and subsequently improved during the next decade, have rapidly gained momentum. Relying on the systematic exploitation of huge corpora of monolingual texts and multilingual bitexts available on the Internet, these approaches appear to be the most effective today for a wide variety of uses. Statistical approaches can handle almost any language pairs, provided a sufficient access to parallel corpora is available. Most studies, nonetheless, focus on machine translation into English.

Textbooks

Very few textbooks are available that deal solely with machine translation. Even though Jurafsky and Martin 2009 contains only a concise introduction to the issue, the volume overall provides an in-depth exposition of the entire conceptual background necessary to understand the vast literature on MT. Koehn 2010 focuses exclusively on statistical approaches, whereas Hutchins and Somers 1992 and Arnold, et al. 1994 are classical references documenting early rule-based approaches in MT.

  • Arnold, Douglas J., Lorna Balkan, Siety Meijer, R. Lee Humphreys, and Louisa Sadler. 1994. Machine translation: An introductory guide. Manchester, UK: Blackwells NCC.

    Save Citation »Export Citation »E-mail Citation »

    Similar in scope to Hutchins and Somers 1992 with a less technical perspective, this book makes a good choice for a more general audience. Also available online.

    Find this resource:

    • Hutchins, W. John, and Harold L. Somers. 1992. An introduction to machine translation. London: Academic Press.

      Save Citation »Export Citation »E-mail Citation »

      A basic course book covering all topics related to the design and development of MT systems, from the linguistic problems to the detailed analysis of some prototypical rule-based engines. Does not cover corpus-based methodology.

      Find this resource:

      • Jurafsky, Daniel, and James H. Martin. 2009. Speech and language processing: An introduction to natural language processing, speech recognition, and computational linguistics. 2d ed. Upper Saddle River, NJ: Pearson Prentice Hall.

        Save Citation »Export Citation »E-mail Citation »

        This general purpose NLP textbook cover a very large spectrum of topics. It notably includes (pp. 799–830) a rather broad, nontechnical introduction to MT and to translation problems from a computational linguistics perspective, with many valuable references.

        Find this resource:

        • Koehn, Philipp. 2010. Statistical machine translation. Cambridge, UK: Cambridge Univ. Press.

          Save Citation »Export Citation »E-mail Citation »

          The most comprehensive reference textbook on statistical machine translation, including many recent extensions of the statistical framework.

          Find this resource:

          Handbooks and Edited Collections

          Since understanding of machine translation systems requires a great deal of knowledge in computational linguistics theory and applications, general purpose natural language processing handbooks might well be the best introduction to the field. Several such collections have been published in recent years, all containing one or several chapters on machine translation. In Mitkov 2003, which is now a classic, MT is viewed from the perspective of experts of the rule-based paradigm, while Indurkhya and Damerau 2010 and Clark, et al. 2010 are more recent handbooks that contain several contributions on, respectively, statistical and example-based MT. Olive, et al. 2011 and Goutte, et al. 2009 are edited collections that are entirely focused on machine translation: The former offers an application-oriented perspective, while the latter is more technical and introduces recent advances in the corpus-based paradigm.

          • Clark, Alexander, Chris Fox, and Shalom Lappin. 2010. The handbook of computational linguistics and natural language processing. Chichester, UK: Wiley Blackwell.

            DOI: 10.1002/9781444324044Save Citation »Export Citation »E-mail Citation »

            This is yet another recent NLP handbook, with a very broad scope. Chapter 31 (pp. 531–573) on MT by A. Way provides a lucid review of the field, with an emphasis on data-based approaches that covers not only statistical MT, but also the less-documented example-based approaches, a field to which the author has greatly contributed.

            Find this resource:

            • Goutte, Cyril, Nicola Cancedda, Marc Dymetman, and George Foster, eds. 2009. Learning machine translation. Cambridge, MA: MIT Press.

              Save Citation »Export Citation »E-mail Citation »

              This collection of high-tech papers presents applications of modern machine learning (ML) techniques to the processing of multilingual text data. The lion’s share of the book is devoted to statistical machine translation, with several very innovative contributions; it also contains various applications in multilingual information mining and extraction.

              Find this resource:

              • Indurkhya, Nitin, and Fred J. Damerau, eds. 2010. Handbook of natural language processing. 2d ed. Boca Raton, FL: Chapman and Hall.

                Save Citation »Export Citation »E-mail Citation »

                Similar in scope to Mitkov 2003 but organized differently. Chapter 16 (pp. 367–408) by D. Wu, chapter 17 (pp. 409–424) by A. Ittycherriah, and chapter 18 (pp. 425–455) by P. Fung are devoted to MT issues. Taken together, they constitute a very good introduction to recent advances in (statistical) machine translation.

                Find this resource:

                • Mitkov, Ruslan, ed. 2003. The Oxford handbook of computational linguistics. Oxford: Oxford Univ. Press.

                  Save Citation »Export Citation »E-mail Citation »

                  This handbook contains two chapters devoted to machine translation: chapter 27 (pp. 501–511) by W. J. Hutchins and chapter 28 (pp. 511–527) by H. Summers. Taken together, they constitute a good introduction to the field, even if the more technical chapter by H. Summers is somewhat outdated.

                  Find this resource:

                  • Olive, Joseph, Caitlin Christianson, and John McCary, eds. 2011. Handbook of natural language processing and machine translation: DARPA global autonomous language exploitation. New York: Springer.

                    DOI: 10.1007/978-1-4419-7713-7Save Citation »Export Citation »E-mail Citation »

                    This collection of texts is the scientific legacy of the large DARPA GALE project. Two parts, consisting of several hundred pages, are devoted to the translation of texts and speech, respectively. They contain a good overview of statistical translation systems and many fine-grained implementation details of some of the best existing systems.

                    Find this resource:

                    Conferences and Workshops

                    The MT community is primarily represented by the International Association for Machine Translation (IAMT), which is organized in three regional associations: the Association for Machine Translation in the Americas (AMTA) for the Pan-American area, the European Association for Machine Translation (EAMT) for Europe, and the Asia-Pacific Association for Machine Translation (AAMT) for this region. Both AMTA and EAMT hold a yearly conference at which both scientific and user-oriented communications are presented; the biannual Machine Translation Summit (since 1987) is a joint conference of the three regional associations, and these conferences and associated workshops are an excellent source of references and publications. Another very selective group of conferences and workshops are organized yearly under the aegis of the Association for Computational Linguistics (ACL). ACL conferences usually include several sessions devoted to machine translation; more focused ACL workshops on Statistical Machine Translation (WMT) and on Syntax and Semantics in Statistical Machine Translation (SSMT) have been running since 2006 and 2007, respectively. Also noteworthy is the workshop on Building and Using Comparable Corpora (BUCC). Likewise, the biannual COLING conference, organized by the International Committee on Computational Linguistics usually includes several sessions dedicated to machine translation. With respect to translation from speech, the primary forum for communication is the annual InterSpeech organized by the International Speech Communication Association (ISCA). The International Workshop on Spoken Language Translation (IWSLT) publishes focused studies on translation from speech; it also hosts a yearly evaluation campaign. The Spoken Language Processing (SLP) workshop series also welcomes communications on speech translation. The most exhaustive source of electronic versions of conference proceedings in machine translation is the Machine Translation Archive, maintained by W. J. Hutchins (cited under Data Sources: Papers); the ACL Anthology (cited under Data Sources: Papers) is also an invaluable source for papers.

                    Data Sources

                    Many resources are now available on the Internet. Regarding scientific publications, two very large electronic archives are maintained, respectively, under the aegis of the Association for Computational Linguistics and the International Association for Machine Translation, both of which make most, if not all, the recent literature on machine translation freely accessible. Several open-source MT packages are also available that represent a large diversity of architectures from rule-based to various types of statistical engines. Internet users interested in implementing data-based systems will need parallel corpora: owing to the effort of institutional agencies or to altruistic individuals, several of them can be downloaded at very little, if any, cost.

                    Papers

                    The Machine Translation Archive and the ACL Anthology are the two main sources of electronic papers and references. A remarkable collaborative effort to develop a comprehensive bibliography of the most recent developments in statistical MT has recently been launched by P. Koehn at Edinburgh (SMT Research Survey Wiki). A somewhat outdated, but extremely exhaustive bibliography of MT evaluation in the 20th century has been compiled as part of the Framework for Evaluation of Machine Translation in ISLE.

                    Software

                    Several very efficient machine translation engines are available, together with all the necessary pre- and post-processing scripts. Regarding statistical machine translation, Moses is by far the most disseminated package. It is both very professionally documented, packaged, and maintained and supported by an active team of top-level researchers and developers. It includes effective implementations of most recent architectures, including statistical phrase-based and hierarchical models. Joshua, and cdec are other mature toolkits that have been used for large-scale tasks. OpenMaTrEx implements a variant of the example-based approach in machine translation; while the most popular open-source implementation of rule-based techniques is available in the Apertium toolkit, which supports an increasing number of language pairs. Efficient alignment tools are also available, both for aligning sentences (e.g., with hunalign) and for aligning words (e.g., with MGiza).

                    Data

                    The Linguistic Data Consortium distributes many annotated corpora and electronic resources: It includes, most notably, several large parallel corpora as well as many reference test data sets. Its European counterpart is the European Language Resources Association. Some parallel corpora can be downloaded for free: This is notably the case for the Europarl Parallel Corpus: Machine Translation and for several other institutional EC corpora prepared and distributed by the European Commission Joint Research Center; many other parallel resources have also been collected and prepared in the project OPUS: The Open Parallel Corpus.

                    Journals

                    With regard to conferences, the two reference journals are the very focused Machine Translation, which concentrates on all aspects of MT technologies, and the (nowadays) more eclectic Computational Linguistics, in which many reference papers on MT have been published over the years. Several other generalist journals in the fields of computational linguistics and speech processing occasionally publish papers, sometimes complete issues, on problems related to MT. They include Computer Speech and Language and Transactions on Speech and Language Processing. Natural Language Engineering is geared more toward application-oriented work; also worth mentioning is Transactions of the Association for Computational Linguistics, which promotes a rapid publication cycle through its links with the ACL series of conferences (see Conferences and Workshops).

                    History

                    Early works in the field of machine translation can be traced back to the 1950s at the onset of the development of computer systems. Early hopes were soon dashed as the true complexity of the task began to be better understood. The famous ALPAC report of 1966 appeared to conclude that the entire enterprise could not be undertaken. Subsequent works in the next decade attempted to better isolate the main sources of complexity and to resolve these problems using the principles of linguistic theory and/or the methods of artificial intelligence. In parallel, more practical developments were undertaken and resulted in several operational systems. Since the 1990s, corpus-based techniques have been developed, and statistical MT systems are nowadays the preferred choice for many uses in providing the best balance among speed, accuracy, robustness, and adaptability to new domains or (sub) languages. Concise retrospectives are provided in Slocum 1985 and Hutchins 2010; these works focus on rule-based approach, which stands in sharp contrast to the account in Jelinek 2009. Readers wishing to delve more deeply into this history should consult Hutchins 1986, which remains the reference source for expert approaches in MT, or the more personal Wilks 2009, which reviews a lifetime of work in machine translation. Finally, Nirenburg, et al. 2003 is an edited collection that contains many foundational papers.

                    • Hutchins, W. John. 1986. Machine translation: Past, present, future. Chichester, UK: Ellis Horwood.

                      Save Citation »Export Citation »E-mail Citation »

                      A full panoply of rule-based MT systems from the pre-ALPAC days up to the (by then) recent successes of artificial intelligence techniques.

                      Find this resource:

                      • Hutchins, W. John. 2010. Machine translation: A concise history. In Special issue: The teaching of computer-aided translation. Edited by Chan Sin Wai. Journal of Translation Studies 13.1–2: 29–70.

                        Save Citation »Export Citation »E-mail Citation »

                        This paper presents a recent historical account of the field by its most authoritative historian. The same author has published many historical surveys on the early days of MT, most of them available from his website online.

                        Find this resource:

                        • Jelinek, Frederick. 2009. ACL lifetime achievement award: The dawn of statistical ASR and MT. Computational Linguistics 35.4: 483–494.

                          DOI: 10.1162/coli.2009.35.4.35401Save Citation »Export Citation »E-mail Citation »

                          A historical account of the development of statistical machine translation by one of its most fervent promoters; recalls how research in machine translation was revolutionized by a fruitful analogy with the automatic speech recognition problem.

                          Find this resource:

                          • Nirenburg, Sergei, Harold Somers, and Yorick Wilks, eds. 2003. Readings in machine translation. Cambridge, MA: MIT Press.

                            Save Citation »Export Citation »E-mail Citation »

                            A collection of papers of great historical significance, containing many out-of-print or difficult to find papers from the early days of “mechanical translation.” Several foundational texts included in this volume are a must read for anyone interested in MT, from the much quoted memo from W. Weaver to the critical assessment of MT by Y. Bar Hillel as well as many other jewels.

                            Find this resource:

                            • Slocum, Jonathan. 1985. A survey of machine translation: Its history, current status, and future prospects. Computational Linguistics 11.1: 1–17.

                              Save Citation »Export Citation »E-mail Citation »

                              An alternative historical overview of the state of the art in MT just before the rise of corpus-based methods.

                              Find this resource:

                              • Wilks, Yorick. 2009. Machine translation: Its scope and limits. New York: Springer.

                                Save Citation »Export Citation »E-mail Citation »

                                This recent book summarizes the impressive works of its author, covering a very large time span (roughly from the early seventies until today). It includes several reprints of older papers as well as more recent contributions, all arguing for knowledge-rich approaches in Machine Translation.

                                Find this resource:

                                Corpora

                                The availability of large annotated corpora, coupled with the development of sophisticated machine learning techniques, has transformed the field of natural language processing. The same is true for machine translation with the rapid development and dissemination of empirical, corpus-based techniques. As far as MT is concerned, the most useful resources are parallel corpora, or bitexts, composed of a text in a source language aligned with a translation in a target language, such as the BAF presented in Simard 1998 (see Data section for some available bitexts). As documented in Véronis 2000, bitexts are used to feed translation memories, to extract bilingual dictionaries and terminologies, and to learn or to evaluate statistical MT systems. They can also provide translators with a rich source of bilingual concordances (Bourdaillet, et al. 2010). Such resources are thus actively sought, for instance, by mining websites (Resnik and Smith 2003). “Pure” bitexts are quite rare, and so weaker forms of parallelisms can be also useful: Comparable corpora, made of loosely “similar” texts in SL and TL, have been used to learn bilingual dictionary entries in Rapp 1995 and Fung and McKeown 1997.

                                • Bourdaillet, Julien, Stéphane Huet, Philippe Langlais, and Guy Lapalme. 2010. TransSearch: From a bilingual concordancer to a translation finder. Machine Translation 24.3–4: 241–271.

                                  DOI: 10.1007/s10590-011-9089-6Save Citation »Export Citation »E-mail Citation »

                                  This paper demonstrates another potential use of parallel corpora: bilingual concordancers, which are of great help to professional or amateur translators. Available online for purchase or by subscription.

                                  Find this resource:

                                  • Fung, Pascale, and Kathleen McKeown. 1997. A technical word- and term-translation aid using noisy parallel corpora across language groups. Machine Translation 12.1–2: 53–87.

                                    DOI: 10.1023/A:1007974605290Save Citation »Export Citation »E-mail Citation »

                                    Explores the continuum between fully parallel and fully comparable corpora: Mining approximately parallel corpus can help to learn useful bilingual associations. Available online for purchase or by subscription.

                                    Find this resource:

                                    • Rapp, Reinhart. 1995. Identifying word translations in non-parallel texts. In Proceedings of the conference: 33rd annual meeting of the Association for Computational Linguistics, 26–30 June 1995, Massachusetts Institute of Technology, Cambridge, Mass. Edited by Hans Uszkoreit, 320–322. Morristown, NJ: Association for Computational Linguistics.

                                      Save Citation »Export Citation »E-mail Citation »

                                      A seminal paper on learning bilingual dictionaries from nonparallel, comparable corpora.

                                      Find this resource:

                                      • Resnik, Philip, and Noah A. Smith. 2003. The web as a parallel corpus. Computational Linguistics 29:349–380.

                                        DOI: 10.1162/089120103322711578Save Citation »Export Citation »E-mail Citation »

                                        Realizing that existing bitexts are too small, the authors explore various methodologies to extract parallel texts by mining the web. As it turns out, many valuable resources can be collected, based on relatively simple heuristics.

                                        Find this resource:

                                        • Simard, Michel. 1998. The BAF: A corpus of English-French bitext. In First International Conference on Language Resources and Evaluation, Granada, Spain, 28–30 May 1998: Proceedings. Vol. 1. Edited by Antonio Rubio, 489–494. Paris: European Language Resources Association.

                                          Save Citation »Export Citation »E-mail Citation »

                                          This paper documents the development and release of a small bilingual parallel corpus, containing documents from diverse sources, from technical manual to novels and texts from international institutions.

                                          Find this resource:

                                          • Véronis, Jean, ed. 2000. Parallel text processing. Text, Speech and Language Technology. Dordrecht, The Netherlands: Kluwer Academic.

                                            DOI: 10.1007/978-94-017-2535-4Save Citation »Export Citation »E-mail Citation »

                                            This collection addresses a variety of issues, ranging from sentence alignment technologies and their evaluation to potential applications of parallel text processing such as the acquisition of bilingual lexicons or terminologies.

                                            Find this resource:

                                            Alignment

                                            Alignment aims at identifying correspondences between textual entities in several languages. As discussed in the recent surveys in Wu 2010 and Tiedemann 2011, alignment can be performed at several levels of granularity. Sentence alignment identifies matching source and target sentences in a bitext—such sentences are termed “parallel.” A sample of applications of sentence-aligned corpora are presented in Melamed 2001, the most significant being the automatic acquisition of bilingual word or term lexicons, the support of bilingual concordancers, and the training example-based machine translation systems. In the context of CAT, sentence-aligned textual fragments are stored in translation memories. More fine-grained alignments can be computed from parallel sentences, yielding translational correspondences at the level of individual words, at the level of word groups, or at the level of syntactic phrases. By extension, document-level alignments can also be sought in comparable collections of texts.

                                            • Melamed, I. Dan. 2001. Empirical methods for exploiting parallel texts. Cambridge, MA: MIT Press.

                                              Save Citation »Export Citation »E-mail Citation »

                                              Summarizes the numerous original and innovative contributions of the author, most notably the “geometrical mapping” approach in sentence alignment and some applications to word and compound alignments. A nonconventional and stimulating presentation of alignment problems.

                                              Find this resource:

                                              • Tiedemann, Jörg. 2011. Bitext alignment. Synthesis Lectures on Human Language Technologies 14. Edited by Graeme Hirst. San Rafael, CA: Morgan and Claypool.

                                                Save Citation »Export Citation »E-mail Citation »

                                                Building on a unique practical expertise in bitext acquisition and annotation, J. Tiedemann’s book gives an exhaustive review of alignment problems and techniques. In particular, the chapter on sentence alignment presents a unified description of existing algorithms.

                                                Find this resource:

                                                • Wu, Dekai. 2010. Alignment. In Handbook of natural language processing. 2d ed. Edited by Nitin Indurkhya and Fred Damerau, 367–408. Boca Raton, FL: Chapman and Hall.

                                                  Save Citation »Export Citation »E-mail Citation »

                                                  A thorough review of alignment techniques covering the entire spectrum of alignment problems and with an emphasis on structural alignments computed by some families of synchronous context-free grammars developed by the author.

                                                  Find this resource:

                                                  Sentence Alignments

                                                  Sentence alignments algorithms rely on simple assumptions to establish their mappings: (1) the sentence order tends to be the same in SL and in TL; (2) sentence alignments tend to be one-to-one; (3) short (resp. long) sentences tend to translate into short (resp. long) sentences; (4) parallel sentences tend to contain words that are mutual translations. Assumptions (1) and (2) warrant the use of effective dynamic programing techniques to compute the optimal alignment, using cues that are either derive from assumption (3), as in the pioneering work of Brown, et al. 1991 and Gale and Church 1993, or from assumption (4), as in Simard, et al. 1993 and Kay and Röscheisen 1993. Moore 2002 describes a more recent synthesis of these approaches.

                                                  • Brown, Peter F., Jenifer C. Lai, and Robert L. Mercer. 1991. Aligning sentences in parallel corpora. In Proceedings of the 29th annual meeting of the Association for Computational Linguistics, 18–21 June 1991, Univ. of California, Berkeley. Edited by Douglas E. Appelt, 169–176. Morristown, NJ: Association for Computational Linguistics.

                                                    Save Citation »Export Citation »E-mail Citation »

                                                    A first formulation of sentence alignment as a dynamic programming problem, relying solely on sentence length similarities.

                                                    Find this resource:

                                                    • Gale, William A., and Kenneth W. Church. 1993. A program for aligning sentences in bilingual corpora. Computational Linguistics 19.1: 75–102.

                                                      Save Citation »Export Citation »E-mail Citation »

                                                      Another, much more detailed, formulation of sentence alignment as a dynamic programming problem relying on sentence length similarities. Includes the source code of the alignment program, which continues to serve as a useful baseline.

                                                      Find this resource:

                                                      • Kay, Martin, and Martin Röscheisen. 1993. Text-translation alignment. Computational Linguistics 19.1: 121–142.

                                                        Save Citation »Export Citation »E-mail Citation »

                                                        Another approach to sentence alignment, which primarily exploits lexical distributional similarities in an iterative fashion.

                                                        Find this resource:

                                                        • Moore, Robert C. 2002. Fast and accurate sentence alignment of bilingual corpora. In Machine translation: From research to real users; 5th conference of the Association for Machine Translation in the Americas, AMTA 2002, Tiburon, CA, USA, 8–12 October 2002: Proceedings. Edited by Stephen D. Richardson, 135–144. Lecture Notes in Computer Science 2499. New York: Springer.

                                                          Save Citation »Export Citation »E-mail Citation »

                                                          A very effective synthesis of various ideas behind sentence alignment algorithms, implemented in a multi-pass, coarse-to-fine methodology. The de facto baseline when one-to-one sentence alignments are sought. The associated code is freely distributed.

                                                          Find this resource:

                                                          • Simard, Michel, George Foster, and Pierre Isabelle. 1993. Using cognates to align sentences in bilingual corpora. In Proceedings of the 1993 conference of the Centre for Advanced Studies on Collaborative Research, Toronto, Ontario, Canada, 24–28 October 1993. 2 vols. Edited by Ann Gawman, Evelyn Kidd, and Per -Åke Larson, 1071–1082. New York: IBM Press.

                                                            Save Citation »Export Citation »E-mail Citation »

                                                            Reveals the effectiveness of using cognate information when aligning sentences in “related” languages, or simply in languages using the same alphabet.

                                                            Find this resource:

                                                            Word Alignments

                                                            Identifying word alignments in parallel sentences is much harder than identifying sentence alignments. This is because the basic assumptions of sentence alignments no longer hold: Word order is rarely the same in SL and in TL; words rarely stand one-to-one correspondences across languages; long words do not necessarily translate into long words, etc. From a computational perspective, these facts make word alignment a NP-hard problem. The observation of co-occurrences in source and target sentences is the strongest sign of a translational equivalence (Melamed 2000), yet additional linguistic hypotheses and constraints must be introduced to make the optimal alignment problem tractable. Statistical alignment models that build asymmetrical (one-to-many) word alignments in an unsupervised fashion, introduced in Brown, et al. 1993 and in Vogel, et al. 1996, constitute the current state of the art, which is thoroughly analyzed in Och and Ney 2003. Evaluating alignments based on the number of correct alignment links reveals that the problem is far from solved, especially for language pairs having diverging morphological and/or syntactical structures. This warrants the exploration of more complex models as in Fraser and Marcu 2007, the introduction of supplementary constraints (Graça, et al. 2010), or the use of supervised learning strategies following the proposal in Taskar, et al. 2005.

                                                            • Brown, Peter F., Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics 19.2: 263–311.

                                                              Save Citation »Export Citation »E-mail Citation »

                                                              The first extensive description by their inventors of the famous IBM alignment models, which still represent the cornerstone of modern statistical machine translation engines.

                                                              Find this resource:

                                                              • Fraser, Alexander, and Daniel Marcu. 2007. Getting the structure right for word alignment: LEAF. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, 28–30 June 2007, Prague, Czech Republic. Edited by Jason Eisner, 51–60. Stroudsburg, PA: Association for Computational Linguistics.

                                                                Save Citation »Export Citation »E-mail Citation »

                                                                A more recent, linguistically sound yet computationally complex attempt to go beyond the IBM models. Still lacks an effective publicly available implementation.

                                                                Find this resource:

                                                                • Graça, Joaõ V., Kuzman Ganchev, and Ben Taskar. 2010. Learning tractable word alignment models with complex constraints. Computational Linguistics 36:481–504.

                                                                  DOI: 10.1162/coli_a_00007Save Citation »Export Citation »E-mail Citation »

                                                                  Another recent important contribution that explains how to integrate linguistic constraints in the “old” IBM models, yielding more realistic and more precise associations between source and target words.

                                                                  Find this resource:

                                                                  • Melamed, I. Dan. 2000. Models of translational equivalence among words. Computational Linguistics 26.2: 221–249.

                                                                    DOI: 10.1162/089120100561683Save Citation »Export Citation »E-mail Citation »

                                                                    A thorough and insightful discussion of word alignment problems; the author also presents the monolink algorithm, one of the most convincing alternatives to date to the computationally demanding IBM model family.

                                                                    Find this resource:

                                                                    • Och, Franz-Josef, and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics 29.1: 19–51.

                                                                      DOI: 10.1162/089120103321337421Save Citation »Export Citation »E-mail Citation »

                                                                      The most comprehensive comparison of word alignment models, including the IBM model family and beyond, as well as models based on association scores. The accompanying open-source software GIZA++ (see Software section) is the reference implementation of word alignment models.

                                                                      Find this resource:

                                                                      • Taskar, Ben, Simon Lacoste-Julien, and Dan Klein. 2005. A discriminative matching approach to word alignment. In Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing: Proceedings of the conference, 6–8 October 2005, Vancouver, British Columbia, Canada. Edited by Raymond J. Mooney, 73–80. Stroudsburg, PA: Association for Computational Linguistics.

                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                        This paper is the starting point of an alternative and successful line of approaches relying on supervision data to train the alignment. The resulting alignments are actually more precise, but the lack of annotated word alignments hinders the practical deployment of this methodology.

                                                                        Find this resource:

                                                                        • Vogel, Stephan, Hermann Ney, and Christof Tillmann. 1996. HMM-based word alignment in statistical translation. In the Proceedings of the 16th Conference on Computational Linguistics, Copenhagen, Denmark, 5–9 August 1996. Edited by Aravind Joshi and Martha Palmer, 836–841. Copenhagen: Center for Sprogteknologi.

                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                          A reformulation of the word alignment problem using the formalism of hidden Markov models (HMMs), thereby taking advantage of many well-known algorithms. HMMs remain the best tractable model for computing asymmetric word alignments.

                                                                          Find this resource:

                                                                          Sub-sentential Alignments

                                                                          Owing to simplifications and approximations, word alignments can yield computationally tractable problems. The resulting mappings are often unsatisfactory due to many-to-many associations, which are poorly captured, and to nonliteral translations. A more principled strategy should try to align word groups, even syntactic structures. The statistical and computational problems that need to be overcome to achieve such goals are, however, much greater, as shown in DeNero and Klein 2008, notwithstanding the linguistic problems of building such alignments, which are analyzed in Dorr 1994 and Wellington, et al. 2006. If most sub-sentential alignment models thus continue to build on preexisting word alignments, alternatives continue to be actively studied: Generalizations of word alignment models are studied in Marcu and Wong 2002 and Deng and Byrne 2008; the ITG formalism introduced in Wu 1997, which relies on latent tree alignments, provides a more sound, albeit computationally more demanding, alternative.

                                                                          • DeNero, John, and Dan Klein. 2008. The complexity of phrase alignment problems. In Proceedings of the 47th annual meeting of the Association for Computational Linguistics: ACL-08, HCT, short papers, June 2008, Columbus, OH. Edited by Johanna D. Moore, Simone Teufel, James Allan, and Sadaoki Furui, 25–28. New Brunswick, NJ: Association for Computational Linguistics.

                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                            A theoretical analysis of the difficulty of phrase alignment problems.

                                                                            Find this resource:

                                                                            • Deng, Yonggang, and William J. Byrne. 2008. HMM word and phrase alignment for statistical machine translation. IEEE Transactions on Audio, Speech and Language Processing 16.3: 494–507.

                                                                              DOI: 10.1109/TASL.2008.916056Save Citation »Export Citation »E-mail Citation »

                                                                              Generalizes the standard HMM alignment model to asymmetric word-to-phrase mappings while preserving the computational tractability of the model. Available online for purchase or by subscription.

                                                                              Find this resource:

                                                                              • Dorr, Bonnie J. 1994. Machine translation divergences: A formal description and proposed solution. Computational Linguistics 20.4: 597–633.

                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                Structural matching at the sentence level might turn out to be impossible: This paper reviews various types of divergences between languages and proposes a model to overcome them.

                                                                                Find this resource:

                                                                                • Marcu, Daniel, and Daniel Wong. 2002. A phrase-based, joint probability model for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 6–7 July 2002, Univ. of Pennsylvania, Philadelphia, PA, USA. Edited by Jan Hajič and Yuji Matsumoto, 133–139. New Brunswick, NJ: Association for Computational Linguistics.

                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                  An early attempt to compute alignments at the level of word groups (“phrases”). Its failure revealed the complexity of the task and paved the way to less naïve approaches.

                                                                                  Find this resource:

                                                                                  • Wellington, Benjamin, Sonjia Waxmonsky, and I. Dan Melamed. 2006. Empirical lower bounds on the complexity of translational equivalence. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, 17–21 July 2006, Sydney, Australia. Edited by Nicoletta Calzolari, Claire Cardie, and Pierre Isabelle, 977–984. Stroudsburg, PA: Association for Computational Linguistics.

                                                                                    Save Citation »Export Citation »E-mail Citation »

                                                                                    Alignments between syntactically meaningful sub-structures are not only difficult to compute, they are also sometimes impossible due to the intricacy of observed alignment patterns.

                                                                                    Find this resource:

                                                                                    • Wu, Dekai. 1997. Stochastic inversion transduction grammar and bilingual parsing of parallel corpora. Computational Linguistics 23.3: 377–404.

                                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                                      The reference publication on inversion transduction grammars, a formalism that enables modeling and computing hierarchical alignments between words and (possibly discontinuous) word groups.

                                                                                      Find this resource:

                                                                                      Evaluating Alignments

                                                                                      The quality of an automatic alignment is a measure of its resemblance with a manually produced reference, notwithstanding the possibility to reliably establish such references. Comparisons are based on metrics such as precision (the proportion of correctly predicted alignment links), recall (the proportion of existing links that are actually predicted), and/or combinations of these measures. The authoritative metric for sentence alignment is the F-measure (Véronis and Langlais 2000), whereas word alignments are usually evaluated (e.g. in Mihalcea and Pedersen 2003) using the alignment error rate (AER), a variant of F-measure that distinguishes sure and possible links. The usability of AER is discussed in Fraser and Marcu 2007.

                                                                                      • Fraser, Alexander, and Daniel Marcu. 2007. Measuring word alignment quality for statistical machine translation. Computational Linguistics 33.3: 293–303.

                                                                                        DOI: 10.1162/coli.2007.33.3.293Save Citation »Export Citation »E-mail Citation »

                                                                                        Questions the relevance of the AER metrics to evaluate alignments when those are computed only to train statistical machine translation engines.

                                                                                        Find this resource:

                                                                                        • Mihalcea, Rada, and Ted Pedersen. 2003. An evaluation exercise for word alignment. In Proceedings of the HLT-NAACL workshop on building and using parallel texts (PARALLEL ’03), Edmonton, Alberta, Canada, 31 May 2003. Edited by Marti Hearst and Mari Ostendorf, 1–10. Morristown, NJ: Association for Computational Linguistics.

                                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                                          An evaluation campaign explicitly targeting the word alignment task. It introduces the metrics and numerical comparisons for two language pairs. The hand-aligned sentences distributed for this campaign have been used in several subsequent evaluations.

                                                                                          Find this resource:

                                                                                          • Véronis, Jean, and Philippe Langlais. 2000. Evaluation of parallel text alignment systems. In Parallel text processing. Edited by Jean Véronis, 369–388. Text, Speech and Language Technology 13. Dordrecht, The Netherlands: Kluwer Academic.

                                                                                            DOI: 10.1007/978-94-017-2535-4_19Save Citation »Export Citation »E-mail Citation »

                                                                                            Reports one of the first systematic evaluations of sentence alignments algorithms; includes a discussion of the corresponding metrics.

                                                                                            Find this resource:

                                                                                            Translation Technologies

                                                                                            Approaches to machine translation are divided into two main traditions: rule-based (“rationalist”) approaches and data-based (“empiricist”) approaches; both traditions are well covered in the introductory survey of Isabelle and Foster 2006. The former expresses the transformation of a source text into a target text into an abstract form, which ultimately condenses the human knowledge of the process into a set of formal transfer rules. These rules can be “word-based” and operate on surface representations of the SL or they can apply on more abstract linguistic representations (such as syntactic trees or even semantic representations), thus necessitating a fair amount of linguistic pre-processing on the source side. Likewise, they can either directly output a valid target string or an abstract representation from which the string will then be generated. Vauquois 1968 is credited for visually synthetizing the various ways to design rule-based systems (the so-called Vauquois pyramid); a more recent and detailed account of rationalist approaches is in Dorr, et al. 1999. Data-based approaches, by contrast, automatically induce (concrete or abstract) transfer rules from a parallel corpus of exemplar translations. All other distinctions regarding the degree of abstraction in source and target apply, and several more do as well: The data-based family is itself subdivided between example-based machine translation, which tends to use “symbolic” representations, and statistical machine translation, which relies on numerical models. Hybridizations of these somewhat “ideal” architectures are numerous: Some even argue that hybrid systems are the future of machine translation; other claim that high-quality mechanical translation is impossible and that the value of translation technologies is in computer-assisted translation (Kay 1997). MT systems have been designed mostly to process textual input; speech-to-speech translation often requires significant adaptations.

                                                                                            • Dorr, Bonnie J., Pamela W. Jordan, and John W. Benoit. 1999. A survey of current paradigms in machine translation. Advances in Computers 49:1–68.

                                                                                              DOI: 10.1016/S0065-2458(08)60282-XSave Citation »Export Citation »E-mail Citation »

                                                                                              This survey contains a very complete presentation of the domain with a focus on the linguistic and computational architectures; in particular, the authors present a very detailed typology of rational approaches that goes well beyond the simple distinctions alluded to above.

                                                                                              Find this resource:

                                                                                              • Isabelle, Pierre, and George Foster. 2006. Machine translation: Overview. In Encyclopedia of language & linguistics. Vol. 7. 2d ed. Edited by E. K. Brown, 404–422. Boston: Elsevier.

                                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                                A clear and concise introduction to the field. The authors present in simple terms both the linguistic problems and the various ways they are addressed in MT systems, keeping a nice and rare balance between the rule-based and the corpus-based approaches.

                                                                                                Find this resource:

                                                                                                • Kay, Martin. 1997. The proper place of men and machines in language translation. Machine Translation 12.1–2: 3–23.

                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                  The reedition of a classic (1980) paper by one of the pioneers in the field, who convincingly argues against the possibility of fully mechanical translation and for the design of systems allowing a more fruitful cooperation between human and machines in translation. A foundational text for computer-assisted translation. Available online for purchase or by subscription.

                                                                                                  Find this resource:

                                                                                                  • Vauquois, Bernard. 1968. A survey of formal grammars and algorithms for recognition and transformation in machine translation. In Proceedings of the IFIP Congress 68, Edinburgh. Edited by Arthur J. H. Morell, 254–260. Laxenburg, Austria: International Federation for Information Processing

                                                                                                    Save Citation »Export Citation »E-mail Citation »

                                                                                                    The “Vauquois pyramid” organizes, in a unified graphical representation, the various computational architectures for MT: between “direct systems,” which resort mostly to transfer rule acting on surface representations, to “interlingua-based” systems, which dispense with transfer rules altogether but require sophisticated analysis strategies and representations; many intermediate strategies (“syntactic transfer,” “semantic transfer,” etc.) are also possible.

                                                                                                    Find this resource:

                                                                                                    Rule-Based Machine Translation

                                                                                                    Rule-based MT constituted the dominant paradigm in the early days of MT. The term is quite generic and encompasses a wide variety of approaches. The “direct transfer” methodology is primarily dictionary-based: its distinctive feature is that most, if not all, ambiguities that need to be solved to select the right dictionary entry can be solved using heuristic, language-dependent rules operating on surface representations. Capitalizing on decades of development of such extended dictionary and transfer rules, several commercial systems (e.g., see Senellart, et al. 2001) using the direct transfer model have attained quite good translation quality, at least in restricted domains. At the other end of this range, interlingua-based systems, surveyed in Dorr, et al. 2006, first perform a full semantic analysis of the SL, from which the TL is directly regenerated: No transfer takes place. The development of interlingual systems requires powerful analysis and generation tools for natural language input as well as formalisms for representing meaning and interacting with knowledge databases (Nirenburg, et al. 1992). Because such tools and formalisms still remain to be developed for general texts, a great number of approaches to cross-lingual transfer need to rely on intermediate, language-dependent representations (Isabelle and Bourbeau 1985).

                                                                                                    • Dorr, Bonnie J., Eduard Hovy, and Lori Levin. 2006. Machine translation: Interlingual methods. In Encyclopedia of language & linguistics. Vol. 7. 2d ed. Edited by E. K. Brown, 383–394. Boston: Elsevier.

                                                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                                                      A recent account of interlingual methods in MT: the authors not only outline the main motivations of such approaches and the difficulties they have faced it the past, but they also show that many of these problems remain and point out various ongoing projects on semantically annotated corpora that might help to revitalize this line of studies.

                                                                                                      Find this resource:

                                                                                                      • Isabelle, Pierre, and Laurent Bourbeau. 1985. TAUM-aviation: Its technical features and some experimental results. Computational Linguistics 11.1: 18–27.

                                                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                                                        A principled implementation of “deep-semantic” transfer rules for an ambitious application, the translation of a technical manual in the aviation domain. This was a follow-up of the TAUM-Meteo system, an emblematic early success of MT in a simpler sub-language.

                                                                                                        Find this resource:

                                                                                                        • Nirenburg, Sergei, Jaime Carbonel, Matsaru Tomita, and Kenneth Goodman, eds. 1992. Machine translation: A knowledge-based approach. San Mateo, CA: Morgan Kaufmann.

                                                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                                                          A defense and illustration of artificial intelligence methods in machine translation: in the author’s KBMT system, an abstract language independent representation of meaning serves as interlingua. The main problems with these approaches are the design of a proper representation of meaning and robust computation from natural language inputs.

                                                                                                          Find this resource:

                                                                                                          • Senellart, Jean, Péter Dienes, and Tamás Váradi. 2001. New generation Systran translation system. In MT Summit VIII: Machine translation in the information age: Proceedings, 18–22 September 2001, Santiago de Compostela, Spain. Edited by Bente Maegaard, 18–22. Geneva, Switzerland: European Association for Machine Translation.

                                                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                                                            A modern presentation of the one of the most successful commercial MT system to date; it describes an efficient and modular implementation of the “direct transfer” approach.

                                                                                                            Find this resource:

                                                                                                            Example-Based Machine Translation

                                                                                                            Example-based machine translation (EBMT), initially introduced in Nagao 1984, borrows ideas from the Artificial Intelligence tradition of “case-based reasoning” into machine translation. An EBMT system performs translation by analogy with a set of existing translations; in essence, it is a data-based methodology. Many implementations of this simple idea have been proposed (see Carl and Way 2003 or Lepage and Denoual 2005), which all share some basic principles that are analyzed in Somers 1999. Fragments of the database that most closely resemble fragments in the source sentence are first retrieved; their translations are then combined and adapted to make up for the observed difference between the source and the closest known cases. Several computational challenges must be thus be tackled (1) to define linguistically relevant and computationally tractable notions of flexible similarity between fragments of source sentences, (2) to identify sub-sentential correspondences between source and target fragments, (3) to design appropriate adaptation strategies for transforming existing translations into new ones. Flexible matching is the analogue of analysis in RBMT, it is also used in computer-assisted translation to retrieve segments from a translation memory. Sub-sentential alignment is the analogue of transfer in RBMT and it is also required in statistical approaches to machine translation. EBMT approaches have nowadays been surpassed by purely statistical techniques; nevertheless, their study provides many insights on how to realize hybrids systems (see Hybrid Machine Translation) combining aspects of rule-based systems and of “purely” statistical systems.

                                                                                                            • Carl, Michael, and Andy Way, eds. 2003. Recent advances in example-based machine translation. Text, Speech and Language Technology 21. New York: Springer.

                                                                                                              DOI: 10.1007/978-94-010-0181-6Save Citation »Export Citation »E-mail Citation »

                                                                                                              This collection of papers demonstrates the (by then) vitality of research in EBMT as well as its great diversity: while some approaches share with RBMT the use of sophisticated linguistic analyses of the source and complex transfer mechanism, others can be viewed as “symbolic” or “heuristic” variants of the statistical approaches to MT (discussed in Statistical Machine Translation).

                                                                                                              Find this resource:

                                                                                                              • Lepage, Yves, and Étienne Denoual. 2005. Purest ever example-based machine translation: Detailed presentation and assessment. Machine Translation 19.3–4: 251–282.

                                                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                                                An elegant implementation of the EBMT founding principles that relies only on the notion of formal analogies between strings. Available online for purchase or by subscription.

                                                                                                                Find this resource:

                                                                                                                • Nagao, Makoto. 1984. A framework of a mechanical translation between Japanese and English by analogy principle. In Artificial and human intelligence. Edited by Alick Elithorn and Ranan Banerji, 173–180. Amsterdam: North-Holland.

                                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                                  The paper that initiated this entire line of research; however, actual implementations of this idea were not published before the end of the decade.

                                                                                                                  Find this resource:

                                                                                                                  • Somers, Harold. 1999. Review article: Example-based machine translation. Machine Translation 14.2: 113–157.

                                                                                                                    DOI: 10.1023/A:1008109312730Save Citation »Export Citation »E-mail Citation »

                                                                                                                    A thorough review of the development of EBMT at the end of the 20th century. Few advances have been made since then. A slightly expanded version is reprinted in Carl and Way 2003. Available online for purchase or by subscription.

                                                                                                                    Find this resource:

                                                                                                                    Statistical Machine Translation

                                                                                                                    The defining principle of statistical MT is the use of a probabilistic model of the transfer component. This means that the transformation of a source into a target representation is modeled by some computational apparatus, the behavior of which is controlled by a set of numerical parameters. The values of these parameters are optimized so as to best reproduce the behavior of the said apparatus on a training corpus made of observed translations. A recent survey of SMT systems is given in Lopez 2008. Simple transfer models are based on rational string rewriting systems (finite-state transducers operating on words implementing a string-to-string translation), but many alternative transfer models have also been designed and explored (see Shieber 2007 or Graehl, et al. 2008), including string-to-tree, tree-to-string, dependency-to-string, etc. These can be computationally much more demanding. Depending on peculiarities of each particular transfer model, a fair amount of pre-processing (analysis) or post-processing (generation) might also be required.

                                                                                                                    • Graehl, Jonathan, Kevin Knight, and Jonathan May. 2008. Training tree transducers. Computational Linguistics 34.3: 391–427.

                                                                                                                      DOI: 10.1162/coli.2008.07-051-R2-03-57Save Citation »Export Citation »E-mail Citation »

                                                                                                                      A formal study of probabilistic tree transducers, which can be used to model tree-to-tree or tree-to-string translation models.

                                                                                                                      Find this resource:

                                                                                                                      • Lopez, Adam. 2008. Statistical machine translation. ACM Computing Surveys 40.3: 1–49.

                                                                                                                        DOI: 10.1145/1380584.1380586Save Citation »Export Citation »E-mail Citation »

                                                                                                                        A concise and precise review of the recent literature on statistical machine translation. A very useful entry point to the domain.

                                                                                                                        Find this resource:

                                                                                                                        • Shieber, Stuart M. 2007. Probabilistic synchronous tree-adjoining grammars for machine translation: The argument from bilingual dictionaries. In Proceedings of the NAACL-2007/AMTA workshop on syntax and structure in statistical translation, SSST’07, 26 April 2007, Rochester, NY. Edited by Dekai Wu and David Chiang, 88–95. Morristown, NJ: Association for Computational Linguistics.

                                                                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                                                                          Another batch of linguistic motivations for considering probabilistic translation models that would apply on structured representations such as syntactic trees. Includes a brief review of the available computational models and many good references to the formal grammar literature.

                                                                                                                          Find this resource:

                                                                                                                          Word-Based

                                                                                                                          Word-based SMT was introduced in the early 1990s in Brown, et al. 1990 and Brown, et al. 1993 as an attempt to transfer, to the MT domain, statistical techniques that had proved successful for automatic speech recognition (ASR). By analogy with ASR, MT is formulated as a statistical decision problem using the “noisy channel” model: in this metaphor, the TL is emitted by a probabilistic emitter, and then corrupted by a noisy communication channel into its SL counterpart. Based on the observation of the SL, translation amounts to reconstructing the original TL signal. Given suitable probabilistic models of the target and of the channel, recovering the TL is a simple matter of decoding. N-gram models are typically used to model the target language; the channel model (or more properly the “translation model”) is more complex and typically decomposes the association between SL and TL using latent alignment models. Computationally, these decompositions usually correspond to rational transductions between strings (Knight and Al-Onaizan 1998), but they can also imply syntactic constructs, as in Yamada and Knight 2001. Knight 1999 shows that even in the simpler model, the decoding problem is, however, NP-hard.

                                                                                                                          • Brown, Peter F., J. Cocke, Stephen A. Della Pietra, et al. 1990. A statistical approach to machine translation. Computational Linguistics 16.2: 79–85.

                                                                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                                                                            The original formulation of the noisy channel model in machine translation, including results of a pilot study for French to English translation. Many technical details of the model are missing and are better sought in Brown, et al. 1993.

                                                                                                                            Find this resource:

                                                                                                                            • Brown, Peter F., Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics 19.2: 263–311.

                                                                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                                                                              A much more detailed description of the IBM translation models, containing a discussion of the complexity of estimation procedures. The underlying alignments between words can model many-to-one mappings and some amount of movement between words in SL and TL.

                                                                                                                              Find this resource:

                                                                                                                              • Knight, Kevin. 1999. Decoding complexity in word-replacement translation models. Computational Linguistics 25.4: 607–615.

                                                                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                                                                Establishes the NP-complexity of decoding with word-based statistical translation models, even under very simple probabilistic models of the emitter. This means that the computation of the optimal translation for the noisy channel model is computationally intractable: The problem needs to be solved using heuristics.

                                                                                                                                Find this resource:

                                                                                                                                • Knight, Kevin, and Yaser Al-Onaizan. 1998. Translation with finite-state devices. In Machine translation and the information soup: Proceedings of the Third Conference of the Association for Machine Translation in the Americas (AMTA ’98), Langhorne, PA, USA, 28–31 October 1998. Edited by David Farwell, Laurie Gerber, and Eduard H. Hovy, 421–437. New York: Springer.

                                                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                                                  A clear reformulation of the noisy-channel model for word translation using the well-understood formalism of finite-state automata and transducers. A similar enterprise was later undertaken for phrase-based models by Kumar and Byrne.

                                                                                                                                  Find this resource:

                                                                                                                                  • Yamada, Kenji, and Kevin Knight. 2001. A syntax-based statistical translation model. In Proceedings of the 39th annual meeting of the Association for Computational Linguistics, July 9th–11th, 2001, Toulouse, France. Edited by Bonnie Lynn Webber, 523–530. San Francisco: Morgan Kaufmann.

                                                                                                                                    Save Citation »Export Citation »E-mail Citation »

                                                                                                                                    This work attempts to improve the alignments between source and target words using knowledge of the syntactic structure on the target side. This means that the translation model defines operations on the node of the target syntactic tree, thereby defining a tree-to-string translation model. Combines parsing tools from a more linguistically oriented tradition with the powerful concepts of statistical decision theory.

                                                                                                                                    Find this resource:

                                                                                                                                    Phrase-Based

                                                                                                                                    Phrase-based SMT constitute the state-of-the-art technology for most language pairs. Formulated originally in Zens, et al. 2002 and Och and Ney 2004, it generalizes Word-Based SMT by considering larger translation units made of variable-length sequences of adjacent words, loosely referred to in the SMT jargon as “phrase,” even though they do not necessarily have any linguistic interpretation. Translating phrases instead of isolated words has several advantages: larger translation units implicitly model local interdependencies in the target side, such as short-range agreement phenomena. It can also passively capture noncompositional translations (such as compound words, terms, and idioms) as well as some local reordering phenomena. Training phrase-based SMT engines, thus, requires accumulating variable-length associations between arbitrary source and target sequences (“bi-phrases”) numerically evaluating the strength of these associations. This information can be usually heuristically derived from phrase-based alignments, as proposed in Koehn, et al. 2003; whereas the implementation of Mariño, et al. 2006 relies on conventional n-gram models, generalized to handle bilingual parallel streams.

                                                                                                                                    • Koehn, Philipp, Franz Joseph Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In HLT-NAACL 2006: Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, 27 May–1 June 1993, Edmonton, Alberta, Canada. Edited by Eduard Hovy, 127–133. East Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                                                                                      The defining framework for phrase-based systems; introduces an original efficient algorithm to translate using a phrase-based system and a comparison of several phrase extraction procedures.

                                                                                                                                      Find this resource:

                                                                                                                                      • Mariño, José B., Raphael E. Banchs, Josep-Maria Crego, et al. 2006. N-gram-based machine translation. Computational Linguistics 32.4: 527–549.

                                                                                                                                        DOI: 10.1162/coli.2006.32.4.527Save Citation »Export Citation »E-mail Citation »

                                                                                                                                        The original presentation of the n-gram based approach, which notably relies on bilingual n-gram models to estimate the translation model. This enables the direct reuse of the large body of existing knowledge available for these models.

                                                                                                                                        Find this resource:

                                                                                                                                        • Och, Franz Joseph, and Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics 30.4: 417–449.

                                                                                                                                          DOI: 10.1162/0891201042544884Save Citation »Export Citation »E-mail Citation »

                                                                                                                                          An extensive presentation of the alignment template model by two of the inventors of the phrase-based approach.

                                                                                                                                          Find this resource:

                                                                                                                                          • Zens, Richard, Franz Joseph Och, and Hermann Ney. 2002. Phrase-based statistical machine translation. In KI-2002: Advances in artificial intelligence: 25th Annual German Conference on AI, KI 2002, Aachen, Germany, 16–20 September 2002: Proceedings. Vol. 2479. Edited by Matthias Jarke, Jana Koehler, and Gerhard Lakemeyer, 18–32. New York: Springer.

                                                                                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                                                                                            The first presentation of the phrase-based model.

                                                                                                                                            Find this resource:

                                                                                                                                            Hierarchical

                                                                                                                                            Hierarchical SMT, formulated in Chiang 2005, departs from the simpler phrase-based approach by resorting to a more powerful computational model known as syntax-directed transduction schemata, sometimes also referred to as synchronous context-free grammars, which was originally introduced in Lewis and Stearns 1968. In this formalism, translation units correspond to bilingual rewriting rules, which are combined recursively through a derivation process. A rule is a pair of “phrases,” optionally containing one (or two) matched gaps: where phrase-based units combine through concatenation, hierarchical units combine through “gap-filling,” meaning that gaps are recursively filled with other units until no gap remains. In this formalism, translation results form a bilingual parsing process (Melamed 2004), which greatly increases the computational complexity in comparison to phrase-based translation (see Satta and Peserico 2005): Implementation issues are discussed in Huang and Chiang 2007; Huang, et al. 2009; and Iglesias, et al. 2009. Hierarchical SMT is the best performing approach for language pairs presenting large differences in word orders, such as English and Japanese.

                                                                                                                                            • Chiang, David. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of the 43rd annual meeting of the Association for Computational Linguistics (ACL’05), 25–30 June 2005, Univ. of Michigan, Ann Arbor. Edited by Kevin Knight, Hwee Tou Ng, and Kemal Oflazer, 263–270. New Brunswick, NJ: Association for Computational Linguistics.

                                                                                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                                                                                              A complete implementation of the idea of “translation as parsing.” It introduces many practical solutions to problems related to synchronous grammar inference and scoring and demonstrates their effectiveness for large-scale translation tasks. An extended version appears in Computational Linguistics 33.2 (2007): 201–228.

                                                                                                                                              Find this resource:

                                                                                                                                              • Huang, Liang, and David Chiang. 2007. Forest rescoring: Faster decoding with integrated language models. In ACL 2007: Proceedings of the 45th annual meeting of the Association for Computational Linguistics, 23–30 June 2007, Prague, Czech Republic. Edited by Annie Zaenen and Antal van den Bosch, 144–151. Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                Algorithmic improvements to the approach in Chiang 2005 to make parsing with integrated language models more efficient.

                                                                                                                                                Find this resource:

                                                                                                                                                • Huang, Liang, Hao Zhang, Dan Gildea, and Kevin Knight. 2009. Binarization of synchronous context-free grammars. Computational Linguistics 35.4: 559–595.

                                                                                                                                                  DOI: 10.1162/coli.2009.35.4.35406Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                  A recent theoretical exploration of the formalism of synchronous context-free grammars for machine translation. Contains attempts to devise an effective algorithm for normalizing the context-free grammar so that all productions are binary

                                                                                                                                                  Find this resource:

                                                                                                                                                  • Iglesias, Gonzalo, Adrià de Gispert, Eduardo R. Banga, and William Byrne. 2009. Hierarchical phrase-based translation with weighted finite state transducers. In Proceedings of human language technologies: The 2009 Annual Conference of the North American chapter of the Association for Computational Linguistics, NAACL HLT 2009: 31 May–5 June 2005, Boulder, CO. Edited by Mari Ostendorf, Michael Collins, Shri Narayanan, Douglas W. Oard, and Lucy Vanderwende, 433–441. Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                    Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                    An elegant and effective implementation of the hierarchical model using finite-state transducers to compactly encode sets of alternative partial translation hypotheses.

                                                                                                                                                    Find this resource:

                                                                                                                                                    • Lewis, Philip M., and Richard E. Stearns. 1968. Syntax directed transduction. Journal of the ACM 15:465–488.

                                                                                                                                                      DOI: 10.1145/321466.321477Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                      An early theoretical analysis of the formalism of syntax-directed transductions, presented here in the context of compiler design. Available online for purchase or by subscription.

                                                                                                                                                      Find this resource:

                                                                                                                                                      • Melamed, I. Dan. 2004. Statistical machine translation by parsing. In Proceedings of the 42nd annual meeting of the Association for Computational Linguistics, 21–26 July 2004, Barcelona, Spain. Edited by Walter Daelemans and Marilyn Walker, 653–660. Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                        Algorithmic exploration of some grammatical formalisms for performing the joint analysis of parallel sentences or, more generally, of “multitexts.”

                                                                                                                                                        Find this resource:

                                                                                                                                                        • Satta, Giorgio, and Enoch Peserico. 2005. Some computational complexity results for synchronous context-free grammars. In Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing: Proceedings of the conference, 6–8 October 2005, Vancouver, British Columbia, Canada. Edited by Raymond J. Mooney, 803–810. East Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                          A theoretical presentation of synchronous context-free grammars, which explains why parsing multitexts is so much more complex than regular monolingual parsing.

                                                                                                                                                          Find this resource:

                                                                                                                                                          Syntax-Based

                                                                                                                                                          Hierarchical SMT remains essentially a string-to-string transduction model in which SL and TL are related through the hidden derivations of a synchronous context-free grammar. Syntax-based SMT aims at designing models in which the input and/or the output of the transfer model are actual syntactic representations in SL or TL and can then be understood as a statistical syntax-based transfer model. Various representations and associated models have been proposed: syntactic trees in Shieber and Schabes 1990 and Galley, et al. 2004 and dependency structures in Ding and Palmer 2005. Syntax-based MT implicitly assumes a tight correspondence between SL and TL syntax, an assumption relaxed in Eisner 2003. Implementing such approaches requires syntax in SL and/or TL; they are also computationally more demanding than string-to-string transfer models.

                                                                                                                                                          • Ding, Yuan, and Martha Palmer. 2005. Machine translation using probabilistic synchronous dependency insertion grammars. In 43rd Annual Meeting of the Association for Computational Linguistics: Proceedings of the conference, 25–30 June 2005, Univ. of Michigan, Ann Arbor. Edited by Kevin Knight, Hwee Tou Ng, and Kemal Oflazer, 541–548. Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                            A transfer model that operates on dependency-based, rather than constituent-based, representations of syntactic structures.

                                                                                                                                                            Find this resource:

                                                                                                                                                            • Eisner, Jason. 2003. Learning non-isomorphic tree mappings for machine translation. In 41st annual meeting of the Association for Computational Linguistics, 7–12 July 2003, Sapporo Convention Center, Sapporo, Japan. Vol. 2. Edited by Jun-ichi Tsujii, 205–208. East Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                              Syntax-based models often rely on the unwarranted hypothesis that syntax in SL and TL should be isomorphic; this paper explains how to make up for non-isomorphic tree correspondences.

                                                                                                                                                              Find this resource:

                                                                                                                                                              • Galley, Michael, Mark Hopkins, Kevin Knight, and Daniel Marcu. 2004. What’s in a translation rule? In Human Language Technology Conference of the North American chapter of the Association for Computational Linguistics, 2–7 May 2004, Boston, MA, USA. Edited by D. M. Susan Dumais and S. Roukos, 273–280. East Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                An intriguing analysis suggesting that, even for the simple case of the French-English pair, the transfer operation may require complex formal devices (and training algorithms) than what is typically used and, thereby, motivating the use of syntax-based models.

                                                                                                                                                                Find this resource:

                                                                                                                                                                • Shieber, Stuart M., and Yves Schabes. 1990. Synchronous tree-adjoining grammars. In Proceedings of the 13th Conference on Computational Linguistics, COLING ’90, Helsinki, 20–25 August 1990. Vol. 3. Edited by Hans Karlgren, 253–258. Helsinki: Yliopistopaino.

                                                                                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                  This generalization of tree-adjoining grammars to the bilingual case has been recently revisited in a SMT context; it defines a simple lexicalized transfer model between SL and TL trees.

                                                                                                                                                                  Find this resource:

                                                                                                                                                                  Hybrid Machine Translation

                                                                                                                                                                  The division between RBMT and data-based MT is somewhat of an idealization: Linguistic transfer rules can take advantage of statistical disambiguation modules of the SL; likewise, SMT engines often include linguistic expertise in their pre-processing modules. A shared feeling among many MT practitioners is that Ynvge’s assertion that high-quality MT would require a deep semantic analysis and interactions with knowledge databases is still true and that the most obvious way to achieve this goal is to use some form of human expertise that would improve the generalization ability of statistical learners and help them learn what can be learned from large bitexts. Practical hybridizations between RMBT and SMT are discussed in Dugast, et al. 2007 and Eisele, et al. 2008; Thurmair 2005 combines EBMT and SMT.

                                                                                                                                                                  • Dugast, Loïc, Jean Senellart, and Philipp Koehn. 2007. Statistical post-editing on Systran’s rule-based translation system. In ACL 2007: Proceedings of the second workshop on statistical machine translation, SMT ’07, Prague, Czech Republic. Edited by Chris Callison-Burch, Philipp Koehn, Cameron Shaw Fordyce, and Christof Monz, 220–223. Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                                    Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                    A simple form of hybrid system, where a statistical system is used to correct the output of the rule-based Systran engine, as if it were a language in its own right (Systranese ?). A practical attempt to combine RBMT adequacy and SMT fluency.

                                                                                                                                                                    Find this resource:

                                                                                                                                                                    • Eisele, Andreas, Christian Federmann, Hans Uszkoreit, et al. 2008. Hybrid machine translation architectures within and beyond the Euromatrix project. In European Association for Machine Translation: Proceedings of the Twelfth EAMT Conference, 22–23 September 2008, Univ. of Hamburg, Germany: Hybrid MT methods in practice: Their use in multilingual extraction, cross-language information retrieval, multilingual summarization, and applications in hand-held devices. Edited by John Hutchins and Walther V. Hahn, 27–34. Hamburg: HITEC.

                                                                                                                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                      Two proposals of hybrid architectures: Several RBMT engines first generate partial or complete hypotheses, which are then recomposed and rescored by a SMT system.

                                                                                                                                                                      Find this resource:

                                                                                                                                                                      • Thurmair, Gregor. 2005. Hybrid architectures for machine translation systems. Language Resources and Evaluation 39.1: 91–108.

                                                                                                                                                                        DOI: 10.1007/s10579-005-2698-zSave Citation »Export Citation »E-mail Citation »

                                                                                                                                                                        An quantitative comparison of RBMT and EBMT systems run in an industrial context: to get the better of both worlds, hybridation is what is needed. Available online for purchase or by subscription.

                                                                                                                                                                        Find this resource:

                                                                                                                                                                        Speech-to-Speech Translation

                                                                                                                                                                        The term machine translation is usually understood as machine translation for text input and outputs. Automatic translation may also be useful to facilitate spoken communications; hence, the need exists for speech-to-speech translation (S2ST) technologies. Their implementation combines several (imperfect) technological bricks (speech recognition, machine translation, and speech synthesis and even comprehension [Gu, et al. 2006]), and this requires robust natural language processing tools. These considerable challenges are minutely analyzed in two collections, Wahlster 2000 and, more recently, Olive, et al. 2011 (see Handbooks and Edited Collections). A simplification is to consider only finalized conversations in restricted domains, such as flight or hotel reservations (Rayner, et al. 2000). In any case, translation transfers only the linguistic content: Affects or attitudes that are conveyed by the prosody are generally disregarded. The most common architecture feeds the output of speech recognition into a MT system and uses speech synthesis to vocalize the proposed translation, thus raising the issue of the interaction between these modules. Alternative, more integrated, architectures are proposed in Vidal 1997, Ney 1999, and Matusov and Ney 2011. A well-studied scenario has been communication over the phone mediated by a S2ST system (Levin, et al. 2000), and it has drawn interest from many large telephone companies. Devices for facilitating cross-lingual face-to-face conversations have also been developed.

                                                                                                                                                                        • Gu, Liang, Yuqin Gao, Fu-Hua Liu, and Michael Picheny. 2006. Concept-based speech-to-speech translation using maximum entropy models for statistical natural concept generation. IEEE Transactions on Audio, Speech, and Language Processing 14.2: 377–392.

                                                                                                                                                                          DOI: 10.1109/TSA.2005.860769Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                          An original architecture that integrates ideas borrowed from the speech understanding literature within the statistical paradigm: Both the analysis of the speech input into some conceptual structure and the generation of the target language translation use statistical decision rules. Available online for purchase or by subscription.

                                                                                                                                                                          Find this resource:

                                                                                                                                                                          • Levin, Lori, Alon Lavie, Monika Woszczyna, et al. 2000. The Janus-III translation system: Speech-to-speech translation in multiple domains. Machine Translation 15.1–2: 3–25.

                                                                                                                                                                            DOI: 10.1023/A:1011186420821Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                            This paper summarizes attempts conducted in the course of the JANUS project(s) to design and deploy computer architectures for translating spontaneous conversations over the phone in several domains. The main challenge is to adapt knowledge-rich translation technologies to potentially noisy inputs; portability issues across language and domains are also documented. Available online for purchase or by subscription.

                                                                                                                                                                            Find this resource:

                                                                                                                                                                            • Matusov, Evgeny, and Hermann Ney. 2011. Lattice-based ASR-MT interface for speech translation. IEEE Transactions on Audio, Speech and Language Processing 19.4: 721–732.

                                                                                                                                                                              DOI: 10.1109/TASL.2010.2060483Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                              This paper explains how to use multiple hypotheses from the speech recognition engine to improve the performance of a statistical MT system. Available online for purchase or by subscription.

                                                                                                                                                                              Find this resource:

                                                                                                                                                                              • Ney, Hermann. 1999. Speech translation: Coupling of recognition and translation. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 99, Phoenix, 15–19 March 1999. Vol. 1, Speedy processing 1. Edited by W. Bastiaan Kleijn and Joe Picone, 517–520. Piscataway, NJ: Institute of Electrical and Electronics Engineers.

                                                                                                                                                                                Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                This paper reformulates the Bayes optimal decision rule for an integrated speech recognition / machine translation architecture.

                                                                                                                                                                                Find this resource:

                                                                                                                                                                                • Rayner, Manny, David Carter, Pierrette Bouillon, Vassilis Digalakis, and Mats Wirén, eds. 2000. The spoken language translator. Cambridge, UK: Cambridge Univ. Press.

                                                                                                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                  This book summarizes the main findings and realizations of another international collaborative project in speech translation in the air travel information domain, the Spoken Language Translator (1992–1999), which gathered several European and American teams. The MT engine is rule-based and robustness is achieved through the stacking of several transfer models operating over increasingly rich representations.

                                                                                                                                                                                  Find this resource:

                                                                                                                                                                                  • Vidal, Enrique. 1997. Finite-state speech-to-speech translation. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 21–24 April 1997, Munich, Germany. Vol. 1. Edited by Heinrich Niemann and Josef A. Nossek, 111–114. Piscataway, NJ: Institute of Electrical and Electronics Engineers.

                                                                                                                                                                                    DOI: 10.1109/ICASSP.1997.599563Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                    A conceptually clean integration of speech recognition and machine translation using the formalism of finite-state transducers. This small-scale system has inspired many subsequent attempts to combine these two modules. Available online for purchase or by subscription.

                                                                                                                                                                                    Find this resource:

                                                                                                                                                                                    • Wahlster, Wolfgang, ed. 2000. Verbmobil: Foundations of speech-to-speech translation. Artificial Intelligence. New York: Springer.

                                                                                                                                                                                      DOI: 10.1007/978-3-662-04230-4Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                      An edited collection summarizing the advances in S2ST achieved in the course of the German Verbmobil project, an eight-year effort involving about twenty teams and several hundred researchers. Contributions concern all aspects of S2ST systems. Several MT architectures were notably compared, including the first attempts at using SMT for speech input by H. Ney’s group. This project had an enormous impact in the development of natural language processing and MT technologies.

                                                                                                                                                                                      Find this resource:

                                                                                                                                                                                      Evaluation

                                                                                                                                                                                      In conjunction with most technological systems, machine translation systems need to be evaluated. Some aspects of MT systems, such as the processing time or the ease of deployment and usage, can use standard evaluation protocols for computer systems or their specialized versions developed for natural language processing systems, which are studied in Spärck Jones and Galliers 1995. As documented in King 1994 or, more recently, in Dorr, et al. 2011, evaluating the quality of a MT system turns out to be a vexing problem because many equally good translations of a given text exist that can be truly compared only with respect to their intended usage and readership. Three main evaluation protocols are considered in the literature: Human (subjective) evaluations are deemed to be more precise, but they are very costly; the more recent automatic evaluations are very crude, but they are produced very quickly and at almost no cost; task-based evaluation indirectly evaluates quality by considering how MT improves the performance of finalized tasks, such as comprehension tests. Public, international evaluation campaigns for written and spoken translation systems are organized yearly, for instance, under the aegis of the US National Institute for Standards and Technology (NIST). A related issue, surveyed in Blatz, et al. 2004, is the automatic prediction of MT quality and the computation of confidence measures.

                                                                                                                                                                                      • Blatz, John, Erin Fitzgerald, George Foster, et al. 2004. Confidence estimation for machine translation. In Proceedings COLING Geneva 2004: 20th International Conference on Computational Linguistics: August 23rd to 27th. Edited by Sergei Nirenburg, 315–321. East Stroudsburg, PA: Association for Computational Linguistics.

                                                                                                                                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                        A collective study aimed at enhancing MT systems with automatic measures of self-assessment. Confidence estimation is an important issue for industrial translation workflows involving post-edition of machine translated texts.

                                                                                                                                                                                        Find this resource:

                                                                                                                                                                                        • Dorr, Bonnie J., Joseph Olive, Caitlin Christianson, and John McCary, eds. 2011. Machine translation evaluation and optimization. In Handbook of natural language processing and machine translation: DARPA global autonomous language exploitation. Edited by Joseph Olive, Caitlin Christianson, and John McCary, 745–843. New York: Springer.

                                                                                                                                                                                          DOI: 10.1007/978-1-4419-7713-7Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                          An up-to-date survey of evaluation for machine translation realized in the context of the GALE/DARPA project, in which semi-automatic metrics, including human-in-the-loop evaluation process, were developed, evaluated, and promoted. Available online for purchase.

                                                                                                                                                                                          Find this resource:

                                                                                                                                                                                          • King, Margaret. 1994. Evaluation of MT software and methods. Aslib Proceedings 46.7–8: 179–183.

                                                                                                                                                                                            DOI: 10.1108/eb051363Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                            A high-level overview of MT evaluation in which the author tries to move away from the great variety of possible evaluation scenarios and put forward general principles for evaluation methodologies.

                                                                                                                                                                                            Find this resource:

                                                                                                                                                                                            • Spärck Jones, Karen, and Julia R. Galliers. 1995. Evaluating natural language processing systems. LNAI State of the Art Survey. New York: Springer.

                                                                                                                                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                              A general framework for evaluating language processing tools that clarifies many important concepts in evaluation design. Includes several discussions related to the evaluations of machine translation systems and their components.

                                                                                                                                                                                              Find this resource:

                                                                                                                                                                                              Human Evaluations

                                                                                                                                                                                              Subjective evaluations typically consist in asking one or more human annotators to judge a translation using a discrete evaluation scale with respect to various subjective criteria. As discussed in van Slype 1979 and Hovy, et al. 2002, two main families of evaluation criteria are usually considered: The first one relates to the adequate transfer of the information content of the source text into the target language asking questions regarding the adequacy, fidelity, or informativity of the translation; the second one independently assesses the correctness of target texts as measured by their intelligibility, acceptability, or fluency. Bilingual judges are required for adequacy assessment as it quantifies the degree of semantic equivalence between the source and target texts. Fluency judgments, on the other hand, look only at the target text and can be performed by monolingual judges; thus, they are easier to collect. Fluency and adequacy are not entirely independent: it may be difficult to judge the adequacy of highly disfluent texts. An alternative evaluation protocol, discussed in White, et al. 1994, is to use comprehension tests in which human subjects answer questions on the original text based on their comprehension of the MT output. King and Falkedal 1990 takes another stance and proposes to design and exploit representative test suites.

                                                                                                                                                                                              • Hovy, Eduard, Margaret King, and Andrei Popescu-Belis. 2002. Principles of context-based machine translation evaluation. Machine Translation 17.1: 43–75.

                                                                                                                                                                                                DOI: 10.1023/A:1025510524115Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                                This is a recent high-level survey in which a unifying view of attempts to formalize and standardize MT human evaluation protocols and metrics is developed. The historical account of MT evaluation is richly documented; the paper also briefly summarizes early discussions raised by the development of fully automatic metrics.

                                                                                                                                                                                                Find this resource:

                                                                                                                                                                                                • King, Margaret, and Kirsten Falkedal. 1990. Using test suites in evaluation of machine translation systems. In Proceedings of the 13th Conference on Computational Linguistics, COLING ’90, Helsinki, Finland, 20–25 August 1990. Vol. 2. Edited by Hans Karlgren, 211–216. Helsinki: Yliopistopaino.

                                                                                                                                                                                                  Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                                  An alternative way to evaluate machine translation systems through the design and use of a test suite of well-chosen examples that should be representative of a large range of difficulties. Test suites can provide valuable insights as to the types of improvements that are needed. Yet, they are hard to design and do not always reflect well on the practical quality of a system.

                                                                                                                                                                                                  Find this resource:

                                                                                                                                                                                                  • van Slype, Georges. 1979. Critical study of methods for evaluating the quality of machine translation. In Commission of the European Communities Directorate General, Scientific and Technical Information and Information Management. Report BR-19142. Brussels: Bureau Marcel van Dijk.

                                                                                                                                                                                                    Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                                    A historical reference reporting attempt at evaluating MT in a real-world application (the deployment of MT systems to be used by services of the European Commission); provides a thorough discussion of various subjective evaluation protocols.

                                                                                                                                                                                                    Find this resource:

                                                                                                                                                                                                    • White, John, Theresa O’Connell, and Francis O’Mara. 1994. The ARPA MT evaluation methodologies: Evolution, lessons, and future approaches. In Proceedings of the First Conference of the Association for Machine Translation in the Americas, 5–8 October 1994, Columbia, Maryland, USA. Edited by Eduard Hovy and Joseph Pentheroudakis, 193–205. Washington, DC: Association for Machine Translation in the Americas.

                                                                                                                                                                                                      Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                                      Some lessons of the first “modern,” large-scale evaluation of MT translation based on subjective evaluation protocols.

                                                                                                                                                                                                      Find this resource:

                                                                                                                                                                                                      Automatic Metrics

                                                                                                                                                                                                      Automatic metrics, such as BLEU (Papineni, et al. 2002), METEOR (Banerjee and Lavie 2005), or TER (Snover, et al. 2006) have been proposed as a useful proxy to expensive human evaluations: They are typically based on surface comparison between an automatic translation and one or several reference translation(s) proposed by individuals. This means that once these human references have been produced, they can be used to evaluate and compare any number of MT systems, a strategy adopted in many recent evaluation campaigns. Automatic metrics should deliver judgments that resemble human evaluations: Their usefulness depends on their ability to produce scores that statistically correlate well with human notes. If automatic metrics are routinely used in standard benchmark tests, they are widely criticized for their lack of precision, especially when it comes to evaluating isolated sentences, as discussed in Callison-Burch, et al. 2010. Their improvement remains an active research area.

                                                                                                                                                                                                      • Banerjee, Satanjeev, and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL05 workshop on intrinsic and extrinsic evaluation measures for machine translation and/ or summarization, 25–30 June 2005, Univ. of Michigan, Ann Arbor. Edited by Jade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare Voss, 65–72. New Brunswick, NJ: Association for Computational Linguistics.

                                                                                                                                                                                                        Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                                        An alternative to BLEU based on flexible matching routines for comparing a MT hypothesis and a human reference. METEOR has been consistently shown to correlate better than BLEU with human judgments.

                                                                                                                                                                                                        Find this resource:

                                                                                                                                                                                                        • Callison-Burch, Chris, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, and Omar Zaidan. 2010. Findings of the 2010 joint workshop on statistical machine translation and metrics for machine translation. In ACL 2010: Proceedings of the joint fifth workshop on statistical machine translation and Metrics Matr, 15–16 July 2010, Uppsala, Sweden. Edited by Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, and Omar Zaidan, 17–53. New Brunswick, NJ: Association for Computational Linguistics.

                                                                                                                                                                                                          Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                                          Provides an overview of many alternative automatic metrics; it also explains why and how evaluation metrics themselves need to be evaluated.

                                                                                                                                                                                                          Find this resource:

                                                                                                                                                                                                          • Papineni, Kishore, Salim Roukos, Tony Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 7–12 July 2002, Philadelphia, PA, USA. Edited by Pierre Isabelle, 311–318. New Brunswick, NJ: Association for Computational Linguistics.

                                                                                                                                                                                                            Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                                            The first successful attempt to promote automatic metrics in MT launched by the IBM statistical MT group. BLEU relies on simple comparison and counting procedures: It is very simple to compute but expectedly delivers crude scores; it remains the most widely used metrics.

                                                                                                                                                                                                            Find this resource:

                                                                                                                                                                                                            • Snover, Matthew G., Bonnie J. Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, AMTA: 2006, 8–12 August 2006, Cambridge, MA, USA. Edited by Nizar Habash and Alon Lavie, 223–231. Stroudsburg, PA: Association for Machine translation in the America.

                                                                                                                                                                                                              Save Citation »Export Citation »E-mail Citation »

                                                                                                                                                                                                              The translation edit rate (TER) implements another algorithm to compare translation and reference sentences. It is especially relevant with respect to references that are chosen to be as close as possible to MT-generated outputs.

                                                                                                                                                                                                              Find this resource:

                                                                                                                                                                                                              back to top

                                                                                                                                                                                                              Article

                                                                                                                                                                                                              Up

                                                                                                                                                                                                              Down