Computational Linguistics

by

Robert Dale

• LAST REVIEWED: 28 October 2011
• LAST MODIFIED: 28 October 2011
• DOI: 10.1093/obo/9780199772810-0046

Introduction

Computational linguistics (CL) is an interdisciplinary mix of computer science and linguistics with additional insights drawn from areas such as psycholinguistics and the philosophy of language. Its primary concern is with the computational modeling of linguistic processes pursued as a theoretically oriented exercise whose purpose is either to provide models that help us gain insight into the nature of language and the human language processing mechanism or to support the development of software applications that do useful things with language (an area sometimes referred to as language technology). For some, the term “computational linguistics” is synonymous with natural language processing (NLP); however, from the perspective of the material provided here, NLP is more applications-oriented than CL. Although we provide some pointers to work that describes NLP applications, our primary focus here is on the theoretical underpinnings that CL provides to activities with a more practical focus. The first work in CL dates from the 1950s, when initial attempts were made to automatically translate Russian into English. Until the late 1980s most work in the field was concerned with what we might call symbolic systems, often involving large collections of handwritten rules to model some linguistic phenomenon. Since the late 1980s there has been a significant shift toward statistical methods, where rules and generalizations are learned from data rather than being produced manually; this has become possible only as a result of the combination of, on one hand, vast amounts of data becoming available, particularly via the World Wide Web, and, on the other hand, the immense increases in computer processing power required to execute many iterations over large data sets to derive information from them. In almost all subareas of CL one can generally divide the work pursued into the periods before and after this “statistical revolution.” The material here is organized around the conventional decomposition of linguistic study into phonology, morphology, syntax, semantics, and pragmatics. Work that is primarily concerned with speech recognition and synthesis is not covered. Primarily concern is with work on the processing of English; again, much of the work in CL is applicable to other languages, but English holds a privileged position as the focus of most research.

Textbooks

In the early 21st century the most widely used general textbook in CL is Jurafsky and Martin 2009; this provides a very comprehensive picture of the field and is the best candidate for a must-have book in the area. A more in-depth introduction to statistical language processing techniques in particular is Manning and Schütze 2005; this is an excellent resource for those who want to understand statistical approaches in some detail. The first textbooks in the area were published in the 1980s, perhaps signaling that the field had reached a certain level of maturity by that time. A perusal of these earlier textbooks provides interesting historical insights into how the field has changed over the years. Worth a look are Winograd 1982, Grishman 1986, Gazdar and Mellish 1989, and Allen 1995. All of these were written prior to the major shift in the field toward statistical processing, but much of what they contain still remains valid with respect to work in the symbolic tradition. The earlier books can often provide illuminating alternative explanations for concepts given less space in later texts.

• Allen, James. 1995. Natural language understanding. 2d ed. Redwood City, CA: Benjamin/Cummings.

  This was for a time the key textbook in the symbolic tradition, although it has been superseded by Jurafsky and Martin 2009 because of the latter’s inclusion of material on statistical processing. Allen’s was the first textbook to attempt detailed coverage of issues in all three areas of syntax, semantics, and pragmatics.

• Gazdar, Gerald, and Christopher Mellish. 1989. Natural language processing in Lisp: An introduction to computational linguistics. Reading, MA: Addison-Wesley.

  One of three books by the authors—the others being Natural Language Processing in Prolog (1989) and Natural Language Processing in POP-11 (1989)—that provide a very hands-on approach to computational linguistics via the programming languages in the titles. Although they contain some material on semantics and pragmatics, their main focus is syntactic processing.

• Grishman, Ralph. 1986. Computational linguistics: An introduction. Cambridge, UK: Cambridge Univ. Press.

  A slim volume of less than two hundred pages, this provides a much briefer introduction to the area than other texts; as such it can serve as a relatively concise primer, although it does not cover any material subsequent to the “statistical revolution.”

• Jurafsky, Daniel, and James H. Martin. 2009. Speech and language processing: An introduction to natural language processing, speech recognition, and computational linguistics. 2d ed. Upper Saddle River, NJ: Pearson Prentice Hall.

  This book is the most comprehensive and up-to-date textbook in the area, with substantial material on all three areas of syntax, semantics, and pragmatics as well as chapters on applications of natural language processing. At almost a thousand pages, the text serves as a thorough and self-contained introduction to the field.

• Manning, Christopher D., and Hinrich Schütze. 2005. Foundations of statistical natural language processing. Cambridge, MA: MIT Press.

  A detailed step-by-step exposition of the techniques and concepts required to understand how statistical techniques can be used in the processing of language. There have been many developments since this book was written, but it still serves as an excellent guide to the area.

• Winograd, Terry. 1982. Language as a cognitive process. Vol. 1, Syntax. Reading, MA: Addison Wesley.

  As suggested by the title, this book focuses on syntax, with subsequent volumes (which ultimately were never written) intended to cover semantics and pragmatics. The book provides an alternative take on the material covered in Gazdar and Mellish 1989 at a similar level of technical detail.