In This Article Connectionism

  • Introduction
  • Introductory Works
  • Reference Works
  • Bibliographies
  • Software
  • Journals
  • History
  • Debates
  • Neural Basis
  • Major Properties
  • Connectionist Language Models
  • Representing Linguistic Features in Models
  • Models of Language Processing
  • Models of Second Language
  • Theoretical Linguistics Inspired by Connectionism

Linguistics Connectionism
Ping Li, Xiaowei Zhao
  • LAST REVIEWED: 10 May 2017
  • LAST MODIFIED: 29 October 2013
  • DOI: 10.1093/obo/9780199772810-0010


Connectionism, also known as parallel distributed processing (PDP) or artificial neural networks, has been an important theoretical framework as well as a computational tool for the study of mind and behavior. It adopts the perspective that human cognition is an emergent property that is due to the interaction of a large number of interconnected processing units (neurons) that operate simultaneously in a network (thus “parallel”). In addition, connectionism advocates that learning, representation, and processing of information are dynamic and distributed across the network. Language as a hallmark of human behavior has received in-depth treatment since the beginning of connectionist research. The acquisition of morphosyntax, the recognition of speech, and the processing of sentences are among the earliest connectionist models. The application of connectionism to second language acquisition has also gathered momentum in the late 20th and early 21st centuries. Learning a language entails complex cognitive and linguistic constraints and interactions, and connectionist models provide insights into how these constraints and interactions may be realized in the natural learning context.

Introductory Works

Many books that introduce the principles of connectionism have appeared since the mid-1980s. Introductions to fundamental algorithms of neural networks can be found in Haykin 1999. There have also been several contributions to readers in linguistics or psychology. The original parallel distributed processing (PDP) volume Rumelhart, et al. 1986 (cited under Reference Works) provides a comprehensive overview of the early models, many of which deal with language and cognition. Spitzer 1999 is an excellent text that provides a clear description of basic theories of connectionism and many of its applications in psychology, linguistics, and neuroscience. This book has very little technical detail and is written for the nonspecialist. Ellis and Humphreys 1999 contains more technical details and in-depth discussions along with selected readings of original articles that emphasize learning, language, and memory. Levine 2000 focuses on general organizing principles underlying neural and cognitive modeling, including competition, association, and categorization. For readers interested in learning how to implement a connectionist network step-by-step, there are a few books and tools at varying levels of difficulty that provide good exercises and examples of phenomena from linguistics, psychology, and neuroscience (see Software). O’Reilly and Munakata 2000 provides a comprehensive discussion of computational cognitive neuroscience that, unlike other texts mentioned in this section, focuses on modeling the neuronal system rather than the cognitive system at an abstract level. This book is written for more advanced readers, perhaps at the advanced graduate student level. There are also a few books that are more focused on applying connectionist approaches to specific topics or domains of study—for example, connectionism and development or connectionist language processing. These include Elman, et al. 1996, which highlights the connectionist framework of learning, interaction, and emergence and provides good illustrations of how connectionism tackles issues in cognitive and language development. Another book of this type is Shultz 2003, focusing more on stages of development and mechanisms of transition. Christiansen and Chater 2001 has useful reviews of connectionism in language studies.

  • Christiansen, Morten, and Nick Chater, eds. 2001. Connectionist psycholinguistics. Westport, CT: Ablex.

    E-mail Citation »

    A collection of papers by connectionist scholars that reviews (up to 2001) connectionist models of language, including spoken word recognition, morphosyntax, language production, reading and dyslexia, and language acquisition.

  • Ellis, Rob, and Glyn W. Humphreys. 1999. Connectionist psychology: A text with readings. New York: Psychology Press.

    E-mail Citation »

    A generally readable textbook with a focus on applying connectionist models to the study of cognitive phenomena, including memory, language, learning, and cognitive disorders.

  • Elman, Jeffrey, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, and Kim Plunkett. 1996. Rethinking innateness: A connectionist perspective on development. Cambridge, MA: MIT Press.

    E-mail Citation »

    This book has been called the “second bible” of connectionism; it focuses on cognitive and language development based on the connectionism framework. It argues for the need to clearly define innateness at different levels and to separate innateness from modularity, domain specificity, and localization.

  • Haykin, Simon. 1999. Neural networks: A comprehensive foundation. 2d ed. Upper Saddle River, NJ: Prentice Hall.

    E-mail Citation »

    A comprehensive textbook introducing many neural network algorithms, though not written specifically for readers in psychology or linguistics.

  • Levine, Daniel. 2000. Introduction to neural and cognitive modeling. 2d ed. Mahwah, NJ: Lawrence Erlbaum.

    E-mail Citation »

    This book focuses on general organizing principles underlying neural and cognitive modeling, including competition, association, and categorization. It contains many more technical and mathematical details than do other books discussed here.

  • O’Reilly, Randall C., and Yuko Munakata. 2000. Computational explorations in cognitive neuroscience: Understanding the mind by simulating the brain. Cambridge, MA: MIT Press.

    E-mail Citation »

    A comprehensive book covering cognitive and neural modeling with the aim of uniting computation, brain, and cognition for a field called computational cognitive neuroscience. The book is more suitable for readers at the graduate student level or higher. Readers who are interested in this text should explore the wiki site provided by the authors.

  • Shultz, Thomas. 2003. Computational developmental psychology. Cambridge, MA: MIT Press.

    E-mail Citation »

    An excellent discussion of using neural networks to study cognitive development, especially stages of development and mechanisms of transition. The book provides a good neural network primer along with mathematical basics of connectionist principles. It also discusses the cascade-correlation model that the author uses, a neural network that can dynamically recruit new, hidden units in response to task demands.

  • Spitzer, Manfred. 1999. The mind within the net: Models of learning, thinking, and acting. Cambridge, MA: MIT Press.

    E-mail Citation »

    An easy-to-read text that provides a discussion of neural networks and cognition in layperson’s terms.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login.

How to Subscribe

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here.