In This Article Expand or collapse the "in this article" section Connectionism

  • Introduction
  • Introductory Works
  • Reference Works
  • Bibliographies
  • Software
  • Journals
  • History
  • Debates
  • Neural Basis
  • Major Properties
  • Connectionist Language Models
  • Representing Linguistic Features in Models
  • Models of Language Processing
  • Models of Second Language
  • Theoretical Linguistics Inspired by Connectionism

Linguistics Connectionism
by
Ping Li, Xiaowei Zhao
  • LAST REVIEWED: 10 May 2017
  • LAST MODIFIED: 29 July 2020
  • DOI: 10.1093/obo/9780199772810-0010

Introduction

Connectionism, also known as parallel distributed processing (PDP) or artificial neural networks, and most recently reengineered as Deep Learning, has been an important theoretical framework as well as a computational tool for the study of mind and behavior. It adopts the perspective that human cognition is an emergent property that is due to the interaction of a large number of interconnected processing units (neurons) that operate simultaneously in a network (thus “parallel”). In addition, connectionism advocates that learning, representation, and processing of information are dynamic and distributed. Language as a hallmark of human behavior has received in-depth treatment since the beginning of connectionist research. The acquisition of morphosyntax, the recognition of speech, and the processing of sentences are among the studies of the earliest connectionist models. The application of connectionism to second language acquisition has also gathered momentum in the late 20th and early 21st centuries. Learning a language entails complex cognitive and linguistic constraints and interactions, and connectionist models provide insights into how these constraints and interactions may be realized in the natural learning context.

Introductory Works

Many books that introduce the principles of connectionism have appeared since the mid-1980s. Introductions to fundamental algorithms of neural networks can be found in Haykin 2009. Goodfellow, et al. 2016 provides comprehensive coverage of many cutting-edge deep learning neural network models, along with a thorough discussion of practical issues related to them. Several contributions also come to readers in linguistics or psychology. The original parallel distributed processing (PDP) volume Rumelhart, et al. 1986 (cited under Reference Works) provides a comprehensive overview of the early models, many of which deal with language and cognition. Spitzer 1999 is an excellent text that provides a clear description of the basic theories of connectionism and many of its applications in psychology, linguistics, and neuroscience. This book has very little technical detail and is written for nonspecialists. Ellis and Humphreys 1999 contains more technical details and in-depth discussions along with selected readings of original articles that emphasize learning, language, and memory. Levine 2000 focuses on general organizing principles underlying neural and cognitive modeling, including competition, association, and categorization. For readers interested in learning how to implement a connectionist network step-by-step, a few books and tools at varying levels of difficulty provide good exercises and examples of phenomena from linguistics, psychology, and neuroscience (see Software). O’Reilly and Munakata 2000 provides a comprehensive discussion of computational cognitive neuroscience that, unlike other texts mentioned in this section, focuses on modeling the neuronal system rather than the cognitive system at an abstract level. This book is written for more advanced readers, perhaps at the advanced graduate student level. In addition, a few books are more focused on applying connectionist approaches to specific topics or domains of study—for example, connectionism and development or connectionist language processing. These include Elman, et al. 1996, which highlights the connectionist framework of learning, interaction, and emergence and provides good illustrations of how connectionism tackles issues in cognitive and language development. Another book of this type is Shultz 2003, focusing more on stages of development and mechanisms of transition. Christiansen and Chater 2001 contains useful reviews of connectionism in language studies. Shirai 2018 provides a recent synthesis of connectionism and second language acquisition, serving as an excellent overview of existing connectionist bilingual models from the past twenty-five years.

  • Christiansen, Morten, and Nick Chater, eds. 2001. Connectionist psycholinguistics. Westport, CT: Ablex.

    E-mail Citation »

    A collection of papers by connectionist scholars that reviews (up to 2001) connectionist models of language, including spoken word recognition, morphosyntax, language production, reading and dyslexia, and language acquisition.

  • Ellis, Rob, and Glyn W. Humphreys. 1999. Connectionist psychology: A text with readings. New York: Psychology Press.

    E-mail Citation »

    A generally readable textbook with a focus on applying connectionist models to the study of cognitive phenomena, including memory, language, learning, and cognitive disorders.

  • Elman, Jeffrey L., Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, and Kim Plunkett. 1996. Rethinking innateness: A connectionist perspective on development. Cambridge, MA: MIT Press.

    E-mail Citation »

    This book has been called the “second bible” of connectionism; it focuses on cognitive and language development based on the connectionism framework. It argues for the need to clearly define innateness at different levels and to separate innateness from modularity, domain specificity, and localization.

  • Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep learning. Cambridge, MA: MIT Press.

    E-mail Citation »

    A comprehensive textbook introducing a broad range of deep learning algorithms. Many practical issues of deep learning are discussed in this book. This book is available via open access online.

  • Haykin, Simon. 2009. Neural networks and learning machines. 3d ed. Upper Saddle River, NJ: Pearson Prentice Hall.

    E-mail Citation »

    A comprehensive textbook introducing many neural network algorithms, though not written specifically for readers in psychology or linguistics.

  • Levine, Daniel. 2000. Introduction to neural and cognitive modeling. 2d ed. Mahwah, NJ: Lawrence Erlbaum.

    DOI: 10.4324/9781410605504E-mail Citation »

    This book focuses on general organizing principles underlying neural and cognitive modeling, including competition, association, and categorization. It contains many more technical and mathematical details than do other books discussed here.

  • O’Reilly, Randall C., and Yuko Munakata. 2000. Computational explorations in cognitive neuroscience: Understanding the mind by simulating the brain. Cambridge, MA: MIT Press.

    DOI: 10.7551/mitpress/2014.001.0001E-mail Citation »

    A comprehensive book covering cognitive and neural modeling with the aim of uniting computation, brain, and cognition for a field called computational cognitive neuroscience. The book is more suitable for readers at the graduate student level or higher. Readers who are interested in this text should explore the wiki site provided by the authors.

  • Shirai, Yasuhiro. 2018. Connectionism and second language acquisition. New York: Routledge.

    DOI: 10.4324/9780203118085E-mail Citation »

    A recent synthesis of connectionism as it applies to the study of bilingualism and second language acquisition. The book not only focuses on the existing models but also situates these models within the larger contexts and theoretical debates of connectionism and language learning.

  • Shultz, Thomas. 2003. Computational developmental psychology. Cambridge, MA: MIT Press.

    E-mail Citation »

    An excellent discussion of using neural networks to study cognitive development, especially stages of development and mechanisms of transition. The book provides a good neural network primer along with mathematical basics of connectionist principles. It also discusses the cascade-correlation model that the author uses, a neural network that can dynamically recruit new, hidden units in response to task demands.

  • Spitzer, Manfred. 1999. The mind within the net: Models of learning, thinking, and acting. Cambridge, MA: MIT Press.

    DOI: 10.7551/mitpress/4632.001.0001E-mail Citation »

    An easy-to-read text that provides a discussion of neural networks and cognition in layperson’s terms.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login.

How to Subscribe

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here.

Article

Up

Down