Connectionism
- LAST REVIEWED: 29 July 2020
- LAST MODIFIED: 29 July 2020
- DOI: 10.1093/obo/9780199772810-0010
- LAST REVIEWED: 29 July 2020
- LAST MODIFIED: 29 July 2020
- DOI: 10.1093/obo/9780199772810-0010
Introduction
Connectionism, also known as parallel distributed processing (PDP) or artificial neural networks, and most recently reengineered as Deep Learning, has been an important theoretical framework as well as a computational tool for the study of mind and behavior. It adopts the perspective that human cognition is an emergent property that is due to the interaction of a large number of interconnected processing units (neurons) that operate simultaneously in a network (thus “parallel”). In addition, connectionism advocates that learning, representation, and processing of information are dynamic and distributed. Language as a hallmark of human behavior has received in-depth treatment since the beginning of connectionist research. The acquisition of morphosyntax, the recognition of speech, and the processing of sentences are among the studies of the earliest connectionist models. The application of connectionism to second language acquisition has also gathered momentum in the late 20th and early 21st centuries. Learning a language entails complex cognitive and linguistic constraints and interactions, and connectionist models provide insights into how these constraints and interactions may be realized in the natural learning context.
Introductory Works
Many books that introduce the principles of connectionism have appeared since the mid-1980s. Introductions to fundamental algorithms of neural networks can be found in Haykin 2009. Goodfellow, et al. 2016 provides comprehensive coverage of many cutting-edge deep learning neural network models, along with a thorough discussion of practical issues related to them. Several contributions also come to readers in linguistics or psychology. The original parallel distributed processing (PDP) volume Rumelhart, et al. 1986 (cited under Reference Works) provides a comprehensive overview of the early models, many of which deal with language and cognition. Spitzer 1999 is an excellent text that provides a clear description of the basic theories of connectionism and many of its applications in psychology, linguistics, and neuroscience. This book has very little technical detail and is written for nonspecialists. Ellis and Humphreys 1999 contains more technical details and in-depth discussions along with selected readings of original articles that emphasize learning, language, and memory. Levine 2000 focuses on general organizing principles underlying neural and cognitive modeling, including competition, association, and categorization. For readers interested in learning how to implement a connectionist network step-by-step, a few books and tools at varying levels of difficulty provide good exercises and examples of phenomena from linguistics, psychology, and neuroscience (see Software). O’Reilly and Munakata 2000 provides a comprehensive discussion of computational cognitive neuroscience that, unlike other texts mentioned in this section, focuses on modeling the neuronal system rather than the cognitive system at an abstract level. This book is written for more advanced readers, perhaps at the advanced graduate student level. In addition, a few books are more focused on applying connectionist approaches to specific topics or domains of study—for example, connectionism and development or connectionist language processing. These include Elman, et al. 1996, which highlights the connectionist framework of learning, interaction, and emergence and provides good illustrations of how connectionism tackles issues in cognitive and language development. Another book of this type is Shultz 2003, focusing more on stages of development and mechanisms of transition. Christiansen and Chater 2001 contains useful reviews of connectionism in language studies. Shirai 2018 provides a recent synthesis of connectionism and second language acquisition, serving as an excellent overview of existing connectionist bilingual models from the past twenty-five years.
Christiansen, Morten, and Nick Chater, eds. 2001. Connectionist psycholinguistics. Westport, CT: Ablex.
A collection of papers by connectionist scholars that reviews (up to 2001) connectionist models of language, including spoken word recognition, morphosyntax, language production, reading and dyslexia, and language acquisition.
Ellis, Rob, and Glyn W. Humphreys. 1999. Connectionist psychology: A text with readings. New York: Psychology Press.
A generally readable textbook with a focus on applying connectionist models to the study of cognitive phenomena, including memory, language, learning, and cognitive disorders.
Elman, Jeffrey L., Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, and Kim Plunkett. 1996. Rethinking innateness: A connectionist perspective on development. Cambridge, MA: MIT Press.
This book has been called the “second bible” of connectionism; it focuses on cognitive and language development based on the connectionism framework. It argues for the need to clearly define innateness at different levels and to separate innateness from modularity, domain specificity, and localization.
Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep learning. Cambridge, MA: MIT Press.
A comprehensive textbook introducing a broad range of deep learning algorithms. Many practical issues of deep learning are discussed in this book. This book is available via open access online.
Haykin, Simon. 2009. Neural networks and learning machines. 3d ed. Upper Saddle River, NJ: Pearson Prentice Hall.
A comprehensive textbook introducing many neural network algorithms, though not written specifically for readers in psychology or linguistics.
Levine, Daniel. 2000. Introduction to neural and cognitive modeling. 2d ed. Mahwah, NJ: Lawrence Erlbaum.
This book focuses on general organizing principles underlying neural and cognitive modeling, including competition, association, and categorization. It contains many more technical and mathematical details than do other books discussed here.
O’Reilly, Randall C., and Yuko Munakata. 2000. Computational explorations in cognitive neuroscience: Understanding the mind by simulating the brain. Cambridge, MA: MIT Press.
DOI: 10.7551/mitpress/2014.001.0001
A comprehensive book covering cognitive and neural modeling with the aim of uniting computation, brain, and cognition for a field called computational cognitive neuroscience. The book is more suitable for readers at the graduate student level or higher. Readers who are interested in this text should explore the wiki site provided by the authors.
Shirai, Yasuhiro. 2018. Connectionism and second language acquisition. New York: Routledge.
A recent synthesis of connectionism as it applies to the study of bilingualism and second language acquisition. The book not only focuses on the existing models but also situates these models within the larger contexts and theoretical debates of connectionism and language learning.
Shultz, Thomas. 2003. Computational developmental psychology. Cambridge, MA: MIT Press.
An excellent discussion of using neural networks to study cognitive development, especially stages of development and mechanisms of transition. The book provides a good neural network primer along with mathematical basics of connectionist principles. It also discusses the cascade-correlation model that the author uses, a neural network that can dynamically recruit new, hidden units in response to task demands.
Spitzer, Manfred. 1999. The mind within the net: Models of learning, thinking, and acting. Cambridge, MA: MIT Press.
DOI: 10.7551/mitpress/4632.001.0001
An easy-to-read text that provides a discussion of neural networks and cognition in layperson’s terms.
Users without a subscription are not able to see the full content on this page. Please subscribe or login.
How to Subscribe
Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here.
Article
- Acceptability Judgments
- Accessibility Theory in Linguistics
- Acquisition, Second Language, and Bilingualism, Psycholin...
- Adjectives
- Adpositions
- Affixation
- African Linguistics
- Afroasiatic Languages
- Agreement
- Algonquian Linguistics
- Altaic Languages
- Ambiguity, Lexical
- Analogy in Language and Linguistics
- Anaphora
- Animal Communication
- Aphasia
- Applicatives
- Applied Linguistics, Critical
- Arawak Languages
- Argument Structure
- Artificial Languages
- Attention and Salience
- Australian Languages
- Austronesian Linguistics
- Auxiliaries
- Balkans, The Languages of the
- Baudouin de Courtenay, Jan
- Berber Languages and Linguistics
- Bilingualism and Multilingualism
- Biology of Language
- Blocking
- Borrowing, Structural
- Caddoan Languages
- Caucasian Languages
- Causatives
- Celtic Languages
- Celtic Mutations
- Chomsky, Noam
- Chumashan Languages
- Classifiers
- Clauses, Relative
- Clinical Linguistics
- Cognitive Linguistics
- Colonial Place Names
- Comparative Reconstruction in Linguistics
- Comparative-Historical Linguistics
- Complementation
- Complexity, Linguistic
- Compositionality
- Compounding
- Comprehension, Sentence
- Computational Linguistics
- Conditionals
- Conjunctions
- Connectionism
- Consonant Epenthesis
- Constructions, Verb-Particle
- Contrastive Analysis in Linguistics
- Conversation Analysis
- Conversation, Maxims of
- Conversational Implicature
- Cooperative Principle
- Coordination
- Copula
- Creoles
- Creoles, Grammatical Categories in
- Critical Periods
- Cross-Language Speech Perception and Production
- Cyberpragmatics
- Default Semantics
- Definiteness
- Dementia and Language
- Dene (Athabaskan) Languages
- Dené-Yeniseian Hypothesis, The
- Dependencies
- Dependencies, Long Distance
- Derivational Morphology
- Determiners
- Dialectology
- Dialogue
- Diglossia
- Disfluency
- Distinctive Features
- Dravidian Languages
- Ellipsis
- Endangered Languages
- English as a Lingua Franca
- English, Early Modern
- English, Old
- Ergativity
- Eskimo-Aleut
- Euphemisms and Dysphemisms
- Evidentials
- Exemplar-Based Models in Linguistics
- Existential
- Existential Wh-Constructions
- Experimental Linguistics
- Fieldwork
- Fieldwork, Sociolinguistic
- Finite State Languages
- First Language Attrition
- Formulaic Language
- Francoprovençal
- French Grammars
- Frisian
- Gabelentz, Georg von der
- Gender
- Genealogical Classification
- Generative Syntax
- Genetics and Language
- Gestures
- Grammar, Categorial
- Grammar, Cognitive
- Grammar, Construction
- Grammar, Descriptive
- Grammar, Functional Discourse
- Grammars, Phrase Structure
- Grammaticalization
- Harris, Zellig
- Heritage Languages
- History of Linguistics
- History of the English Language
- Hmong-Mien Languages
- Hokan Languages
- Honorifics
- Humor in Language
- Hungarian Vowel Harmony
- Iconicity
- Ideophones
- Idiolect
- Idiom and Phraseology
- Imperatives
- Indefiniteness
- Indo-European Etymology
- Inflected Infinitives
- Information Structure
- Innateness
- Interface Between Phonology and Phonetics
- Interjections
- Intonation
- IPA
- Irony
- Iroquoian Languages
- Islands
- Isolates, Language
- Jakobson, Roman
- Japanese Word Accent
- Jones, Daniel
- Juncture and Boundary
- Khoisan Languages
- Kiowa-Tanoan Languages
- Kra-Dai Languages
- Labov, William
- Language Acquisition
- Language and Law
- Language Contact
- Language Documentation
- Language, Embodiment and
- Language for Specific Purposes/Specialized Communication
- Language, Gender, and Sexuality
- Language Geography
- Language Ideologies and Language Attitudes
- Language in Autism Spectrum Disorders
- Language Nests
- Language Revitalization
- Language Shift
- Language Standardization
- Language, Synesthesia and
- Languages of Africa
- Languages of the Americas, Indigenous
- Languages of the World
- Learnability
- Lexemes
- Lexical Access, Cognitive Mechanisms for
- Lexical Semantics
- Lexical-Functional Grammar
- Lexicography
- Lexicography, Bilingual
- Lexicon
- Linguistic Accommodation
- Linguistic Anthropology
- Linguistic Areas
- Linguistic Landscapes
- Linguistic Prescriptivism
- Linguistic Profiling and Language-Based Discrimination
- Linguistic Relativity
- Linguistics, Educational
- Listening, Second Language
- Literature and Linguistics
- Loanwords
- Machine Translation
- Maintenance, Language
- Mande Languages
- Markedness
- Mass-Count Distinction
- Mathematical Linguistics
- Mayan Languages
- Mental Health Disorders, Language in
- Mental Lexicon, The
- Mesoamerican Languages
- Metaphor
- Metathesis
- Metonymy
- Minority Languages
- Mixed Languages
- Mixe-Zoquean Languages
- Modification
- Mon-Khmer Languages
- Morphological Change
- Morphology
- Morphology, Blending in
- Morphology, Subtractive
- Movement
- Munda Languages
- Muskogean Languages
- Nasals and Nasalization
- Negation
- Niger-Congo Languages
- Non-Pama-Nyungan Languages
- Northeast Caucasian Languages
- Nostratic
- Number
- Numerals
- Oceanic Languages
- Papuan Languages
- Penutian Languages
- Philosophy of Language
- Phonetics
- Phonetics, Acoustic
- Phonetics, Articulatory
- Phonological Research, Psycholinguistic Methodology in
- Phonology
- Phonology, Computational
- Phonology, Early Child
- Pidgins
- Polarity
- Policy and Planning, Language
- Politeness in Language
- Polysemy
- Positive Discourse Analysis
- Possessives, Acquisition of
- Pragmatics, Acquisition of
- Pragmatics, Cognitive
- Pragmatics, Computational
- Pragmatics, Cross-Cultural
- Pragmatics, Developmental
- Pragmatics, Experimental
- Pragmatics, Game Theory in
- Pragmatics, Historical
- Pragmatics, Institutional
- Pragmatics, Second Language
- Pragmatics, Teaching
- Prague Linguistic Circle, The
- Presupposition
- Pronouns
- Psycholinguistics
- Quechuan and Aymaran Languages
- Questions
- Reading, Second-Language
- Reciprocals
- Reduplication
- Reflexives and Reflexivity
- Register and Register Variation
- Relevance Theory
- Representation and Processing of Multi-Word Expressions in...
- Salish Languages
- Sapir, Edward
- Saussure, Ferdinand de
- Second Language Acquisition, Anaphora Resolution in
- Semantic Maps
- Semantic Roles
- Semantic-Pragmatic Change
- Semantics, Cognitive
- Sentence Processing in Monolingual and Bilingual Speakers
- Sign Language Linguistics
- Slang
- Sociolinguistics
- Sociolinguistics, Variationist
- Sociopragmatics
- Sonority
- Sound Change
- South American Indian Languages
- Specific Language Impairment
- Speech, Deceptive
- Speech Perception
- Speech Production
- Speech Synthesis
- Suppletion
- Switch-Reference
- Syllables
- Syncretism
- Synonymy
- Syntactic Change
- Syntactic Knowledge, Children’s Acquisition of
- Tense, Aspect, and Mood
- Text Mining
- Tone
- Tone Sandhi
- Topic
- Transcription
- Transitivity and Voice
- Translanguaging
- Translation
- Trubetzkoy, Nikolai
- Tucanoan Languages
- Tupian Languages
- Typology
- Usage-Based Linguistics
- Uto-Aztecan Languages
- Valency Theory
- Verbs, Serial
- Vocabulary, Second Language
- Voice and Voice Quality
- Vowel Harmony
- Whitney, William Dwight
- Word Classes
- Word Formation in Japanese
- Word Recognition, Spoken
- Word Recognition, Visual
- Word Stress
- Writing, Second Language
- Writing Systems
- Yiddish
- Zapotecan Languages