Reliability–Contemporary Psychometric Conceptions
- LAST REVIEWED: 26 May 2023
- LAST MODIFIED: 26 May 2023
- DOI: 10.1093/obo/9780199828340-0314
- LAST REVIEWED: 26 May 2023
- LAST MODIFIED: 26 May 2023
- DOI: 10.1093/obo/9780199828340-0314
Introduction
Reliability is a major index of quality of behavioral measurement. Informally, reliability of a behavior measuring device—whether an item, question, scale, inventory, self-report, or test—reflects the repeatability of the results obtainable with it. The reliability index—often referred to as an index of reliability and defined as the (positive) square root of the reliability coefficient—is substantially less often used than the reliability coefficient in the behavioral sciences. This index represents the extent to which an observation, considered as a random variable, correlates with the associated true value it aims to evaluate. The reliability coefficient, which is at present nearly always employed in theoretical and empirical measurement-related discussions and treatments, can be thought of as the degree to which observed individual differences (of the units of analysis) are the result of true underlying individual differences.
General Overviews
Based on the classical test theory (CTT) decomposition X = T + E for an observed score X, associated true score T, and pertinent error score E, the reliability coefficient is defined as the ratio Var(T)/Var(X) where Var(.) denotes variance (in a relevant population). That is, the reliability coefficient is defined whenever there are observed individual differences, i.e., Var(X) > 0 holds. This can be assumed to be the case in most if not all contemporary empirical behavioral studies—see McDonald 1999 and Zimmerman 1975. Reliability is not defined when Var(X) = 0, which is unlikely in the overwhelming majority of contemporary behavioral studies unless measuring instruments are used that are highly insensitive to individual differences—see Raykov and Marcoulides 2011. An equivalent definition of the reliability coefficient is the following: This coefficient is the squared correlation of true with observed score, that is Corr2(T,X) where Corr(.,.) denotes correlation, whenever observed and error variance are positive—see Crocker and Algina 2006 and Lord and Novick 1968. If either of these latter two variances vanishes, the reliability index is not defined, but the reliability coefficient is defined as 0 if Var(T) = 0 and Var(X) > 0 holds—see Allen and Yen 1979. Due to limited uses of the reliability index in the behavioral and social science literature, unless otherwise indicated the remainder of this discussion utilizes the reference “reliability” as synonymous to the reliability coefficient. This reference will be typically used for multi-component measuring instruments (psychometric scales), which will be mostly of relevance in the sequel. Further, all measuring instruments are assumed consisting of components that are pre-fixed rather than selected randomly or otherwise from pre-existing larger pools of such measures, to which inferences are sought.
Allen, M. J., and W. M. Yen. 1979. Introduction to measurement theory. Long Grove, IL: Waveland Press.
Provides a comprehensive discussion of the basics of behavioral measurement and its applications, including detailed coverage of reliability at a relatively introductory level.
Crocker, L., and J. Algina. 2006. Introduction to classical and modern test theory. Fort Worth, TX: Harcourt College Publishers.
Offers a rigorous discussion of modern and classical approaches to behavioral measurement and attends in detail to reliability-related matters.
Lord, F. M., and M. Novick. 1968. Statistical theories of mental test scores. Reading, MA: Wesley.
A thorough treatment of the theoretical framework of mental testing and pertinent statistical theories, including the foundations of CTT, within which framework reliability is readily definable as the true to observed variances ratio.
McDonald, R. P. 1999. Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
Discusses reliability within the context of a unified methodology for behavioral measurement, which connects item response theory modeling-based approaches with those grounded in factor analysis.
Raykov, T., and G. A. Marcoulides. 2011. Introduction to psychometric theory. New York: Taylor & Francis.
A largely introductory treatment of psychometric theory, which includes interpretations of the reliability coefficient and discussions of its relationships to conceptual regression models of true and observed scores.
Zimmerman, D. W. 1975. Probability spaces, Hilbert spaces, and the axioms of test theory. Psychometrika 40:395–412.
DOI: 10.1007/BF02291765
Provides the arguably most rigorous treatment of CTT, based on the concepts of Hilbert space, projection, and conditional expectation, and offers a very general definition of the reliability coefficient within that formal approach.
Users without a subscription are not able to see the full content on this page. Please subscribe or login.
How to Subscribe
Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here.
Article
- Abnormal Psychology
- Academic Assessment
- Acculturation and Health
- Action Regulation Theory
- Action Research
- Addictive Behavior
- Adolescence
- Adoption, Social, Psychological, and Evolutionary Perspect...
- Adulthood
- Advanced Theory of Mind
- Affective Forecasting
- Affirmative Action
- Ageism
- Ageism at Work
- Aggression
- Allport, Gordon
- Alzheimer’s Disease
- Ambulatory Assessment in Behavioral Science
- Analysis of Covariance (ANCOVA)
- Anger
- Animal Behavior
- Animal Learning
- Anxiety Disorders
- Art and Aesthetics, Psychology of
- Artificial Intelligence, Machine Learning, and Psychology
- Assessment and Clinical Applications of Individual Differe...
- Attachment in Social and Emotional Development across the ...
- Attention-Deficit/Hyperactivity Disorder (ADHD) in Adults
- Attention-Deficit/Hyperactivity Disorder (ADHD) in Childre...
- Attitudes
- Attitudinal Ambivalence
- Attraction in Close Relationships
- Attribution Theory
- Authoritarian Personality
- Autism
- Bayesian Statistical Methods in Psychology
- Behavior Therapy, Rational Emotive
- Behavioral Economics
- Behavioral Genetics
- Belief Perseverance
- Bereavement and Grief
- Biological Psychology
- Birth Order
- Body Image in Men and Women
- Burnout
- Bystander Effect
- Categorical Data Analysis in Psychology
- Childhood and Adolescence, Peer Victimization and Bullying...
- Clark, Mamie Phipps
- Clinical Neuropsychology
- Clinical Psychology
- Cognitive Consistency Theories
- Cognitive Dissonance Theory
- Cognitive Neuroscience
- Communication, Nonverbal Cues and
- Comparative Psychology
- Competence to Stand Trial: Restoration Services
- Competency to Stand Trial
- Computational Psychology
- Conflict Management in the Workplace
- Conformity, Compliance, and Obedience
- Consciousness
- Coping Processes
- Correspondence Analysis in Psychology
- Counseling Psychology
- Courage
- Creativity
- Creativity at Work
- Critical Thinking
- Cross-Cultural Psychology
- Cultural Psychology
- Daily Life, Research Methods for Studying
- Data Science Methods for Psychology
- Data Sharing in Psychology
- Death and Dying
- Deceiving and Detecting Deceit
- Defensive Processes
- DEI in Organizations
- Depression
- Depressive Disorders
- Development, Prenatal
- Developmental Psychology (Cognitive)
- Developmental Psychology (Social)
- Diagnostic and Statistical Manual of Mental Disorders (DSM...
- Discrimination
- Disgust
- Dissociative Disorders
- Drugs and Behavior
- Eating Disorders
- Ecological Psychology
- Ecopsychology
- Educational Settings, Assessment of Thinking in
- Effect Size
- Embodiment and Embodied Cognition
- Emerging Adulthood
- Emotion
- Emotional Intelligence
- Empathy and Altruism
- Employee Stress and Well-Being
- Environmental Neuroscience and Environmental Psychology
- Ethics in Psychological Practice
- Event Perception
- Evolutionary Psychology
- Expansive Posture
- Experimental Existential Psychology
- Exploratory Data Analysis
- Eyewitness Testimony
- Eysenck, Hans
- Factor Analysis
- Festinger, Leon
- Five-Factor Model of Personality
- Flynn Effect, The
- Forensic Psychology
- Forgiveness
- Friendships, Children's
- Fundamental Attribution Error/Correspondence Bias
- Gambler's Fallacy
- Game Theory and Psychology
- Geropsychology, Clinical
- Global Mental Health
- Habit Formation and Behavior Change
- Happiness
- Health Psychology
- Health Psychology Research and Practice, Measurement in
- Heider, Fritz
- Heuristics and Biases
- History of Psychology
- Human Factors
- Humanistic Psychology
- Humor
- Hypnosis
- Implicit Association Test (IAT)
- Industrial and Organizational Psychology
- Inferential Statistics in Psychology
- Insanity Defense, The
- Intelligence
- Intelligence, Crystallized and Fluid
- Intercultural Psychology
- Intergroup Conflict
- International Classification of Diseases and Related Healt...
- International Psychology
- Interviewing in Forensic Settings
- Intimate Partner Violence, Psychological Perspectives on
- Introversion–Extraversion
- Item Response Theory
- Kurtosis
- Language
- Laughter
- Law, Psychology and
- Lazarus, Richard
- Leadership
- Learned Helplessness
- Learning Theory
- Learning versus Performance
- LGBTQ+ Romantic Relationships
- Lie Detection in a Forensic Context
- Life-Span Development
- Lineups
- Locus of Control
- Loneliness and Health
- Mathematical Psychology
- Meaning in Life
- Mechanisms and Processes of Peer Contagion
- Media Violence, Psychological Perspectives on
- Mediation Analysis
- Meditation
- Memories, Autobiographical
- Memories, Flashbulb
- Memories, Repressed and Recovered
- Memory, False
- Memory, Human
- Memory, Implicit versus Explicit
- Memory in Educational Settings
- Memory, Semantic
- Meta-Analysis
- Metacognition
- Metamemory
- Metaphor, Psychological Perspectives on
- Microaggressions
- Military Psychology
- Mindfulness
- Mindfulness and Education
- Minnesota Multiphasic Personality Inventory (MMPI)
- Money, Psychology of
- Moral Conviction
- Moral Development
- Moral Psychology
- Moral Reasoning
- Motivation
- Music
- Narcissism
- Narrative
- Nature versus Nurture Debate in Psychology
- Neuroscience of Associative Learning
- Nonergodicity in Psychology and Neuroscience
- Nonparametric Statistical Analysis in Psychology
- Observational (Non-Randomized) Studies
- Obsessive-Complusive Disorder (OCD)
- Occupational Health Psychology
- Older Workers
- Olfaction, Human
- Operant Conditioning
- Optimism and Pessimism
- Organizational Justice
- Parenting Stress
- Parenting Styles
- Parents' Beliefs about Children
- Path Models
- Peace Psychology
- Perception
- Perception, Person
- Performance Appraisal
- Personality and Health
- Personality Disorders
- Personality Psychology
- Person-Centered and Experiential Psychotherapies: From Car...
- Phenomenological Psychology
- Placebo Effects in Psychology
- Play Behavior
- Positive Psychological Capital (PsyCap)
- Positive Psychology
- Posttraumatic Stress Disorder (PTSD)
- Prejudice and Stereotyping
- Pretrial Publicity
- Prisoner's Dilemma
- Problem Solving and Decision Making
- Procrastination
- Prosocial Behavior
- Prosocial Spending and Well-Being
- Protocol Analysis
- Psycholinguistics
- Psychological Literacy
- Psychological Perspectives on Food and Eating
- Psychology, Political
- Psychoneuroimmunology
- Psychophysics, Visual
- Psychotherapy
- Psychotic Disorders
- Publication Bias in Psychology
- Race
- Reasoning, Counterfactual
- Rehabilitation Psychology
- Relationships
- Reliability–Contemporary Psychometric Conceptions
- Religion, Psychology and
- Remote Work
- Replication Initiatives in Psychology
- Research Methods
- Resilience
- Risk Taking
- Role of the Expert Witness in Forensic Psychology, The
- Rumination
- Sample Size Planning for Statistical Power and Accurate Es...
- Savoring
- Schizophrenic Disorders
- School Psychology
- School Psychology, Counseling Services in
- Self, Gender and
- Self, Psychology of the
- Self-Construal
- Self-Control
- Self-Deception
- Self-Determination Theory
- Self-Efficacy
- Self-Esteem
- Self-Monitoring
- Self-Regulation in Educational Settings
- Self-Report Tests, Measures, and Inventories in Clinical P...
- Sensation Seeking
- Sex and Gender
- Sexual Minority Parenting
- Sexual Orientation
- Signal Detection Theory and its Applications
- Simpson's Paradox in Psychology
- Single People
- Single-Case Experimental Designs
- Situational Strength
- Skinner, B.F.
- Sleep and Dreaming
- Small Groups
- Social Class and Social Status
- Social Cognition
- Social Neuroscience
- Social Support
- Social Touch and Massage Therapy Research
- Somatoform Disorders
- Spatial Attention
- Sports Psychology
- Stanford Prison Experiment (SPE): Icon and Controversy
- Stereotype Threat
- Stereotypes
- Stress and Coping, Psychology of
- Student Success in College
- Subjective Wellbeing Homeostasis
- Suicide
- Taste, Psychological Perspectives on
- Teaching of Psychology
- Terror Management Theory
- Testing and Assessment
- The Concept of Validity in Psychological Assessment
- The Neuroscience of Emotion Regulation
- The Reasoned Action Approach and the Theories of Reasoned ...
- The Weapon Focus Effect in Eyewitness Memory
- Theory of Mind
- Therapy, Cognitive-Behavioral
- Thinking Skills in Educational Settings
- Time Perception
- Trait Perspective
- Trauma Psychology
- Twin Studies
- Type A Behavior Pattern (Coronary Prone Personality)
- Unconscious Processes
- Video Games and Violent Content
- Virtues and Character Strengths
- Wisdom
- Women and Science, Technology, Engineering, and Math (STEM...
- Women, Psychology of
- Work Well-Being
- Workforce Training Evaluation
- Wundt, Wilhelm