Sample Size Planning for Statistical Power and Accurate Estimation
- LAST REVIEWED: 27 October 2022
- LAST MODIFIED: 27 October 2022
- DOI: 10.1093/obo/9780199828340-0296
- LAST REVIEWED: 27 October 2022
- LAST MODIFIED: 27 October 2022
- DOI: 10.1093/obo/9780199828340-0296
Introduction
At the most general level, sample size refers to the number of participants (typically human or animal subjects) who provide data for a scientific experiment or study. Selecting an appropriate sample size is a fundamental feature of a well-designed study, and, as the title suggests, sample size can be carefully planned ahead of time. Sample size planning is important because the sample size of a study has implications for statistical power as well as accuracy. In conventional terms, statistical power reflects the probability of rejecting the null hypothesis, assuming that the purported effect is non-null (non-zero) in reality. Although power can be treated as a function of unknown parameter values, power is often defined conditionally on a specific value of population effect size. (On a technical level, power is conditional upon the population noncentrality parameter, which is a combination of sample size and effect size, but relying on effect size as a proxy is generally acceptable.) Power is calculated from the statistical significance threshold (otherwise known as the nominal Type I error rate or alpha-level), the population effect size, and the sample size. Because of the connection between sample size and power, sample size planning is sometimes called a priori power analysis. For example, if a new therapy developed to treat depression is truly beneficial, the sample size of a study assessing this therapy should be large enough to demonstrate the beneficial outcome as statistically significant. Accuracy reflects how close an estimate is to the parameter it aims to estimate. Accurate estimates are expressed with narrow confidence intervals (small margins of error). Sample size also has a connection to accuracy. For example, if the investigators in the previous example want to estimate how much the therapy reduces depressive symptoms, the sample size should be large enough so that the effect size estimate derived from the study is an accurate estimate of the true magnitude of the treatment effect. An underlying challenge to sample size planning for both goals is that the population effect size is unknown. An important corollary is that whether a particular sample size provides appropriate power (and in some cases, accuracy) depends on the population effect size, and as such, sample size cannot completely be judged without context. Although there are sometimes limitations to sample size, such as when working with a limited budget or specialized populations, and although sample size is often selected based on convenience or historical precedent, when possible it is desirable to directly plan the study sample size at the study design phase. Explicitly planning sample size helps to ensure that the sample size will be effective in meeting the investigators’ goals and answering the scientific questions of interest.
Books and Book Chapters
Given its important role in the design of an effective research study, several books and chapters have been devoted to sample size planning and/or power analysis. Although full books on sample size are more commonly written for biomedical researchers and clinical trial designs, the selected books and chapters cited here offer excellent introductions to sample size planning geared toward psychologists or serve as thorough reference texts. Cohen 1988 is considered by many to be the quintessential text on statistical power analysis in the psychological and behavioral sciences, containing sample size tables and formulas for numerous designs and effect sizes. For readers looking for a thinner, but reader-friendly, alternative, Kraemer and Blasey 2016 carefully contextualizes, describes, and demonstrates sample size planning for designs of interest to psychologists. Despite its focus on biomedical research, Chow, et al. 2008 represents a more technical resource for those interested in equivalence and non-inferiority testing, in addition to general two-group or crossover designs common in clinical and applied research. Lipsey 1990 also focuses more on applied contexts, but provides a broader context for sample size planning as a part of good study design. Kelley and Maxwell 2012, a chapter rather than a full-length text, provides an accessible introduction to sample size planning, multiple intuitive and carefully worked through examples, and strategies for planning for power and accuracy.
Chow, S.-C., J. Shao, and H. Wang. 2008. Sample size calculations in clinical research. 2d ed., Vol. 11. Boca Raton, FL: CRC Press.
Although geared slightly more toward researchers in the biomedical sciences, represents a thorough guide to sample size formulas for a variety of designs common to clinical and experimental domains, including tests of mean differences, proportions, and survival analysis. Contains formulas for conducting equivalence testing as well as a section on Bayesian sample size planning.
Cohen, J. 1988. Statistical power analysis for the behavioral sciences. 2d ed. Hillsdale, NJ: Erlbaum.
Considered by many to be the fundamental text on sample size and statistical power. Contains formulas for calculating statistical power and tables of necessary sample sizes for an array of experimental designs. Additionally, includes information and formulas for calculating various effect sizes needed for sample size calculations.
Kelley, K., and S. E. Maxwell. 2012. Sample size. In APA handbook of research methods in psychology. Vol. 1. Edited by H. Cooper, 181–202. Washington, DC: American Psychological Association.
Thorough overview of the basic issues and processes involved in sample size planning, written for a general audience. Covers sample size planning for statistical power and sample size planning for accuracy, and includes several step-by-step examples using freely available software.
Kraemer, H. C., and C. M. Blasey. 2016. How many subjects? Statistical power analysis in research. 2d ed. Thousand Oaks, CA: SAGE.
A concise and well-written guide to sample size planning for the ANOVA and regression designs of interest to psychologists. Includes a thoughtful introduction to hypothesis testing to better contextualize the role of sample size planning for power. Uses the same example topic multiple times throughout to illustrate sample size planning for different questions and designs.
Lipsey, M. W. 1990. Design sensitivity: Statistical power for experimental research. Newbury Park, CA: SAGE.
With a focus on applied treatment research, this work focuses on the broader issue of designing studies that are sensitive to detect the effect of interest, a goal for which sample size and statistical power are fundamental features. Includes an entire chapter focusing on the ways in which effect size can be approached with regard to statistical power.
Users without a subscription are not able to see the full content on this page. Please subscribe or login.
How to Subscribe
Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here.
Article
- Abnormal Psychology
- Academic Assessment
- Acculturation and Health
- Action Regulation Theory
- Action Research
- Addictive Behavior
- Adolescence
- Adoption, Social, Psychological, and Evolutionary Perspect...
- Adulthood
- Advanced Theory of Mind
- Affective Forecasting
- Affirmative Action
- Ageism
- Ageism at Work
- Aggression
- Allport, Gordon
- Alzheimer’s Disease
- Ambulatory Assessment in Behavioral Science
- Analysis of Covariance (ANCOVA)
- Anger
- Animal Behavior
- Animal Learning
- Anxiety Disorders
- Art and Aesthetics, Psychology of
- Artificial Intelligence, Machine Learning, and Psychology
- Assessment and Clinical Applications of Individual Differe...
- Attachment in Social and Emotional Development across the ...
- Attention-Deficit/Hyperactivity Disorder (ADHD) in Adults
- Attention-Deficit/Hyperactivity Disorder (ADHD) in Childre...
- Attitudes
- Attitudinal Ambivalence
- Attraction in Close Relationships
- Attribution Theory
- Authoritarian Personality
- Autism
- Bayesian Statistical Methods in Psychology
- Behavior Therapy, Rational Emotive
- Behavioral Economics
- Behavioral Genetics
- Belief Perseverance
- Bereavement and Grief
- Biological Psychology
- Birth Order
- Body Image in Men and Women
- Burnout
- Bystander Effect
- Categorical Data Analysis in Psychology
- Childhood and Adolescence, Peer Victimization and Bullying...
- Clark, Mamie Phipps
- Clinical Neuropsychology
- Clinical Psychology
- Cognitive Consistency Theories
- Cognitive Dissonance Theory
- Cognitive Neuroscience
- Communication, Nonverbal Cues and
- Comparative Psychology
- Competence to Stand Trial: Restoration Services
- Competency to Stand Trial
- Computational Psychology
- Conflict Management in the Workplace
- Conformity, Compliance, and Obedience
- Consciousness
- Coping Processes
- Correspondence Analysis in Psychology
- Counseling Psychology
- Courage
- Creativity
- Creativity at Work
- Critical Thinking
- Cross-Cultural Psychology
- Cultural Psychology
- Daily Life, Research Methods for Studying
- Data Science Methods for Psychology
- Data Sharing in Psychology
- Death and Dying
- Deceiving and Detecting Deceit
- Defensive Processes
- Depression
- Depressive Disorders
- Development, Prenatal
- Developmental Psychology (Cognitive)
- Developmental Psychology (Social)
- Diagnostic and Statistical Manual of Mental Disorders (DSM...
- Discrimination
- Disgust
- Dissociative Disorders
- Drugs and Behavior
- Eating Disorders
- Ecological Psychology
- Ecopsychology
- Educational Settings, Assessment of Thinking in
- Effect Size
- Embodiment and Embodied Cognition
- Emerging Adulthood
- Emotion
- Emotional Intelligence
- Empathy and Altruism
- Employee Stress and Well-Being
- Environmental Neuroscience and Environmental Psychology
- Ethics in Psychological Practice
- Event Perception
- Evolutionary Psychology
- Expansive Posture
- Experimental Existential Psychology
- Exploratory Data Analysis
- Eyewitness Testimony
- Eysenck, Hans
- Factor Analysis
- Festinger, Leon
- Five-Factor Model of Personality
- Flynn Effect, The
- Forensic Psychology
- Forgiveness
- Friendships, Children's
- Fundamental Attribution Error/Correspondence Bias
- Gambler's Fallacy
- Game Theory and Psychology
- Geropsychology, Clinical
- Global Mental Health
- Habit Formation and Behavior Change
- Happiness
- Health Psychology
- Health Psychology Research and Practice, Measurement in
- Heider, Fritz
- Heuristics and Biases
- History of Psychology
- Human Factors
- Humanistic Psychology
- Humor
- Hypnosis
- Implicit Association Test (IAT)
- Industrial and Organizational Psychology
- Inferential Statistics in Psychology
- Insanity Defense, The
- Intelligence
- Intelligence, Crystallized and Fluid
- Intercultural Psychology
- Intergroup Conflict
- International Classification of Diseases and Related Healt...
- International Psychology
- Interviewing in Forensic Settings
- Intimate Partner Violence, Psychological Perspectives on
- Introversion–Extraversion
- Item Response Theory
- Kurtosis
- Language
- Laughter
- Law, Psychology and
- Lazarus, Richard
- Leadership
- Learned Helplessness
- Learning Theory
- Learning versus Performance
- LGBTQ+ Romantic Relationships
- Lie Detection in a Forensic Context
- Life-Span Development
- Lineups
- Locus of Control
- Loneliness and Health
- Mathematical Psychology
- Meaning in Life
- Mechanisms and Processes of Peer Contagion
- Media Violence, Psychological Perspectives on
- Mediation Analysis
- Meditation
- Memories, Autobiographical
- Memories, Flashbulb
- Memories, Repressed and Recovered
- Memory, False
- Memory, Human
- Memory, Implicit versus Explicit
- Memory in Educational Settings
- Memory, Semantic
- Meta-Analysis
- Metacognition
- Metamemory
- Metaphor, Psychological Perspectives on
- Microaggressions
- Military Psychology
- Mindfulness
- Mindfulness and Education
- Minnesota Multiphasic Personality Inventory (MMPI)
- Money, Psychology of
- Moral Conviction
- Moral Development
- Moral Psychology
- Moral Reasoning
- Motivation
- Music
- Narcissism
- Narrative
- Nature versus Nurture Debate in Psychology
- Neuroscience of Associative Learning
- Nonergodicity in Psychology and Neuroscience
- Nonparametric Statistical Analysis in Psychology
- Observational (Non-Randomized) Studies
- Obsessive-Complusive Disorder (OCD)
- Occupational Health Psychology
- Older Workers
- Olfaction, Human
- Operant Conditioning
- Optimism and Pessimism
- Organizational Justice
- Parenting Stress
- Parenting Styles
- Parents' Beliefs about Children
- Path Models
- Peace Psychology
- Perception
- Perception, Person
- Performance Appraisal
- Personality and Health
- Personality Disorders
- Personality Psychology
- Person-Centered and Experiential Psychotherapies: From Car...
- Phenomenological Psychology
- Placebo Effects in Psychology
- Play Behavior
- Positive Psychological Capital (PsyCap)
- Positive Psychology
- Posttraumatic Stress Disorder (PTSD)
- Prejudice and Stereotyping
- Pretrial Publicity
- Prisoner's Dilemma
- Problem Solving and Decision Making
- Procrastination
- Prosocial Behavior
- Prosocial Spending and Well-Being
- Protocol Analysis
- Psycholinguistics
- Psychological Literacy
- Psychological Perspectives on Food and Eating
- Psychology, Political
- Psychoneuroimmunology
- Psychophysics, Visual
- Psychotherapy
- Psychotic Disorders
- Publication Bias in Psychology
- Race
- Reasoning, Counterfactual
- Rehabilitation Psychology
- Relationships
- Reliability–Contemporary Psychometric Conceptions
- Religion, Psychology and
- Replication Initiatives in Psychology
- Research Methods
- Resilience
- Risk Taking
- Role of the Expert Witness in Forensic Psychology, The
- Rumination
- Sample Size Planning for Statistical Power and Accurate Es...
- Savoring
- Schizophrenic Disorders
- School Psychology
- School Psychology, Counseling Services in
- Self, Gender and
- Self, Psychology of the
- Self-Construal
- Self-Control
- Self-Deception
- Self-Determination Theory
- Self-Efficacy
- Self-Esteem
- Self-Monitoring
- Self-Regulation in Educational Settings
- Self-Report Tests, Measures, and Inventories in Clinical P...
- Sensation Seeking
- Sex and Gender
- Sexual Minority Parenting
- Sexual Orientation
- Signal Detection Theory and its Applications
- Simpson's Paradox in Psychology
- Single People
- Single-Case Experimental Designs
- Situational Strength
- Skinner, B.F.
- Sleep and Dreaming
- Small Groups
- Social Class and Social Status
- Social Cognition
- Social Neuroscience
- Social Support
- Social Touch and Massage Therapy Research
- Somatoform Disorders
- Spatial Attention
- Sports Psychology
- Stanford Prison Experiment (SPE): Icon and Controversy
- Stereotype Threat
- Stereotypes
- Stress and Coping, Psychology of
- Student Success in College
- Subjective Wellbeing Homeostasis
- Suicide
- Taste, Psychological Perspectives on
- Teaching of Psychology
- Terror Management Theory
- Testing and Assessment
- The Concept of Validity in Psychological Assessment
- The Neuroscience of Emotion Regulation
- The Reasoned Action Approach and the Theories of Reasoned ...
- The Weapon Focus Effect in Eyewitness Memory
- Theory of Mind
- Therapy, Cognitive-Behavioral
- Thinking Skills in Educational Settings
- Time Perception
- Trait Perspective
- Trauma Psychology
- Twin Studies
- Type A Behavior Pattern (Coronary Prone Personality)
- Unconscious Processes
- Video Games and Violent Content
- Virtues and Character Strengths
- Wisdom
- Women and Science, Technology, Engineering, and Math (STEM...
- Women, Psychology of
- Work Well-Being
- Workforce Training Evaluation
- Wundt, Wilhelm