In This Article Expand or collapse the "in this article" section Sample Size Planning for Statistical Power and Accurate Estimation

  • Introduction
  • Books and Book Chapters
  • Software
  • Effect Size: An Ingredient of Sample Size Planning
  • Sample Size and Low Statistical Power in Published Research
  • Sample Size Planning for Statistical Power: Using a Hypothetical Population Effect Size
  • Sample Size Planning for Statistical Power: Using Prior Information
  • Sample Size Planning for Accuracy: Obtaining Narrow Confidence Intervals
  • Bayesian Approaches to Sample Size Planning
  • Sample Size Planning for Equivalence Testing
  • Sequential Methods for Sample Size Planning
  • Sample Size Guidelines for Specialized Designs: Mediation, Moderation, and SEM
  • Sample Size Guidelines for Specialized Designs: Other Designs
  • Additional Consequences of Poor Choice of Sample Size

Psychology Sample Size Planning for Statistical Power and Accurate Estimation
by
Samantha F. Anderson, Sophia J. Lamp
  • LAST REVIEWED: 27 October 2022
  • LAST MODIFIED: 27 October 2022
  • DOI: 10.1093/obo/9780199828340-0296

Introduction

At the most general level, sample size refers to the number of participants (typically human or animal subjects) who provide data for a scientific experiment or study. Selecting an appropriate sample size is a fundamental feature of a well-designed study, and, as the title suggests, sample size can be carefully planned ahead of time. Sample size planning is important because the sample size of a study has implications for statistical power as well as accuracy. In conventional terms, statistical power reflects the probability of rejecting the null hypothesis, assuming that the purported effect is non-null (non-zero) in reality. Although power can be treated as a function of unknown parameter values, power is often defined conditionally on a specific value of population effect size. (On a technical level, power is conditional upon the population noncentrality parameter, which is a combination of sample size and effect size, but relying on effect size as a proxy is generally acceptable.) Power is calculated from the statistical significance threshold (otherwise known as the nominal Type I error rate or alpha-level), the population effect size, and the sample size. Because of the connection between sample size and power, sample size planning is sometimes called a priori power analysis. For example, if a new therapy developed to treat depression is truly beneficial, the sample size of a study assessing this therapy should be large enough to demonstrate the beneficial outcome as statistically significant. Accuracy reflects how close an estimate is to the parameter it aims to estimate. Accurate estimates are expressed with narrow confidence intervals (small margins of error). Sample size also has a connection to accuracy. For example, if the investigators in the previous example want to estimate how much the therapy reduces depressive symptoms, the sample size should be large enough so that the effect size estimate derived from the study is an accurate estimate of the true magnitude of the treatment effect. An underlying challenge to sample size planning for both goals is that the population effect size is unknown. An important corollary is that whether a particular sample size provides appropriate power (and in some cases, accuracy) depends on the population effect size, and as such, sample size cannot completely be judged without context. Although there are sometimes limitations to sample size, such as when working with a limited budget or specialized populations, and although sample size is often selected based on convenience or historical precedent, when possible it is desirable to directly plan the study sample size at the study design phase. Explicitly planning sample size helps to ensure that the sample size will be effective in meeting the investigators’ goals and answering the scientific questions of interest.

Books and Book Chapters

Given its important role in the design of an effective research study, several books and chapters have been devoted to sample size planning and/or power analysis. Although full books on sample size are more commonly written for biomedical researchers and clinical trial designs, the selected books and chapters cited here offer excellent introductions to sample size planning geared toward psychologists or serve as thorough reference texts. Cohen 1988 is considered by many to be the quintessential text on statistical power analysis in the psychological and behavioral sciences, containing sample size tables and formulas for numerous designs and effect sizes. For readers looking for a thinner, but reader-friendly, alternative, Kraemer and Blasey 2016 carefully contextualizes, describes, and demonstrates sample size planning for designs of interest to psychologists. Despite its focus on biomedical research, Chow, et al. 2008 represents a more technical resource for those interested in equivalence and non-inferiority testing, in addition to general two-group or crossover designs common in clinical and applied research. Lipsey 1990 also focuses more on applied contexts, but provides a broader context for sample size planning as a part of good study design. Kelley and Maxwell 2012, a chapter rather than a full-length text, provides an accessible introduction to sample size planning, multiple intuitive and carefully worked through examples, and strategies for planning for power and accuracy.

  • Chow, S.-C., J. Shao, and H. Wang. 2008. Sample size calculations in clinical research. 2d ed., Vol. 11. Boca Raton, FL: CRC Press.

    DOI: 10.1201/9780203911341

    Although geared slightly more toward researchers in the biomedical sciences, represents a thorough guide to sample size formulas for a variety of designs common to clinical and experimental domains, including tests of mean differences, proportions, and survival analysis. Contains formulas for conducting equivalence testing as well as a section on Bayesian sample size planning.

  • Cohen, J. 1988. Statistical power analysis for the behavioral sciences. 2d ed. Hillsdale, NJ: Erlbaum.

    Considered by many to be the fundamental text on sample size and statistical power. Contains formulas for calculating statistical power and tables of necessary sample sizes for an array of experimental designs. Additionally, includes information and formulas for calculating various effect sizes needed for sample size calculations.

  • Kelley, K., and S. E. Maxwell. 2012. Sample size. In APA handbook of research methods in psychology. Vol. 1. Edited by H. Cooper, 181–202. Washington, DC: American Psychological Association.

    Thorough overview of the basic issues and processes involved in sample size planning, written for a general audience. Covers sample size planning for statistical power and sample size planning for accuracy, and includes several step-by-step examples using freely available software.

  • Kraemer, H. C., and C. M. Blasey. 2016. How many subjects? Statistical power analysis in research. 2d ed. Thousand Oaks, CA: SAGE.

    DOI: 10.4135/9781483398761

    A concise and well-written guide to sample size planning for the ANOVA and regression designs of interest to psychologists. Includes a thoughtful introduction to hypothesis testing to better contextualize the role of sample size planning for power. Uses the same example topic multiple times throughout to illustrate sample size planning for different questions and designs.

  • Lipsey, M. W. 1990. Design sensitivity: Statistical power for experimental research. Newbury Park, CA: SAGE.

    With a focus on applied treatment research, this work focuses on the broader issue of designing studies that are sensitive to detect the effect of interest, a goal for which sample size and statistical power are fundamental features. Includes an entire chapter focusing on the ways in which effect size can be approached with regard to statistical power.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login.

How to Subscribe

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here.

Article

Up

Down