Meta-Analysis

by

Larry V. Hedges

• LAST REVIEWED: 27 July 2016
• LAST MODIFIED: 27 July 2016
• DOI: 10.1093/obo/9780199828340-0181

Introduction

Meta-analysis is the use of statistical methods to combine evidence across studies for the purposes of drawing general conclusions. The results of each study are summarized by a quantitative index called an “effect size,” and then the effect sizes from each study are combined across studies to obtain an overall summary effect. Meta-analysis differs from secondary analysis because the latter involves analysis of the raw data from each study, but meta-analysis typically uses only summaries of the raw data from each study (the effect sizes), not the raw data itself. Meta-analyses may have any of several primary objectives. In many cases, meta-analyses are carried out to obtain a summary of average effects. In other cases, they may be carried out to study differential effects of a treatment on different subject populations, different treatment contexts, different variations of a treatment, or different outcome measures. It is also common for meta-analysts to use their results to identify areas of research that not been the object of sufficient study, for example, units or contexts that have gone unstudied. In some cases meta-analyses are carried out to determine whether there is variation of results across studies that are carried out in different contexts.

General Overviews

There are now many general references on meta-analysis. Cooper, et al. 2009 provides detailed coverage of both statistical and non-statistical aspects of systematic research synthesis, with chapters authored by leading scholars in their respective areas. Higgins and Green 2011 provides another comprehensive introduction to systematics reviews and meta-analysis that is more focused on health sciences research. This reference was designed to provide procedures for authors of reviews and meta-analyses carried out as part of the Cochrane Collaboration in medicine, and therefore some of its detailed guidelines are not applicable to meta-analyses in psychology and the social sciences. An important application of meta-analysis is in the study of validity generalization in personnel psychology. A comprehensive coverage of meta-analysis for validity generalization is given by Hunter and Schmidt 2004. Pigott 2012 offers particularly strong coverage of more advanced topics, such as power analysis in meta-analysis, dealing with missing data, and individual participant meta-analyses. Hartung, et al. 2008 addresses some more technical issues in meta-analysis but may be too focused and technical for general readers.

• Cooper, Harris, Larry V. Hedges, and Jeffrey C. Valentine. 2009. The handbook of research synthesis and meta-analysis. New York: Russell Sage.

  A comprehensive reference on the entire systematic review process. It is best used to investigate particular topics of interest. Chapters share a common notation and terminology but have separate bibliographies and can stand alone.

• Hartung, Joachim, Guido Knapp, and Bimal K. Sinha. 2008. Statistical meta-analysis with applications. Hoboken, NJ: John Wiley.

  DOI: 10.1002/9780470386347

  A more statistical reference. This book is most useful to the reader with greater methodological sophistication. It is less comprehensive than other overviews listed here but thorough in the topics covered.

• Higgins, Julian P. T., and Sally Green, eds. 2011. Cochrane handbook for systematic reviews of interventions. (Version 5.1.0 [updated March 2011]). Cochrane Collaboration

  Designed to support Cochrane Collaboration systematic reviews in medicine but has much useful content for psychologists and social scientists. The primary focus is on meta-analyses of randomized controlled trials but is useful when considering meta-analyses of non-randomized studies.

• Hunter, John E., and Frank L. Schmidt. 2004. Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: SAGE.

  Comprehensive reference on meta-analyses of validity studies (so-called validity generalization). It provides a guide to the methods for estimating effect sizes that are attenuated due to methodological artifacts (e.g., sample range restrictions, unreliability of measures) and the terminology used in validity generalization.

• Pigott, Terri. 2012. Advances in meta-analysis. New York: Springer Science and Business Media.

  DOI: 10.1007/978-1-4614-2278-5

  Another advanced reference but presents the topics of power analysis, missing data, and individual participant meta-analyses clearly.