Meta-analysis

by

Paul Montgomery, Matthew Morton

• LAST REVIEWED: 28 April 2017
• LAST MODIFIED: 14 December 2009
• DOI: 10.1093/obo/9780195389678-0054

Introduction

This article organizes a wide range of selected resources for understanding and executing meta-analyses. Meta-analysis refers to “statistical techniques used to analyse and synthesise results of multiple studies on the same topic” (see Introductory Works, Littell, et al. 2008, pp. 176). Although meta-analysis encompasses the statistical aspects of research synthesis, the process is situated within other processes involving the selection and inclusion of studies as well as the reporting and interpretation of results. In particular meta-analysis should generally be conducted within the context of a systematic review in order to ensure that the process of gathering and appraising data for meta-analysis is as credible and transparent as possible. Therefore this bibliography also provides resources related to bias, assessing study quality, unpublished data, and other nonquantitative aspects of research synthesis that have direct implications for meta-analysis.

Introductory Works

This section highlights a few of the many articles and texts that have been written in recent decades in an effort to introduce the concept, principles, and procedures of meta-analysis to a wider audience. Together the British Medical Journal’s “Meta-Analysis: Education and Debate” series on meta-analysis (Egger, et al. 1998) may provide the most concise and Internet accessible overview introduction on the topic. The Littell, et al. 2008 pocket guide text should also be prioritized; it is the most up-to-date introductory text available on meta-analysis, is nontechnical, and helps readers understand how meta-analysis and systematic reviews are unique but complementary endeavors. The Cooper, et al. 2000 and Schmidt and Hunter 2003 chapters offer nonmathematical overviews of how meta-analysis has developed over time, controversies involved, and key principles and language regarding statistical procedures. These chapters are written for specific research areas (education and psychology, respectively) but would be useful for a more general audience. Greenhalgh 1997 helps guide nonexperts in making sense of meta-analytic studies and appraising the quality of the paper. Though targeted toward a medical audience, Chaturvedi, et al. 2007 and Zwahlen, et al. 2008 effectively illustrate key limitations argued of meta-analyses when not conducted or presented appropriately.

• Chaturvedi, Nishi, Rudy Bilous, Rebecca Hardy, Giuseppe Remuzzi, Piero Ruggenenti, and Giancarlo C. Viberti. 2007. Misleading meta-analysis: A need to look beyond the headlines. Diabetic Medicine 24 (6): 587–591.

  DOI: 10.1111/j.1464-5491.2007.02139.x

  Using a meta-analytic example from medicine, the authors illustrate the limitations of meta-analysis when executed or presented inadequately and provide guidance for overcoming, or at least being transparent with, these issues.

• Cooper, H. M., J. C. Valentine, and K. Charlton. 2000. The methodology of meta-analysis. In Contemporary Special Education Research: Syntheses of the Knowledge Base on Critical Instructional Issues. Edited by Russell Gersten, Ellen P. Schiller, and Sharon Vaughn, 263–281. Mahwah, NJ: Lawrence Erlbaum.

  This chapter provides a brief overview of the history of meta-analysis, how methods and approaches have developed over time, and key concepts for understanding meta-analysis.

• Cwikel, Julie, Lynn Behar, and June Rabson-Hare. 2000. A comparison of a vote count and a meta-analysis review of intervention research with adult cancer patients. Research on Social Work Practice 10 1 (1): 139–158.

  Within a social work context, the authors explore the merits and limitations of meta-analysis as well as an alternative research synthesis procedure (vote counting) and compare the two approaches for social workers and social work researchers. A research example of a care intervention for adult cancer patients is used for demonstration.

• Egger, Matthias, et al. 1998. Meta-Analysis: Education and Debate.

  During 1997–1998 the British Medical Journal printed a series of six articles, all written by Matthias Egger along with other colleagues, examining the procedures in conducting reliable meta-analysis in medical research. The articles are available free online and together provide a useful introductory guide through the major issues involved with meta-analysis. One complementary article also provides a review of meta-analysis software packages, but given the rapid advances of computer technology, this review is now largely outdated.

• Greenhalgh, Trisha. 1997. How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses). British Medical Journal 315: 672–675.

  The How to Read a Paper series of ten articles is geared toward helping nonexperts appraise the quality of studies—in this case, systematic reviews and meta-analyses.

• Littell, Julia H., Jacqueline Corcoran, and Vijayan Pillai. 2008. Systematic reviews and meta-analysis. Oxford: Oxford Univ. Press.

  DOI: 10.1093/acprof:oso/9780195326543.001.0001

  This pocket guide text provides a step-by-step introduction to the process of research synthesis using systematic review and meta-analysis that employs specific social work examples and is geared toward students and researchers alike.

• Lundahl, Brad, and Joanne Yaffe. 2007. Use of meta-analysis in social work and allied disciplines. Journal of Social Service Research 33:1–11.

  DOI: 10.1300/J079v33n03_01

  This study analyzes publication patterns of meta-analyses across five disciplines—family studies, nursing, psychology, psychiatry, and social work—and finds that in all disciplines meta-analyses are rising, though social work journals have done much less publishing and commenting on meta-analyses. Implications and recommendations are discussed for improving effective use of meta-analysis in the social work context.

• Schmidt, Frank L., and John E. Hunter. 2003. Meta-analysis. In Handbook of Psychology. Vol. 2: Research Methods in Psychology. Editor in chief Irving B. Weiner, volume editors John A. Schinka and Wayne F. Velicer, 533–554. New York: Wiley.

  Provides a succinct overview of the purpose and methods behind meta-analysis and discusses the application of meta-analysis to the field of psychology.

• Zwahlen, Marcel, Andrew Renehan, and Matthias Egger. 2008. Meta-analysis in medical research: Potentials and limitations. Urologic Oncology: Seminars and Original Investigations 26 (3): 320–329.

  DOI: 10.1016/j.urolonc.2006.12.001

  Though geared toward a medical audience, this article provides an updated overview of common forms of bias and quality issues that can compromise the value and validity of meta-analyses. The authors argue that meta-analyses should generally be conducted in the context of a systematic review to try to avoid these limitations as much as possible.