Meta-Analysis and Research Synthesis in Education

by

Harsh Suri, John Hattie

• LAST REVIEWED: 24 July 2013
• LAST MODIFIED: 24 July 2013
• DOI: 10.1093/obo/9780199756810-0091

Introduction

This bibliography begins with this introduction followed by a general overview of meta-analysis and research synthesis methods in education. The third section cites references to illustrate how the concept of statistical integration of research findings dates back to early 20th century. Citations in the fourth section highlight early calls from educational researchers to recognize the process of synthesizing research as a scholarly endeavor in its own right. As evident from the citations in the fifth section, it was in the 1980s when monographs exclusively devoted to research synthesis methods started to be published. Since then, a number of books have been published on research synthesis methods. The sixth section cites some of the most comprehensive books on research synthesis methods and includes contributions from key players of meta-analysis current at the time. The citations in the seventh section illustrate how meta-analysis has become very popular over time. Several large organizations have been set up and software developed to support systematic reviews of research. These are cited in the eighth section. Citations in the ninth section illustrate critiques of systematic reviews. As qualitative research is becoming popular in education, sophisticated discussions of issues associated with synthesizing qualitative research have also been published, some of which are cited in the tenth section. The eleventh section presents a methodologically inclusive account of current developments in research synthesis methods. The final section cites examples of journals exclusively devoted to publishing research reviews and exemplary research synthesis with different methodologies. Individual methods of research synthesis are discussed chronologically as they became popular in educational research. Accordingly, this bibliography starts with a discussion of statistical methods of integrating research that parallels the dominance of quantitative research in education until the 1970s. As the popularity and diversity of qualitative research methods have been increasing in educational research, more methodologically inclusive discussions of research synthesis methods are becoming popular as described in the later sections of this bibliography.

General Overview

Suri and Clarke 2009 comprehensively maps how meta-analysis and research synthesis methods have advanced in education. As noted in most texts on meta-analysis (many of which are listed in later sections), the authors observe that meta-analysts have played an important role in formalizing the methodology of research synthesis. To facilitate comparisons across studies, meta-analysis has allowed the bringing together of many seemingly diverse findings from individual primary research studies onto a common metric called an effect size. After appropriate statistical adjustments of individual effect sizes according to their respective study features, a cumulative effect size can be calculated that is indicative of the direction and the magnitude of the overall trend of findings. This is followed by tests to identify substantive, methodological, or contextual study features that may be potential moderators of the effect size. Meta-analysts have systematized the entire process of research synthesis by identifying the main tasks in each phase of a research synthesis, highlighting critical decision points within each phase, and allowing discussion of the relative merits of different choices at each decision point. They advocate explicit statement and justification of the decisions made at each stage of the research synthesis from hypothesis formulation, data selection, evaluation, analysis, and interpretation to public presentation. There are several types of sensitivity analyses that can examine the dependence of the findings on the assumptions made about the nature of the data. Over the past three decades, meta-analysts have conducted numerous investigations to examine the robustness of their techniques and have explored ways of refining these techniques, as well as examining many substantive uses in the field of education. Meta-analysis now has become but one (important) method in integrative research synthesis. As a large proportion of primary research in education does not lend itself to a meta-analysis, there is a growing interest in methodologically inclusive discussions of quantitative and qualitative research synthesis in education. The field is now rich in methods, debates, and uses of research synthesis of primary literature. It is further continuing to become more diverse as more researchers are drawing insights from interpretive, participatory, and critical traditions to the process of synthesizing research.

• Suri, Harsh, and David Clarke. 2009. Advancements in research synthesis methods: From a methodologically inclusive perspective. Review of Educational Research 79.1: 395–430.

  DOI: 10.3102/0034654308326349

  In addition to summarizing the key advancements in research synthesis methods from different methodological persuasions, the authors also identify suitable domains of applicability and present common critiques raised over these developments. They deliberately highlight the contributions that are less frequently cited in the dominant literature on meta-analysis. Available online for purchase or by subscription.