Experiments

by

Josh Pasek, Sun Joo (Grace) Ahn

• LAST REVIEWED: 30 September 2013
• LAST MODIFIED: 30 September 2013
• DOI: 10.1093/obo/9780199756841-0138

Introduction

Experimentation is one of the central scientific techniques for understanding the world. Although researchers often have ideas about how the things they study are related, experiments make it possible to test the relations between a proposed “cause” and its expected “effect.” By manipulating purported causes and evaluating the effects, communication researchers have learned enormous amounts about the roles media content and presentation play in citizens’ attitudes and behaviors. Early experiments in communication focused on understanding the influence of propaganda during the Second World War. From that time, the use of communication experiments has ballooned from a technique primarily focused on assessing media effects in laboratory environments to a broadly used set of strategies exploring factors as diverse as racial attitudes, voter turnout, health information, and the violation of social norms. Of course, no research method is without its limitations and flaws; many problems are ill-suited for experimental methods and many attempts to induce experimental design fall short of answering the questions researchers are asking. This article attempts to showcase both the virtues and the limitations of experiments. It is hoped that the references cited will help to illuminate the power of good designs and of solid questions while exposing the potential pitfalls of the experimental framework. To this end, the references presented serve as sources for good experimental design and analysis, illustrate ways to improve experimental techniques, and contrast experiments with other inferential methods, showing where each can provide useful scientific insights.

General Overviews

General reference books on experimentation vary considerably in their focus on statistical methods, their target audience, and their comprehensiveness. Researchers looking to design experiments should probably start with Shadish, et al. 2001, which provides the most comprehensive overview of a large range of designs and accompanying assumptions, or Brown and Melamed 1990, a shorter reference text. Those interested in a more complete statistical treatment of experimentation would be well served to explore Keppel and Wickens 2004 or Ryan 2007, both of which could serve as graduate textbooks. Those interested in teaching undergraduates might find the general research methods text Babbie 2013 useful; another good undergraduate-level general text is Frey, et al. 1991. The more experimentally focused Harris 2008 may prove useful for undergraduates running, and writing up, their own experiments. Druckman, et al. 2011 provides a strong introduction for researchers interested in understanding the variety of experimental approaches.

• Babbie, E. R. 2013. The basics of social research. 5th ed. Belmont, CA: Wadsworth/Cengage Learning.

  This undergraduate textbook provides a straightforward basic overview of a variety of social research methods. The text provides a good introduction to experiments and various experimental concepts as well as alternative methods for data collection.

• Brown, S. R., and L. E. Melamed. 1990. Experimental design and analysis. Newbury Park, CA: SAGE.

  Similar to many of the SAGE series on quantitative applications in the social sciences, this little green book provides a pithy overview of simple analytical techniques for experiments and a handful of true experimental designs.

• Druckman, J. N., D. P. Green, J. H. Kuklinski, and A. Lupia, eds. 2011. Cambridge handbook of experimental political science. New York: Cambridge Univ. Press.

  DOI: 10.1017/CBO9780511921452

  Although targeted for political scientists, this reference text provides chapters reviewing a variety of considerations when designing and analyzing experiments, with considerable focus on nonlaboratory experimental designs. A number of creative applications of experimentation are also showcased, including some from political communication.

• Frey, L. R., C. H. Botan, P. G. Friedman, and G. L. Krepa. 1991. Investigating communication: An introduction to research methods. Englewood Cliffs, NJ: Prentice Hall.

  This general textbook introducing both qualitative and quantitative communication research methods is well tooled for an undergraduate class. Although the section on experiments is not large, the text is well pitched for an introductory research methods class.

• Harris, P. 2008. Designing and reporting experiments in psychology. 3d ed. New York: Open Univ. Press.

  Describing experimental design and running simple experiments are the focus of this how-to manual of an undergraduate textbook. It is particularly useful for undergraduates engaging in experimental research during the course of a term.

• Keppel, G., and T. D. Wickens. 2004. Design and analysis: A researcher’s handbook. 4th ed. Upper Saddle River, NJ: Pearson Prentice Hall.

  This textbook focuses largely on the statistical analysis of, and assumptions behind, various experimental designs. The book is well suited to a graduate course or can be used as a general reference text.

• Ryan, T. P. 2007. Modern experimental design. Hoboken, NJ: Wiley-Interscience.

  DOI: 10.1002/0470074353

  In looking across a variety of disciplines in both the natural and the social sciences, this textbook focuses on the statistical principles in both designing and analyzing experiments. It is useful as a reference text or graduate-level textbook, although the author does not discuss the many particular challenges that social scientists can encounter.

• Shadish, W. R., T. D. Cook, and D. T. Campbell. 2001. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

  Unmatched as a reference text or graduate textbook; the authors carefully illustrate many of the choices in experimental design, the inferences that can be made with various design decisions, and considerations for researchers using all types of experiments.