Statistical Assumptions

by

Vincent Reitano

• LAST REVIEWED: 23 August 2018
• LAST MODIFIED: 28 September 2016
• DOI: 10.1093/obo/9780199756810-0165

Introduction

Statistical assumptions underlie valid inference in education research. All statistical procedures have underlying assumptions, some more stringent than others. In some cases, violations of these assumptions will not change substantive research conclusions. In other cases, violations of assumptions will undermine meaningful research. Establishing that one’s data meet the assumptions of the procedure one is using is an expected component of all quantitatively based research. To ensure valid inference using a statistical procedure, sound education research requires that all statistical assumptions be met as a part of quasi-experimental and experimental research designs. However, some statistical procedures are robust to violations of statistical assumptions. Still, fulfillment of statistical assumptions in education research is increasingly recognized to ensure that valid conclusions are reached, which generalize to the sample being studied (or to be aware that generalization is not warranted). As the range of statistical procedures is great, researchers should look to the corresponding range of literature on the theory of statistical assumptions for any given procedure. Further, researchers should review literature on the application of statistical procedures in research guidelines released by educational research agencies, such as the Institute of Educational Sciences. Currently, the gold standard for educational research is experimental research design, which requires a different set of statistical assumptions than the more common observational or quasi-experimental designs.

General Overviews

Statistical assumptions are critical to conducting valid inference on a sample. Typical statistical assumptions of hypothesis testing include normality, linearity, exogeneity, homoskedasticity, and correct model specification, as noted by Garson 2014. For a discussion of these assumptions in the context of regression modeling, which is a common technique in education research, see Berry 1993. For a broader discussion of assumptions beyond the linear regression model, a comprehensive reference overview of the various statistical assumptions underlying parametric and semi-parametric statistical techniques can be found in the work of David Sheskin (See Sheskin 2011). In the case of violations of statistical assumptions, Wilcox 2010 provides a thorough discussion of modern robust statistical techniques for conducting inference on a sample. A more advanced theoretical exposition regarding consequences of violations of assumptions addressed with the bootstrap and maximum likelihood methods can be found in Freedman 2009. For a general discussion of how to approach violations of statistical assumptions and how to conduct statistical inference in an applied setting, see van Belle 2011. Additionally, once the researcher has determined that possible violations of statistical assumptions are addressed, it may be possible to generalize findings beyond the sample being studied, or to make causal assertions, depending on the sample being studied and selected methodology. Experimental and quasi-experimental methodologies, which have distinct assumptions from observational studies in regard to randomization, may permit generalization and causal inference. A thorough and modern discussion of the methodologies and assumptions that are used for causal inference in the social sciences is provided by Imbens and Rubin 2015. For a discussion of design-based and statistical assumptions requisite to causal inference in education research, see Murnane and Willett 2010.

• Berry, William D. 1993. Understanding regression assumptions. Sage University Paper Series on Quantitative Applications in the Social Sciences 07–092. Newbury Park, CA: SAGE.

  DOI: 10.4135/9781412986427

  A concise primer on the theory and application of testing for statistical assumptions in the regression context. Emphasis is placed on the substantive meaning of meeting regression assumptions in an applied setting.

• Freedman, David A. 2009. Statistical models: Theory and practice. 2d ed. New York: Cambridge Univ. Press.

  DOI: 10.1017/CBO9780511815867

  Comprehensive graduate-level discussion of statistical assumptions and common methods such as regression and path analysis with matrix algebra. Chapters on maximum likelihood and bootstrap methods will prove relevant to social science researchers seeking to understand the role of the asymptotic normality assumption.

• Garson, G. David. 2014. Multiple regression. Asheboro, NC: Statistical Associates.

  An introductory graduate-level text that covers application of multiple regression methodology in various statistical packages, such as SAS, Stata, and SPSS. Includes a comprehensive overview of statistical assumptions underlying multiple regression motivated by numerous applied social science datasets.

• Imbens, G. W., and Donald B. Rubin. 2015. Causal inference for statistics, social, and biomedical sciences: An introduction. New York: Cambridge Univ. Press.

  DOI: 10.1017/CBO9781139025751

  A new but seminal text that develops the theory of causal inference as applied in various domains of social and natural science research. Extensive discussion of the classical randomized experiment design from the research design phase to statistical analysis. Discussion of causality is motivated by a discussion spanning Fisher and Rubin to the assumptions underlying application of modern statistical analysis to conduct valid inference.

• Murnane, Richard J., and John B. Willett. 2010. Methods matter: Improving causal inference in educational and social science research. New York: Oxford Univ. Press.

  Clear discussion of the steps in conducting causal inference in education and the social sciences, with strong introductory sections on assumptions in the design stage and subsequent statistical analysis. Nontechnical applied exposition on various aspects and assumptions of experimental, quasi-experimental designs, and observational designs, including randomized controlled trials and regression discontinuity designs.

• Sheskin, David J. 2011. Handbook of parametric and nonparametric statistical procedures. 5th ed. New York: Chapman and Hall/CRC.

  A seminal reference text for applied statistics that encompasses theory, application, and interpretation of virtually all parametric and nonparametric statistical tests underlying much of modern social science inference today. Brief introductory chapter provides an overview of various subsets of statistical analysis, which is followed by numerous chapters on single sample, multiple sample, factorial design, correlational, and multivariate statistical procedures.

• van Belle, Gerald. 2011. Statistical rules of thumb. 2d ed. Hoboken, NJ: John Wiley.

  Provides over one hundred formal and informal suggestions regarding statistical assumptions, analysis, and interpretation in an applied setting. Chapters on sample size, observational studies, and covariation prove relevant to the modern education researcher, in addition to concluding chapters on how to present statistical analysis and conduct professional statistical consulting.

• Wilcox, Rand R. 2010. Fundamentals of modern statistical methods: Substantially improving power and accuracy. 2d ed. New York: Springer.

  DOI: 10.1007/978-1-4419-5525-8

  A text that delves into theoretical and practical considerations of applied statistical methodology, from discussion of distributions and hypothesis testing to conducting robust multivariate analysis. Particular emphasis is placed on robust methods, which are key to reaching valid inference in the social sciences in relation to statistical power.