The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences Volume 1: Building a Program of Research
Book contents
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
- Cambridge Handbooks in Psychology
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
- Copyright page
- Dedication
- Contents
- Figures
- Tables
- Contributors
- Preface
- Part I From Idea to Reality: The Basics of Research
- Part II The Building Blocks of a Study
- Part III Data Collection
- 13 Cross-Sectional Studies
- 14 Quasi-Experimental Research
- 15 Non-equivalent Control Group Pretest–Posttest Design in Social and Behavioral Research
- 16 Experimental Methods
- 17 Longitudinal Research: A World to Explore
- 18 Online Research Methods
- 19 Archival Data
- 20 Qualitative Research Design
- Part IV Statistical Approaches
- Part V Tips for a Successful Research Career
- Index
- References
14 - Quasi-Experimental Research
from Part III - Data Collection
Published online by Cambridge University Press: 25 May 2023
Book contents
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
- Cambridge Handbooks in Psychology
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
- Copyright page
- Dedication
- Contents
- Figures
- Tables
- Contributors
- Preface
- Part I From Idea to Reality: The Basics of Research
- Part II The Building Blocks of a Study
- Part III Data Collection
- 13 Cross-Sectional Studies
- 14 Quasi-Experimental Research
- 15 Non-equivalent Control Group Pretest–Posttest Design in Social and Behavioral Research
- 16 Experimental Methods
- 17 Longitudinal Research: A World to Explore
- 18 Online Research Methods
- 19 Archival Data
- 20 Qualitative Research Design
- Part IV Statistical Approaches
- Part V Tips for a Successful Research Career
- Index
- References
Summary
In this chapter, we discuss the logic and practice of quasi-experimentation. Specifically, we describe four quasi-experimental designs – one-group pretest–posttest designs, non-equivalent group designs, regression discontinuity designs, and interrupted time-series designs – and their statistical analyses in detail. Both simple quasi-experimental designs and embellishments of these simple designs are presented. Potential threats to internal validity are illustrated along with means of addressing their potentially biasing effects so that these effects can be minimized. In contrast to quasi-experiments, randomized experiments are often thought to be the gold standard when estimating the effects of treatment interventions. However, circumstances frequently arise where quasi-experiments can usefully supplement randomized experiments or when quasi-experiments can fruitfully be used in place of randomized experiments. Researchers need to appreciate the relative strengths and weaknesses of the various quasi-experiments so they can choose among pre-specified designs or craft their own unique quasi-experiments.
Keywords
- Type
- Chapter
- Information
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral SciencesVolume 1: Building a Program of Research, pp. 292 - 313Publisher: Cambridge University PressPrint publication year: 2023
References
Aiken, L. S., West, S. G., Schwalm, D. E., Carroll, J. L., & Hsiung, S. (1998). Comparison of a randomized and two quasi-experimental designs in a single outcome evaluation: Efficacy of a university-level remedial writing program. Evaluation Review, 22(2), 207–244.Google Scholar
Angrist, J. D. & Pischke, J-S. (2015). Mastering ‘Metrics: The Path from Cause to Effect. Princeton University Press.Google Scholar
Arum, R. & Roksa, J. (2010). Academically Adrift: Limited Learning on College Campuses. University of Chicago Press.CrossRefGoogle Scholar
Berk, R., Barnes, G., Ahlman, L., & Kurtz, E. (2010). When second best is good enough: A comparison between a true experiment and a regression discontinuity quasi-experiment. Journal of Experimental Criminology, 6(2), 191–208.CrossRefGoogle Scholar
Bloom, H. S. (2003). Using “short” interrupted time-series analysis to measure the impacts of whole-school reforms: With application to a study of accelerated schools. Evaluation Review, 27(1), 3–49.CrossRefGoogle Scholar
Braden, J. P. & Bryant, T. J. (1990). Regression discontinuity designs: Applications for school psychologists. School Psychology Review, 19(2), 232–239.Google Scholar
Cook, T. D. (2008). “Waiting for life to arrive”: A history of the regression–discontinuity designs in psychology, statistics and economics. Journal of Econometrics, 142(2), 636–654.Google Scholar
Cook, T. D., Shadish, W. R., & Wong, V. C. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. Journal of Policy Analysis and Management, 27(4), 724–750.CrossRefGoogle Scholar
Cook, T. D., Steiner, P. M., & Pohl, S. (2009). Assessing how bias reduction is influenced by covariate choice, unreliability and data analysis mode: An analysis of different kinds of within-study comparisons in different substantive domains. Multivariate Behavioral Research, 44(6), 828–847.Google Scholar
Eckert, W. A. (2000). Situational enhancement of design validity: The case of training evaluation at the World Bank Institute. American Journal of Evaluation, 21(2) 185–193.Google Scholar
Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16(5), 319–324.CrossRefGoogle ScholarPubMed
Goldberger, A. S. (2008). Selection bias in evaluation treatment effects: Some formal illustrations. In Fomby, T., Hill, R. C., Millimet, D. L., Smith, J. A., & Vytlacil, E. J. (eds.), Modeling and Evaluating Treatment Effects in Economics (pp. 1–31). JAI Press.Google Scholar
Goplan, M., Rosinger, K., & Ahn, J. B. (2020). Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education, 44(1), 218–243.Google Scholar
Heinsman, D. T. & Shadish, W. R. (1996). Assignment methods in experimentation: When do nonrandomized experiments approximate answers from randomized experiments? Psychological Methods, 1(2), 154–169.Google Scholar
Henry, G. T., Fortner, C. K., & Thompson, C. L. (2010). Targeted funding for educationally disadvantaged students: A regression discontinuity estimate of the impact on high school student achievement. Educational Evaluation and Policy Analysis, 32(2), 183–204.CrossRefGoogle Scholar
Henry, G. T. & Harbatkin, E. (2020). The next generation of state reforms to improve their lowest performing schools: An evaluation of North Carolina’s school transformation intervention. Journal of Research on Educational Effectiveness, 13(4), 702–730.CrossRefGoogle Scholar
Hudson, J., Fielding, S., & Ramsay, C.R. (2019). Methodology and reporting characteristics of studies using interrupted time series design in healthcare. BMC Medical Research Methodology, 19(1), 137.CrossRefGoogle ScholarPubMed
Imbens, G. W. & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice, Journal of Econometrics, 142(2), 615–635.Google Scholar
Jacob, R., Zhu, P., Somers, M-A., & Bloom, H. (2012). A Practical Guide to Regression Discontinuity. Manpower Demonstration Research Corporation.Google Scholar
Kazden, A. E. (2011). Single-Case Research Designs: Methods for Clinical and Applied Settings, 2nd ed. Oxford University Press.Google Scholar
Lee, D. S. & Lemieux, T. (2010). Regression discontinuity designs in economics. Journal of Economic Literature, 48(2), 281–355.Google Scholar
Lehman, D. R., Lempert, R. O., & Nisbett, R. E. (1988). The effects of graduate training on reasoning: Formal discipline and thinking about everyday-life events. American Psychologist, 43(6), 431–442.Google Scholar
Lipsey, M.W., Cordray, D.S., & Berger, D.E. (1981). Evaluation of a juvenile diversion program: Using multiple lines of evidence. Evaluation Review, 5(3), 283–306.CrossRefGoogle Scholar
Mark, M. M. & Mellor, S. (1991). The effect of self-relevance of an event on hindsight bias: The foreseeability of a layoff. Journal of Applied Psychology, 76(4), 569–577.CrossRefGoogle Scholar
Matthews, M. S., Peters, S. J., & Housand, A. M. (2012). Regression discontinuity design in gifted and talented education research. Gifted Child Quarterly, 56(2), 105–112.Google Scholar
McCleary, R. & McDowall, D. (2012). Time-series designs. In Cooper, H., Camic, P. M., Long, D. L., et al. (eds.), APA Handbook of Research Methods in Psychology, Volume 2. Research Designs: Quantitative, Qualitative, Neuropsychological, and Biological (pp. 613–627). American Psychological Association.Google Scholar
McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 698–714.Google Scholar
Palmgreen, P. (2009) Interrupted time-series designs for evaluating health communication campaigns. Communication Methods and Measures, 3(1–2), 29–46.Google Scholar
Paluck, E. L. & Green, D. P. (2009). Prejudice reduction: What works? A review and assessment of research practice. Annual Review of Psychology, 60, 339–367.CrossRefGoogle Scholar
Reichardt, C. S. (2019). Quasi-Experimentation: A Guide to Design and Analysis. Guilford Press.Google Scholar
Reynolds, K. D. & West, S. G. (1987). A multiplist strategy for strengthening nonequivalent control group designs. Evaluation Review, 11(6), 691–714.Google Scholar
Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469), 322–331.CrossRefGoogle Scholar
Sagarin, B. J., West, S. G., Ratnikov, A., Homan, W. K., & Ritchie, T. D. (2014). Treatment noncompliance in randomized experiments: Statistical approaches and design issues. Psychological Methods, 19(3), 317–333.Google Scholar
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton-Mifflin.Google Scholar
Somers, M.-A., Zhu, P., Jacob, R., & Bloom, H. (2013). The Validity and Precision of the Comparative Interrupted Time Series Design and the Difference-in-Difference Design in Educational Evaluation. Manpower Demonstration Research Corporation.Google Scholar
St. Pierre, R. G., Ricciuti, A., & Creps, C. (1999). Synthesis of Local and State Even Start Evaluations. Abt Associates.Google Scholar
Thistlewaite, D. L. & Campbell, D. T. (1960). Regression–discontinuity analysis: An alternative to the ex-post-facto experiment. Journal of Educational Psychology, 51(2), 309–317.CrossRefGoogle Scholar
Trochim, W. M. K. (1984). Research Designs for Program Evaluation: The Regression–Discontinuity Approach. SAGE Publications.Google Scholar