Hostname: page-component-cd9895bd7-gbm5v Total loading time: 0 Render date: 2025-01-05T14:53:11.865Z Has data issue: false hasContentIssue false
  • English
  • Français

Meta-analysis of Comparative Therapy Outcome Research: a Critical Appraisal*

Published online by Cambridge University Press:  16 June 2009

David A. Shapiro  and
Diana Shapiro
David A. Shapiro
Affiliation:
M.R.C./S.S.R.C. Social and Applied Psychology Unit, Department of Psychology, University of Sheffield, Sheffield S10 2TN
Diana Shapiro
Affiliation:
M.R.C./S.S.R.C. Social and Applied Psychology Unit, Department of Psychology, University of Sheffield, Sheffield S10 2TN
Get access Rights & Permissions [Opens in a new window]

Extract

Meta-analysis is a systematic and quantitative approach to reviewing empirical literature. Its claimed benefits of superior objectivity and dependability arise from its adoption of the methodological principles and methods of primary empirical research. This paper offers a critical appraisal of these claims, with special reference to the application of meta-analysis to the therapy outcome literature. Features examined include the numerical combination of results obtained in independent studies; search procedures and inclusion criteria; and the coding of objective and qualitative features of the source studies, which are then used to study the correlates of outcome via disaggregation and multiple regression analysis. The major criticisms reviewed include allegations of overgeneralization, indiscriminate inclusion of low quality data, and idiosyncratic and unacceptable conclusions. It is concluded that the continued application and refinement of meta-analysis in the field of therapy outcomes has an important contribution to make, although not to the exclusion of other methods of integrating data. It is predicted that methods and principles currently associated with meta-analysis will increasingly find a place in all reviews of extensive tracts of empirical literature.

Type
Research Article
Information
Behavioural and Cognitive Psychotherapy , Volume 10 , Issue 1 , January 1982 , pp. 4 - 25
Copyright
Copyright © British Association for Behavioural and Cognitive Psychotherapies 1982

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Arkin, R., Cooper, H. and Kolditz, T. (1980). A statistical review of the literature concerning the self-serving bias in interpersonal influence situations. Journal of Personality 48, 435–448.Google Scholar
Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin 66, 423–437.Google Scholar
Barber, T. X. (1978). Commentary on Rosenthal and Rubin. Behavioral and Brain Sciences 3, 388–390.CrossRefGoogle Scholar
Bergin, A. E. (1971). The evaluation of therapeutic outcomes. In Handbook of Psychotherapy and Behavior Change, Bergin, A. E. and Garfield, S. L. (Eds), New York: Wiley.Google Scholar
Carlier, M. and Gottesdiener, H. (1978). Commentary on Rosenthal and Rubin. Brain and Behavioral Sciences 3, 390.CrossRefGoogle Scholar
Cohen, J. (1962). The statistical power of abnormal-social psychological research: a review. Journal of Abnormal and Social Psychology 65, 145–153.Google Scholar
Ċohen, J. (1977). Statistical Power Analysis for the Behavioral Sciences (revised edition), New York: Academic Press.Google Scholar
Cohen, J. and Cohen, P. (1975). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, New York: Wiley.Google Scholar
Cook, T. D. and Leviton, L. C. (1980). Reviewing the literature: a comparison of traditional methods with meta-analysis. Journal of Personality 48, 449–472.CrossRefGoogle Scholar
Cooper, H. M. (1979). Statistically combining independent studies: a meta-analysis of sex differences in conformity research. Journal of Personality and Social Psychology 37, 131–135.CrossRefGoogle Scholar
Cooper, H. M. and Arkin, R. M. (1981). On quantitative reviewing. Journal of Personality 49, 225–230.Google Scholar
Cooper, H. M. and Rosenthal, R. (1980). Statistical versus traditional procedures for summarizing research findings. Psychological Bulletin 87, 442–449.CrossRefGoogle ScholarPubMed
Elashoff, J. D. (1978). Commentary on Rosenthal and Rubin. Behavioral and Brain Sciences 3, 392.CrossRefGoogle Scholar
Eysenck, H. J. (1952). The effects of psychotherapy: an evaluation. Journal of Consulting Psychology 16, 319–324.Google Scholar
Eysenck, H. J. (1961). The effects of psychotherapy. In Handbook of Abnormal Psychology, Eysenck, H. J. (Ed.), New York: Basic Books.CrossRefGoogle Scholar
Eysenck, H. J. (1965). The effects of psychotherapy. International Journal of Psychiatry 1, 97–178.Google Scholar
Eysenck, H. J. (1978). An exercise in mega-silliness. American Psychologist 33, 517.Google Scholar
Gallo, P. S. (1978). Meta-analysis – a mixed metaphor. American Psychologist 33, 515–517.CrossRefGoogle Scholar
Glass, G. V. (1976). Primary, secondary and meta-analysis of research. The Educational Researcher 10, 3–8.CrossRefGoogle Scholar
Glass, G. V. and Smith, M. L. (1979). The effects of class size on achievement. Journal of Education and Policy Studies 1, 2–16.Google Scholar
Goldstein, A. P. and Stein, N. (1976). Prescriptive Psychotherapies, New York: Pergamon.Google Scholar
Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin 82, 1–20.Google Scholar
Hayes, S. C. (1981). Single case experimental design and empirical clinical practice. Journal of Consulting and Clinical Psychology 49, 193–211.Google Scholar
Hedges, L. V. and Olkin, I. (1980). Vote counting methods in research synthesis. Psychological Bulletin 88, 359–369.CrossRefGoogle Scholar
Herink, R. (Ed.) (1980). The Psychotherapy Handbook: The A to Z Guide to More Than 250 Different Therapies in Use Today, New York: New American Library.Google Scholar
Kazdin, A. E. (1981). Drawing valid inferences from case studies. Journal of Consulting and Clinical Psychology 49, 183–192.Google Scholar
Kazdin, A. E. and Wilson, G. T. (1978). Evaluation of Behavior Therapy. Issues, Evidence and Research Strategies, Cambridge, Mass: Ballinger.Google Scholar
Kazrin, A., Durac, J. and Agteros, T. (1979). Meta-meta analysis: a new method of evaluating therapy outcome. Behavior Research and Therapy 17, 397–399.CrossRefGoogle ScholarPubMed
Kiesler, D. J. (1966). Some myths of psychotherapy research and the search for a paradigm. Psychological Bulletin 65, 110–136.CrossRefGoogle Scholar
Ledwidge, B. (1978). Cognitive behavior modification: a step in the wrong direction? Psychological Bulletin 85, 353–375.CrossRefGoogle ScholarPubMed
Leviton, L. C. and Cook, T. D. (1981). What differentiates meta-analysis from other forms of review. Journal of Personality 49, 231–236.CrossRefGoogle Scholar
Luborsky, L., Singer, B. and Luborsky, L. (1975). Comparative studies of psychotherapies. Archives of General Psychiatry 32, 995–1008.CrossRefGoogle ScholarPubMed
Mansfield, R. S. and Bussey, T. V. (1977). Meta-analysis in research: a rejoinder to Glass. The Educational Researcher 6, 3.Google Scholar
Meltzoff, J. and Kornreich, M. (1970). Research in Psychotherapy, New York: Atherton.Google Scholar
Mosteller, F. M. and Bush, R. R. (1954). Selected quantitative techniques. In Handbook of Social Psychology. Volume I: Linzey, G. (Ed.), Cambridge, Mass: Addison-Wesley.Google Scholar
Presby, S. (1978). Overly broad categories obscure important differences between therapies. American Psychologist 33, 514–515.Google Scholar
Rachman, S. J. (1971). The Effects of Psychotherapy, New York: Pergamon.Google Scholar
Rachman, S. J. and Wilson, G. T. (1980). The Effects of Psychological Therapy: Second Enlarged Edition. New York: Pergamon Press.Google Scholar
Rosenthal, R. (1978). Combining results of independent studies. Psychological Bulletin 85, 185–193.CrossRefGoogle Scholar
Rosenthal, R. (1979). The “file drawer” problem and tolerance for null results. Psychological Bulletin 86, 638–641.Google Scholar
Rosenthal, R. and Rubin, D. B. (1978). Interpersonal expectancy effects: the first 345 studies. Behavioral and Brain Sciences 3, 377–415.Google Scholar
Shapiro, D. A. and Shapiro, D. (1982). Meta-analysis of comparative therapy outcome studies: a replication and refinement. Psychological Bulletin (in press).CrossRefGoogle ScholarPubMed
Skelton-Robinson, M. (1980). This interminable debate. Bulletin of the British Psychological Society 33, 376–379.Google Scholar
Smart, R. G. (1964). The importance of negative results in psychological research. The Canadian Psychologist 5, 225–232.CrossRefGoogle Scholar
Smith, M. L. and Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist 32, 752–760.Google Scholar
Smith, M. L., Glass, G. V. and Miller, T. I. (1980). The Benefits of Psychotherapy, Baltimore: Johns Hopkins University Press.Google Scholar
Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance – or vice versa. Journal of the American Statistical Association 54, 30–34.Google Scholar
Stouffer, S. A. (1949). The American Soldier. Volume I: Adjustment During Army Life, Princeton, NJ: Princeton University Press.Google Scholar
Taveggia, T. C. (1974). Resolving research controversy through empirical cumulation. Sociological Methods and Research 2, 395–407.Google Scholar
White, K. (1977). The relationship between socio-economic status and academic underachievement. (Doctoral Dissertation University of Colorado, 1977). Dissertation Abstracts International, 37, 5076A. (University Microfilms No. 77–03250).Google Scholar
Submit a response

Comments

No Comments have been published for this article.