Hostname: page-component-cd9895bd7-gvvz8 Total loading time: 0 Render date: 2024-12-27T13:26:37.943Z Has data issue: false hasContentIssue false
  • English
  • Français

When Multi-Method Research Subverts Methodological Pluralism—or, Why We Still Need Single-Method Research

Published online by Cambridge University Press:  19 December 2012

Amel Ahmed  and
Rudra Sil
Amel Ahmed
Affiliation:
University of Massachusetts at Amherst
Rudra Sil
Affiliation:
University of Pennsylvania
Get access Rights & Permissions [Opens in a new window]

Abstract

While acknowledging the many forms and contributions of multi-method research (MMR), we examine the costs of treating it as best practice on the grounds that it reduces method-specific weaknesses and increases external validity for findings. Focusing on MMR that combines some type of qualitative analysis with statistical or formal approaches, we demonstrate that error-reduction and cross-validation are not feasible where methods are not sufficiently similar in their basic ontologies and their conceptions of causality. In such cases, MMR may still yield important benefits—such as uncovering related insights or improving the coding of variables—but these can be readily obtained through collaboration among scholars specializing in single-method research (SMR). Such scholars often set the standards for the application of particular methods and produce distinctive insights that can elude researchers concerned about competently deploying different methods and producing coherent findings. Thus, the unchecked proliferation of multi-method skill sets risks forefeiting the benefits of SMR and marginalizing idiographically-oriented qualitative research that fits less well with formal or quantitative approaches. This would effectively subvert the pluralism that once gave impetus to MMR unless disciplinary expectations and professional rewards are predicated on a more balanced and nuanced understanding of what various forms of SMR and MMR bring to the table.

Type
Research Article
Information
Perspectives on Politics , Volume 10 , Issue 4 , December 2012 , pp. 935 - 953
Copyright
Copyright © American Political Science Association 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abbott, Andrew. 2004. Methods of Discovery. New York: W. W. Norton.Google Scholar
Adcock, Robert. 2006. “Generalization in Comparative and Historical Social Science: The Difference That Interpretivism Makes.” In Interpretation and Method: Empirical Research Methods and the Interpretive Turn, ed. Yanow, Dvora and Schwartz-Shea, Peregrine. Armonk, NY: M.E. Sharpe.Google Scholar
Ahram, Ariel. 2011. “Concepts and Measurement in Multi-Method Research.” Political Research Quarterly http://prq.sagepub.com/content/early/2011/11/06/1065912911427453.full.pdf+html.Google Scholar
Ames, Barry. 1996. “Comparative Politics and the Replication Controversy.” APSA-CP: Newsletter of the APSA Organized Section in Comparative Politics 7(1): 8, 11–12.Google Scholar
Bartels, Larry. 2008. Unequal Democracy: The Political Economy of the New Gilded Age. Princeton, NJ: Princeton University Press.Google Scholar
Bates, Robert H., Greif, Avner, Levi, Margaret, Rosenthal, Jean-Laurent, and Weingast, Barry. 1998. Analytic Narratives. Princeton, NJ: Princeton University Press.Google Scholar
Beck, Nathaniel. 2006. “Is Causal-Process Observation An Oxymoron?” Political Analysis 14(3): 347–52.Google Scholar
Beck, Nathaniel. 2010. “Causal Process ‘Observations’: Oxymoron or (Fine) Old Wine.” Political Analysis 18(4): 499–505.CrossRefGoogle Scholar
Bennett, Andrew, and Braumoeller, Bear. 2006. “Where the Model Frequently Meets the Road: Combining Statistical, Formal and Case Study Methods.” Presented at the Annual Meeting of the American Political Science Association, Philadelphia, PA August 31–September 3.Google Scholar
Bevir, Mark, and Kedar, Asaf. 2008. “Concept Formation in Political Science: An Anti-Naturalist Critique of Qualitative Methodolog.” Perspectives on Politics 6(3): 503–17.CrossRefGoogle Scholar
Brady, Henry, and Collier, David, eds. 2004. Rethinking Social Inquiry: Diverse Tools: Shared Standards. Lanham, MD: Rowman & Littlefield.Google Scholar
Brewer, John, and Hunter, Albert. 2006. Foundations of Multimethod Research: Synthesizing Styles. Thousand Oaks, CA: Sage.Google Scholar
Bueno de Mesquita, Bruce. 1982. The War Trap. New Haven: Yale University Press.Google Scholar
Bueno de Mesquita, Bruce. 1985. “Toward a Scientific Understanding of International Conflict.” International Studies Quarterly 29(2): 121–36.Google Scholar
Bunce, Valerie. 1981. Do New Leaders Make a Difference? Exclusive Succession and Public Policy Under Capitalism and Socialism. Princeton: Princeton University Press.Google Scholar
Cameron, Charles, and Morton, Rebecca. 2002. “Formal Theory Meets Data.” In Political Science: The State of the Discipline, ed. Katznelson, Ira and Milner, Helen. New York: W. W. Norton.Google Scholar
Campbell, Donald, and Fiske, Donald. 1959. “Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix.” Psychological Bulletin 56: 81–105.Google Scholar
Caporaso, James. 1995. “Research Design, Falsification, and the Qualitative-Quantitative Divide.” American Political Science Review 89(2): 457–60.Google Scholar
Caporaso, James. 2009. “Is There a Quantitative-Qualitative Divide in Comparative Politics?” In The Sage Handbook of Comparative Politics, ed. Landman, Todd and Robinson, Neil. Thousand Oaks: Sage.Google Scholar
Carpenter, Daniel. 2007. “Some Observations on the Interweaving of Model and Narrative.” Newsletter of the APSA Organized Section on Qualitative Methods 5(1): 17–19.Google Scholar
Chatterjee, Abhishek. 2009. “Ontology, Epistemology, and Multi-Methods.” Newsletter of the APSA Organized Section in Qualitative & Multi-Method Research 7(2): 11–15.Google Scholar
Collier, David. 1995. “Translating Quantitative Methods for Qualitative Researchers: The Case of Selection Bias”; in American Political Science Review 89(2): 461–66.Google Scholar
Collier, David, Brady, Henry, and Seawright, Jason. 2010. “Outdated Views of Qualitative Methods: Time to Move On.” Political Analysis 18(4): 506–13.Google Scholar
Collier, David, and Elman, Colin. 2008. “Qualitative and Multi-Method Research: Organizations, Publication, and Reflections on Integration.” In The Oxford Handbook of Political Methodology. ed. Box-Steffensmeier, Janet M., Brady, Henry, and Collier, David. Oxford, UK: Oxford University Press.Google Scholar
Crasnow, Sharon. 2011. “Evidence for Use: Causal Pluralism and the Role of Case Studies in Political Science Research.” Philosophy of the Social Sciences 41(1): 26–49.Google Scholar
Cusack, Thomas, Iversen, Torben, and Soskice, David. 2010. “Coevolution of Capitalism and Political Representation: The Choice of Electoral Systems.” American Political Science Review 104(2): 393–403.Google Scholar
Denzin, Norman. 1978 [1970]. The Research Act In Sociology: A Theoretical Introduction To Sociological Methods. 2d ed. New York: McGraw-Hill.Google Scholar
Dunning, Thad. 2007. “The Role of Iteration in Multi-Method Research.” Newsletter of the APSA Organized Section on Qualitative Methods 5(1): 22–24.Google Scholar
Dunning, Thad. 2012. Natural Experiments in the Social Sciences: A Design-Based Approach. Strategies for Social Inquiry series. New York: Cambridge University Press.Google Scholar
Edling, Christofer. 2000. “Rational Choice Theory and Quantitative Analysis: A Comment on Goldthorpe's Sociological Alliance.” European Sociological Review 16(1): 1–8.Google Scholar
Falleti, Tulia, and Lynch, Julia. 2009. “Context and Causal Mechanisms in Political Analysis.” Comparative Political Studies 42(9): 1143–66.Google Scholar
Fearon, James, and Laitin, David. 2008. “Integrating Qualitative and Quantitative Methods.” In The Oxford Handbook of Political Methodology, ed. Box-Steffensmeier, Janet M., Brady, Henry, and Collier, David. New York: Oxford University Press.Google Scholar
Fiss, Peer C. 2007. “A Set-theoretic Approach to Organizational Configurations.” Academy of Management Review 32: 1180–98.Google Scholar
Flyvbjerg, Bent. 2001. Making Social Science Matter: Why Social Inquiry Fails and How It Can Succeed Again. New York: Cambridge University Press.Google Scholar
Flyvbjerg, Bent. 2004. “A Perestroikan Strawman Answers Back: David Laitin and Phronetic Political Science.” Politics & Society 32(3): 389–416.Google Scholar
Garfinkel, Harold. 1967. Studies in Ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Geertz, Clifford. 1971. Islam Observed: Religious Development in Morocco and Indonesia. Chicago: University of Chicago Press.Google Scholar
George, Alexander, and Bennett, Andrew. 2005. Case Studies and Theory Development in the Social Sciences. Cambridge, MA: M.I.T. Press.Google Scholar
Gerring, John. 2011a. Social Science Methodology: A Unified Framework. New York: Cambridge University Press.Google Scholar
Gerring, John. 2011b. “How Good is Good Enough? A Multidimensional Best-Possible Standard for Research Design.” Political Research Quarterly 64(3): 625–36.CrossRefGoogle Scholar
Goemans, Hein. 2007. “Qualitative Methods as an Essential Component to Quantitative Methods.” Newsletter of the APSA Organized Section on Qualitative Methods 5(1): 11–13.Google Scholar
Goertz, Gary, and Mahoney, James. 2005. “Two-Level Theories and Fuzzy-Set Analysis.” Sociological Methods and Research 33(4): 497–538.Google Scholar
Goertz, Gary, and Mahoney, James. 2012. A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences. Princeton: Princeton University Press.Google Scholar
Goldthorpe, John. 1996. “The Quantitative Analysis of Large-Scale Data-sets and Rational Actor Theory: For a Sociological Alliance.” European Sociological Review 12(2): 109–26.Google Scholar
Grzymala-Busse, Anna. 2011. “Time Will Tell? Temporality and the Analysis of Causal Mechanisms and Processes.” Comparative Political Studies 44(9): 1267–97.Google Scholar
Hall, Peter. 2003. “Aligning Ontology and Methodology in Comparative Politics.” In Comparative Historical Analysis in the Social Sciences, ed. Mahoney, James and Reuschemeyer, Dietrich. New York: Cambridge University Press.Google Scholar
Hall, Peter. 2007. “The Dilemmas of Contemporary Social Science.” boundary 2 34(3): 121–41.Google Scholar
Harrits, Gitte Sommer. 2011. “More Than Method? A Discussion of Paradigm Differences within Mixed Methods Research.” Journal of Mixed Methods Research 5(2): 150–66.Google Scholar
Herrera, Yoshiko. 2005. Imagined Economies: The Sources of Russian Regionalism. Cambridge: Cambridge University Press.Google Scholar
Hopf, Ted. 2007. “The Limits of Interpreting Evidence.” In Theory and Evidence in Comparative Politics and International Relations, ed. Lebow, Richard Ned and Lichbach, Mark. New York: Palgrave Macmillan.Google Scholar
Jick, Todd D. 1979. “Mixing Qualitative and Quantitative Methods: Triangulation in Action.” Administrative Science Quarterly 24(4): 602–11.Google Scholar
Johnson, James. 2002. “How Conceptual Problems Migrate: Rational Choice, Interpretation, and the Hazards of Pluralism.” Annual Review of Political Science 5: 223–48.Google Scholar
Johnson, R. Burke, Onwuegbuzie, Anthony J., and Turner, Lisa A.. 2007. “Toward a Definition of Mixed Methods Research.” Journal of Mixed Method Research 1(2): 112–33.CrossRefGoogle Scholar
King, Gary, Keohane, Robert, and Verba, Sidney. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press.Google Scholar
Kreuzer, Marcus. 2010. “Historical Knowledge and Quantitative Analysis: the Case of the Origins of Proportional Representation.” American Political Science Review 104(2): 369–92.Google Scholar
Laitin, David D. 1995. “Disciplining Political Science. Review of ‘Designing Social Inquiry: Scientific Inference in Qualitative Research’ by Gary King, Robert O. Keohane, and Sidney Verba.” The American Political Science Review 89(2): 454–456.Google Scholar
Laitin, David. 2003. “The Perestroikan Challenge to Social Science.” Politics & Society 31(1): 163–84.Google Scholar
Levi, Margaret. 2004. “An Analytic Narrative Approach to Puzzles and Problems.” In Problems and Methods in the Study of Politics, ed. Shapiro, Ian, Smith, Rogers, and Masoud, Tarek. New York: Cambridge University Press.Google Scholar
Levy, Jack S. 2007. “Qualitative Methods and Cross-Method Dialogue in Political Science.” Comparative Political Studies 40(2): 196–214.CrossRefGoogle Scholar
Lieberman, Evan. 2003. Race and Regionalism in the Politics of Taxation in Brazil and South Africa. New York: Cambridge University Press.Google Scholar
Lieberman, Evan. 2005. “Nested Analysis as a Mixed-Method Strategy in Comparative Politics.” American Political Science Review 99(3): 435–52.Google Scholar
Lieberman, Evan. 2009. Boundaries of Contagion: How Ethnic Politics Have Shaped Government Responses to AIDS. Princeton: Princeton University Press.Google Scholar
Lijphart, Arend. 1971. “Comparative Politics and the Comparative Method.” American Political Science Review 65(3): 682–93.Google Scholar
Lohmann, Susanne. 2007. “The Trouble with Multi-Methodism.” Newsletter of the APSA Organized Section on Qualitative Methods 5(1): 13–17.Google Scholar
Lustick, Ian. 1996. “History, Historiography, and Political Science: Multiple Historical Records and the Problem of Selection Bias.” American Political Science Review 90(3): 605–18.CrossRefGoogle Scholar
Mahoney, James. 2008. “Toward a Unified Theory of Causality.” Comparative Political Studies 41 (4-5): 412–36.Google Scholar
Mahoney, James. 2010. “After KKV: The New Methodology of Qualitative Research.” World Politics 62(1): 120–47.Google Scholar
Mahoney, James, and Goertz, Gary. 2006. “A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research.” Political Analysis 14(3): 227–49.Google Scholar
March, James, and Olsen, Johan. 2004. “The Logic of Appropriateness.” ARENA Working Papers 04/09, Centre for European Studies, University of Oslo, Norway.Google Scholar
McKeown, Timothy. 2004. “Case Studies and the Limits of the Quantitative Worldview.” In Rethinking Social Inquiry: Diverse Tools: Shared Standards, ed. Brady, Henry and Collier, David. Lanham, MD: Rowman & Littlefield.Google Scholar
Munck, Gerardo. 2010. “Comparative Politics: Taking Stock and Looking Forward.” Working Paper No. 30 on Political Methodology, Committee on Concepts and Methods of the International Political Science Association (December): 1–37.Google Scholar
Ostrom, Elinor. 2002. “Some Thoughts About Shaking Things Up: Future Directions in Political Science.” PS: Political Science and Politics 35(2): 191–92.Google Scholar
Ostrom, Elinor. 2005. Understanding Institutional Diversity. Princeton: Princeton University Press.Google Scholar
Pevehouse, Jon. 2005. Democracy from Above: Regional Organizations and Democratization. New York: Cambridge University Press.Google Scholar
Pierson, Paul. 2004. Politics in Time: History, Institutions, and Social Analysis. Princeton: Princeton University Press.Google Scholar
Poteete, Amy, Janssen, Marco, and Ostrom, Elinor. 2010. Working Together: Collective Action, the Commons, and Multiple Methods in Practice. Princeton: Princeton University Press.Google Scholar
Przeworski, Adam, and Teune, Henry. 1970. The Logic of Comparative Social Inquiry. New York: John Wiley & Sons.Google Scholar
Ragin, Charles. 2008. Redesigning Social Inquiry: Fuzzy Sets and Beyond. Chicago: University of Chicago Press.Google Scholar
Reiss, Julian. 2009. “Causation in the Social Sciences: Evidence, Inference, and Purpose.” Philosophy of the Social Sciences 39(1): 20–40.Google Scholar
Rohlfing, Ingo. 2008. “What You See and What You Get: Pitfalls and Principles of Nested Analysis in Comparative Research.” Comparative Political Studies 41(11): 1492–514.Google Scholar
Rohner, Ronald. 1977. “Advantages of the Comparative Method of Anthropology.” Behavior Science Research 12: 117–44.Google Scholar
Schatz, Edward, ed. 2009. Political Ethnography: What Immersion Contributes to the Study of Power. Chicago: University of Chicago Press.Google Scholar
Schultz, Kenneth. 2001. Democracy and Coercive Diplomacy. New York: Cambridge University Press.Google Scholar
Scott, James. 1990. Domination and the Arts of Resistance: Hidden Transcripts. New Haven: Yale University Press.Google Scholar
Shapiro, Ian, Smith, Rogers, and Masoud, Tarek, eds. 2004. Problems and Methods in the Study of Politics. New York: Cambridge University Press.Google Scholar
Shapiro, Ian, and Wendt, Alexander. 2005. “The Difference that Realism Makes: Social Science and the Politics of Consent.” In The Flight From Reality in the Human Sciences, ed. Shapiro, Ian. Princeton: Princeton University Press.Google Scholar
Sil, Rudra. 2004. “Problems Chasing Methods or Methods Chasing Problems? Research Communities, Constrained Pluralism, and the Role of Eclecticism.” In Problems and Methods in the Study of Politics, ed. Shapiro, Ian, Smith, Rogers, and Masoud, Tarek. New York: Cambridge University Press.Google Scholar
Sil, Rudra, and Katzenstein, Peter J.. 2010. “Analytic Eclecticism in the Study of World Politics: Reconfiguring Problems and Mechanisms across Research Traditions.” Perspectives on Politics 8(2): 411–31.Google Scholar
Tarrow, Sidney. 1989. Democracy and Disorder: Protest and Politics in Italy, 1965–1975. New York: Oxford University Press.Google Scholar
Tarrow, Sidney. 1995. “Bridging the Quantitative-Qualitative Divide in Political Science.” American Political Science Review 89(2): 471–74.Google Scholar
Tashakkori, Abbas, and Teddlie, Charles, eds. 2003. Handbook of Mixed-Method Social and Behavioral Research. Thousand Oaks, CA: Sage Publications.Google Scholar
Webb, Eugene, Campbell, Donald, Schwartz, Richard, and Sechrest, Lee. 1966. Unobtrusive Measures: Non-reactive Research in the Social Sciences. Chicago: Rand McNally.Google Scholar
Wedeen, Lisa. 2008. Peripheral Visions: Publics, Power and Performance in Yemen. Chicago: University of Chicago Press.Google Scholar
Yanow, Dvora. 2006. “Thinking Interpretively: Philosophical Presuppositions and the Human Sciences.” In Interpretation and Method: Empirical Research Methods and the Interpretive Turn, ed. Yanow, Dvora and Schwartz-Shea, Peregrine. Armonk, NY: M.E. Sharpe.Google Scholar
Yanow, Dvora, and Schwartz-Shea, Peregrine, eds. 2006. Interpretation and Method: Empirical Research Methods and the Interpretive Turn. Armonk, NY: M.E. Sharpe.Google Scholar