Hostname: page-component-5cf477f64f-2wr7h Total loading time: 0 Render date: 2025-04-02T17:35:53.295Z Has data issue: false hasContentIssue false
  • English
  • Français

Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest

Published online by Cambridge University Press:  16 December 2021

Flavien Ganter Open the ORCID record for Flavien Ganter [Opens in a new window]
Flavien Ganter*
Affiliation:
Department of Sociology, Columbia University, New York City, NY, USA. Email: [email protected]
*
Corresponding author Flavien Ganter
Get access Rights & Permissions [Opens in a new window]

Abstract

Forced-choice conjoint experiments have become a standard component of the experimental toolbox in political science and sociology. Yet the literature has largely overlooked the fact that conjoint experiments can be used for two distinct purposes: to uncover respondents’ multidimensional preferences, and to estimate the causal effects of some attributes on a profile’s selection probability in a multidimensional choice setting. This paper makes the argument that this distinction is both analytically and practically relevant, because the quantity of interest is contingent on the purpose of the study. The vast majority of social scientists relying on conjoint analyses, including most scholars interested in studying preferences, have adopted the average marginal component effect (AMCE) as their main quantity of interest. The paper shows that the AMCE is neither conceptually nor practically suited to explore respondents’ preferences. Not only is it essentially a causal quantity conceptually at odds with the goal of describing patterns of preferences, but it also does generally not identify preferences, mixing them with compositional effects unrelated to preferences. This paper proposes a novel estimand—the average component preference—designed to explore patterns of preferences, and it presents a method for estimating it.

Keywords

conjoint analysisfactorial experimentsurvey experimentpreferences
Type
Article
Information
Political Analysis , Volume 31 , Issue 1 , January 2023 , pp. 98 - 112
Copyright
© The Author(s) 2021. Published by Cambridge University Press on behalf of the Society for Political Methodology

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Edited by Jeff Gill

References

Abramson, S. F., Koçak, K., and Magazinnik, A.. 2019. “What Do We Learn About Voter Preferences From Conjoint Experiments?” Working Paper. https://scholar.princeton.edu/sites/default/files/kkocak/files/conjoint_draft.pdf.Google Scholar
Abramson, S. F., Kocak, K., Magazinnik, A., and Strezhnev, A.. 2020. “Improving Preference Elicitation in Conjoint Designs Using Machine Learning for Heterogeneous Effects.” Working Paper. https://polmeth.theopenscholar.com/files/polmeth2020/files/polmeth_magazinnik.pdf.Google Scholar
Bansak, K., Hainmueller, J., Hopkins, D. J., and Yamamoto, T.. 2020. “Using Conjoint Experiments to Analyze Elections: The Essential Role of the Average Marginal Component Effect (AMCE).” Working Paper. https://cpb-us-w2.wpmucdn.com/web.sas.upenn.edu/dist/f/49/files/2020/04/amce_04302020.pdf.CrossRefGoogle Scholar
Carey, J. M., Carman, K. R., Clayton, K. P., Horiuchi, Y., Htun, M., and Ortiz, B.. 2018. “Who Wants to Hire a More Diverse Faculty? A Conjoint Analysis of Faculty and Student Preferences for Gender and Racial/Ethnic Diversity.” Politics, Groups, and Identities 8 (3): 535–553.CrossRefGoogle Scholar
Carlson, E. 2015. “Ethnic Voting and Accountability in Africa: A Choice Experiment in Uganda.” World Politics 67 (2): 353–385.CrossRefGoogle Scholar
de la Cuesta, B., Egami, N., and Imai, K.. 2021. “Improving the External Validity of Conjoint Analysis: The Essential Role of Profile Distribution.” Political Analysis, 1–27. https://doi.org/10.1017/pan.2020.40.Google Scholar
Flores, R. D. and Schachter, A.. 2018. “Who Are the ‘Illegals’? The Social Construction of Illegality in the United States.” American Sociological Review 83 (5): 839–868.CrossRefGoogle Scholar
Ganter, F. 2021. “Replication Data for: Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest.” https://doi.org/10.7910/DVN/3RNU1Q, Harvard Dataverse, V1, UNF:6:6L5nUOugK9UYIOiQd/DOsA== [fileUNF].CrossRefGoogle Scholar
Hainmueller, J., Hangartner, D., and Yamamoto, T.. 2015. “Validating Vignette and Conjoint Survey Experiments against Real-World Behavior.” Proceedings of the National Academy of Sciences 112 (8): 2395–2400.CrossRefGoogle ScholarPubMed
Hainmueller, J. and Hopkins, D. J.. 2015. “The Hidden American Immigration Consensus: A Conjoint Analysis of Attitudes toward Immigrants.” American Journal of Political Science 59 (3): 529–548.CrossRefGoogle Scholar
Hainmueller, J., Hopkins, D. J., and Yamamoto, T.. 2013. “Replication data for: Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments.” https://doi.org/10.7910/DVN/THJYQR, Harvard Dataverse, V2, UNF:5:A5Jv8XldU7R73/D1xTL3Gw== [fileUNF].Google Scholar
Hainmueller, J., Hopkins, D. J., and Yamamoto, T.. 2014. “Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments.” Political Analysis 22 (1): 1–30.CrossRefGoogle Scholar
Hankinson, M. 2018. “When Do Renters Behave Like Homeowners? High Rent, Price Anxiety, and NIMBYism.” American Political Science Review 112 (3): 473–493.CrossRefGoogle Scholar
Jasso, G. (2006). “Factorial Survey Methods for Studying Beliefs and Judgments.” Sociological Methods & Research 34 (3): 334–423.10.1177/0049124105283121CrossRefGoogle Scholar
Leeper, T. J., Hobolt, S. B., and Tilley, J.. 2019. “Measuring Subgroup Preferences in Conjoint Experiments.” Political Analysis 28 (2): 1–15.Google Scholar
Mummolo, J. and Nall, C.. 2017. “Why Partisans Do Not Sort: The Constraints on Political Segregation.” The Journal of Politics 79(1): 45–59.CrossRefGoogle Scholar
Nall, C. and Mummolo, J.. 2016. “Replication Data for ‘Why Partisans Don’t Sort: The Constraints on Partisan Segregation’.” https://doi.org/10.7910/DVN/EDGRDC, Harvard Dataverse, V1, UNF:6:DeEDwKhm55BMi0wsgEhluA== [fileUNF].Google Scholar
Supplementary material: PDF

Ganter supplementary material

Ganter supplementary material

Download Ganter supplementary material(PDF)
PDF 139.7 KB