Element contents
Advances in Efficient Design of Experiments in Economics
Published online by Cambridge University Press: 04 December 2024
Summary
Amidst concerns about replicability but also thanks to the professionalisation of labs, the rise of pre-registration, the switch to online experiments, and enhanced computational power, experimental economics is undergoing rapid changes. They all call for efficient designs and data analysis, that is, they require that, given the constraints on participants' time, experiments provide as rich information as possible. In this Element the authors explore some ways in which this goal may be reached.
Keywords
- Type
- Element
- Information
- Online ISBN: 9781009263030Publisher: Cambridge University PressPrint publication: 16 January 2025
References
Abdellaoui, Mohammed. Parameter-free elicitation of utility and probability weighting functions. Management Science, 46(11):1497–1512, 2000.CrossRefGoogle Scholar
Abdi, Hervé. Bonferroni and Šidák corrections for multiple comparisons. Encyclopedia of Measurement and Statistics, 3:103–107, 2007.Google Scholar
Agranov, Marina, Caplin, Andrew, and Tergiman, Chloe. Naive play and the process of choice in guessing games. Journal of the Economic Science Association, 1:146–157, 2015.CrossRefGoogle Scholar
Akaichi, Faical, Costa-Font, Joan, and Frank, Richard. Uninsured by choice? A choice experiment on long term care insurance. Journal of Economic Behavior & Organization, 173:422–434, 2020.CrossRefGoogle Scholar
Andreoni, James. Why free ride? Strategies and learning in public goods experiments. Journal of Public Economics, 37(3):291–304, 1988.CrossRefGoogle Scholar
Bleichrodt, Han, and Luis Pinto, Jose. A parameter-free elicitation of the probability weighting function in medical decision analysis. Management Science, 46(11):1485–1496, 2000.CrossRefGoogle Scholar
Box, E. P. George, Hunter, William Gordon, and Stuart Hunter, J.. Statistics for experimenters: Design, innovation, and discovery, 2nd edition. Wiley, 2005.Google Scholar
Brady, Henry E. Causation and explanation in social science. In Robert, E. Goodin, editor, The Oxford handbook of political science, Oxford University Press, 2008.Google Scholar
John, Bucknell, White, Justin S., and Shang, Ce. Can incentive-compatibility reduce hypothetical bias in smokers’ experimental choice behavior? A randomized discrete choice experiment. Journal of Choice Modelling, 37:100255, 2020.Google Scholar
Cavagnaro, Daniel R., Gonzalez, Richard, Myung, Jay I., and Pitt, Mark A.. Optimal decision stimuli for risky choice experiments: An adaptive approach. Management Science, 59(2):358–375, 2013.CrossRefGoogle ScholarPubMed
Cavagnaro, Daniel R., Myung, Jay I., Pitt, Mark A., and Kujala, Janne V.. Adaptive design optimization: A mutual information-based approach to model discrimination in cognitive science. Neural Computation, 22(4):887–905, 2010.CrossRefGoogle ScholarPubMed
Cetre, Sophie, Lobeck, Max, Senik, Claudia, and Verdier, Thierry. Preferences over income distribution: Evidence from a choice experiment. Journal of Economic Psychology, 74:102202, 2019.CrossRefGoogle Scholar
Chapman, Jonathan, Snowberg, Erik, Wang, Stephanie, and Camerer, Colin. Loss attitudes in the U.S. population: Evidence from Dynamically Optimized Sequential Experimentation (DOSE). Technical report, National Bureau of Economic Research, 2018.Google Scholar
Gary, Charness, Gneezy, Uri, and Kuhn, Michael A.. Experimental methods: Between-subject and within-subject design. Journal of Economic Behavior & Organization, 81(1):1–8, 2012.Google Scholar
Clark, Andrew E., Senik, Claudia, and Yamada, Katsunori. When experienced and decision utility concur: The case of income comparisons. Journal of Behavioral and Experimental Economics, 70:1–9, 2017.CrossRefGoogle Scholar
Croissant, Yves. Estimation of random utility models in r: The mlogit package. Journal of Statistical Software, 95(11):1–41, 2020. https://doi.org/10.18637/jss.v095.i11.CrossRefGoogle Scholar
Mikolaj, Czajkowski, Giergiczny, Marek, and Greene, William H.. Learning and fatigue effects revisited: Investigating the effects of accounting for unobservable preference and scale heterogeneity. Land Economics, 90(2):324–351, 2014.Google Scholar
Ding, Min. An incentive-aligned mechanism for conjoint analysis. Journal of Marketing Research, 44(2):214–223, 2007.CrossRefGoogle Scholar
Fox, Armando, Gribble, Steven D., Chawathe, Yatin, and Brewer, Eric A.. Adapting to network and client variation using infrastructural proxies: Lessons and perspectives. IEEE Personal Communications, 5(4):10–19, 1998.CrossRefGoogle Scholar
Frydman, Cary, Camerer, Colin, Bossaerts, Peter, and Rangel, Antonio. Maoa-l carriers are better at making optimal financial decisions under risk. Proceedings of the Royal Society B: Biological Sciences, 278(1714):2053–2059, 2011.CrossRefGoogle Scholar
Harrison, Glenn W., Lau, Morten I., and Elisabet Rutström, E.. Risk attitudes, randomization to treatment, and self-selection into experiments. Journal of Economic Behavior & Organization, 70(3):498–507, 2009.CrossRefGoogle Scholar
Hess, Stephane, and Palma, David. Apollo: A flexible, powerful and customisable freeware package for choice model estimation and application. Journal of Choice Modelling, 32:100170, 2019.CrossRefGoogle Scholar
Holland, Paul W. Causation and race. ETS Research Report Series, 2003(1):i–21, 2003.CrossRefGoogle Scholar
Holland, Paul W. Statistics and causal inference. Journal of the American Statistical Association, 81(396):945–960, 1986.CrossRefGoogle Scholar
Horiuchi, Yusaku, Markovich, Zachary, and Yamamoto, Teppei. Does conjoint analysis mitigate social desirability bias? Political Analysis, 30(4):535–549, 2022.CrossRefGoogle Scholar
Horowitz, John K. The Becker–DeGroot–Marschak mechanism is not necessarily incentive compatible, even for non-random goods. Economics Letters, 93(1):6–11, 2006.CrossRefGoogle Scholar
Jacquemet, Nicolas, and l’Haridon, Olivier. Experimental economics. Cambridge University Press, 2018.CrossRefGoogle Scholar
Cathleen, Johnson, Baillon, Aurélien, Bleichrodt, Han, Zhihua, Li, Van Dolder, Dennie, and Wakker, Peter P. PRINCE: An improved method for measuring incentivized preferences. Journal of Risk and Uncertainty, 62(1):1–28, 2021.Google Scholar
Kachurka, Raman, Krawczyk, Michał, and Rachubik, Joanna. State lottery in the lab: an experiment in external validity. Experimental Economics, 24: 1242–1266, 2021.CrossRefGoogle Scholar
Krawczyk, Michał. What should be regarded as deception in experimental economics? Evidence from a survey of researchers and subjects. Journal of Behavioral and Experimental Economics, 79:110–118, 2019.CrossRefGoogle Scholar
Krawczyk, Michał, Blasco, Andrea, Gajderowicz, Tomasz, and Giergiczny, Marek. Europeans’ attitudes towards displaced populations: Evidence from a conjoint experiment on support for temporary protection, 2023. Available at https://ssrn.com/abstract=4564737 or http://dx.doi.org/10.2139/ssrn.4564737.CrossRefGoogle Scholar
Krawczyk, Michał, and Sylwestrzak, Marta. Exploring the role of deliberation time in non-selfish behavior: The double response method. Journal of Behavioral and Experimental Economics, 72:121–134, 2018.CrossRefGoogle Scholar
Levitt, Steven D., and List, John A.. Field experiments in economics: The past, the present, and the future. European Economic Review, 53(1):1–18, 2009.CrossRefGoogle Scholar
Lindley, Dennis V. On a measure of the information provided by an experiment. The Annals of Mathematical Statistics, 27(4):986–1005, 1956.CrossRefGoogle Scholar
List, John. Sometimes winning means knowing when to quit. Wall Street Journal, 30 December 2021.Google Scholar
Mahoney, James, and Acosta, Laura. A regularity theory of causality for the social sciences. Quality & Quantity, 56:1889–1911, 2021.CrossRefGoogle Scholar
Mariel, Petr, Hoyos, David, Meyerhoff, Jürgen, Czajkowski, Mikolaj, Dekker, Thijs, Glenk, Klaus, Bredahl Jacobsen, Jette, Liebe, Ulf, Bøye Olsen, Søren, Sagebiel, Julian, and Thiene, Mara. Environmental valuation with discrete choice experiments: Guidance on design, implementation and data analysis. Springer Nature, 2021.CrossRefGoogle Scholar
Marks, David F., and Colwell, John. The psychic staring effect. Skeptical Inquirer, 24(5):41–49, 2000.Google Scholar
McFadden, Daniel. The measurement of urban travel demand. Journal of Public Economics, 3(4):303–328, 1974.CrossRefGoogle Scholar
McFadden, Daniel. Modeling the choice of residential location. In Karlqvist, Anders, Snickars, Folke, and Weibull, Jürgen, editors, Spatial interaction theory and planning models, pp. 75–96. North Holland, 1978.Google Scholar
Menapace, Luisa, and Raffaelli, Roberta. Unraveling hypothetical bias in discrete choice experiments. Journal of Economic Behavior & Organization, 176:416–430, 2020.CrossRefGoogle Scholar
Meyerhoff, Jürgen, Oehlmann, Malte, and Weller, Priska. The influence of design dimensions on stated choices in an environmental context. Environmental and Resource Economics, 61(3):385–407, 2015.CrossRefGoogle Scholar
Moffatt, Peter G. Experimetrics: Econometrics for experimental economics. Macmillan International Higher Education, 2015.Google Scholar
Overstall, Antony M., and Woods, David C.. Bayesian design of experiments using approximate coordinate exchange. Technometrics, 59(4):458–470, 2017.CrossRefGoogle Scholar
Papoutsi, Georgia Rodolfo, S. Nayga Jr, M., Lazaridis, Panagiotis, and Andreas, C. Drichoutis. Fat tax, subsidy or both? The role of information and children’s pester power in food choice. Journal of Economic Behavior & Organization, 117:196–208, 2015.CrossRefGoogle Scholar
Perny, Patrice, Viappiani, Paolo, and Boukhatem, Abdellah. Incremental preference elicitation for decision making under risk with the rank-dependent utility model. In Ihler, Alexander and Janzing, Dominik, editors, Uncertainty in artificial intelligence, pp. 597–606. AUAI Press for Association for Uncertainty in Artificial Intelligence, 2016.Google Scholar
Shannon, Claude Elwood. A mathematical theory of communication. The Bell System Technical Journal, 27(3):379–423, 1948.CrossRefGoogle Scholar
Shigeoka, Hitoshi, and Yamada, Katsunori. Income-comparison attitudes in the United States and the United Kingdom: Evidence from discrete-choice experiments. Journal of Economic Behavior & Organization, 164:414–438, 2019.CrossRefGoogle Scholar
Smith, Mike D. Biased coin randomization. In Balakrishnan, Narayanaswamy, editor, Methods and applications of statistics in clinical trials. Volume 1: Concepts, principles, trials, and design, pp. 90–105. Wiley, 2014.CrossRefGoogle Scholar
Sokol-Hessner, Peter, Hsu, Ming, Curley, Nina G., Delgado, Mauricio R., Camerer, Colin F., and Phelps, Elizabeth A.. Thinking like a trader selectively reduces individuals’ loss aversion. Proceedings of the National Academy of Sciences, 106(13):5035–5040, 2009.CrossRefGoogle Scholar
Spirtes, Peter, Glymour, Clark, and Scheines, Richard, with additional material by David Heckerman. Causation, prediction, and search. MIT Press, 2001.CrossRefGoogle Scholar
Steimle, Lauren N., Sun, Yuming, Johnson, Lauren, Besedeš, Tibor, Mokhtarian, Patricia, and Nazzal, Dima. Students’ preferences for returning to colleges and universities during the COVID-19 pandemic: A discrete choice experiment. Socio-economic Planning Sciences, page 101266, 2022.CrossRefGoogle Scholar
, Student. Appendix to Mercer and Hall’s paper ‘The experimental error of field trials’. Journal of Agricultural Science, 4, 128–131.Google Scholar
Vandana, Subroy, Rogers, Abbie A., and Kragt, Marit E.. To bait or not to bait: A discrete choice experiment on public preferences for native wildlife and conservation management in Western Australia. Ecological Economics, 147:114–122, 2018.Google Scholar
Thye, Shane. Logical and philosophical foundations of experimental research in the social sciences. In Webster, Murray and Sell, Jane, editors, Laboratory experiments in the social sciences, pp. 53–82. Elsevier, 2014.CrossRefGoogle Scholar
Titchener, Edward Bradford. Experimental psychology: A retrospect. American Journal of Psychology, 36(3):313–323, 1925.CrossRefGoogle Scholar
Toubia, Olivier, Hauser, John, and Garcia, Rosanna. Probabilistic polyhedral methods for adaptive choice-based conjoint analysis: Theory and application. Marketing Science, 26(5):596–610, 2007.CrossRefGoogle Scholar
Toubia, Olivier, Johnson, Eric, Evgeniou, Theodoros, and Delquié, Philippe. Dynamic experiments for estimating preferences: An adaptive method of eliciting time and risk parameters. Management Science, 59(3):613–640, 2013.CrossRefGoogle Scholar
Train, Kenneth E. Recreation demand models with taste differences over people. Land economics, University of Wisconsin Press, 74(2), 230–239, 1998.CrossRefGoogle Scholar
Tversky, Amos, and Kahneman, Daniel. Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5(4):297–323, 1992.CrossRefGoogle Scholar
Amos, Tversky, and Thaler, Richard H.. Anomalies: Preference reversals. Journal of Economic Perspectives, 4(2):201–211, 1990.Google Scholar
Vossler, Christian A., Doyon, Maurice, and Rondeau, Daniel. Truth in consequentiality: Theory and field evidence on discrete choice experiments. American Economic Journal: Microeconomics, 4(4):145–171, 2012.Google Scholar
Vossler, Christian A., and Evans, Mary F.. Bridging the gap between the field and the lab: Environmental goods, policy maker input, and consequentiality. Journal of Environmental Economics and Management, 58(3):338–345, 2009.CrossRefGoogle Scholar
Wakker, Peter, and Deneffe, Daniel. Eliciting von Neumann–Morgenstern utilities when probabilities are distorted or unknown. Management Science, 42(8):1131–1150, 1996.CrossRefGoogle Scholar
Woodward, James. Causation and manipulability. In Edward, N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter edition, 2016.Google Scholar
Ziliak, Stephen T. Field balanced versus randomized field experiments in economics: Why W. S. Gosset aka ‘Student’ matters. Review of Behavioral Economics, 1(1–2):167–208.CrossRefGoogle Scholar