Hostname: page-component-669899f699-7tmb6 Total loading time: 0 Render date: 2025-04-30T15:13:52.075Z Has data issue: false hasContentIssue false

Do participation rates vary with participation payments in laboratory experiments?

Published online by Cambridge University Press:  02 April 2025

Huizhen Zhong*
Affiliation:
College of Economics and Management, South China Agricultural University, Guangzhou 510642, China College of Business, Shaoguan University, Shaoguan 512005, China
Cary Deck*
Affiliation:
Department of Economics, Finance and Legal Studies, University of Alabama, Tuscaloosa 35487-0224, USA Economic Science Institute, Chapman University, Orange, USA
Daniel J. Henderson*
Affiliation:
Department of Economics, Finance and Legal Studies, University of Alabama, Tuscaloosa 35487-0224, USA Institute for the Study of Labor (IZA), Bonn, Germany

Abstract

This paper reports a series of experiments designed to evaluate how the advertised participation payment impacts participation rates in laboratory experiments. Our initial goal was to generate variation in the participation rate as a means to control for selection bias when evaluating treatment effects in common laboratory experiments. Initially, we varied the advertised participation payment to 1734 people from $5 to $15 using standard email recruitment procedures, but found no statistical evidence this impacted the participation rate. A second study increased the advertised payment up to $100. Here, we find marginally significant statistical evidence that the advertised participation payment affects the participation rate when payments are large. To combat skepticism of our results, we also conducted a third study in which verbal offers were made. Here, we found no statistically significant increase in participation rates when the participation payment increased from $5 to $10. Finally, we conducted an experiment similar to the first one at a separate university. We found no statistically significant increase in participation rates when the participation payment increased from $7 to $15. The combined results from our four experiments suggest moderate variation in the advertised participation payment from standard levels has little impact on participation rates in typical laboratory experiments. Rather, generating useful variation in participation rates likely requires much larger participation payments and/or larger potential subject pools than are common in laboratory experiments.

Type
Original Paper
Copyright
Copyright © The Author(s), under exclusive licence to Economic Science Association 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Allred, S., Duffy, S., & Smith, J.. (2016). Cognitive load and strategic sophistication. Journal of Economic Behavior & Organization, 125, 162178, 10.1016/j.jebo.2016.02.006.CrossRefGoogle Scholar
Andersen, S., Harrison, G.W., Lau, M.I., & Elisabet Rutström, E.. (2010). Preference heterogeneity in experiments: Comparing the field and laboratory. Journal of Economic Behavior & Organization, 73, (2, 209224, 10.1016/j.jebo.2009.09.006.CrossRefGoogle Scholar
Azar, O.. (2019). Do fixed payments affect effort? Examining relative thinking in mixed compensation schemes. Journal of Economic Psychology, 70, 5266, 10.1016/j.joep.2018.10.004.CrossRefGoogle Scholar
Carpenter, J., Graham, M., & Wolf, J.. (2013). Cognitive ability and strategic sophistication. Games and Economic Behavior, 80, 115130, 10.1016/j.geb.2013.02.012.CrossRefGoogle Scholar
Cleave, B.L., Nikiforakis, N., & Slonim, R.. (2013). Is there selection bias in laboratory experiments? the case of social and risk preferences. Experimental Economics, 16, (3, 372382, 10.1007/s10683-012-9342-8.CrossRefGoogle Scholar
Crosetto, P., & Filippin, A.. (2013). The “bomb” risk elicitation task. Journal of Risk and Uncertainty, 47, (1, 3165, 10.1007/s11166-013-9170-z.CrossRefGoogle Scholar
Crosetto, P., & Filippin, A.. (2016). A theoretical and experimental appraisal of four risk elicitation methods. Experimental Economics, 19, (3, 613641, 10.1007/s10683-015-9457-9.CrossRefGoogle Scholar
Duffy, S., & Smith, J.. (2014). Cognitive load in the multi-player prisoner's dilemma game: Are there brains in games?. Journal of Behavioral and Experimental Economics, 51, 4756, 10.1016/j.socec.2014.01.006.CrossRefGoogle Scholar
Eckel, C.C., & Grossman, P.J.. (1996). Altruism in anonymous dictator games. Games and Economic Behavior, 16, (2, 181191, 10.1006/game.1996.0081.CrossRefGoogle Scholar
Eckel, C.C., & Grossman, P.J.. (2003). Rebate versus matching: Does how we subsidize charitable contributions matter?. Journal of Public Economics, 87, (3–4, 681701, 10.1016/S0047-2727(01)00094-9.CrossRefGoogle Scholar
Eckel, C.C., & Grossman, P.J.. (2006). Subsidizing charitable giving with rebates or matching: Further laboratory evidence. Southern Economic Journal, 72, (4, 794807.Google Scholar
Eckel, C.C., & Grossman, P.J.. (2008). Subsidizing charitable contributions: A natural field experiment comparing matching and rebate subsidies. Experimental Economics, 11, (3, 234252, 10.1007/s10683-008-9198-0.CrossRefGoogle Scholar
Falk, A., Meier, S., & Zehnder, C.. (2013). Do lab experiments misrepresent social preferences? The case of self-selected student samples. Journal of the European Economic Association, 11, (4, 839852, 10.1111/jeea.12019.CrossRefGoogle Scholar
Harrison, G.W., Lau, M.I., & Elisabet Rutström, E.. (2009). Risk attitudes, randomization to treatment, and self-selection into experiments. Journal of Economic Behavior & Organization, 70, (3, 498507, 10.1016/j.jebo.2008.02.011.CrossRefGoogle Scholar
Harrison, G.W., Lau, M.I., & Yoo, H.I.. (2020). Risk attitudes, sample selection, and attrition in a longitudinal field experiment. Review of Economics and Statistics, 102, (3, 552568, 10.1162/rest_a_00845.CrossRefGoogle Scholar
Heckman, J.J.. (1979). Sample selection bias as a specification error. Econometrica, 47, 153161, 10.2307/1912352.CrossRefGoogle Scholar
Henderson, D.J., & Sperlich, S.. (2023). A complete framework for model-free difference-in-differences estimation. Foundations and Trends® in Econometrics, 12, (3, 232323, 10.1561/0800000046.CrossRefGoogle Scholar
Holt, C.A., & Laury, S.K.. (2002). Risk aversion and incentive effects. American Economic Review, 92, (5, 16441655, 10.1257/000282802762024700.CrossRefGoogle Scholar
Santos Silva, J.M.C., & Tenreyro, S.. (2006). The log of gravity. Review of Economics and Statistics, 88, (4, 641658, 10.1162/rest.88.4.641.CrossRefGoogle Scholar
Slonim, R., Wang, C., Garbarino, E., & Merrett, D.. (2013). Opting-in: Participation bias in economic experiments. Journal of Economic Behavior & Organization, 90, 4370, 10.1016/j.jebo.2013.03.013.CrossRefGoogle Scholar
Snowberg, E., & Yariv, L.. (2021). Testing the waters: Behavior across participant pools. American Economic Review, 111, (2, 687719, 10.1257/aer.20181065.CrossRefGoogle Scholar
Supplementary material: File

Zhong et al. supplementary material

Zhong et al. supplementary material
Download Zhong et al. supplementary material(File)
File 188.1 KB