Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2025-01-03T21:21:40.008Z Has data issue: false hasContentIssue false
  • English
  • Français

The Measurement and Communication of Effect Sizes in Management Research

Published online by Cambridge University Press:  21 April 2022

Carl F. Fey Open the ORCID record for Carl F. Fey [Opens in a new window] ,
Tianyou Hu Open the ORCID record for Tianyou Hu [Opens in a new window]  and
Andrew Delios
Carl F. Fey*
Affiliation:
Aalto University, Finland
Tianyou Hu
Affiliation:
Maynooth University, UK
Andrew Delios
Affiliation:
National University of Singapore, Singapore
*
Corresponding author: Cary F. Fey ([email protected])
Get access Rights & Permissions [Opens in a new window]

Abstract

The measurement and communication of the effect size of an independent variable on a dependent variable is critical to effective statistical analysis in the Social Sciences. We develop ideas about how to extend traditional methods of evaluating relationships in multivariate models to explain and illustrate the statistical power of a focal independent variable. Even with a growing acceptance of the need to report effect sizes, scholars in the management community have few well-established protocols or guidelines for reporting effect sizes. In this editorial essay, we: (1) review the necessity of reporting effect sizes; (2) discuss commonly used measures of effect size and accepted cut-offs for large, medium, and small effect sizes; (3) recommend standards for reporting effect sizes via verbal descriptions and graphical presentations; and (4) present best practice examples of reporting and discussing effect size. In summary, we provide guidance for authors on how to report and interpret effect sizes, advocating for rigor and completeness in statistical analysis.

摘要

在社会科学研究中进行有效的数据分析，一个关键点是测量和汇报自变量对应变量影响的强度大小，即效应强度。通过对多变量数据分析方法的汇总和拓展，我们致力于如何阐释、展示效应强度——这个重要的统计学功效。管理学界虽然逐步认可接受汇报效应强度，但仍缺乏完整的操作方法和汇报准则。在此篇社论中，我们主要做了以下工作：1）综述和重申汇报效应强度的必要性； 2）讨论常用的效应强度测量指标，以及学界普遍认可的大、中、小强度的临界值； 3）提出汇报效应强度的文字说明和绘图等具体操作标准； 4）列举一些优秀例子辅以说明。总之，我们在此提出汇报和阐述效应强度的指导性原则，以推进管理学研究中严谨和完整的数据分析。

Keywords

confidence intervaldata visualizationeffect sizestatistical analysisstatistical reporting效应强度数据分析数据汇报置信区间数据可视化
Type
Perspectives
Information
Management and Organization Review , Volume 19 , Issue 1 , February 2023 , pp. 176 - 197
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press on behalf of The International Association for Chinese Management Research

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

ACCEPTED BY Editor-in-Chief Arie Y. Lewin

References

REFERENCES

Acock, A. C. 2014. A gentle introduction to stata. College Station, Texas: Stata Press.Google Scholar
Aguinis, H., Beaty, J. C., Boik, R. J., & Pierce, C. A. 2005. Effect size and power in assessing moderating effects of categorical variables using multiple regression: A 30-year review. Journal of Applied Psychology, 90(1): 94–107.CrossRefGoogle ScholarPubMed
American Educational Research Association. 2006. Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6): 33–40.CrossRefGoogle Scholar
APA. 2020. Publication manual of the American Psychological Association, 7th ed. Washington, DC: American Psychological Association.Google Scholar
Asgari, N., Tandon, V., Singh, K., & Mitchell, W. 2018. Creating and taming discord: How firms manage embedded competition in alliance portfolios to limit alliance termination. Strategic Management Journal, 39(12): 3273–3299.CrossRefGoogle Scholar
Baum, J., & Bromiley, P. 2019. P-hacking in top-tier management journals. Academy of Management Annual Meeting Proceedings. doi:10.5465/AMBPP.2019.10810abstractGoogle Scholar
Bettis, R. A., Ethiraj, S., Gambardella, A., Helfat, C., & Mitchell, W. 2016. Creating repeatable cumulative knowledge in strategic management. Strategic Management Journal, 37(2): 257–26.CrossRefGoogle Scholar
Boßow-Thies, S., & Albers, S. 2010. Application of PLS in marketing: Content strategies on the internet. In Vinzi, V. E., Chin, W. W., Henseler, J., & Wang, H. (Eds.), Handbook of partial least squares: 589–604. Berlin, Germany: Springer.Google Scholar
Card, D. 1992. Do minimum wages reduce employment? A case study of California, 1987–89. Industrial and Labor Relations Review, 46(1): 38–54.Google Scholar
Chatterji, A., Delecourt, S., Hasan, S., & Koning, R. 2019. When does advice impact startup performance? Strategic Management Journal, 40(3): 331–356.Google Scholar
Chen, D., Dai, L., & Li, D. 2019. A delicate balance for innovation: Competition and collaboration in R&D consortia. Management and Organization Review, 15(1): 145–176.CrossRefGoogle Scholar
Chin, W. W. 1998. The partial least squares approach to structural equation modeling. In Marcoulides, G. A. (Ed.), Modern methods for business research: 295–336. New York: Taylor & Francis Group.Google Scholar
Cohen, J. 1988. Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.Google Scholar
Cohen, J. 1992. Quantitative methods in psychology: A power primer. Psychological Bulletin, 112(1): 155–159.Google Scholar
Dezsö, C. L., & Ross, D. G. 2012. Does female representation in top management improve firm performance? A panel data investigation. Strategic Management Journal, 33(9): 1072–1089.Google Scholar
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. 1996. Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1(2): 170–177.Google Scholar
Ellis, P. D. 2010. Effect sizes and the interpretation of research results in international business. Journal of International Business Studies, 41(9): 1581–1588.CrossRefGoogle Scholar
Ferguson, C. J. 2016. An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40(5): 532–538.CrossRefGoogle Scholar
Fern, E. F., & Monroe, K. B. 1996. Effect-size estimates: Issues and problems in interpretation. Journal of Consumer Research, 23(2): 89–105.CrossRefGoogle Scholar
Fisher, R. A. 1946. Statistical methods for research workers. Statistical methods for research workers, 10th ed. London, UK: Gyan Books.Google Scholar
Gignac, G. E., & Szodorai, E. T. 2016. Effect size guidelines for individual differences researchers. Personality and Individual Differences, 102: 74–78.CrossRefGoogle Scholar
Gupta, A., & Misangyi, V. F. 2018. Follow the leader (or not): The influence of peer CEOs’ characteristics on interorganizational imitation. Strategic Management Journal, 39(5): 1437–1472.CrossRefGoogle Scholar
Hemphill, J. F. 2003. Interpreting the magnitudes of correlation coefficients. American Psychologist, 58(1): 78–79.Google ScholarPubMed
Hu, Q., Zhang, Y., & Yao, J. 2018. Family involvement in middle management and its impact on the labor productivity of family firms. Management and Organization Review, 14(2): 249–274.CrossRefGoogle Scholar
Kaiser, U., Kongsted, H. C., Laursen, K., & Ejsing, A. K. 2018. Experience matters: The role of academic scientist mobility for industrial innovation. Strategic Management Journal, 39(7): 1935–1958.CrossRefGoogle Scholar
Kelley, K., & Preacher, K. J. 2012. On effect size. Psychological Methods, 17(2): 137–152.CrossRefGoogle ScholarPubMed
Lewin, A. Y., Chiu, C.-Y., Fey, C. F., Levine, S. S., Mcdermott, G., Murmann, J. P., & Tsang, E. 2016. The critique of empirical social science: New policies at management and organization review. Management and Organization Review, 12(4): 649–658.CrossRefGoogle Scholar
Li-Ying, J., Zhang, Z., & Long, Q. 2018. An alternative way to make knowledge sharing work in online communities? The effects of hidden knowledge facilitators. Management and Organization Review, 14(4): 781–825.CrossRefGoogle Scholar
Makino, S., & Delios, A. 1996. Local knowledge transfer and performance: Implications for alliance formation in Asia. Journal of International Business Studies, 27(5): 905–927.CrossRefGoogle Scholar
Meyer, K. E., Van Witteloostuijn, A., & Beugelsdijk, S. 2017. What's in a p? Reassessing best practices for conducting and reporting hypothesis-testing research. Journal of International Business Studies, 48(5): 535–551.CrossRefGoogle Scholar
Nakagawa, S., & Cuthill, I. C. 2007. Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews, 82(4): 591–605.CrossRefGoogle ScholarPubMed
Nieminen, P., Lehtiniemi, H., Vähäkangas, K., Huusko, A., & Rautio, A. 2013. Standardised regression coefficient as an effect size index in summarising findings in epidemiological studies. Epidemiology, Biostatistics and Public Health, 10(4). doi:10.2427/8854.Google Scholar
OBHDP Editors. 2021. Author guidelines for Organizational Behavior and Human Decision Processes. [Cited 24 December 2021]. Available from URL: www.elsevier.com/wps/find/journaldescription.cws_home/622929?generatepdf=trueGoogle Scholar
Okada, K. 2013. Is omega squared less biased? A comparison of three major effect size indices in one-way ANOVA. Behaviormetrika, 40(2): 129–147.CrossRefGoogle Scholar
Olejnik, S., & Algina, J. 2003. Generalized eta and omega squared statistics: Measures of effect size for some common research designs. Psychological Methods, 8(4): 434–447.CrossRefGoogle ScholarPubMed
Pearson, K. 1911. On a correction to be made to the correlation ratio η. Biometrika, 8(1/2): 254–256.Google Scholar
Prashantham, S., Zhou, A. J., & Dhanaraj, C. 2020. Depth vs. breadth: Network strategy in emerging markets. Management and Organization Review, 16(2): 229–260.CrossRefGoogle Scholar
Rea, L. M., & Parker, R. A. 2014. Designing and conducting survey research: A comprehensive guide. San Francisco: Jossey-Bass.Google Scholar
Rosenthal, R. 1994. Parametric measures of effect size. In Cooper, H., & Hedges, L. (Eds.), The handbook of research synthesis: 231–244. New York: Sage.Google Scholar
Rosenthal, R., Cooper, H., & Hedges, L. 1994. Parametric measures of effect size. The Handbook of Research Synthesis, 621(2): 231–244.Google Scholar
Sampson, R. C. 2007. R&D alliances and firm performance: The impact of technological diversity and alliance organization on innovation. Academy of Management Journal, 50(2): 364–386.CrossRefGoogle Scholar
Song, Y., Liu, Y., Wang, M., Lanaj, K., Johnson, R. E., & Shi, J. 2018. A social mindfulness approach to understanding experienced customer mistreatment: A within-person field experiment. Academy of Management Journal, 61(3): 994–1020.CrossRefGoogle Scholar
Sullivan, G. M., & Feinn, R. 2012. Using effect size – or why the P value is not enough. Journal of Graduate Medical Education, 4(3): 279–282.CrossRefGoogle ScholarPubMed
Sun, S., Pan, W., & Wang, L. L. 2010. A comprehensive review of effect size reporting and interpreting practices in academic journals in education and psychology. Journal of Educational Psychology, 102(4): 989–1004.CrossRefGoogle Scholar
Thompson, D., Bhatt, R., Lazarus, M. M., Cullen, M. K., Baranowski, D. J., & Baranowski, M. T. 1994. Guidelines for authors. Educational and Psychological Measurement, 54: 837–847.Google Scholar
Vanneste, B., & Gulati, R. 2021. Generalized trust, external sourcing, and firm performance in economic downturns. Organization Science. doi:10.1287/orsc.2021.1500Google Scholar
Wetzels, M., Odekerken-Schröder, G., & Van Oppen, C. 2009. Using PLS path modeling for assessing hierarchical construct models: Guidelines and empirical illustration. Management Information Systems Quarterly, 33(1): 177–195.CrossRefGoogle Scholar
Wilkinson, L. 1999. Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8): 594–604.CrossRefGoogle Scholar
Wilson, D. B. 2010. Meta-analysis. In Piquero, A. R. & Weisburd, D. (Eds.), Handbook of quantitative criminology: 181–208. New York: Springer.CrossRefGoogle Scholar
Yigit, S., & Mendes, M. 2018. Which effect size measure is appropriate for one-way and two-way ANOVA models? A Monte Carlo simulation study. Revstat Statistical Journal, 16(3): 295–313.Google Scholar