Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-6bf8c574d5-b4m5d Total loading time: 0 Render date: 2025-02-26T18:09:56.771Z Has data issue: false hasContentIssue false

2 - Investigating Differences between Groups

Using ANOVA in EMI Research

from Part I - Theoretical Chapters

Published online by Cambridge University Press:  14 December 2024

By
Gene Thompson  and
Victoria Kim
Edited by
Samantha M. Curle  and
Jack Pun
Samantha M. Curle
Affiliation:
University of Bath
Jack Pun
Affiliation:
City University of Hong Kong
Get access

Summary

Given the various differences between learners, teachers, and instructional methods in English Medium Instruction (EMI), a common purpose of EMI research involves investigating the potential variation between groups. The analysis of variance (ANOVA) test is a common technique used to address such a research aim, as it tests whether there are significant differences between the means of different groups. This chapter introduces the ANOVA test to readers by highlighting how it has been used in research within the field of EMI. To illustrate how different forms of the ANOVA test can be employed, the chapter then provides two case studies: (1) the use of a one-way between subjects ANOVA to examine the differences between three groups of students with respect to their perceptions of the role of English in their academic and career goals and comprehension level of EMI courses; and (2) the use of a mixed ANOVA in a quasi-experimental study that examined the differences in pre- and posttest writing performance and academic motivation of two groups provided with different types of feedback. Each of the case studies summarizes the assumptions required for the use of ANOVA, discusses potential problems that may face EMI researchers, and introduces alternative procedures.

Keywords

analysis of variance (ANOVA)English Medium Instruction (EMI)between-subjects and mixed designsacademic motivationresearch methodology in EMI
Type
Chapter
Information
Researching English Medium Instruction
Quantitative Methods for Students and Researchers
 , pp. 27 - 39
Publisher: Cambridge University Press
Print publication year: 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aizawa, I., Rose, H., Thompson, G., & Curle, S. (2023). Beyond the threshold: Exploring English language proficiency, linguistic challenges, and academic language skills of Japanese students in an English medium instruction programme. Language Teaching Research, 27(4), 837–861. https://doi.org/10.1177/1362168820965510CrossRefGoogle Scholar
Ament, J., Vidal, C. P., & Parés, J. B. (2018). The effects of English-medium instruction on the use of textual and interpersonal pragmatic markers. Pragmatics, 28(4), 517–545. https://doi.org/10.1075/prag.17042.ameGoogle Scholar
Armstrong, R. A. (2017). Recommendations for analysis of repeated‐measures designs: Testing and correcting for sphericity and use of MANOVA and mixed model analysis. Ophthalmic and Physiological Optics, 37(5), 585–593. https://doi.org/10.1111/opo.12399CrossRefGoogle ScholarPubMed
Barrios, E., & López-Gutiérrez, A. (2023). Language learning in a partially English-taught teacher education programme: Language gains and student perceptions. International Journal of Multilingualism, 20(2), 504–521. https://doi.org/10.1080/14790718.2021.1888956CrossRefGoogle Scholar
Brown, J. D. (2008). Effect size and eta squared. Shiken: JALT Testing & Evaluation SIG Newsletter, 12(2), 38–43.Google Scholar
Brown, J. D. (2011). Likert items and scales of measurement? Shiken: JALT Testing & Evaluation SIG Newsletter, 15(1), 10–14.Google Scholar
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155CrossRefGoogle ScholarPubMed
Curle, S., Jablonkai, R., Mittelmeier, J., Sahan, K., & Veitch, A. (2020). English Medium Part 1: Literature review. In N. Galloway (Ed.) English in higher education (Report No. 978-0-86355-977–8). British Council. https://www.teachingenglish.org.uk/article/english-higher-education-%E2%80%93-english-medium-part-1-literature-reviewGoogle Scholar
Dafouz, E., & Camacho-Miñano, M. M. (2016). Exploring the impact of English-medium instruction on university student academic achievement: The case of accounting. English for Specific Purposes, 44, 57–67. https://doi.org/10.1016/j.esp.2016.06.001CrossRefGoogle Scholar
Farrel, P. J., & Stewart, K. R. (2006). Comprehensive study of tests for normality and symmetry: Extending the Spiegelhalter test. Journal of Statistical Computation and Simulation, 76(9), 803–816. https://doi.org/10.1080/10629360500109023CrossRefGoogle Scholar
Field, A. (2013). Discovering statistics using IBM SPSS statistics (4th ed.). Sage.Google Scholar
Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R (1st ed.). Sage.Google Scholar
Fisher, R. A. (1918). The correlation between relatives on the supposition of Mendelian inheritance. Transactions of the Royal Society of Edinburgh, 52, 399–433.CrossRefGoogle Scholar
Galloway, N., Curle, S., Jablonkai, R. R., Mittelmeier, J., Sahan, K., & Veitch, A. (2020a). English in higher education: English medium. Part 1 - Literature review. London: British Council. www.teachingenglish.org.uk/sites/teacheng/files/L020_English_HE_lit_review_FINAL.pdfGoogle Scholar
Galloway, N., Numajiri, T., & Rees, N. (2020b). The “internationalisation,” or “Englishisation,” of higher education in East Asia. Higher Education, 80(3), 395–414. https://doi.org/10.1007/s10734–019-00486-1CrossRefGoogle Scholar
Glantz, S. A., Slinker, B. K., & Neilands, T. B. (2016). Primer of applied regression & analysis of variance (3rd ed.). New York: McGraw-Hill.Google Scholar
Guo, H., Tong, F., Wang, Z., Min, Y., & Tang, S. (2018). English- vs. Chinese-medium instruction in Chinese higher education: A quasi-experimental comparison. Sustainability, 10(11), 4230. https://doi.org/10.3390/su10114230CrossRefGoogle Scholar
Hendriks, B., & van Meurs, F. (2022). Dutch students’ evaluations of EMI and L1MOI lectures: The role of non-native pronunciation. System, 108, 102849.CrossRefGoogle Scholar
Hennebry, M., & Gao, X. (2021). Interactions between medium of instruction and language learning motivation. International Journal of Bilingual Education and Bilingualism, 24(7), 976–989. https://doi.org/10.1080/13670050.2018.1530190CrossRefGoogle Scholar
Howell, D. C. (2002). Statistical methods for psychology. Belmont, CA: Duxbury Press.Google Scholar
Keppel, G. (1991). Design and analysis: A researcher’s handbook (3rd ed.). Englewood Cliffs, New York: Prentice Hall.Google Scholar
Kim, V. (2018). Technology-enhanced feedback on student writing in the English-medium instruction classroom. English Teaching, 73(4), 29–53. https://doi.org/10.15858/engtea.73.4.201812.29CrossRefGoogle Scholar
Kim, V., & Kim, J. Y. (2020). Roles of teacher feedback in promoting effective English-medium instruction of a business subject. Journal of Asia TEFL, 17(3), 889–905. http://dx.doi.org/10.18823/asiatefl.2020.17.3.9.889Google Scholar
Kim, V., & Thompson, G. (2022). The interplay between gender and major on content knowledge and self-beliefs in the English-medium instruction context: A comparative study between university students from Japan and South Korea. System, 107, 102824. http://dx.doi.org/10.1016/J.SYSTEM.2022.102824CrossRefGoogle Scholar
Kojima, N., & Yashima, T. (2017). Motivation in English medium instruction classrooms from the perspective of self-determination theory and the ideal self. JACET Journal, 61, 23–39. https://doi.org/10.32234/jacetjournal.61.0_23Google Scholar
Köylü, Z., & Tracy-Ventura, N. (2022). Learning English in today’s global world: A comparative study of at home, anglophone, and lingua franca study abroad. Studies in Second Language Acquisition, 44(5), 1330–1355.CrossRefGoogle Scholar
Kym, I., & Kym, M. H. (2014). Students’ perceptions of EMI in higher education in Korea. Journal of Asia TEFL, 11(2), 35–61.Google Scholar
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863.CrossRefGoogle Scholar
Larson, M. G. (2008). Analysis of variance. Circulation, 117(1), 115–121. https://doi.org/10.1161/CIRCULATIONAHA.107.654335CrossRefGoogle ScholarPubMed
Lasagabaster, D. (2016). The relationship between motivation, gender, L1 and possible selves in English-medium instruction. International Journal of Multilingualism, 13(3), 315–332. https://doi.org/10.1080/14790718.2015.1105806CrossRefGoogle Scholar
Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared and the misreporting of effect size in communication research. Human Communication Research, 28, 612–625. https://doi.org/10.1111/j.1468-2958.2002.tb00828.xCrossRefGoogle Scholar
Macaro, E., & Akincioglu, M. (2018). Turkish university students’ perceptions about English medium instruction: Exploring year group, gender and university type as variables. Journal of Multilingual and Multicultural Development, 39(3), 256–270. https://doi.org/10.1080/01434632.2017.1367398CrossRefGoogle Scholar
Macaro, E., Curle, S., Pun, J., An, J., & Dearden, J. (2018). A systematic review of English medium instruction in higher education. Language Teaching, 51(01), 36–76. https://doi.org/10.1017/S0261444817000350CrossRefGoogle Scholar
Moulton, S. T. (2012). Mauchly test. In Salkind, N. (Ed.), Encyclopedia of Research Design. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781412961288Google Scholar
Norouzian, R., & Plonsky, L. (2018). Eta-and partial eta-squared in L2 research: A cautionary review and guide to more appropriate usage. Second Language Research, 34(2), 257–271. https://doi.org/10.1177/0267658316684904CrossRefGoogle Scholar
Razali, N. M., & Wah, Y. B. (2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. Journal of Statistical Modeling and Analytics, 2(1), 21–33.Google Scholar
Sahan, K., & Şahan, Ö. (2021). Investigating student and alumni perspectives on language learning and career prospects through English medium instruction. Teaching in Higher Education, 26, 1–22. https://doi.org/10.1080/13562517.2021.1973407Google Scholar
Soruç, A., Altay, M., Curle, S., & Yuksel, D. (2021). Students’ academic language-related challenges in English medium instruction: The role of English proficiency and language gain. System, 103, 102651. https://doi.org/10.1016/j.system.2021.102651CrossRefGoogle Scholar
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston: Allyn & Bacon.Google Scholar
Tabachnick, B. G., & Fidell, L. S. (2020). Experimental designs using ANOVA (3rd ed.). Belmont, CA: Duxbury Press.Google Scholar
Thompson, G. (2018). Insights for efficacy development from an exploration of Japanese business management students’ EAP self-efficacy beliefs. The Asian ESP Journal, 14(7.1), 244–284.Google Scholar
Wei, R., Hu, Y., & Xiong, J. (2019). Effect size reporting practices in applied linguistics: A study of one major journal. SAGE Open, 9, 1–19. https://doi.org/10.1177/2158244019850035CrossRefGoogle Scholar
Zhou, S., Thompson, G., & Zhou, S. (2023). Transition from secondary school to an English medium transnational university in China: A longitudinal study of student self-efficacy and motivational beliefs. International Journal of Bilingual Education and Bilingualism. https://doi.org/10.1080/13670050.2023.2213376CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×