Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-22T21:15:42.097Z Has data issue: false hasContentIssue false

7 - Analyzing Questionnaire Data through Many-Facet Rasch Measurement: A Pilot Study of Students’ Attitudes toward EMI in the Chinese Higher Education Context

from Part II - Empirical Chapters (Case Studies)

Published online by Cambridge University Press:  14 December 2024

By
Jie Gao
Edited by
Samantha M. Curle  and
Jack Pun
Samantha M. Curle
Affiliation:
University of Bath
Jack Pun
Affiliation:
City University of Hong Kong
Get access

Summary

Questionnaires have been widely used to tap into fine-grained themes of educational studies (see also Chapters 3, 8, and 11 of the this book). Unfitted use of parametric methods, however, would result in mistakenly interpreting ordinal scales as equal intervals, or difficulties in resolving issues such as missing raw data. Rasch measurement, as one of several item response theoretic models, is strongly suggested for application prior to the conduction of parametric statistical tests (Boone, Staver, & Yale, 2014).

This chapter starts with introducing basic facts about Rasch modelling, and discusses three study cases that applied Rasch modeling for scale development and validation.

In addition, this chapter features a step-by-step analysis procedure for data collected from a questionnaire, which is administered among 102 undergraduate students enrolled in a university located in Shanghai, China. All of the students have registered for at least one course instructed in English, which is related to their major area of study. The questionnaire was adapted from the Japanese English Medium of Instruction Attitude Scale (JEMIAS) (Curle, 2018). Multi-facet Rasch Measurement (MFRM) analysis was conducted to investigate the possible influence of students’ academic major on their attitude toward EMI, as well as the functioning of individual items on the scale. Analysis results show that students’ disciplinary background has limited influence on their attitudes. Items demonstrating different logit scales, however, provide practical implications for designing EMI courses in Chinese higher education institutions.

Keywords

English Medium of InstructionMulti-facet Rasch MeasurementQuestionnaire
Type
Chapter
Information
Researching English Medium Instruction
Quantitative Methods for Students and Researchers
 , pp. 107 - 133
Publisher: Cambridge University Press
Print publication year: 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Andrich, D. (2004). Controversy and the Rasch model: A characteristic of incompatible paradigms? In Smith, E. V., Jr., & Smith, R. M., (Eds.), Introduction to Rasch measurement (pp. 143–166). Maple Grove, MN: JAM Press.Google Scholar
Bond, T., & Fox, C. (2001). Applying the Rasch model: Fundamental measures in human sciences. Mahwah, NJ: Lawrence Erlbaum Associates.CrossRefGoogle Scholar
Boone, W., & Staver, J. (2020). Advances in Rasch analyses in the human sciences. Cham, Switzerland: Springer.CrossRefGoogle Scholar
Boone, W., Staver, J., & Yale, M. (2014). Rasch analysis in the human sciences. Cham, Switzerland: Springer.CrossRefGoogle Scholar
Boone, W., Townsend, J. S., & Staver, J. (2011). Using Rasch theory to guide the practice of survey development and survey data analysis in science education and to inform science reform efforts: An exemplar utilizing STEBI self-efficacy data. Science Education, 95(2), 258–280.CrossRefGoogle Scholar
Coleman, J., Hultgren, K., Li, W., Tsui, C.-F. C., & Shaw, P. (2018). Forum on English-medium Instruction. TESOL Quarterly, 52(3), 701–720.CrossRefGoogle Scholar
Curle, S. (2018). Developing and validating a Japanese English Medium of Instruction; Scale (JEMIAS). The Journal of Asia TEFL, 15(4), 1195–1221.Google Scholar
Curle, S., & Derakhshan, A. (2022). Trends in using questionnaires for EMI research. In Pun, J. K. H. & Curle, S. (Eds.), Research methods in English medium instruction (pp. 32–45). New York: Routledge.Google Scholar
Dafouz, E. (2014). Integrating content and language in European higher education: An overview of recurrent research concerns and pending issues. In Psaltou-Joycey, A., Agathopoulou, E., & Mattheoudakis, M. (Eds.) Cross-curricular approaches to language education (pp. 289–305). Newcastle, UK: Cambridge Scholars.Google Scholar
Dörnyei, Z., & Taguchi, T. (2009). Questionnaires in second language research: Construction, administration, and processing. New York: Routledge.CrossRefGoogle Scholar
Eckes, T. (2011). Introduction to many-facet Rasch measurement: Analyzing evaluation rater-mediated assessments. Frankfurt, Germany: Peter Lang.CrossRefGoogle Scholar
Enochs, L. G., & Riggs, I. M. (1990). Further development of an elementary science teaching efficacy belief instrument: A preservice elementary scale. School Science and Mathematics, 90(8), 694–706.CrossRefGoogle Scholar
Houston, J., Harris, P., McIntire, S., & Francis, D. (2002). Revising the competitiveness index using factor analysis. Psychological Reports, 90(1), 31–34.CrossRefGoogle ScholarPubMed
Khabbazbashi, N. (2017). Topic and background knowledge effects on performance in speaking assessment. Language Testing, 34(1), 23–48.CrossRefGoogle Scholar
Kong, M., & Wei, R. (2019). EFL learners’ attitudes toward English medium instruction in China: The influence of sociobiographical variables. Linguistic and Education, 52, 44–51.CrossRefGoogle Scholar
Krägeloh, C. D., Medvedev, O. N., Hill, E., Webster, C. S., Booth, R., & Henning, M. A. (2018). Improving measurement of trait competitiveness: A Rasch analysis of the revised competitiveness index with samples from New Zealand and US university students, Psychological Reports, 122(2), 689–708.CrossRefGoogle ScholarPubMed
Kreiner, S. (2013). The Rasch model for dichotomous items. In Chirstenson, K. B., Kreiner, S., & Mesbah, M. (Eds.), Rasch models in health. London, UK: Wiley.Google Scholar
Linacre, J. M. (2012). Many-facet Rasch Measurement: facets Tutorial. http://winsteps.com/tutorials.htmGoogle Scholar
McKinley, J., Rose, H., & Zhou, S. (2021). Transnational universities and English Medium Instruction in China: How admission, language support and language use differ in Chinese universities. RELC Journal, 52(2), 236–252.CrossRefGoogle Scholar
Nielson, T., Garcia, I. M., & Alastor, E. (2021). Critical thinking of psychology students: A within and cross-cultural study using Rasch models. Scandinavian Journal of Psychology, 62, 426–435.CrossRefGoogle Scholar
Shaw, F. (1991). Descriptive IRT vs. prescriptive Rasch. Rasch Measurement Transactions, 10, 512–514.Google Scholar
Upegui-Arango, L., Forkmann, T., Nielson, T., Hallensleben, N., Glaesmer, H., Spangenberg, L., Telsmann, T., Jukef, G., & Boecker, M. (2019). Psychometric evaluation of the Interpersonal Needs Questionnaire (INQ) using item analysis according to the Rasch model. Plos One, 15(8), e0232030–e0232030. https://doi.org/10.1371/journal.pone.0232030CrossRefGoogle Scholar
Yan, X. (2014). An examination of rater performance on a local oral English proficiency test: A mixed-methods approach. Language Testing, 31(4), 501–527.CrossRefGoogle Scholar
Zhao, Y., Huen, J. M. Y., & Chan, Y. W. (2017). Measuring longitudinal gains in student learning: A comparison of Rasch scoring and summative scoring approaches. Research in Higher Education, 58(6), 605–616.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×