Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-19T06:26:59.734Z Has data issue: false hasContentIssue false
  • English
  • Français

ELICITED IMITATION AS A MEASURE OF L2 PROFICIENCY

NEW INSIGHTS FROM A COMPARISON OF TWO L2 ENGLISH PARALLEL FORMS

Published online by Cambridge University Press:  28 July 2021

Shu-Ling Wu Open the ORCID record for Shu-Ling Wu [Opens in a new window] ,
Yee Pin Tio  and
Lourdes Ortega
Shu-Ling Wu*
Affiliation:
Southern Illinois University Carbondale
Yee Pin Tio*
Affiliation:
Southern Illinois University Carbondale
Lourdes Ortega*
Affiliation:
Georgetown University
*
*Correspondence concerning this article should be addressed to Shu-Ling Wu, PhD, Associate Professor and Chinese Section Head, Director of Foreign Language and International Trade Program, Department of Languages, Cultures, and International Trade, Southern Illinois University Carbondale, Carbondale, Illinois 62901. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Elicited imitation (EI), a short-cut measure of global proficiency in second language (L2) research, requires participants to listen to sentences and repeat them as closely as possible. To support instrument sharing and assessment of L2 proficiency for longitudinal and crosslinguistic research, we created a parallel form of an EI task (EIT) for L2 English originally developed by the third author and colleagues and investigated the reliability and validity of the original and new forms. Eighty-two participants completed the two EITs, an oral narrative task, and a self-diagnostic survey. Both forms exhibited high reliability and good alignment with external criterion measures. Both distinguished well among four proficiency levels in the sample. Further, participants’ perception of EI difficulty aligned well with their EI scores. We suggest some improvements to boost forms equivalence and discuss new insights about the nature of EI as reconstructive, integrative, modality independent, and with indirect links to communicative abilities. Our study seeks to make this English EIT instrument widely useful to the L2 research community.

Type
Methods Forum
Information
Studies in Second Language Acquisition , Volume 44 , Issue 1 , March 2022 , pp. 271 - 300
Check for updates
Open Practices
Open materials
Copyright
© The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The experiment in this article earned an Open Materials badge for transparent practices. The materials are available at the IRIS database (http://www.iris-database.org).

The authors thank I-Ru Su, Chien-Hui Hsu, Chenyue Guo, Shu-Ping Wu, and Wentao Li for their help with data collection, and all participants for their time. We are also very grateful to the editors and the anonymous reviewers for their constructive comments and suggestions. Any remaining errors or oversights are our own.

†

This article has been updated since its original publication. See https://doi.org/10.1017/S0272263121000607.

References

REFERENCES

ACTFL. (2015). NCSSFL-ACTFL Can-Do statements: progress indicators for language learners [electronic version]. American Council on the Teaching of Foreign Languages.Google Scholar
Aldenderfer, M. S., & Blashfield, R. K. (1984). Cluster analysis. Sage.CrossRefGoogle Scholar
Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge University Press.Google Scholar
Audacity Team. (2012). Audacity® version 2.0.0. audio editor and recorder. http://audacityteam.org/ Google Scholar
Barcroft, J. (2015). Lexical input processing and vocabulary learning. John Benjamins.CrossRefGoogle Scholar
Baten, K., & Cornillie, F. (2019). Elicited imitation as a window into developmental stages. Journal of the European Second Language Association, 3, 23–34.CrossRefGoogle Scholar
Berman, R. A., & Slobin, D. I. (1994). Relating events in narrative: A crosslinguistic developmental study. Lawrence Erlbaum.Google Scholar
Bley-Vroman, R., & Chaudron, C. (1994). Elicited imitation as a measure of second-language competence. In Tarone, E. E., Gass, S. M., & Cohen, A. D. (Eds.), Research methodology in second-language acquisition (pp. 245–261). Lawrence Erlbaum.Google Scholar
Bock, J. K. (1986). Syntactic persistence in language production. Cognitive Psychology, 18, 355–387.CrossRefGoogle Scholar
Bowden, H. W. (2016). Assessing second-language oral proficiency for research: The Spanish elicited imitation task. Studies in Second Language Acquisition, 38, 647–675.CrossRefGoogle Scholar
Brown, J. D. (2003). Norm-referenced item analysis (item facilitation and item discrimination). Shiken: JALT Testing and Evaluation SIG Newsletter , 7, 16–19.Google Scholar
Bulté, B., & Roothooft, H. (2020). Investigating the interrelationship between rated L2 proficiency and linguistic complexity in L2 speech. System, 91, 102246.CrossRefGoogle Scholar
Clark, J. L. D. (1981). Language. In Barrows, T. S. (Ed.), College students’ knowledge and beliefs: A survey of global understanding. The final report of the Global Understanding Project (pp. 25–35, 87–100). Change Magazine Press.Google Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.Google Scholar
Collins Online Dictionary. (2020). HarperCollins Publishers. https://www.collinsdictionary.com/us/dictionary/english Google Scholar
Denies, K., & Janssen, R. (2016). Country and gender differences in the functioning of CEFR-based Can-Do statements as a tool for self-assessing English proficiency. Language Assessment Quarterly, 13, 251–276.CrossRefGoogle Scholar
DeVellis, R. F. (2003). Scale development theory and applications (2nd ed.). Sage.Google Scholar
Deygers, B. (2020). Elicited imitation: A test for all learners? Examining the EI performance of learners with diverging educational backgrounds. Studies in Second Language Acquisition, 42, 933–957. https://doi.org/10.1017/S027226312000008X CrossRefGoogle Scholar
Díez-Bedmar, M. B., & Pérez-Paredes, P. (2020). Noun phrase complexity in young Spanish EFL learners’ writing: Complementing syntactic complexity indices with corpus-driven analyses. International Journal of Corpus Linguistics, 25, 4–35.CrossRefGoogle Scholar
Drackert, A. (2015). Validating language proficiency assessments in second language acquisition research. Peter Lang.Google Scholar
Drackert, A., & Timukova, A. (2020). What does the analysis of C-test gaps tell us about the construct of a C-test? A comparison of foreign and heritage language learners’ performance. Language Testing, 37, 107–132.CrossRefGoogle Scholar
Duran-Karaoz, Z., & Tavakoli, P. (2020). Predicting L2 fluency from L1 fluency behaviour: The case of L1 Turkish and L2 English speakers. Studies in Second Language Acquisition, 42, 671–695.CrossRefGoogle Scholar
Ebel, R. L., & Frisbie, D. A. (1986). Essentials of educational measurement. Prentice-Hall.Google Scholar
Ellis, N. C. (2017). Cognition, corpora, and computing: Triangulating research in usage-based language learning. Language Learning, 67, 40–65.CrossRefGoogle Scholar
Ellis, R. (2005). Measuring implicit and explicit knowledge of a second language: A psychometric study. Studies in Second Language Acquisition, 27, 141–172.CrossRefGoogle Scholar
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.CrossRefGoogle ScholarPubMed
Fraser, C., Bellugi, U., & Brown, R. (1963). Control of grammar in imitation, comprehension, and production. Journal of Verbal Learning and Verbal Behavior, 2, 121–135.CrossRefGoogle Scholar
Gaillard, S., & Tremblay, A. (2016). Linguistic proficiency assessment in second language acquisition research: The elicited imitation task. Language Learning, 66, 419–447.CrossRefGoogle Scholar
Graham, C. R., McGhee, J., & Millard, B. (2010). The role of lexical choice in elicited imitation item difficulty. In Prior, M. T., Watanabe, Y., & Lee, S.-K. (Eds.), Selected proceedings of the 2008 Second Language Research Forum (pp. 57–72). Cascadilla Proceedings Project.Google Scholar
Huensch, A., & Tracy-Ventura, N. (2017). Understanding second language fluency behavior: The effects of individual differences in first language fluency, cross-linguistic differences, and proficiency over time. Applied PsychoLinguistics, 38, 755–785.CrossRefGoogle Scholar
Hulstijn, J. H. (2012). The construct of language proficiency in the study of bilingualism from a cognitive perspective. Bilingualism: Language and Cognition, 15, 422–433.CrossRefGoogle Scholar
Hulstijn, J. H. (2015). Language proficiency in native and non-native speakers: Theory and research. John Benjamins.CrossRefGoogle Scholar
Iyengar, S., & Greenhouse, J. B. (2009). Sensitivity analysis and diagnostics. In Harris, C., , L. Hedges, , & Valentine, J. (Eds.), Handbook of research synthesis and meta-analysis (pp. 417–433). Russell Sage Foundation.Google Scholar
Iwashita, N. (2006). Syntactic complexity measures and their relation to oral proficiency in Japanese as a foreign language. Language Assessment Quarterly , 3, 151–169.CrossRefGoogle Scholar
Kim, Y., Tracy-Ventura, N., & Jung, Y. (2016). A measure of proficiency or short-term memory? Validation of an elicited imitation test for SLA research. Modern Language Journal, 100, 655–673.CrossRefGoogle Scholar
Klem, M., Melby-Lervåg, M., Hagtvet, B., Lyster, S. A. H., Gustafsson, J. E., & Hulme, C. (2015). Sentence repetition is a measure of children’s language skills rather than working memory limitations. Developmental Science, 18, 146–154.CrossRefGoogle ScholarPubMed
Leclercq, P., Edmonds, A., & Hilton, H. (Eds.). (2014). Measuring L2 proficiency: Perspectives from SLA. Multilingual Matters.CrossRefGoogle Scholar
Long, A. Y., & Geeslin, K. (2017). Spanish second language acquisition across the globe: What future research on non-English speaking learners will tell us. Hispania, 100, 205–210.CrossRefGoogle Scholar
Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15, 474–496.CrossRefGoogle Scholar
Ma, W., & Winke, P. (2019). Self-assessment: How reliable is it in assessing oral proficiency over time? Foreign Language Annals, 52, 66–86.CrossRefGoogle Scholar
Mahowald, K., James, A., Futrell, R., & Gibson, E. (2016). A meta-analysis of syntactic priming in language production. Journal of Memory and Language, 91, 5–27.CrossRefGoogle Scholar
Marsden, E., Mackey, A., & Plonsky, L. D. (2016). The IRIS repository: Advancing research practice and methodology. In Mackey, A. & Marsden, E. (Eds.), Advancing methodology and practice: The IRIS repository of instruments for research into second languages (pp. 1–21). Routledge.Google Scholar
Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68, 321–391.CrossRefGoogle Scholar
McCowan, R., & McCowan, S. (1999). Item analysis for criterion-referenced tests. Research Foundation of SUNY/Center for Development of Human Services.Google Scholar
McManus, K., & Liu, Y. (2020). Using elicited imitation to measure global oral proficiency in SLA research: A close replication study. Language Teaching. Advance online publication. https://doi.org/10.1017/S026144482000021X Google Scholar
Naiman, N. (1974). The use of elicited imitation in second language acquisition research. Working Papers on Bilingualism, 2, 1–37.Google Scholar
Norris, J. M., & Ortega, L. (2012). Assessing learner knowledge. In Gass, S. M. & Mackey, A. (Eds.), The Routledge handbook of second language acquisition (pp. 573–589). Routledge.Google Scholar
Okura, E., & Lonsdale, D. (2012). Working memory’s meager involvement in sentence repetition tests. In Miyake, N., Peebles, D, & Cooper, R. P. (Eds.), Proceedings of the 34th Annual Conference of the Cognitive Science Society (pp. 2132–2137). Cognitive Science Society.Google Scholar
Ortega, L., Iwashita, N., Norris, J. M., & Rabie, S. (2002). An investigation of elicited imitation tasks in crosslinguistic SLA research. Paper presented at the 20th Second Language Research Forum, Toronto, October 3–6, 2002 [Unpublished handout retrieved from IRIS].Google Scholar
Oswald, F. L., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 30, 85–110.CrossRefGoogle Scholar
Park, H. I. (2020). How do Korean–English bilinguals speak and think about motion events? Evidence from verbal and non-verbal tasks. Bilingualism: Language and Cognition, 23, 483–499.CrossRefGoogle Scholar
Park, H. I., Solon, M., Henderson, C., & Dehghan-Chaleshtori, M. (2020). The roles of working memory and oral language abilities in elicited imitation performance. Modern Language Journal, 104, 133–151.CrossRefGoogle Scholar
Plonsky, L., Marsden, E., Crowther, D., Gass, S. M., & Spinner, P. (2019). A methodological synthesis and meta-analysis of judgment tasks in second language research. Second Language Research, 1–39. https://doi.org/10.1177/0267658319828413 Google Scholar
Porte, G., & McManus, K. (2018). Doing replication research in applied linguistics. Routledge.CrossRefGoogle Scholar
Price, P., Jhangiani, R., & Chiang, I. (2015). Research methods in psychology (2nd Canadian ed.). BC Campus. https://kora.kpu.ca/islandora/object/kora%3A384/datastream/PDF/view Google Scholar
Purpura, J. E. (2016). Second and foreign language assessment. Modern Language Journal, 100, 190–208.CrossRefGoogle Scholar
Qian, D. D., & Lin, L. H. F. (2020). The relationship between vocabulary knowledge and language proficiency. In Webb, S. (Ed.), The Routledge handbook of vocabulary studies (pp. 66–80). Routledge.Google Scholar
Robinson, P., Cadierno, T., & Shirai, Y. (2009). Time and motion: Measuring the effects of the conceptual demands of tasks on second language speech production. Applied Linguistics, 30, 533–554.CrossRefGoogle Scholar
Schepens, J., Van der Slik, F., & Van Hout, R. (2016). L1 and L2 distance effects in learning L3 Dutch. Language Learning, 66, 224–256.CrossRefGoogle Scholar
Serafini, E. J. & Sanz, C. (2016). Evidence for the decreasing impact of cognitive ability on second language development as proficiency increases. Studies in Second Language Acquisition, 38, 607–646.CrossRefGoogle Scholar
Skehan, P. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30, 510–532.CrossRefGoogle Scholar
Solon, M., Park, H., Henderson, C., & Dehghan-Chaleshtori, M. (2019). Revisiting the Spanish elicited imitation task: A tool for assessing advanced language learners? Studies in Second Language Acquisition, 41, 1027–1053.CrossRefGoogle Scholar
Staples, S., & Biber, D. (2015). Cluster analysis. In Plonsky, L. (Ed.), Advancing quantitative methods in second language research (pp. 243–274). Routledge.CrossRefGoogle Scholar
Thomas, M. (2006). Research synthesis and historiography: The case of assessment of second language proficiency. In Norris, J. M. & Ortega, L. (Eds.), Synthesizing research on language learning and teaching (pp. 279–298) John Benjamins.CrossRefGoogle Scholar
Tracy-Ventura, N., McManus, K., Norris, J. M., & Ortega, L. (2014). “Repeat as much as you can”: Elicited imitation as a measure of oral proficiency in L2 French. In Leclercq, P., Hilton, H., & Edmonds, A. (Eds.), Measuring L2 proficiency: Perspectives from SLA (pp. 143–166). Multilingual Matters.CrossRefGoogle Scholar
Tremblay, A. (2011). Proficiency assessment standards in second language acquisition research: “Clozing” the gap. Studies in Second Language Acquisition, 33, 339–372.CrossRefGoogle Scholar
Webb, N. M., Shavelson, R. J., & Haertel, E. H. (2006). Reliability coefficients and generalizability theory. Handbook of Statistics, 26, 81–124.CrossRefGoogle Scholar
Weir, C. J., & Wu, J. R. W. (2006). Establishing test form and individual task comparability: A case study of a semi-direct speaking test. Language Testing, 23, 167–197.CrossRefGoogle Scholar
Westergaard, M. (2021). Microvariation in multilingual situations: The importance of property-by-property acquisition. Second Language Research, 37, 379–407. https://doi.org/10.1177/0267658319884116 CrossRefGoogle Scholar
Wu, S.-L. (2017). The planning, implementation, and assessment of an international internship program: An exploratory case study. Foreign Language Annals, 50, 567–583.CrossRefGoogle Scholar
Wu, S.-L., & Ortega, L. (2013). Measuring global oral proficiency in SLA research: A new elicited imitation test of L2 Chinese. Foreign Language Annals, 46, 680–704.CrossRefGoogle Scholar
Yan, X. (2020). Unpacking the relationship between formulaic sequences and speech fluency on elicited imitation tasks: Proficiency level, sentence length, and fluency dimensions. TESOL Quarterly, 54, 460–487.CrossRefGoogle Scholar
Yan, X., Maeda, Y., Lv, J., & Ginther, A. (2016). Elicited imitation as a measure of second language proficiency: A narrative review and meta-analysis. Language Testing, 33, 497–528.CrossRefGoogle Scholar
Yu, Q. (2021). An organic syntactic complexity measure for the Chinese language: The TC-unit. Applied Linguistics 42, 60–92. https://doi.org/10.1093/applin/amz064 CrossRefGoogle Scholar
Supplementary material: File

Wu et al. supplementary material

Wu et al. supplementary material

Download Wu et al. supplementary material(File)
File 950.4 KB