Hostname: page-component-586b7cd67f-2plfb Total loading time: 0 Render date: 2024-11-22T08:23:05.391Z Has data issue: false hasContentIssue false

Review of washback research literature within Kane's argument-based validation framework

Published online by Cambridge University Press:  08 September 2015

Liying Cheng
Affiliation:
Queen's University, Kingston, [email protected]
Youyi Sun
Affiliation:
Queen's University, Kingston, [email protected]
Jia Ma
Affiliation:
Queen's University, Kingston, [email protected]

Abstract

No area of language assessment research in the past 20 years has received a greater increase in attention than washback research. Beginning with the seminal work of Alderson & Wall (Alderson & Wall 1993; Wall & Alderson 1993), an evolving body of empirical washback studies has been conducted worldwide, especially in countries where English is not the dominant language. A systematic search of the pertinent literature between 1993 and 2013 identified a total of 123 publications consisting of 36 review articles and 87 empirical studies. The focus of this review is on the empirical studies. A further breakdown of these empirical studies reveals 11 books and monographs, 27 doctoral dissertations, 40 journal articles, and 9 book chapters. This intensity of research activity underscores the timeliness and importance of this research topic and highlights its maturity, which in turn calls for this systematic review.

Type
State-of-the-Art Article
Copyright
Copyright © Cambridge University Press 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abdul Kadir, K. (2008). Framing a validity argument for test use and impact: The Malaysian public service experience. Ph.D. dissertation, University of Illinois at Urbana-Champaign.Google Scholar
Adair-Hauck, B., Glisan, E. W., Koda, K., Swender, E. B. & Sandrock, P. (2006). The Integrated Performance Assessment (IPA): Connecting assessment to instruction and learning. Foreign Language Annals 39, 359382.CrossRefGoogle Scholar
Alderson, J. C. & Hamp-Lyons, L. (1996). TOEFL preparation courses: A case study. Language Testing 13, 280297.CrossRefGoogle Scholar
Alderson, J. C. & Wall, D. (1993). ‘Does washback exist?Applied Linguistics 14, 115129.CrossRefGoogle Scholar
Alderson, J. C. & Wall, D. (1996) (eds.). Language Testing 13.3, 239240.CrossRefGoogle Scholar
Andrews, S. J., Fullilove, J. & Wong, Y. (2002). Targeting washback: A case study. System 30, 207233.CrossRefGoogle Scholar
Babcock, M. S. (2006). The impact of a culture of large-scale assessment and high-stakes results accountability for elementary principals in one Ontario school board. Master's thesis, Queen's University.Google Scholar
Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly 2.1, 134.CrossRefGoogle Scholar
Bachman, L. F. & Palmer, A. S. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford: Oxford University Press.Google Scholar
Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing. Language Testing 13, 257279.CrossRefGoogle Scholar
Ben Nkosana, L. M. (2008). Washback effects of an ESL examination: The impact of curriculum reform and assessment on teaching: The case of the BGCSE English curriculum in Botswana. Lambert Academic Publishing.Google Scholar
Booth, D. (2012). Exploring the washback of TOEIC in South Korea. Ph.D. dissertation, University of Auckland.Google Scholar
Brindley, G. (2008). Educational reform and language testing. In Shohamy, E. (ed.), Language testing and assessment. Encyclopedia of language and education (vol. 7). New York: Springer, 365378.Google Scholar
Brookhart, S. M. (2004). Classroom assessment: Tensions and intersections in theory and practice. Teachers College Record 106, 429458.CrossRefGoogle Scholar
Burrows, C. (2004). Washback in classroom-based assessment: A study of the washback effect in the Australian adult migrant English program. In Cheng, L., Watanabe, Y. & Curtis, A. (eds.), Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum Associates, 113128.Google Scholar
Chen, L.-M. D. (2002). Taiwanese junior high school English teachers’ perceptions of the washback effect of the Basic Competence Test in English. Ph.D. dissertation, The Ohio State University.Google Scholar
Cheng, L. (1997). How does washback influence teaching? Implications for Hong Kong. Language and Education 11, 3854.CrossRefGoogle Scholar
Cheng, L. (1998). Impact of a public English examination change on students’ perceptions and attitudes toward their English learning. Studies in Educational Evaluation 24.3, 279301.CrossRefGoogle Scholar
Cheng, L. (1999). Changing assessment: Washback on teacher perspectives and actions. Teaching and Teacher Education 15.3, 253271.CrossRefGoogle Scholar
Cheng, L. (2002). The washback effect on classroom teaching of changes in public examinations. In Savignon, S. J. (eds.), Interpreting communicative language teaching: Contexts and concerns in teacher education. New Haven, CT: Yale University Press, 91111.Google Scholar
Cheng, L. (2003). Looking at the impact of a public examination change on secondary classroom teaching: A Hong Kong case study. Journal of Classroom Interaction 38.1, 110.Google Scholar
Cheng, L. (2004). The washback effect of a public examination change on teachers’ perceptions toward their classroom teaching. In Cheng, L., Watanabe, Y. & Curtis, A. (eds.), Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum Associates, 146170.CrossRefGoogle Scholar
Cheng, L. (2005). Changing language teaching through language testing: A washback study. Studies in Language Testing 21. Cambridge: Cambridge University Press and Cambridge ESOL.Google Scholar
Cheng, L. (2008). Washback, impact and consequences. In Shohamy, E. & Hornberger, N. H. (eds.), Encyclopedia of Language and Education (2nd edn). Language Testing and Assessment (vol. 7), New York, NY: Springer, 349364.Google Scholar
Cheng, L. & Curtis, A. (2004). Washback or backwash: A review of the impact of testing on teaching and learning. In Cheng et al. (eds.) 3–18.CrossRefGoogle Scholar
Cheng, L. & DeLuca, C. (2011). Voices from test-takers: Further evidence for test validation and test use. Educational Assessment 16.2, 104122.CrossRefGoogle Scholar
Cheng, L., Andrews, S. & Yu, Y. (2011). Impact and consequences of school-based assessment in Hong Kong: Views from students and their parents. Language Testing 28.2, 221250.CrossRefGoogle Scholar
Cheng, L., Klinger, D. & Zheng, Y. (2007). The challenges of the Ontario Secondary School Literacy Test for second language students. Language Testing 24.2, 185208.CrossRefGoogle Scholar
Cheng, L., Watanabe, Y. & Curtis, A. (2004). Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum Associates.CrossRefGoogle Scholar
Chik, A. & Besser, S. (2011). International language test taking among young learners: A Hong Kong case study. Language Assessment Quarterly 8.1, 7391.CrossRefGoogle Scholar
Eckstein, M. A. & Noah, H. J. (eds.) (1992). Examinations: Comparative and international studies. Oxford: Pergamon Press.Google Scholar
Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and learning. In Cheng, L., Watanabe, Y. & Curtis, A. (eds.), Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum Associates, 191210.Google Scholar
Fournier-Kowaleski, L. A. (2005). Depicting washback in the intermediate Spanish language classroom: A descriptive study of teacher's instructional behaviors as they relate to tests. Ph.D. dissertation, The Ohio State University.Google Scholar
Fox, J. & Cheng, L. (2007). Did we take the same test? Differing accounts of the Ontario Secondary School Literacy Test by first and second language test-takers. Assessment in Education: Principles, Policy and Practice 14.1, 926.Google Scholar
Froehlich, V. (2010). Washback of an oral exam on teaching and learning in Germany: The EuroCom in Realschulen in Baden-Wuerttemberg. VDM Verlag Dr. Mueller.Google Scholar
Fullilove, J. (1992). The tail that wags. Institute of Language in Education Journal 9, 131147.Google Scholar
Gibson, C. & Swan, A. (2008).The significance of the sociolinguistic backgrounds of teachers of IELTS test preparation courses in selected Malaysian institutions. www.ielts.org/pdf/Volume8_Report2.pdf.Google Scholar
Gosa, C. M. C. (2004). Investigating washback: A case study using student diaries. Ph.D. dissertation, Lancaster University.Google Scholar
Green, A. (2006a). Washback to the learner: Learner and teacher perspectives on IELTS preparation course expectations and outcomes. Assessing Writing 11, 113134.CrossRefGoogle Scholar
Green, A. (2006b). Watching for washback: Observing the influence of the International English Language Testing System academic writing test in the classroom. Language Assessment Quarterly 3, 333368.CrossRefGoogle Scholar
Green, A. (2007a). IELTS washback in context: Preparation for academic writing in higher education. Studies in Language Testing 25. Cambridge: Cambridge University Press and Cambridge ESOL.Google Scholar
Green, A. (2007b). Washback to learning outcomes: A comparative study of IELTS preparation and university pre-sessional language courses. Assessment in Education 14, 7597.Google Scholar
Hamp-Lyons, L. (1997). Washback, impact and validity: Ethical concerns. Language Testing 14.3, 295303.CrossRefGoogle Scholar
Hamp-Lyons, L. & Brown, A. (2007).The effect of changes in the new TOEFL format on the teaching and learning of EFL/ESL: Stage 2 (20032005). Entering innovation. Educational Testing Service.Google Scholar
Hawkey, R. A. H. (2006). Impact theory and practice: Studies of the IELTS test and ProgettoLingue 2000. Studies in Language Testing 24. Cambridge: Cambridge University Press and Cambridge ESOL.Google Scholar
Hayes, B. M. (2003). IELTS preparation in New Zealand: An investigation into the nature of the courses and evidence of washback. Ph.D. dissertation, Victoria University of Wellington.Google Scholar
Hayes, B. & Read, J. (2004). IELTS test preparation in New Zealand: Preparing students for the IELTS academic module. In Cheng, L., Watanabe, Y. & Curtis, A. (eds.), Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum Associates, 97112.Google Scholar
Henrichsen, L. E. (1989). Diffusion of innovations in English language teaching: The ELEC effort in Japan, 1956–1968. New York: Greenwood Press.Google Scholar
Hirai, A. & Koizumi, R. (2009). Development of a practical speaking test with a positive impact on learning using a story retelling technique. Language Assessment Quarterly 6, 151167.CrossRefGoogle Scholar
Huang, L. (2009). Washback on teacher beliefs and behaviour: Investigating the process from a social psychology perspective. Ph.D. dissertation, Lancaster University.Google Scholar
Huang, S. (2011). Convergent vs. divergent assessment: Impact on college EFL students’ motivation and self-regulated learning strategies. Language Testing 28, 251271.CrossRefGoogle Scholar
Huang, X. (2011). The washback effects of national college entrance examination: Chinese language teachers’ beliefs and practices. Ph.D. dissertation, The Chinese University of Hong Kong.Google Scholar
Hughes, A. (1993). Backwash and TOEFL 2000. Unpublished manuscript, University of Reading.Google Scholar
Hung, S.-T. A. (2012). A washback study on e-portfolio assessment in an English as a foreign language teacher preparation program. Computer Assisted Language Learning 25, 2136.CrossRefGoogle Scholar
Kane, M. T. (2006). Validation. In Brennan, R. L. (eds.), Educational measurement (4th edn). Westport, CT: American Council on Education/Praeger, 1764.Google Scholar
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement 50.1, 173.CrossRefGoogle Scholar
Klinger, D., DeLuca, C. & Miller, T. (2008). The evolving culture of large-scale assessments in Canadian education. Canadian Journal of Educational Administration and Policy 76, 134.Google Scholar
Kunnan, A. J. (2004). Test fairness. In Milanovic, M., Weir, C. & Bolton, S. (eds.), Europe language testing in a global context: Selected papers from the ALTE conference in Barcelona. Cambridge: Cambridge University Press, 2748.Google Scholar
Latham, H. (1877). On the action of examinations considered as a means of selection, Cambridge: Deighton, Bell and Company.Google Scholar
Latimer, D. (2009). Washback effects of the Cambridge Preliminary English Test at an Argentinean bilingual school. Ph.D. dissertation, University of Georgia.Google Scholar
Liauh, Y.-H. E. (2011). A study of the perceptions of English faculty and students of exit English examinations at Taiwan's technological and vocational higher education institutions. Ph.D. dissertation, University of Montana.Google Scholar
Linn, R. L. (2000). Assessments and accountability. Educational Researcher 29.2, 416.CrossRefGoogle Scholar
Lopez, A. A. (2005). Examining the washback of a new large-scale English proficiency test: aligning tests, teaching, and learning. Ph.D. dissertation, University of Illinois at Urbana-Champaign.Google Scholar
Lumley, T. & Stoneman, B. (2000). Conflicting perspectives on the role of test preparation in relation to learning? Hong Kong Journal of Applied Linguistics 5.1, 5080.Google Scholar
Madaus, G. F. (1985). Public policy and the testing profession: You’ve never had it so good? Educational Measurement: Issues and Practice 4.4, 511.CrossRefGoogle Scholar
McNamara, T. (2000). Language testing. Oxford: Oxford University Press.Google Scholar
McMillan, J. H., Hellsten, L. & Klinger, D. (2010). Classroom assessment: Principles and practice for effective standards-based instruction. New York: Pearson.Google Scholar
Menken, K. (2006). Teaching to the test: How No Child Left Behind impacts language policy, curriculum, and instruction for English language learners. Bilingual Research Journal 30, 521546.CrossRefGoogle Scholar
Messick, S. (1996).Validity and washback in language testing. Language Testing 13, 243256.CrossRefGoogle Scholar
Messick, S. (1989). Validity. In Linn, R. (eds.), Educational measurement (3rd edn), New York: MacMillan, 13103.Google Scholar
Mickan, P. & Motteram, J. (2008). An ethnographic study of classroom instruction in an IELTS preparation program. www.ielts.org/pdf/Volume8_Report1.pdf.Google Scholar
Mickan, P. & Motteram, J. (2009). The preparation practices of IELTS candidates: Case studies. www.ielts.org/pdf/Vol10_Report5.pdf.Google Scholar
Mishan, F. (2010). Withstanding washback: Thinking outside the box in materials development. In Tomlinson, B. & Masuhara, H. (eds.), Research in materials development for language learning: Evidence for best practice. London: Continuum, 353368.Google Scholar
Muñoz, A. P. & Álvarez, M. E. (2010). Washback of an oral assessment system in the EFL classroom. Language Testing 27.1, 3349.CrossRefGoogle Scholar
Murray, J., Riazi, A. & Cross, J. (2012). Test candidates’ attitudes and their relationship to demographic and experiential variables: The case of overseas trained teachers in NSW, Australia. Language Testing 29.4, 577595.CrossRefGoogle Scholar
Nemati, M. (2003). The positive washback effect of introducing essay writing tests in EFL environments. Indian Journal of Applied Linguistics 29, 4962.Google Scholar
Nguyen, P. (1997). Washback effects of the International English Language Testing System at the Vietnam National University. Unpublished doctoral dissertation, University of Melbourne.Google Scholar
K-M., O. (2001). Washback on high school classrooms of the English tests within Korean university entrance exams. Unpublished Doctor of Education dissertation, Teachers College, Columbia University.Google Scholar
Pan, Y. (2010). Consequences of test use: Educational and societal effects of English certification exit requirements in Taiwan. Unpublished doctoral dissertation, The University of Melbourne.Google Scholar
Perrone, M. (2010). The impact of the First Certificate of English (FCE) on the EFL classroom: A washback study.Unpublished doctoral dissertation, Teachers College, Columbia University.Google Scholar
Perrone, M. (2011). The effect of classroom-based assessment and language processing on the second language acquisition of EFL students. Journal of Adult Education 40, 2033.Google Scholar
Qi, L. (2004). Has a high-stakes test produced the intended changes? In Cheng, L., Watanabe, Y. & Curtis, A. (eds.), Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum Associates, 171190.Google Scholar
Qi, L. (2005). Stakeholders’ conflicting aims undermine the washback function of a high-stakes test. Language Testing 22, 142173.Google Scholar
Qi, L. (2007). Is testing an efficient agent for pedagogical change? Examining the intended washback of the writing task in a high-stakes English test in China. Assessment in Education 14.1, 5174.Google Scholar
Quesada, I. R. (2001). Washback overrides the curriculum: An exploratory study on the washback effect of a high-stakes standardized test in the Costa Rican EFL high school context. Unpublished doctoral dissertation, The Pennsylvania State University.Google Scholar
Rao, C., McPherson, K., Chand, R. & Khan, V. (2003). Assessing the impact of IELTS preparation programs on the General Training reading and writing test modules. IELTS Research Report (vol. 4). www.ielts.org/pdf/Vol5Report5.pdf.Google Scholar
Read, J. & Hayes, B. (2003). The impact of IELTS on preparation for academic study in New Zealand. IELTS Research Reports (vol. 4). www.ielts.org/pdf/Vol4Report5.pdf.Google Scholar
Saif, S. (2006). Aiming for positive washback: A case study of international teaching assistants, Language Testing 23, 134.CrossRefGoogle Scholar
Saville, N. & Hawkey, R. (2004). The IELTS impact study: Investigating washback on teaching materials. In Cheng et al. (eds.), 73–96.Google Scholar
Schissel, J. L. (2012). The pedagogical practice of test accommodations with emergent bilinguals: Policy-enforced washback in two urban schools. Unpublished doctoral dissertation, University of Pennsylvania.Google Scholar
Scott, C. (2005). Washback in the UK primary context with EAL learners: Exploratory case studies. Unpublished doctoral dissertation, University of Bristol.Google Scholar
Shih, C. M. (2006). Perceptions of the General English Proficiency Test and its washback: A case study at the two Taiwan technological institutes. Unpublished doctoral dissertation, University of Toronto.Google Scholar
Shih, C. (2007). A new washback model of students’ learning. The Canadian Modern Language Review 64.1, 135162.CrossRefGoogle Scholar
Shih, C. M. (2009). How tests change teaching: A model for reference. English Teaching: Practice and Critique 8, 188206.Google Scholar
Shih, C. M. (2010). The washback of the General English Proficiency Test on university policies: A Taiwan case study. Language Assessment Quarterly 7, 234254.CrossRefGoogle Scholar
Shohamy, E. (2001). The power of tests: A critical perspective on the uses of language tests. Essex: Longman.Google Scholar
Shohamy, E., Donitsa-Schmidt, S. & Ferman, I. (1996). Test impact revisited: Washback effect over time. Language Testing 13, 298317.CrossRefGoogle Scholar
Stecher, B., Chun, T. & Barron, S. (2004). The effects of assessment-driven reform on the teaching of writing in Washington State. In Cheng et al. (eds.), 53–72.Google Scholar
Stoneman, B. W. H. (2006). The impact of an exit English test on Hong Kong undergraduates: A study investigating the effects of test status on students’ test preparation behaviors. Unpublished doctoral dissertation, Hong Kong Polytechnic University.Google Scholar
Tan, M. H. (2009). Changing the language of instruction for mathematics and science in Malaysia: The PPSMI policy and the washback effect of bilingual high-stakes secondary school exit exams. Unpublished doctoral dissertation, University of McGill, Montreal.Google Scholar
Torrance, H. & Pryor, J. (2001). Developing formative assessment in the classroom: Using action research to explore and modify theory. British Educational Research Journal 27.5, 615631.CrossRefGoogle Scholar
Tsagari, D. (2007). Investigating the washback effect of a high-stakes EFL exam in the Greek context: Participants’ perceptions, material design and classroom applications. Unpublished doctoral dissertation, Lancaster University.Google Scholar
Turner, C. (2006). Professionalism and high-stakes tests: Teachers’ perspectives when dealing with educational change introduced through provincial exams. TESL Canada Journal 23, 5476.CrossRefGoogle Scholar
Ungerleider, C. (2003). Failing our kids: How we are ruining our public schools. Toronto: McClelland & Stewart.Google Scholar
Virkiru, L. I. (2011). From assessment to learning: The teaching of English beyond examinations. Educational Forum 75, 129142.CrossRefGoogle Scholar
Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education and from innovation theory. Language Testing 13, 334354.CrossRefGoogle Scholar
Wall, D. (1999). The impact of high-stakes examinations on classroom teaching: A case study using insights from testing and innovation theory. Unpublished Ph.D. thesis, Department of Linguistics and Modern English Language, Lancaster University. Lancaster.Google Scholar
Wall, D. (2005). The impact of high-stakes examinations on classroom teaching: A case study using insights from testing and innovation theory. Studies in Language Testing 22. Cambridge: Cambridge University Press and Cambridge ESOL.Google Scholar
Wall, D. & Alderson, J. C. (1993). Examining washback: The Sri Lankan impact study. Language Testing 10, 4169.CrossRefGoogle Scholar
Wall, D. & Horák, T. (2006). The impact of changes in the TOEFL examination on teaching and learning in Central and Eastern Europe – Phase 1: The baseline study. TOEFL Monograph Series, MS-34. Princeton, NJ: Educational Testing Service.CrossRefGoogle Scholar
Wall, D. & Horák, T. (2008). The impact of changes in the TOEFL examination on teaching and learning in Central and Eastern Europe – Phase 2: Coping with change. (Report No. RR-08–37). www.ets.org/Media/Research/pdf/RR-08-37.pdf.CrossRefGoogle Scholar
Wang, H. (2010). Investigating the justifiability of an additional test use: An application of assessment use argument to an English as a foreign language test. Unpublished doctoral dissertation, University of California, Los Angeles.Google Scholar
Wang, J. (2011). A study of role of the ‘teacher factor’ in washback. Unpublished doctoral dissertation, University of McGill, Montreal.Google Scholar
Watanabe, Y. (1996). Does grammar translation come from the entrance examination? Preliminary findings from classroom-based research. Language Testing 13.3, 318333.CrossRefGoogle Scholar
Watanabe, Y. (2004). Teacher factors mediating washback. In Cheng et al. (eds.), 129–146.Google Scholar
Wei, W. (2011). The washback effect and impact of a high-stakes test on classroom teaching : A case study from China. Unpublished doctoral dissertation, University of Leeds.Google Scholar
Weigle, S. C. (2004). Integrating reading and writing in a competency test for non-native speakers of English. Assessing Writing 9, 2755.CrossRefGoogle Scholar
Wigfield, A. (1994). Expectancy-value theory of achievement motivation: A developmental perspective. Educational Psychology Review 6.1, 4978.CrossRefGoogle Scholar
Wigfield, A. & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation Contemporary Educational Psychology 25.1, 6881.CrossRefGoogle ScholarPubMed
Winke, P. (2011). Evaluating the validity of a high-stakes ESL test: Why teachers’ perceptions matter. TESOL Quarterly 45, 628660.CrossRefGoogle Scholar
Xie, Q. (2008). Students’ perception of the CET4 listening and test preparation practices: Implications for washback. Research Studies in Education 6, 3247.Google Scholar
Xie, Q. (2010). Test design and use, preparation, and performance: A structural equation modeling study of consequential validity. Unpublished doctoral dissertation, University of Hong Kong.Google Scholar
Xie, Q. & Andrews, S. (2012). Do test design and uses influence test preparation? Testing a model of washback with structural equation modeling. Language Testing 30.1, 4970.CrossRefGoogle Scholar
Yu, Y. (2010). The washback effects of school-based assessment on teaching and learning: A case study. Unpublished doctoral dissertation, University of Hong Kong.Google Scholar
Zhan, Y. & Wan, Z. H. (2013). Dynamic nature of washback on individual learners: The role of possible selves. Assessment & Evaluation in Higher Education, DOI:10.1080/02602938.2013.872769.Google Scholar
Zhao, S. F. (2005). A consideration of the washback effect of English as a foreign language oral examinations at tertiary level in China: Teaching, testing, and syllabus change. Unpublished doctoral dissertation, Cardiff University.Google Scholar