Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-23T08:20:22.144Z Has data issue: false hasContentIssue false
  • English
  • Français

TRENDS IN COMPUTER-BASED SECOND LANGUAGE ASSESSMENT

Published online by Cambridge University Press:  21 July 2005

Joan Jamieson
Get access Rights & Permissions [Opens in a new window]

Abstract

In the last 20 years, several authors have described the possible changes that computers may effect in language testing. Since ARAL's last review of general language testing trends (Clapham, 2000), authors in the Cambridge Language Assessment Series have offered various visions of how computer technology could alter the testing of second language skills. This chapter reflects these perspectives as it charts the paths recently taken in the field. Initial steps were made in the conversion of existing item types and constructs already known from paper-and- pencil testing into formats suitable for computer delivery. This conversion was closely followed by the introduction of computer-adaptive tests, which aim to make more, and perhaps, better, use of computer capabilities to tailor tests more closely to individual abilities and interests. Movement toward greater use of computers in assessment has been coupled with an assumption that computer-based tests should be better than their traditional predecessors, and some related steps have been taken. Corpus linguistics has provided tools to create more authentic assessments; the quest for authenticity has also motivated inclusion of more complex tasks and constructs. Both these innovations have begun to be incorporated into computer-based language tests. Natural language processing has also provided some tools for computerized scoring of essays, particularly relevant in large-scale language testing programs. Although computer use has not revolutionized all aspects of language testing, recent efforts have produced some of the research, technological advances, and improved pedagogical understanding needed to support progress.

Type
RESEARCH IN LANGUAGE ASSESSMENT
Information
Annual Review of Applied Linguistics , Volume 25 , March 2005 , pp. 228 - 242
Copyright
© 2005 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

ACT, Inc. 2000. COMPASS/ESL reference manual. Iowa City, Iowa: Author.
Alderson J. C. 2000. Assessing reading. New York: Cambridge University Press.
Attali Y., & Burstein J. 2004. Automated essay scoring with E-rater V.2.0. Available on-line at http://www.ets.org/research/erater.html.
Bachman L., & Palmer A. 1996. Language testing in practice. New York: Oxford University Press.
Biber D., Conrad S., Reppen R., Byrd P., & Helt M. 2002. Speaking and writing in the university: A multidimensional comparison. TESOL Quarterly, 36, 9– 48.Google Scholar
Biber D., Conrad S., Reppen R., Byrd P., Helt M., Clark V., Cortez V., Csomay E., Urzua A. 2004. Representing language use in the university: Analysis of the TOEFL 2000 Spoken and Written Academic Language Corpus. TOEFL Monograph MS-25. Princeton, NJ: Educational Testing Service.
Brown J. D. 2004. For computerized language tests, potential benefits outweigh problems. Essential Teacher, 1, 4, 37– 40.Google Scholar
Buck G. 2001. Assessing listening. New York: Cambridge University Press.
Burstein J., & Chodorow M. (1999, June). Automated essay scoring for nonnative English speakers. In Proceedings of the ACL99 Workshop on Computer-Mediated Language Assessment and Evaluation of Natural Language Processing. College Park, MD. Available online at http://www.ets.org/research/dload/acl99rev.pdf.
Burstein J., Chodorow M., & Leacock C. 2003. Criterion online essay evaluation: An application for automated evaluation of student essays. Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence, Acapulco, Mexico. Available online at: http://www.ets.org/research/dload/iaai03bursteinj.pdf.
Burstein J., Frase L., Ginther A., & Grant L. 1996. Technologies for language assessment. Annual Review of Applied Linguistics, 16, 240– 260.Google Scholar
Canale M. 1986. The promise and threat of computerized adaptive assessment of reading comprehension. In C. Stansfield(Ed.), Technology and language testing (pp. 30– 45). Washington, DC: TESOL Publications.
Chalhoub-Deville M. (Ed.), 1999. Issues in computer adaptive testing of reading proficiency . Studies in Language Testing 10, . New York: Cambridge University Press.
Chalhoub-Deville M., & Deville C. 1999. Computer adaptive testing in second language contexts. Annual Review of Applied Linguistics, 19, 273– 299.Google Scholar
Chapelle C. A. 2004. English language learning and technology. Philadelphia: John Benjamins.
Chapelle C. A. 2001. Computer applications in second language acquisition. New York: Cambridge University Press.
Chapelle C. A., Enright M. K., & Jamieson J. (forthcoming). Challenges in developing a test of academic English. In C. A. Chapelle M. K. Enright, & J. Jamieson(Eds.), Building a validity argument for TOEFL. Mahwah, NJ: Erlbaum.
Chapelle C. A., Jamieson J., & Hegelheimer V. 2003. Validation of a web-based ESL test. Language Testing, 20, 409– 439.Google Scholar
Chodorow M., & Burstein J. 2004. Beyond essay length: Evaluating e-rater's performance on TOEFL essays. TOEFL Research Reports, Report 73. Princeton, NJ: Educational Testing Service.
Choi I-C., Sung Kim K., & Boo J. 2003. Comparability of a paper-based language test and a computer-based language test. Language Testing, 20, 295– 320.Google Scholar
Clapham C. 2000. Assessment and testing. Annual Review of Applied Linguistics, 20, 147– 161.Google Scholar
Douglas D. 2000. Assessing languages for specific purposes. New York: Cambridge University Press.
Educational Testing Service (ETS). 2000. The computer-based TOEFL score user guide. Princeton, NJ: Author.
Eignor D. 1999. Selected technical issues in the creation of computer-adaptive tests of second language reading proficiency. In Issues in computer adaptive testing of reading proficiency: Studies in Language Testing 10, (pp. 167– 181). New York: Cambridge University Press.
Enright M. K., Bridgeman B., Eignor D., Lee & Powers D. E. (forthcoming). Designing measures of listening, reading, writing, and speaking. In C. A. Chapelle M. K. Enright, & J. Jamieson(Eds.), Building a validity argument for TOEFL. Mahwah, NJ: Erlbaum.
Godwin-Jones R. 2001. Emerging technologies. Language Learning & Technology, 5 (2), 8– 12.Google Scholar
Jamieson J., & Chapelle C. A. 2002. Longman English Assessment. New York: Pearson Longman.
Jamieson J., Eignor D., Grabe W., & Kunnan A. (forthcoming). The frameworks for reconceptualization of TOEFL. In C. A. Chapelle, M. K. Enright, & J. Jamieson(Eds.), Building a validity argument for TOEFL. Mahwah, NJ: Erlbaum.
Jamieson J., Jones S., Kirsch I., Mosenthal P., & Taylor C. 2000. TOEFL 2000 framework: A working paper. TOEFL Monograph Series 16. Princeton, NJ: Educational Testing Service.
Kenyon D., & Malabonga, V. 2001. Comparing examinee attitudes toward computer-assisted and other oral proficiency assessments. Language Learning & Technology, 5 (2), 60– 83.Google Scholar
Kenyon D., Malabonga V., & Carpenter H. 2001. Response to the Norris commentary. Language Learning & Technology, 5 (2), 106– 108.Google Scholar
Lanfer B., Elder C., Hill K., & Congdon P. (2004a). Size and strength: Do we need both to measure vocabulary knowledge? Language Testing, 21, 202– 226.Google Scholar
Lanfer B., & Goldstein Z. (2004b). Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54, 399– 436.Google Scholar
Leacock C. 2004. Scoring free-responses automatically: A case study of a large-scale assessment. Examens, 1, 3. English version available online at http://www.ets.org/research/erater.html.
LeLoup J., & Pontierio R. 2001. On the net, language testing resources. Language Learning & Technology, 5 (2), 4– 7.Google Scholar
Loumi S. 2004. Assessing speaking. New York: Cambridge University Press.
Madsen H. 1986. Evaluating a computer-adaptive ESL placement test. CALICO Journal, 4(2) 41– 50.Google Scholar
Norris J. 2001. Concerns with computerized adaptive oral proficiency assessment. Language Learning & Technology, 5 (2) 99– 105.
Ordinate Corp. 1999. PhonePass testing: Structure and content. Ordinate Corporation Technical Report. Menlo Park, CA: Author.
Polio C. 2001. Review of Test Pilot. Language Learning & Technology, 5 (2), 34– 37.Google Scholar
Powers D. E., Roever C., Huff K. L., & Trapani C. S. 2003. Validating LanguEdge courseware scores against faculty ratings and student self-assessments. (ETS Research Report 03–11). Princeton, NJ: Educational Testing Service.
Read J. 2000. Assessing vocabulary. New York: Cambridge University Press.
Roever C. 2001. Web-based language testing. Language Learning & Technology, 5 (2), 84– 94.Google Scholar
Rosenfeld M., Leung S., & Oltman P. 2001. The reading, writing, speaking, and listening tasks important for academic success at the undergraduate and graduate levels. (TOEFL Monograph Series, MS-21). Princeton, NJ: Educational Testing Service.
Rost M., & Fuchs M. 2003. Longman English Interactive. New York: Pearson Education.
Rupp A., Garcia P., & Jamieson J. 2002. Combining multiple regression and CART to understand difficulty in second language reading and listening comprehension test items. International Journal of Language Testing, 1, 185– 216.Google Scholar
Sawaki Y. 2001. Comparability of conventional and computerized tests of reading in a second language. Language Learning & Technology, 5 (2) 38– 59.Google Scholar
Stoynoff S., & Chapelle C. A. 2005. ESOL tests and testing: A resource for teachers and administrators. Alexandria, VA: Teachers of English to Speakers of Other Languages.
Tang K. L. 1996. Polytomous item response theory models and their applications in large-scale testing programs: Review of literature. (TOEFL Monograph Series MS-2). Princeton, NJ: Educational Testing Service.
Tang K. L., & Eignor D. R. 1997. Concurrent calibration of dichotomously and polytomously scored TOEFL items using IRT models. (TOEFL Technical Report TR-13). Princeton, NJ: Educational Testing Service.
Taylor C., Kirsch I., Jamieson J., & Eignor D. 1999. Examining the relationship between computer familiarity and performance on computer-based language tasks. Language Learning, 49, 219– 274.Google Scholar
Thompson I. 2001. From the special issue editor (Introduction). Language Learning & Technology, 5, (2), 2– 3.Google Scholar
Wainer H., Dorans N., Eignor D., Flaugher R., Green B., Mislevy R., Steinberg L., & Thissen D. 2000. Computer adaptive testing: A primer (2nd ed.). Mahwah, NJ: Erlbaum.
Weigle S. C. 2002. Assessing writing. New York: Cambridge University Press.
Winke P., & MacGregor D. 2001. Review of Version 5, Hot Potatoes. Language Learning & Technology, 5 (2) 28– 33.Google Scholar