Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-11-22T05:54:04.077Z Has data issue: false hasContentIssue false
  • English
  • Français

ASSESSING LANGUAGE USING COMPUTER TECHNOLOGY

Published online by Cambridge University Press:  28 March 2008

Dan Douglas  and
Volker Hegelheimer
Get access Rights & Permissions [Opens in a new window]

Abstract

In this article, we propose to follow up on the most recent ARAL survey article on trends in computer-based second language assessment (Jamieson, 2005) and review developments in the use of technology in the creation, delivery, and scoring of language tests. We will discuss the promise and threats associated with computer-based language testing, including the language construct in relation to computer-based delivery and response technologies; computer-based authoring options; current developments; scoring, feedback, and reporting systems; and validation issues.

Type
Research Article
Information
Annual Review of Applied Linguistics , Volume 27 , March 2007 , pp. 115 - 132
Copyright
Copyright © Cambridge University Press 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

ANNOTATED REFERENCES

Alderson, J. C. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. London: Continuum.Google Scholar
Carr, N. T. (2006). Computer-based testing: Prospects for innovative assessment. In Ducate, L. & Arnold, N. (Eds.), Calling on CALL: From theory and research to new directions in foreign language teaching (CALICO Monograph Series, Vol. 5, pp. 289–312). San Marcos, TX: CALICO.Google Scholar
Chapelle, C., & Douglas, D. (2006). Assessing language through computer technology. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Stoynoff, S., & Chapelle, C. (2005). ESOL tests and testing. Alexandria, VA: TESOL.Google Scholar
Warschauer, M., & Ware, P. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10 (2), 1–24.CrossRefGoogle Scholar

OTHER REFERENCES

ACT. (2007a). ACT ESL Placement Test. Retrieved June 4, 2007, from http://www.act.org/esl/index.htmlGoogle Scholar
ACT. (2007b). ACT Listening Proficiency Descriptors. Retrieved June 4, 2007, from http://www.act.org/esl/desc/listen.htmlGoogle Scholar
Alderson, J. C. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. London: Continuum.Google Scholar
Alderson, J. C., & Huhta, A. (2005). The development of a suite of computer-based diagnostic tests based on the common European framework. Language Testing, 22 (3), 301–320.CrossRefGoogle Scholar
Alderson, J. C., Krahnke, K., & Stansfield, C. (Eds.) (1987). Reviews of English language proficiency tests. Washington, D.C.: TESOL Publications.Google Scholar
Alderson, J. C., Percsich, R., & Szabo, G. (2000). Sequencing as an item type. Language Testing, 17 (4), 423–447.CrossRefGoogle Scholar
Bachman, L., & Palmer, A. (1996). Language testing in practice. Oxford: Oxford University Press.Google Scholar
Brandl, K. (2005). Are you ready to “MOODLE”? Language Learning & Technology, 9 (2), 16–23.Google Scholar
Breland, H., Lee, Y., & Muraki, E. (2004). Comparability of TOEFL CBT writing prompts: Response mode analyses (TOEFL Research Report No. RR-75). Princeton, NJ: ETS.Google Scholar
Burgoon, J. (1994). Non-verbal signals. In Knapp, M. & Miller, G. (Eds.), Handbook of interpersonal communication (pp. 344–393). London: Routledge.Google Scholar
Burstein, J., & Chodorow, M. (2002). Directions in automated essay analysis. In Kaplan, R. (Ed.), The Oxford handbook of applied linguistics (pp. 487–497). New York: Oxford University Press.Google Scholar
Cambridge ESOL. (2007). Business language testing service. Retrieved June 15, 2007, from http://www.bulats.org/Google Scholar
Canale, M. (1986). The promise and threat of computerized adaptive assessment of reading comprehension. In Stansfield, C. (Ed.), Technology and language testing (pp. 30–45). Washington, DC: TESOL.Google Scholar
Carr, N. T. (2006). Computer-based testing: Prospects for innovative assessment. In Ducate, L. & Arnold, N. (Eds.), Calling on CALL: From theory and research to new directions in foreign language teaching (CALICO Monograph Series Vol. 5, pp. 289–312). San Marcos, TX: CALICO.Google Scholar
Carrell, P. (2007). Notetaking strategies and their relationship to performance on listening comprehension and communicative assessment tasks (ETS Research Report No. RR-07-01, TOEFL Monograph No. MS-35). Princeton, NJ: ETS.CrossRefGoogle Scholar
Chapelle, C., & Douglas, D. (2006). Assessing language through computer technology. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Chapelle, C. A. (2006). Review of DIALANG. Language Testing, 23 (4), 544–550.CrossRefGoogle Scholar
Chodorow, H., & Burstein, J. (2004). Beyond essay length: Evaluating E-rater®'s performance on TOEFL essays (TOEFL Research Report No. RR-73). Princeton, NJ: ETS.Google Scholar
Choi, I.-C., Kim, K. S., & Boo, J. (2003). Comparability of a paper-based language test and a computer-based language test. Language Testing, 20 (3), 295–320.CrossRefGoogle Scholar
Chun, C. W. (2006). Commentary: An analysis of a language test for employment: The authenticity of the PhonePass Test. Language Assessment Quarterly: An International Journal, 3 (3), 295–306.CrossRefGoogle Scholar
Cohen, Y., Ben-Simon, A., & Hovav, M. (2003, October). The effect of specific language features on the complexity of systems for automated essay scoring. Paper presented at the International Association of Educational Assessment Annual Conference, Manchester, England.Google Scholar
Cohen, A. D., & Upton, A. T. (2006). Strategies in responding to the new TOEFL reading tasks (TOEFL Monograph Series MS-33). Princeton, NJ: ETS.CrossRefGoogle Scholar
Coniam, D. (2006). Evaluating computer-based and paper-based versions of an English-language listening test. ReCALL, 18, 193–211.CrossRefGoogle Scholar
Cumming, A., Kantor, R., Baba, K., Eouanzoui, K., Erdosy, U. S., & James, M. (2006). Analysis of discourse features and verification of scoring levels for independent and integrated prototype written tasks for the new TOEFL (TOEFL Monograph Series MS-30). Princeton, NJ: ETS.Google Scholar
DIALANG. (2003). Retrieved June 15, 2007, from http://www.dialang.orgGoogle Scholar
Dikli, S. (2006). An overview of automated scoring of essays. Journal of Technology, Learning, and Assessment, 5 (1). Retrieved January 30, 2007, from http://www.jtla.orgGoogle Scholar
Douglas, D., & Hegelheimer, V. (2006, July). Strategies and use of knowledge in performing new TOEFL listening tasks: The main study. Paper presented at Language Testing Research Colloquium, Melbourne, Australia.Google Scholar
Fulcher, G. (2005). Better communications test will silence critics. Guardian Weekly, Friday, February 18, 2005. http://education.guardian.co.uk/tefl/story/0,5500,1645011,00.htmlGoogle Scholar
Green, T. (2006, April 3). Comment on LTEST-L. LTEST-L Archives. Retrieved from http://lists.psu.edu/archives/ltest-l.htmlGoogle Scholar
Gruba, P. (1997). Exploring digital video material. In Debski, R., Gassin, J., & Smith, M. (Eds.), Language learning through social computing (pp. 109–140). Parkville, Victoria, Australia: Applied Linguistics Association of Australia.Google Scholar
Harcourt Assessment. (2006). Versant. Retrieved June 5, 2007, from http://harcourtassessment.ca/haiweb/Cultures/en-US/dotCom/Versant/Google Scholar
Higgins, D., Burstein, J., & Attali, Y. (2006). Identifying off-topic student essays without topic-specific training data. Natural Language Engineering, 12 (2), 145–159.CrossRefGoogle Scholar
Ishioka, T., & Kameda, M. (2004). Automated Japanese essay scoring system: JESS. Proceedings of the 15th International Workshop on Database and Expert Systems Applications, Zaragosa, Spain, 4–8.CrossRefGoogle Scholar
Jamieson, J. (2005). Trends in computer-based second language assessment. Annual Review of Applied Linguistics, 25, 228–242.CrossRefGoogle Scholar
LTEST-L Archive. (2006a, April–May). Whatever happened to adaptive tests? Retrieved June 14, 2007, from http://lists.psu.edu/archives/ltest-l.htmlGoogle Scholar
LTEST-L Archive. (2006b, November). Questionmark Perception. Retrieved June 14, 2007, from http://lists.psu.edu/archives/ltest-l.htmlGoogle Scholar
Monaghan, W., & Bridgeman, B. (2005, April). E-rater as a quality control on human scores. ETS RD Connections. Retrieved January 30, 2007, from http://www.ets.org/Media/Research/pdf/RD_Connections2.pdfGoogle Scholar
Ordinate. (2006). Retrieved June 15, 2007, from http://www.ordinate.com/Google Scholar
Pearson Knowledge Technologies. (2006). Reliability and validity of the KAT engine. Retrieved January 30, 2007, from http://www.pearsonkt.com/researchVRSum.shtmlGoogle Scholar
Roever, C. (2005). Review of the ESL Computer Adaptive Placement Exam (ESL-CAPE). In Stoynoff, S. & Chapelle, C., ESOL tests and testing (pp. 66–68). Alexandria, VA: TESOL.Google Scholar
Rudner, L. M., Garcia, V., & Welch, C. (2005). An evaluation of IntelliMetric essay scoring system using responses to GMAT® AWA prompts (GMAC Research Reports RR-05-08). McLean, VA: Graduate Management Admission Council.Google Scholar
Sim, G., Horton, M., & Strong, S. (2004, April). Interfaces for online assessment: Friend or foe? Paper presented at The 7th HCI Educators Workshop: Effective Teaching and Training in HCI, University of Central Lancashire, Preston, England. Retrieved February 4, 2007, from http://www.ics.heacademy.ac.uk/events/presentations/382_grsim_interface.pdfGoogle Scholar
SoftStudy. (2006). Web-Computer-Adaptive Placement Examination (Web-CAPE). Retrieved February 2, 2007, from http://www.softstudy.com//products/CAPE.cfmGoogle Scholar
Stoynoff, S., & Chapelle, C. (2005). ESOL tests and testing. Alexandria, VA: TESOL.Google Scholar
Taylor, C., Kirsch, I., Eignor, D., & Jamieson, J. (1999). Examining the relationship between computer familiarity and performance on computer-based language tasks. Language Learning, 49 (2), 219–274.CrossRefGoogle Scholar
Taylor, C., Jamieson, J., & Eignor, D. (2000). Trends in computer use among international students. TESOL Quarterly, 34 (3), 575–585.CrossRefGoogle Scholar
Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language Learning & Technology, 11 (1), 67–86.Google Scholar
Warschauer, M., & Ware, P. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10 (2), 1–24.CrossRefGoogle Scholar
Winke, P., & MacGregor, D. (2001). Review of hot potatoes. Language Learning & Technology, 5 (2), 28–33. Retrieved January 24, 2007, from http://llt.msu.edu/vol5num2/review3/default.htmlGoogle Scholar
Wolfe, E. W., & Manalo, J. R. (2005). An investigation of the impact of composition medium on the quality of TOEFL writing scores (TOEFL Research Report No. RR-72). Princeton, NJ: ETS.Google Scholar
Zechner, K., Bejar, I., & Hemat, R. (2007). Toward an understanding of the role of speech recognition in non-native speech assessment (ETS Research Report No. RR-07-02, TOEFL iBT Report No. TOEFLiBT-02). Princeton, NJ: ETS.Google Scholar