This argument-based validation research investigates the validity of score interpretations on a computer-based, graphic-prompt writing test, focusing on the generalization inference. The graphic-prompt writing test assesses examinees’ ability to incorporate visual graphic information into their writing,. Both analytic ratings on Graph Description, Content Development, Organization, and Grammar/Vocabulary (n = 2,424) and composite ratings (n = 606) on written test responses from 101 ESL students were analyzed using Generalizability (G) Theory and Multi-Faceted Rasch Measurement (MFRM). Findings indicated three of the four analytic scales and the composites yielded dependable scores. In addition, the results of the G-studies and MFRM analysis revealed the relative effects of the raters on the total score variance was not trivial for both composite and analytic scores and the three raters were not quite equivalent in their rating severity. Nevertheless, the findings support the generalization inference to a large extent. Thus, it can be claimed the graphic-prompt writing task scores were dependable enough to be used for the intended purposes, particularly with the two-rater and three-task test administration design.

Following the argument-based approach to validity, this chapter focuses on the inference about the positive consequences of an academic ESL placement test (EPT) score use on major stakeholders: ESL instructors and ESL students at a university. The assumptions underlying this inference are that the decisions about ESL course placement are justifiable and comprehensible to test stakeholders, that the decisions are beneficial for ESL learners’ improvement of academic English proficiency, and that the decisions are beneficial for ESL learners’ academic achievement at the university. The assumptions were examined using mixed methods that included interviews, pre- and post-course analyses of students’ performances in ESL classes, and an analysis of the predictive power of the EPT results on students’ academic achievement. The results indicated the positive perceptions of the EPT placement decisions as reported by most of the ESL students and instructors, but variation in the degree of improvement was identified in the ESL courses. ESL students’ performance on the EPT predicted their first-year academic achievement at the university to some degree, highlighting the importance of English proficiency in ESL students’ academic success and the potential impact of the EPT placement decisions on students’ English language learning and overall academic performance.