Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-19T15:22:36.787Z Has data issue: false hasContentIssue false
  • English
  • Français

Flawed self-assessment: Investigating self- and other-perception of second language speech*

Published online by Cambridge University Press:  29 December 2014

PAVEL TROFIMOVICH ,
TALIA ISAACS ,
SARA KENNEDY ,
KAZUYA SAITO  and
DUSTIN CROWTHER
PAVEL TROFIMOVICH*
Affiliation:
Concordia University
TALIA ISAACS
Affiliation:
University of Bristol
SARA KENNEDY
Affiliation:
Concordia University
KAZUYA SAITO
Affiliation:
Waseda University
DUSTIN CROWTHER
Affiliation:
Michigan State University
*
Address for correspondence: Pavel Trofimovich, Concordia University, Department of Education, 1445 de Maisonneuve Blvd., West Montreal, Quebec, CanadaH3G 1M8[email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

This study targeted the relationship between self- and other-assessment of accentedness and comprehensibility in second language (L2) speech, extending prior social and cognitive research documenting weak or non-existing links between people's self-assessment and objective measures of performance. Results of two experiments (N = 134) revealed mostly inaccurate self-assessment: speakers at the low end of the accentedness and comprehensibility scales overestimated their performance; speakers at the high end of each scale underestimated it. For both accent and comprehensibility, discrepancies in self- versus other-assessment were associated with listener-rated measures of phonological accuracy and temporal fluency but not with listener-rated measures of lexical appropriateness and richness, grammatical accuracy and complexity, or discourse structure. Findings suggest that inaccurate self-assessment is linked to the inherent complexity of L2 perception and production as cognitive skills and point to several ways of helping L2 speakers align or calibrate their self-assessment with their actual performance.

Type
Research Article
Information
Bilingualism: Language and Cognition , Volume 19 , Issue 1 , January 2016 , pp. 122 - 140
Check for updates
Copyright
Copyright © Cambridge University Press 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

This study was funded by grants from the Social Sciences and Humanities Research Council of Canada (SSHRC) and Fonds de recherche sur la société et la culture (FRQSC) awarded to the first, second, and third authors, a Marie Curie Career Integration Grant (European Commission) awarded to the second author, and by the Grant-in-Aid for Scientific Research in Japan (No. 26770202) awarded to the fourth author. We are grateful to participants, Ze Shan Yao for his technical help, and to anonymous BLC reviewers for their helpful input and feedback on the content of this manuscript.

References

Alderson, C. J. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. London: Continuum.Google Scholar
Ammar, A., & Spada, N. (2006). One size fits all? Recasts, prompts, and L2 learning. Studies in Second Language Acquisition, 28, 543–574. doi:10.1017/S0272263106060268.CrossRefGoogle Scholar
Anderson-Hsieh, J., Johnson, R., & Koehler, K. (1992). The relationship between native speaker judgments of nonnative pronunciation and deviance in segmentals, prosody, and syllable structure. Language Learning, 42, 529–555. doi:10.1111/j.1467-1770.1992.tb01043.x CrossRefGoogle Scholar
Bachman, L. F., & Palmer, A. S. (1989). The construct validation of self-ratings of communicative language ability. Language Testing, 6, 14–29. doi:10.1177/026553228900600104 CrossRefGoogle Scholar
Blanche, P. & Merino, B. J. (1989). Self assessment of foreign-language skills: Implications for teachers and researchers. Language Learning, 39, 313–340. doi:10.1111/j.1467-1770.1989.tb00595.x CrossRefGoogle Scholar
Brantmeier, C., Vanderplank, R., & Strube, M. (2012). What about me? Individual self-assessment by skill and level of language instruction. System, 40, 144–160. doi:10.1016/j.system.2012.01.003 CrossRefGoogle Scholar
Brown, P., & Levinson, S. (1987). Politeness: Some universals in language usage. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
Burson, K. A., Larrick, R. P., & Klayman, J. (2006). Skilled or unskilled, but still unaware of it: How perceptions of difficulty drive miscalibration in relative comparisons. Journal of Personality and Social Psychology, 90, 60–77. doi:10.1037/0022-3514.90.1.60 CrossRefGoogle ScholarPubMed
Carter, T. J., & Dunning, D. (2008). Faulty self-assessment: Why evaluating one's own competence is an intrinsically difficult task. Social and Personality Psychology Compass, 2, 346–360. doi:10.1111/j.1751-9004.2007.00031.x CrossRefGoogle Scholar
Chalhoub-Deville, M., & Turner, C. E. (2000). What to look for in ESL admission tests: Cambridge certificate exams, IELTS, and TOEFL. System, 28, 523–539. doi:10.1016/S0346-251X(00)00036-1 CrossRefGoogle Scholar
Critcher, C. R., & Dunning, D. (2009). How chronic self-views influence (and mislead) self-assessments of task performance: Self-views shape bottom-up experiences with the task. Journal of Personality and Social Psychology, 97, 931–945. doi:10.1037/a0017452 CrossRefGoogle ScholarPubMed
Crossley, S. A., Salsbury, T., & McNamara, D. S. (2014). Assessing lexical proficiency using analytic ratings: A case for collocation accuracy. Applied Linguistics. doi:10.1093/applin/amt056 CrossRefGoogle Scholar
Crowther, D., Trofimovich, P., Saito, K., & Isaacs, T. (2014). Second language comprehensibility revisited: Investigating the effects of learner background. TESOL Quarterly. Published online 28 October 2014. doi:10.1002/tesq.203 Google Scholar
Davidson, F., & Henning, G. (1985). A self-rating scale of English difficulty: Rasch scalar analysis of items and rating categories. Language Testing, 2, 164–179. doi:10.1177/026553228500200205 CrossRefGoogle Scholar
de Saint Léger, D. (2009). Self-assessment of speaking skills and participation in a foreign language class. Foreign Language Annals, 42, 158–178. doi:10.1111/j.1944-9720.2009.01013.x CrossRefGoogle Scholar
Derwing, T. M., & Munro, M. J. (2009). Putting accent in its place: Rethinking obstacles to communication. Language Teaching, 42, 476–490. doi:10.1017/S026144480800551X CrossRefGoogle Scholar
Derwing, T. M., & Rossiter, M. J. (2002). ESL learners’ perceptions of their pronunciation needs and strategies. System, 30, 155–166. doi:10.1016/S0346-251X(02)00012-X CrossRefGoogle Scholar
Derwing, T. M., Rossiter, M. J., Munro, M. J., & Thomson, R. I. (2004). Second language fluency: Judgments on different tasks. Language Learning, 54, 655–679. doi:10.1111/j.1467-9922.2004.00282.x CrossRefGoogle Scholar
Dlaska, A., & Krekeler, C. (2008). Self-assessment of pronunciation. System, 36, 506–516. doi:10.1016/j.system.2008.03.003 CrossRefGoogle Scholar
Dunning, D. (2005). Self-insight: Roadblocks and detours on the path to knowing thyself. New York: Psychology Press.Google Scholar
Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment: Implications for health, education, and the workplace. Psychological Science in the Public Interest, 5, 69–106. doi:10.1111/j.1529-1006.2004.00018.x CrossRefGoogle ScholarPubMed
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12, 83–87. doi:10.1111/1467-8721.01235 CrossRefGoogle Scholar
Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1989). Ambiguity and self-evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability. Journal of Personality and Social Psychology, 57, 1082–1090. doi:10.1037/0022-3514.57.6.1082 CrossRefGoogle Scholar
Eckman, F. (2004). From phonemic differences to constraint rankings: Research on second language phonology. Studies in Second Language Acquisition, 26, 513–549. doi:10.1017/S027226310404001X CrossRefGoogle Scholar
Ehrlinger, J., & Dunning, D. (2003). How chronic self-views influence (and potentially mislead) estimates of performance. Journal of Personality and Social Psychology, 84, 5–17. doi:10.1037/0022-3514.84.1.5 CrossRefGoogle ScholarPubMed
Ehrlinger, J., Johnson, K., Banner, M., Dunning, D., & Kruger, J. (2008). Why the unskilled are unaware: Further exploration of (absent) self-insight among the incompetent. Organizational Behavior and Human Decision Processes, 105, 98–121. doi:10.1016/j.obhdp.2007.05.002 CrossRefGoogle ScholarPubMed
Eid, M. & Diener, E. (2001). Norms for experiencing emotions in different cultures: Inter- and intranational differences. Journal of Personality and Social Psychology, 81, 869–885. doi:10.1037/0022-3514.81.5.869 CrossRefGoogle ScholarPubMed
Falchikov, N., & Boud, D. (1989). Student self-assessment in higher education: A meta-analysis. Review of Education Research, 59, 395–430. doi:10.3102/00346543059004395 CrossRefGoogle Scholar
Fay, A. J., Jordan, A. H., & Ehrlinger, J. (2012). How social norms promote misleading social feedback and inaccurate self-assessment. Social and Personality Psychology Compass, 6, 206–216. doi:10.1111/j.1751-9004.2011.00420.x CrossRefGoogle Scholar
Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Thousand Oaks, CA: Sage.Google Scholar
Flege, J. (2003). Assessing constraints on second-language segmental production and perception. In Meyer, A. & Schiller, N. (Eds.), Phonetics and phonology in language comprehension and production: Differences and similarities (pp. 319–355). Berlin: Mouton de Gruyter.CrossRefGoogle Scholar
Flege, J. E., Munro, M. J., & MacKay, I. R. A. (1995). Effects of age of second-language learning on the production of English consonants. Speech Communication, 16, 1–26. doi:10.1016/0167-6393(94)00044-B CrossRefGoogle Scholar
Foote, J. A., Trofimovich, P., Collins, L., & Soler Urzúa, F. (2013). Pronunciation teaching practices in communicative ESL classes. The Language Learning Journal. Published online 16 April 2013. doi:10.1080/09571736.2013.784345 Google Scholar
Fussell, S. R., & Krauss, R. M. (1992). Coordination of knowledge in communication: Effects of speakers’ assumptions about what others know. Journal of Personality and Social Psychology, 62, 378–391. doi:10.1037/0022-3514.62.3.378 CrossRefGoogle Scholar
Glover, P. (2011). Using CEFR level descriptors to raise university students’ awareness of their speaking skills. Language Awareness, 20, 121–133. doi:10.1080/09658416.2011.555556 CrossRefGoogle Scholar
Guenther, C. L., & Alicke, M. D. (2010). Deconstructing the better-than-average effect. Journal of Personality and Social Psychology, 99, 755–770. doi:10.1037/a0020959 CrossRefGoogle ScholarPubMed
Hacker, D. J., Bol, L., Horgan, D., & Rakow, E. (2000). Test prediction and performance in a classroom context. Journal of Educational Psychology, 92, 160–170. doi:10.1037/0022-0663.92.1.160 CrossRefGoogle Scholar
Harris, M. M., & Schaubroek, J. (1988). A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. Personnel Psychology, 41, 43–62. doi:10.1111/j.1744-6570.1998.tb00631.x CrossRefGoogle Scholar
Hayes, A. F., & Dunning, D. (1997). Construal processes and trait ambiguity: Implications for self-peer agreement in personality judgment. Journal of Personality and Social Psychology, 72, 664–677. doi:10.1037/0022-3514.72.3.664 CrossRefGoogle ScholarPubMed
Heilenman, L. K. (1990). Self-assessment of second language ability: The role of response effects. Language Testing, 7, 174–201. doi:10.1177/02655322900070004 CrossRefGoogle Scholar
Heine, S. J., Kitayama, S., & Lehman, D. R. (2001). Cultural differences in self-evaluation: Japanese readily accept negative self-relevant information. Journal of Cross-Cultural Psychology, 32, 434–443. doi:10.1177/0022022101032004004 CrossRefGoogle Scholar
Hodges, B., Regehr, G., & Martin, D. (2001). Difficulties in recognizing one's own incompetence: Novice physicians who are unskilled and unaware of it. Academic Medicine, 76, S87–S89.CrossRefGoogle Scholar
Hutcheson, G. D., & Sofroniou, N. (1999). The multivariate social scientist. London: Sage.CrossRefGoogle Scholar
Isaacs, T., & Trofimovich, P. (2011). International students at Canadian universities: Validating a pedagogically-oriented pronunciation scale. Unpublished corpus of second language speech.Google Scholar
Isaacs, T., & Trofimovich, P. (2012). Deconstructing comprehensibility: Identifying the linguistic influences on listeners’ L2 comprehensibility ratings. Studies in Second Language Acquisition, 34, 475–505. doi:10.1017/S0272263112000150 CrossRefGoogle Scholar
Janssen van Dieten, A.-M. (1989). The development of a test of Dutch as a second language: The validity of self-assessment by inexperienced subjects. Language Testing, 6, 30–46. doi:10.1177/026553228900600105 CrossRefGoogle Scholar
Jun, S.-A. (2005). Prosodic typology. In Jun, S.-A. (Ed.), Prosodic typology: The phonology of intonation and phrasing (pp. 430–458). Oxford: Oxford University Press.CrossRefGoogle Scholar
King, C. (1994). One language, two scripts: The Hindi movement in nineteenth century North India. Delhi: Oxford University Press.Google Scholar
Klin, C. M., Guzman, A. E., & Levine, W. H. (1997). Knowing that you don't know: Metamemory and discourse processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 1378–1393. doi:10.1037/0278-7393.23.6.1378 Google ScholarPubMed
Kruger, J., & Dunning, D. (1999). Unskilled or unaware of it: Difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121–1134. doi:10.1037/0022-3514.77.6.1121 CrossRefGoogle ScholarPubMed
Kruger, J., & Mueller, R. A. (2002). Unskilled, unaware, or both? The better-than-average heuristic and statistical regression predict errors in estimates of own performance. Journal of Personality and Social Psychology, 82, 180–188. doi:10.1037/0022-3514.82.2.180 CrossRefGoogle ScholarPubMed
Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. New York: Routledge.Google Scholar
Levis, J. M. (2005). Changing contexts and shifting paradigms in pronunciation teaching. TESOL Quarterly, 39, 369–377. doi:10.2307/3588485 CrossRefGoogle Scholar
Lichtenstein, S., & Fischoff, B. (1977). Do those who know more also know more about how much they know? Organization Behavior and Human Performance, 20, 159–183. doi:10.1016/0030-5073(77)90001-0 CrossRefGoogle Scholar
Little, D. (2005). The Common European Framework and the European Language Portfolio: Involving learners and their judgements in the assessment process. Language Testing, 22, 321–336. doi:10.1191/0265532205lt311oa CrossRefGoogle Scholar
Long, M. (1996). The role of the linguistic environment in second language acquisition. In Ritchie, W. C. & Bhatia, T. K. (Eds.), Handbook of second language acquisition (pp. 413–468). New York: Academic Press.Google Scholar
Long, M. H. (1991). Focus on form: A design feature in language teaching methodology. In de Bot, K., Coste, D., Ginsberg, R., & Kramsch, C. (Eds.), Foreign language research in cross-cultural perspective (pp. 39–52). Amsterdam: John Benjamins.CrossRefGoogle Scholar
Lyster, R. (2001). Negotiation of form, recasts, and explicit correction in relation to error types and learner repair in immersion classrooms. Language Learning, 51, 265–301. doi:10.1111/j.1467-1770.2001.tb00019.x CrossRefGoogle Scholar
Mabe, P. A., & West, S. G. (1982). Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology, 67, 280–296. doi:10.1037/0021-9010.67.3.280 CrossRefGoogle Scholar
Mackey, A. & Goo, J. (2007). Interaction research in SLA: A meta-analysis and research synthesis. In Mackey, A. (Ed.), Conversational interaction in second language acquisition: A collection of empirical studies (pp. 407–452). Oxford: Oxford University Press.Google Scholar
Mattern, K. D., Burrus, J., & Shaw, E. (2010). When both skilled and unskilled are unaware: Consequences for academic performance. Self and Identity, 9, 129–141. doi:10.1080/15298860802618963 CrossRefGoogle Scholar
Moyer, A. (2013). Foreign accent: The phenomenon of non-native speech. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Ochs, E. (1979). Transcription as theory. In Orchs, E. & Schieffelin, B. B. (Eds.), Developmental pragmatics (pp. 43–72). New York: Academic Press.Google Scholar
Ross, S. (1998). Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 15, 1–20. doi:10.1177/026553229801500101 CrossRefGoogle Scholar
Saito, K. (2013). Communicative focus on second language phonetic form: Teaching Japanese learners to perceive and produce English/ɹ/without explicit instruction. Applied Psycholinguistics. Published online 28 April 2013. doi:10.1017/S0142716413000271 Google Scholar
Saito, K., Trofimovich, P., & Isaacs, T. (forthcoming). Using listener judgements to investigate linguistic influences on L2 comprehensibility and accentedness: A validation and generalization study. Applied Linguistics. Google Scholar
Schmidt, R. (2001). Attention. In Robinson, P. (Ed.), Cognition and second language instruction (pp. 3–32). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Segalowitz, N. (2010). Cognitive bases of second language fluency. New York: Routledge.CrossRefGoogle Scholar
Shackle, C. (2001). Speakers of South Asian languages. In Swan, M. & Smith, B. (Eds.), Learner English: A teacher's guide to interference and other problems (pp. 310–324). Cambridge: Cambridge University Press.Google Scholar
Simons, D. J. (2013). Unskilled and optimistic: Overconfident predictions despite calibrated knowledge of relative skill. Psychonomic Bulletin & Review, 20, 601–607.CrossRefGoogle ScholarPubMed
Sinkavich, F. J. (1995). Performance and metamemory: Do students know what they don't know? Instructional Psychology, 22, 77–87.Google Scholar
Smiljanic, R., & Bradlow, A. R. (2009). Speaking and hearing clearly: Talker and listener factors in speaking style changes. Linguistics and Language Compass, 3, 236–264. doi:10.1111/j.1749-818X.2008.00112.x CrossRefGoogle ScholarPubMed
Stemler, S.E., & Tsai, J. (2008). Best practices in estimating interrater reliability. In Osborne, J. (Ed.): Best practices in quantitative methods (pp. 29–49). Thousand Oaks, CA: Sage.CrossRefGoogle Scholar
Tajima, K., Port, R., & Dalby, J. (1997). Effects of temporal correction on intelligibility of foreign-accented English. Journal of Phonetics, 25, 1–24. doi:10.1006/jpho.1996.0031 CrossRefGoogle Scholar
Tesser, A., & Rosen, S. (1975). The reluctance to transmit bad news. In Berkowitz, L. (Ed.), Advances in experimental social psychology (pp. 193–232). New York: Academic Press.Google Scholar
Tokumoto, M., & Shibata, M. (2011). Asian varieties of English: Attitudes towards pronunciation. World Englishes, 30, 392–408. doi:10.1111/j.1467-971X.2011.01710.x CrossRefGoogle Scholar
Trofimovich, P., & Isaacs, T. (2012). Disentangling accent from comprehensibility. Bilingualism: Language and Cognition, 15, 905–916. doi:10.1017/S1366728912000168 CrossRefGoogle Scholar
Upshur, J. (1975). Objective evaluation of oral proficiency in the ESOL classroom. In Palmer, L. & Spolsky, B. (Eds.), Papers on language testing (pp. 1967–1974). Washington: TESOL.Google Scholar
Weigle, S. C. (2010). Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability. Language Testing, 27, 335–353. doi:10.1177/0265532210364406 CrossRefGoogle Scholar
Yule, G., Damico, J., & Hoffman, P. (1987). Learners in transition: Evidence from the interaction of accuracy and self-monitoring skill in a listening task. Language Learning, 37, 511–521. doi:10.1111/j.1467-1770.1987.tb00582.x CrossRefGoogle Scholar
Yule, G., Hoffman, P., & Damico, J. (1987). Paying attention to pronunciation: The role of self-monitoring in perception. TESOL Quarterly, 21, 765–768. doi:10.2307/3586994 CrossRefGoogle Scholar