Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-22T23:51:24.289Z Has data issue: false hasContentIssue false
  • English
  • Français

Visual salience effects on speaker choices: Direct or indirect influences on linguistic processing?

Published online by Cambridge University Press:  11 October 2016

INÉS ANTÓN-MÉNDEZ
INÉS ANTÓN-MÉNDEZ*
Affiliation:
University of New England, Australia
*
ADDRESS FOR CORRESPONDENCE Inés Antón-Méndez, Linguistics (E11), University of New England, Armidale, New South Wales 2351, Australia. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The effect of visual salience on speakers’ choices is investigated by contrasting the effects of both visual and linguistic manipulations on picture descriptions and eye movements. Two-character pictures were used, which can be described in one of two complementary ways (e.g., a cop chasing a robber can be described either from a chasing or from a fleeing perspective), and using simple actives or other alternative syntactic structures (e.g., “a robber is being chased by a cop”). The pictures were preceded by a verb priming one of the two perspectives and/or a preview of one of the two characters. The results show that the visual manipulation affects looks to the characters regardless of which perspective had been linguistically primed, but it only affects verbal descriptions in the absence of a linguistic prime. Linguistically priming one of the perspectives, in contrast, has a reliable effect on both looks to the characters and verbal descriptions. These results suggest that visual salience does not influence linguistic choices directly.

Type
Articles
Information
Applied Psycholinguistics , Volume 38 , Issue 3 , May 2017 , pp. 601 - 631
Check for updates
Copyright
Copyright © Cambridge University Press 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Antón-Méndez, I., Gerfen, C., & Ramos, M. (2016). Salience effects: L2 sentence production as a window on L1 speech planning. Journal of Psycholinguistic Research, 45, 537–552. doi:10.1007/s10936-015-9361-7 CrossRefGoogle ScholarPubMed
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278. doi:10.1016/j.jml.2012.11.001 Google Scholar
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2014). lme4: Linear mixed-effects models using Eigen and S4. Journal of Statistical Software, 67.Google Scholar
Bock, K., & Ferreira, V. S. (2014). Syntactically speaking. In Goldrick, M., Ferreira, V., & Miozzo, M. (Eds.), The Oxford handbook of language production (pp. 21–46). Oxford: Oxford University Press.Google Scholar
Bock, K., Irwin, D. E., & Davidson, D. J. (2004). Putting first things first. In Henderson, J. M. & Ferreira, F. (Eds.), The interface of language, vision and action: Eye movements and the visual world. New York: Psychology Press.Google Scholar
Bock, K., Irwin, D. E., Davidson, D. J., & Levelt, W. J. M. (2003). Minding the clock. Journal of Memory and Language, 48, 653–685.Google Scholar
Branigan, H. P., Pickering, M. J., & Tanaka, M. N. (2008). Contributions of animacy to grammatical function assignment and word order during production. Lingua, 118, 172–189. doi:10.1016/j.lingua.2007.02.003 Google Scholar
Brawley, H. (2012). What informs event descriptions: Language, salience, and discourse in English and Japanese. Unpubished manuscript, Ohio State University. Retrieved from https://etd.ohiolink.edu/ap/10?0::NO:10:P10_ACCESSION_NUM:osu1338275303 Google Scholar
Bunger, A., Papafragou, A., & Trueswell, J. C. (2013). Event structure influences language production: Evidence from structural priming in motion event description. Journal of Memory and Language, 69, 299–323. doi:10.1016/j.jml.2013.04.002 Google Scholar
Coco, M. I., & Keller, F. (2009). The impact of visual information on reference assignment in sentence production. Paper presented at the 31st Annual Conference of the Cognitive Science Society, Amsterdam.Google Scholar
Coco, M. I., & Keller, F. (2012). Scan patterns predict sentence production in the cross-modal processing of visual scenes. Cognitive Science, 36, 1204–1223. doi:10.1111/j.1551-6709.2012.01246.x Google Scholar
Coco, M. I., & Keller, F. (2015). Integrating mechanisms of visual guidance in naturalistic language production. Cognitive Processing, 16, 131–150. doi:10.1007/s10339-014-0642-0 Google Scholar
Coco, M. I., Malcolm, G. L., & Keller, F. (2014). The interplay of bottom-up and top-down mechanisms in visual guidance during object naming. Quarterly Journal of Experimental Psychology, 67, 1096–1200. doi:10.1080/17470218.2013.844843 Google Scholar
Ferreira, F., & Swets, B. (2002). How incremental is language production? Evidence from the production of utterances requiring the computation of arithmetic sums. Journal of Memory and Language, 46, 57–84. doi.org/10.1006/jmla.2001.2797 Google Scholar
Fischer, B., & Ramsperger, E. (1984). Human express saccades: Extremely short reaction times of goal directed eye movements. Experimental Brain Research, 57. doi:10.1007/BF0023114 Google Scholar
Fukumura, K., van Gompel, R. P. G., & Pickering, M. J. (2010). The use of visual context during the production of referring expressions. Quarterly Journal of Experimental Psychology, 63, 1700–1715. doi:10.1080/17470210903490969 CrossRefGoogle ScholarPubMed
Gleitman, L. R., January, D., Nappa, R., & Trueswell, J. C. (2007). On the give and take between event apprehension and utterance formulation. Journal of Memory and Language, 57, 544–569. doi:10.1016/j.jml.2007.01.007 Google Scholar
Griffin, Z. M. (2001). Gaze durations during speech reflect word selection and phonological encoding. Cognition, 82, B1–B14. doi:10.1016/S0010-0277(01)00138-X Google Scholar
Griffin, Z. M., & Bock, K. (2000). What the eyes say about speaking. Psychological Science, 11, 274–279. doi:10.1111/1467-9280.00255 Google Scholar
Haskell, T. R. (2005). Speaker's eye movements during the production of adjectives. Paper presented at the 18th Annual CUNY Sentence Processing Conference, Tucson, AZ.Google Scholar
Hickmann, M., Taranne, P., & Bonnet, P. (2009). Motion in first language acquisition: Manner and path in French and English child language. Journal of Child Language, 36, 705–741. doi:10.1017/S0305000908009215 Google Scholar
Ibbotson, P., Lieven, E. V. M., & Tomasello, M. (2013). The attention–grammar interface: Eye-gaze cues structural choice in children and adults. Cognitive Linguistics, 24. doi:10.1515/cog-2013-0020 Google Scholar
Jackendoff, R. (1987). On beyond zebra: The relation of linguistic and visual information. Cognition, 26, 89–114.Google Scholar
Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs (transformational or not) and towards logit mixed models. Journal of Memory and Language, 59, 434–446. doi:10.1016/j.jml.2007.11.007 Google Scholar
Kempen, G., & Hoenkamp, E. (1987). An incremental procedural grammar for sentence formulation. Cognitive Science, 11, 173–218.CrossRefGoogle Scholar
Klein, R. M. (2000). Inhibition of return. Trends in Cognitive Sciences, 4, 138–147. doi:10.1016/S1364-6613(00)01452-2 Google Scholar
Kuchinsky, S. E. (2009). From seeing to saying: Perceiving, planning, producing. Ann Arbor, MI: ProQuest. Retrieved from http://eric.ed.gov/?id=ED526717 Google Scholar
Kuchinsky, S. E., & Bock, K. (2010). From seeing to saying: Perceiving, planning, producing. Paper presented at the CUNY Sentence Processing Conference, New York.Google Scholar
Kuchinsky, S. E., Bock, K., & Irwin, D. E. (2011). Reversing the hands of time: Changing the mapping from seeing to saying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 1–16. doi:10.1037/a0022637.Google ScholarPubMed
Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207, 203–205.Google Scholar
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2014). lmerTest: Tests for random and fixed effects for linear mixed effect models (lmer objects of lme4 package). Retrieved from http://CRAN.R-project.org/package=lmerTest Google Scholar
Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.Google Scholar
Meyer, A., van der Meulen, F., & Brooks, A. (2004). Eye movements during speech planning: Talking about present and remembered objects. Visual Cognition, 11, 553–576. doi:10.1080/13506280344000248 CrossRefGoogle Scholar
Mirman, D., Dixon, J. A., & Magnuson, J. S. (2008). Statistical and computational models of the visual world paradigm: Growth curves and individual differences. Journal of Memory and Language, 59, 475–494. doi:10.1016/j.jml.2007.11.006 Google Scholar
Montag, J. L., & MacDonald, M. C. (2013). Visual salience modulates structure choice in relative clause production. Language and Speech, 57, 163–180. doi:10.1177/0023830913495656 Google Scholar
Myachykov, A., & Garrod, S. (2008). Perception and word order in Russian and Finnish sentence production. In Ramm, W. & Fabricius-Jansen, C. (Eds.), Linearisation and segmentation in discourse: Multidisciplinary approaches to discourse 2008 (pp. 87–97). Oslo: University of Oslo.Google Scholar
Myachykov, A., Garrod, S., & Scheepers, C. (2011). Perceptual priming of structural choice during English and Finnish sentence production. In Mishra, R. K. & Srinivasan, N. (Eds.), Language and cognition: State of the art (pp. 53–71). Munich: Lincom Europa.Google Scholar
Myachykov, A., Garrod, S., & Scheepers, C. (2012). Determinants of structural choice in visually situated sentence production. Acta Psychologica, 141, 304–315. doi:10.1016/j.actpsy.2012.09.006 Google Scholar
Myachykov, A., Posner, M. I., & Tomlin, R. S. (2007). A parallel interface for language and cognition in sentence production: Theory, method, and experimental evidence. Linguistic Review, 24, 457–474. doi:10.1515/TLR.2007.020 Google Scholar
Myachykov, A., Thompson, D., Garrod, S., & Scheepers, C. (2011). Referential and visual cues to structural choice in visually situated sentence production. Frontiers in Psychology, 2, 1–9. doi:10.3389/fpsyg.2011.00396 Google ScholarPubMed
Myachykov, A., Thompson, D., Scheepers, C., & Garrod, S. (2011). Visual attention and structural choice in sentence production across languages. Language and Linguistics Compass, 5, 95–107. doi:10.1111/j.1749-818X.2010.00265.x Google Scholar
Prat-Sala, M., & Branigan, H. P. (2000). Discourse constraints on syntactic processing in language production: A cross-linguistic study in English and Spanish. Journal of Memory and Language, 42, 168–182. doi:10.1006/jmla.1999.2668 Google Scholar
Qualtrics LLC. (2005). Qualtrics [Computer software]. Retrieved from http://www.qualtrics.com/ Google Scholar
Core Team, R. (2015). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.Google Scholar
Smith, M., & Wheeldon, L. R. (1999). High level processing scope in spoken sentence production. Cognition, 73, 205–246. doi:10.1016/S0010-0277(99)00053-0 Google Scholar
Tanaka, M. N., Branigan, H. P., McLean, J. F., & Pickering, M. J. (2011). Conceptual influences on word order and voice in sentence production: Evidence from Japanese. Journal of Memory and Language, 65, 318–330. doi:10.1016/j.jml.2011.04.009 Google Scholar
Tomlin, R. S. (1997). Mapping conceptual representations into linguistic representations: The role of attention in grammar. In Nuyts, J. & Pederson, E. (Eds.), Language and conceptualization (pp. 162–189). Cambridge: Cambridge University Press.Google Scholar
van der Meulen, F. F. (2001). Moving eyes and naming objects. Retrieved from http://repository.ubn.ru.nl/handle/2066/19013 Google Scholar
van de Velde, M., Meyer, A. S., & Konopka, A. E. (2014). Message formulation and structural assembly: Describing “easy” and “hard” events with preferred and dispreferred syntactic structures. Journal of Memory and Language, 71, 124–144. doi:10.1016/j.jml.2013.11.001 Google Scholar
Vogels, J., Krahmer, E., & Maes, A. (2013). Who is where referred to how, and why? The influence of visual saliency on referent accessibility in spoken language production. Language and Cognitive Processes, 28, 1323–1349. doi:10.1080/01690965.2012.682072 Google Scholar
Wheeldon, L. R., Ohlson, N., Ashby, A., & Gator, S. (2013). Lexical availability and grammatical encoding scope during spoken sentence production. Quarterly Journal of Experimental Psychology, 66, 37–41. doi:10.1080/17470218.2012.754913 Google Scholar