Hostname: page-component-cd9895bd7-7cvxr Total loading time: 0 Render date: 2024-12-23T08:30:19.745Z Has data issue: false hasContentIssue false
  • English
  • Français

A cross-language study on feedforward and feedback control of voice intensity in Chinese–English bilinguals

Published online by Cambridge University Press:  10 July 2020

Xiao Cai Open the ORCID record for Xiao Cai [Opens in a new window] ,
Yulong Yin  and
Qingfang Zhang
Xiao Cai
Affiliation:
Renmin University of China
Yulong Yin
Affiliation:
Renmin University of China
Qingfang Zhang*
Affiliation:
Renmin University of China
*
*Corresponding author. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Speech production requires the combined efforts of feedforward and feedback control, but it remains unclear whether the relative weighting of feedforward and feedback control is organized differently between the first language (L1) and the second language (L2). In the present study, a group of Chinese–English bilinguals named pictures in their L1 and L2, while being exposed to multitalker noise. Experiment 1 compared feedforward control between L1 and L2 speech production by examining intensity increases in response to a masking noise (90 dB SPL). Experiment 2 compared feedback control between L1 and L2 speech production by examining intensity increases in response to a weak (30 dB SPL) or strong noise (60 dB SPL). We also examined a potential relationship between L2 fluency and the relative weighting of feedforward and feedback systems. The results indicated that L2 speech production relies less on feedforward control relative to L1, exhibiting attenuated Lombard effects to the masking noise. In contrast, L2 speech production relies more on feedback control than L1, producing larger Lombard effects to the weak and strong noise. The relative weighting of feedforward and feedback control is dynamically changed as second language learning progresses.

Keywords

L2 speech motor controlfeedforward controlfeedback controlLombard effectvoice intensity
Type
Original Article
Information
Applied Psycholinguistics , Volume 41 , Issue 4 , July 2020 , pp. 771 - 795
Check for updates
Copyright
© The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abutalebi, J., Cappa, S. F., & Perani, D. (2001). The bilingual brain as revealed by functional neuroimaging. Bilingualism: Language and Cognition, 4, 179–190.CrossRefGoogle Scholar
Alm, P. A. (2004). Stuttering and the basal ganglia circuits: A critical review of possible relations. Journal of Communication Disorders, 37, 325–369.CrossRefGoogle ScholarPubMed
Alm, P. A. (2005). On the causal mechanisms of stuttering. Lund University.Google Scholar
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.CrossRefGoogle Scholar
Ballard, K. J., Halaki, M., Sowman, P. F., Kha, A., Daliri, A., Robin, D., … & Guenther, F. (2018). An investigation of compensation and adaptation to auditory perturbations in individuals with acquired apraxia of speech. Frontiers in Human Neuroscience, 12, 510.CrossRefGoogle ScholarPubMed
Bates, D. M. (2005). Fitting linear mixed models in R. R News, 5, 27–30.Google Scholar
Bates, D. M., Maechler, M., Bolker, B., & Walker, S. (2014). lme4: Linear mixed-effects models using Eigen and S4. R package version 1, 1–7.Google Scholar
Bauer, J. J., Mittal, J., Larson, C. R., & Hain, T. C. (2006). Vocal responses to unanticipated perturbations in voice loudness feedback: An automatic mechanism for stabilizing voice amplitude. Journal of the Acoustical Society of America, 119, 2363–2371.CrossRefGoogle ScholarPubMed
Bays, P. M., & Wolpert, D. M. (2006). Computational principles of sensorimotor control that minimize uncertainty and variability. Journal of Physiology, 578, 387–396.CrossRefGoogle ScholarPubMed
Behroozmand, R., Shebek, R., Hansen, D. R., Oya, H., Robin, D. A., Howard, M. A III., & Greenlee, J. D. (2015). Sensory-motor networks involved in speech production and motor control: An fMRI study. NeuroImage, 109, 418–428.CrossRefGoogle ScholarPubMed
Bergmann, C., Sprenger, S. A., & Schmid, M. S. (2015). The impact of language co-activation on L1 and L2 speech fluency. Acta Psychologica, 161, 25–35.CrossRefGoogle ScholarPubMed
Birdsong, D., & Molis, M. (2001). On the evidence for maturational constraints in second-language acquisition. Journal of Memory and Language, 44, 235–249.CrossRefGoogle Scholar
Boersma, P., & Weenink, D. (2013). Praat: Doing Phonetics by Computer [Computer program]. http://www.praat.org Google Scholar
Cai, S., Beal, D. S., Ghosh, S. S., Tiede, M. K., & Guenther, F. H. (2012). Weak responses to auditory feedback perturbation during articulation in persons who stutter: Evidence for abnormal auditory-motor transformation. PLoS ONE, 7, e41830.CrossRefGoogle ScholarPubMed
Chang, E. F., Niziolek, C. A., Knight, R. T., Nagarajan, S. S., & Houde, J. F. (2013). Human cortical sensorimotor network underlying feedback control of vocal pitch. Proceedings of the National Academy of Sciences, 110, 2653–2658.CrossRefGoogle ScholarPubMed
Chang-Yit, R., Pick, H. L., & Siegel, G. M. (1975). Reliability of sidetone amplification effect in vocal intensity. Journal of Communication Disorders, 8, 317–324.CrossRefGoogle ScholarPubMed
Chen, Y., Robb, M. P., Gilbert, H. R., & Lerman, J. W. (2001). Vowel production by Mandarin speakers of English. Clinical Linguistics & Phonetics, 15, 427–440.Google Scholar
Chen, Z., Liu, P., Wang, E. Q., Larson, C. R., Huang, D., & Liu, H. (2012). ERP correlates of language-specific processing of auditory pitch feedback during self-vocalization. Brain and Language, 121, 25–34.CrossRefGoogle ScholarPubMed
Chen, Z., Wong, F. C. K., Jones, J. A., Li, W., Liu, P., Chen, X., et al. (2015). Transfer effect of speech-sound learning on auditory-motor processing of perceived vocal pitch errors. Scientific Reports, 5, 13134.CrossRefGoogle ScholarPubMed
Christoffels, I. K., Formisano, E., & Schiller, N. O. (2007). Neural correlates of verbal feedback processing: An fMRI study employing overt speech. Hum Brain Mapping, 28, 868–879.CrossRefGoogle ScholarPubMed
Civier, O. (2010). Computational modeling of the neural substrates of stuttering and induced fluency (Unpublished doctoral dissertation, Boston University).Google Scholar
Civier, O., Tasko, S. M., & Guenther, F. H. (2010). Overreliance on auditory feedback may lead to sound/syllable repetitions: simulations of stuttering and fluency-inducing conditions with a neural model of speech production. Journal of Fluency Disorders, 35, 246–279.CrossRefGoogle ScholarPubMed
Daliri, A., & Max, L. (2015a). Electrophysiological evidence for a general auditory prediction deficit in adults who stutter. Brain and Language, 150, 37–44.CrossRefGoogle ScholarPubMed
Daliri, A., & Max, L. (2015b). Modulation of auditory processing during speech movement planning is limited in adults who stutter. Brain and Language, 143, 59–68.CrossRefGoogle ScholarPubMed
De Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H. (2012). Facets of speaking proficiency. Studies in Second Language Acquisition, 34, 5–34.CrossRefGoogle Scholar
Epstein, S., Flynn, S., & Martohardjono, G. (1996). Second language acquisition: Theoretical and experimental issues in contemporary research. Behavioral and Brain Sciences, 19, 677–714.CrossRefGoogle Scholar
Ganushchak, L. Y., & Schiller, N. O. (2009). Speaking one’s second language under time pressure: An ERP study on verbal self-monitoring in German–Dutch bilinguals. Psychophysiology, 46, 410–419.CrossRefGoogle Scholar
Götz, S. (2013). Fluency in native and nonnative English speech. John Benjamins.CrossRefGoogle Scholar
Grosjean, F. (2010). Bilingual. Harvard University Press.CrossRefGoogle Scholar
Guenther, F. H. (2006). Cortical interactions underlying the production of speech sounds. Journal of Communication Disorders, 39, 350–365.CrossRefGoogle ScholarPubMed
Guenther, F. H. (2016). Neural control of speech. MIT Press.CrossRefGoogle Scholar
Guenther, F. H., & Vladusich, T. (2012). A neural theory of speech acquisition and Production. Journal of Neurolinguistics, 25, 408–422.CrossRefGoogle Scholar
Guenther, F. H., Ghosh, S. S., & Tourville, J. A. (2006). Neural modeling and imaging of the cortical interactions underlying syllable production. Brain and Language, 96, 280–301.CrossRefGoogle ScholarPubMed
Heinks-Maldonado, T. H., & Houde, J. F. (2005). Compensatory responses to brief perturbations of speech amplitude. Acoustics Research Letters Online, 6, 131–137.CrossRefGoogle Scholar
Hickok, G. (2012). Computational neuroanatomy of speech production. Nature Reviews Neuroscience, 13, 135–145.CrossRefGoogle ScholarPubMed
Hickok, G., Houde, J., & Rong, F. (2011). Sensorimotor integration in speech processing: Computational basis and neural organization. Neuron, 69, 407–422.CrossRefGoogle ScholarPubMed
Hilton, H. (2014). Oral fluency and spoken proficiency: Considerations for research and testing. In Leclercq, Pascale, Edmonds, Amanda, & Hilton, Heather (Eds.), Measuring L2 proficiency: Perspectives from SLA (pp. 27–53). Multilingual Matters.CrossRefGoogle Scholar
Hincks, R. (2008). Presenting in English or Swedish: Differences in speaking rate. Proceedings of FONETIK 2008 (pp. 21–24). Department of Linguistics, Gothenburg University.Google Scholar
Houde, J. F., & Jordan, M. I. (2002). Sensorimotor adaptation of speech I: Compensation and adaptation. Journal of Speech Language and Hearing Research, 45, 295–310.CrossRefGoogle ScholarPubMed
Houde, J. F., & Nagarajan, S. S. (2011). Speech production as state feedback control. Frontiers in Human Neuroscience, 5, 82.CrossRefGoogle ScholarPubMed
Houde, J. F., Nagarajan, S. S., Sekihara, K., & Merzenich, M. M. (2002). Modulation of the auditory cortex during speech: An MEG study. Journal of Cognitive Neuroscience, 14, 1125–1138.CrossRefGoogle Scholar
Howell, P., & Powell, D. J. (1984). Hearing your voice through bone and air: Implications for explanations of stuttering behaviour from studies of normal speakers. Journal of Fluency Disorders, 9, 247–264.CrossRefGoogle Scholar
Indefrey, P., & Levelt, W. J. M. (2004). The spatial and temporal signatures of word production components. Cognition, 92, 101–144.CrossRefGoogle ScholarPubMed
Kearney, E., & Guenther, F. H. (2019). Articulating: The neural mechanisms of speech production. Language, Cognition and Neuroscience, 34, 1214–1229.CrossRefGoogle ScholarPubMed
Kent, R. D., Kent, J. F., Weismer, G., & Duffy, J. R. (2000). What dysarthrias can tell us about the neural control of speech. Journal of Phonetics, 28, 273–302.CrossRefGoogle Scholar
Kormos, J. (2006). Speech production and second language acquisition. Lawrence Erlbaum Associates.Google Scholar
Lametti, D. R., Nasir, S. M., & Ostry, D. J. (2012). Sensory preference in speech production revealed by simultaneous alteration of auditory and somatosensory feedback. Journal of Neuroscience, 32, 9351–9358.CrossRefGoogle ScholarPubMed
Lametti, D. R., Krol, S. A., Shiller, D. M., & Ostry, D. J. (2014). Brief periods of auditory perceptual training can determine the sensory targets of speech motor learning. Psychological Science, 25, 1325–1336.CrossRefGoogle ScholarPubMed
Levelt, W. J. M., Roelofs, A., & Meyer, A.S. (1999). A theory of lexical access in speech production. Behavioral and Brain Sciences, 22, 1–75.CrossRefGoogle ScholarPubMed
Lin, I-F., Mochida, T., Asada, K., Ayaya, S., Kumagaya, S. I., & Kato, M. (2015). Atypical delayed auditory feedback effect and Lombard effect on speech production in high-functioning adults with autism spectrum disorder. Frontiers in Human Neuroscience, 9, 510.CrossRefGoogle ScholarPubMed
Liu, H., Russo, N., & Larson, C. R. (2010a). Age-related differences in vocal responses to pitch feedback perturbations: A preliminary study. Journal of the Acoustical Society of America, 127, 1042–1046.CrossRefGoogle ScholarPubMed
Liu, H., Zhang, Q., Xu, Y., & Larson, C. R. (2007). Compensatory responses to loudness-shifted voice feedback during production of Mandarin speech. Journal of the Acoustical Society of America, 122, 2405–2412.CrossRefGoogle ScholarPubMed
Liu, H., Wang, E. Q., Chen, Z., Liu, P., Larson, C. R., & Huang, D. (2010b). Effect of tonal native language on voice fundamental frequency responses to pitch feedback perturbations during vocalization. Journal of the Acoustical Society of America, 128, 3739–3746.CrossRefGoogle Scholar
Liu, P., Chen, Z., Larson, C. R., Huang, D. & Liu, H. (2010c). Auditory feedback control of voice fundamental frequency in school children. Journal of the Acoustical Society of America, 128, 1306–1312.CrossRefGoogle ScholarPubMed
Liu, X., & Tian, X. (2018). The functional relations among motor-based prediction, sensory goals and feedback in learning non-native speech sounds: Evidence from adult Mandarin Chinese speakers with an auditory feedback masking paradigm. Scientific Reports, 8, 11910.CrossRefGoogle ScholarPubMed
Lombard, E. (1911). The sign of the elevation of the voice [in French: Le signe de l’elevation de la voix], Annales des Maladies de l’Oreille, du Larynx, du Nez et du Pharynx, 37, 101–119, English translation: http://paul.sobriquet.net/wp-content/uploads/2007/02/lombard-1911-p-h-mason-2006.pdf Google Scholar
Loucks, T., Chon, H., & Han, W. (2012). Audiovocal integration in adults who stutter. International Journal of Language & Communication Disorders, 47, 451–456.CrossRefGoogle ScholarPubMed
Maas, E., Mailend, M. L., & Guenther, F. H. (2015). Feedforward and feedback control in apraxia of speech: Effects of noise masking on vowel production. Journal of Speech, Language, and Hearing Research, 58, 185–200.CrossRefGoogle ScholarPubMed
Mackay, D. G. (1970). How does language familiarity influence stuttering under delayed auditory feedback? Perceptual and Motor Skills, 30, 655–669.CrossRefGoogle ScholarPubMed
Mitsuya, T., MacDonald, E. N., Purcell, D. W., & Munhall, K. G. (2011). A cross-language study of compensation in response to real-time formant perturbation. Journal of the Acoustical Society of America, 130, 2978–2986.CrossRefGoogle ScholarPubMed
Moser, D., Fridriksson, J., Bonhilha, L., Healy, E. W., Baylis, G., Baker, J. M., & Rorden, C. (2009). Neural recruitment for the production of native and novel speech sounds. NeuroImage, 46, 549–557.CrossRefGoogle ScholarPubMed
Ng, M. L., Chen, Y., & Sadaka, J. (2008). Vowel features in Turkish accented English. International Journal of Speech-Language Pathology, 10, 404–413.CrossRefGoogle ScholarPubMed
Ning, L-H., Loucks, T. M., & Shih, C. (2015). The effects of language learning and vocal training on sensorimotor control of lexical tone. Journal of Phonetics, 51, 50–69.CrossRefGoogle Scholar
Ning, L.-H., Shih, C., & Loucks, T. M. (2014). Mandarin tone learning in L2 adults: A test of perceptual and sensorimotor contributions. Speech Communication, 63–64, 55–69.CrossRefGoogle Scholar
Parker Jones, Ō., Green, D. W., Grogan, A., Pliatsikas, C., Filippopolitis, K., Ali, N., … & Seghier, M. L. (2012). Where, when and why brain activation differs for bilinguals and monolinguals during picture naming and reading aloud. Cerebral Cortex, 22, 892–902.CrossRefGoogle ScholarPubMed
Parrell, B., Lammert, A. C., Ciccarelli, G., & Quatieri, T. F. (2019). Current models of speech motor control: A control-theoretic overview of architectures and properties. Journal of the Acoustical Society of America, 145, 1456–1481.CrossRefGoogle ScholarPubMed
Patel, R., & Schell, K. W. (2008). The influence of linguistic content on the Lombard effect. Journal of Speech, Language, and Hearing Research, 51, 209–220.CrossRefGoogle ScholarPubMed
Perkell, J. S. (2012). Movement goals and feedback and feedforward control mechanisms in speech production. Journal of Neurolinguistics, 25, 382–407.CrossRefGoogle ScholarPubMed
R Core Team. (2015). R: A language and environment for statistical computing. R foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/ Google Scholar
Scheerer, N. E., & Jones, J. A. (2014). The predictability of frequency-altered auditory feedback changes the weighting of feedback and feedforward input for speech motor control. European Journal of Neuroscience, 40, 3793–3806.CrossRefGoogle ScholarPubMed
Scheerer, N. E., Liu, H., & Jones, J. (2013). The development trajectory of vocal and event-related potential response to frequency altered auditory feedback. European Journal of Neuroscience, 38, 3189–3200.CrossRefGoogle Scholar
Schmidt, R. A. & Lee, T. D. (2005). Motor control and learning: A behavioral emphasis. Human Kinetics Publishers.Google Scholar
Segalowitz, N. (2010). Cognitive bases of second language fluency. Routledge.CrossRefGoogle Scholar
Simmonds, A. J., Wise, R. J., & Leech, R. (2011a). Two tongues, one brain: Imaging bilingual speech production. Frontiers in Psychology, 2, 166.CrossRefGoogle ScholarPubMed
Simmonds, A. J., Wise, R. J., Dhanjal, N. S., & Leech, R. (2011b). A comparison of sensory-motor activity during speech in first and second languages. Journal of Neurophysiology, 106, 470–478.CrossRefGoogle ScholarPubMed
Terband, H., Rodd, J., & Maas, E. (2015). Simulations of feedforward and feedback control in apraxia of speech (AOS): Effects of noise masking on vowel production in the DIVA model. In The 18th International Congress of Phonetic Sciences. Google Scholar
Tian, X., & Poeppel, D. (2010). Mental imagery of speech and movement implicates the dynamics of internal forward models. Frontiers in Psychology, 1, 166.CrossRefGoogle ScholarPubMed
Tian, X., & Poeppel, D. (2012). Mental imagery of speech: Linking motor and perceptual systems through internal simulation and estimation. Frontiers in Human Neuroscience, 6, 314.CrossRefGoogle ScholarPubMed
Tian, X., & Poeppel, D. (2013). The effect of imagination on stimulation: The functional specificity of efference copies in speech processing. Journal of Cognitive Neuroscience, 25, 1020–1036.CrossRefGoogle ScholarPubMed
Tian, X., & Poeppel, D. (2015). Dynamics of self-monitoring and error detection in speech production: Evidence from mental imagery and MEG. Journal of Cognitive Neuroscience, 27, 352–364.CrossRefGoogle ScholarPubMed
Tian, X., Ding, N., Teng, X., Bai, F., & Poeppel, D. (2018). Imagined speech influences perceived loudness of sound. Nature Human Behaviour, 2, 225–234.CrossRefGoogle Scholar
Tourville, J. A., & Guenther, F. H. (2011). The DIVA model: A neural theory of speech acquisition and production. Language and Cognitive Processes, 26, 952–981.CrossRefGoogle Scholar
Tourville, J. A., Reilly, K. J., & Guenther, F. H. (2008). Neural mechanisms underlying auditory feedback control of speech. NeuroImage, 39, 1429–1443.CrossRefGoogle Scholar
Toyomura, A., Koyama, S., Miyamaoto, T., Terao, A., Omori, T., Murohashi, H., et al. (2007). Neural correlates of auditory feedback control in human. Neuroscience, 146, 499–503.CrossRefGoogle ScholarPubMed
Van Borsel, J., Sunaert, R., & Engelen, S. (2005). Speech disruption under delayed auditory feedback in multilingual speakers. Journal of Fluency Disorders, 30, 201–217.CrossRefGoogle ScholarPubMed
Wang, H., & van Heuven, V. J. (2006). Acoustic analysis of English vowels produced by Chinese, Dutch and American speakers, Linguistics in Netherlands, 23, 237–248.CrossRefGoogle Scholar
Wiese, R. (1984). Language production in foreign and native languages: Same or different? In Dechert, H. W., Möhle, D., & Raupach, M. (Eds.), Second Language Productions (pp. 11–25). Gunter Narr Verlag.Google Scholar
Woumans, E., Santens, P., Sieben, A., Versijpt, J., Stevens, M., & Duyck, W. (2015). Bilingualism delays clinical manifestation of Alzheimer’s disease. Bilingualism: Language and Cognition, 18, 568–574.CrossRefGoogle Scholar
Zhang, Q., & Yang, Y. (2003). The determiners of picture-naming latency. Acta Psychologica Sinica, 35, 447–454.Google Scholar
Zheng, Z. Z., Munhall, K. G., & Johnsrude, I. S. (2010). Functional overlap between regions involved in speech perception and in monitoring one’s own voice during speech production. Journal of Cognitive Neuroscience, 22, 1770–1781.CrossRefGoogle ScholarPubMed