Introduction
English speakers say that things go together like bread and butter, not like butter and bread. Lexical patterns like these (often referred to as formulaic language or multiword sequences) account for up to half spoken discourse (Erman & Warren, Reference Erman and Warren2000; Pawley & Syder, Reference Pawley, Syder, Richards and Schmidt1983). One example of such lexical patterns is binomials, defined as “coordinated word pairs whose lexical elements share the same word class” (Mollin, Reference Mollin2014, p. 1). Binomials abound in English (e.g., aches and pains, fair and square, high and low, life and death) and vary in their degree of reversibility along a cline (Malkiel, Reference Malkiel1959). At one end of the cline are frozen binomials, which are very clearly irreversible (e.g., hit and run, chalk and cheese) because of their highly idiomatic meaning. However, the focus of the present study is not on such idiomatic binomials, but rather on semantically compositional (i.e., transparent) binomials. Transparent binomials exist along a continuum of fixedness, where components may have a preferred sequence even when the order could in theory be reversed without fundamentally changing the meaning (e.g., public and private, mother and father). Therefore, binomials have two important properties: co-occurrence restrictions (like other lexical patterns) and configuration restrictions, and this unique nature has led to interest in how they are processed and acquired.
One line of research has explored the factors that determine the preferred order of binomials, particularly in terms of diachronic changes (e.g., Goldberg & Lee, Reference Goldberg and Lee2021; Mollin, Reference Mollin2014). Goldberg and Lee (Reference Goldberg and Lee2021) found that binomials like uncles and aunts/nephews and nieces, which were in common use prior to the 1930s, have more recently reversed their preferred order to aunts and uncles/nieces and nephews. Goldberg and Lee proposed several cognitive explanations for the change, including the accessibility of the individual components of binomials in memory and their cluster strength. Others have attributed the ordering preferences in binomials to sociocultural factors (e.g., Mollin, Reference Mollin2013) and factors such as the semantic, phonological and lexical properties of component words, as well as how much experience an individual has with the binomial in question (Morgan & Levy, Reference Morgan and Levy2016).
Another research area is concerned with how binomials are acquired by native and nonnative speakers. Some studies have addressed this question using post-treatment tasks (Alotaibi et al., Reference Alotaibi, Pellicer-Sánchez and Conklin2022) or using eye-tracking to examine the processing of novel binomials as it unfolds in real time (Alotaibi, Reference Alotaibi2020; Conklin & Carrol, Reference Conklin and Carrol2021). The current study aims to contribute to research on the processing of novel binomials (i.e., infrequent phrases which do not have a conventionalized word order) by extending the work of Conklin and Carrol (Reference Conklin and Carrol2021), who examined the processing of novel binomials in native speakers, to a population of nonnative speakers of English. The study investigates whether non-native speakers show a processing sensitivity (i.e., speeded recognition) to novel binomials in a natural reading context, as was the case with native speakers in the original study. More specifically, we examine whether nonnative speakers can associate pairs of words in memory and register their preferred word order (e.g., wires and pipes instead of pipes and wires) over the course of reading short texts seeded with the novel binomials. Thus, the current study examines L2 processing of novel binomials rather than their acquisition (i.e., there is no post-treatment measure of knowledge).
The following sections will review two relevant strands of literature to situate the current study: the processing of lexical patterns by native and nonnative speakers and the acquisition of single-word vocabulary and lexical patterns by nonnative speakers.
Literature review
Processing of lexical patterns
It is well-established that native speakers recognize lexical patterns faster and process their phrase-level meaning more easily than other nonrecurrent combinations of words that do not show any significant degree of cohesion or fixedness. This has been shown for idioms (e.g., Carrol & Conklin, Reference Carrol and Conklin2020; Conklin & Schmitt, Reference Conklin and Schmitt2008; Libben & Titone, Reference Libben and Titone2008; Rommers et al., Reference Rommers, Dijkstra and Bastiaansen2013), phrasal verbs (e.g., Blais & Gonnerman, Reference Blais and Gonnerman2013; Matlock & Heredia, Reference Matlock, Heredia, Herediaand and Altarriba2002; Tiv et al., Reference Tiv, Gonnerman, Whitford, Friesen, Jared and Titone2019) and binomials (e.g., Arcara et al., Reference Arcara, Lacaita, Mattaloni, Passarini, Mondini, Beninca and Semenza2012; Carrol & Conklin, Reference Carrol and Conklin2020). The processing advantage for lexical patterns by native speakers has prompted researchers to explore these patterns in nonnative speakers.
Several studies have explored the determinants of nonnative processing of lexical patterns. One important factor is first language (L1)–second language (L2) similarity. When single words are the same (or very similar) across languages (e.g., piano in both English and French), there is considerable evidence of cross-language activation, or a cognate effect, in nonnative lexical processing (for an overview, see van Hell & Tanner, Reference van Hell and Tanner2012). More recently, researchers have started to examine the effect of L1-L2 similarity on the processing of lexical patterns. This has been referred to as the congruency effect (i.e., the availability of a literal translation equivalent). Studies have examined the congruency effect using different types of lexical patterns including collocations (e.g., Wolter & Gyllstad, Reference Wolter and Gyllstad2011, Reference Wolter and Gyllstad2013; Yamashita & Jiang, Reference Yamashita and Jiang2010) and idioms (e.g., Carrol & Conklin, Reference Carrol and Conklin2014; Carrol et al., Reference Carrol, Conklin and Gyllstad2016; Irujo, Reference Irujo1986; Pritchett et al., Reference Pritchett, Vaid and Tosun2016; Titone et al., Reference Titone, Columbus, Whitford, Mercier, Libben, Heredia and Cieślicka2015). The general finding is that congruency has a clear influence on the processing of lexical patterns in the L2 with an advantage for congruent items (L1 = L2) over incongruent items (L1 ≠ L2).Footnote 1
Another important determinant of the processing of lexical patterns is transparency. One relevant study here is by Gyllstad and Wolter (Reference Gyllstad and Wolter2016), who employed a semantic judgment task to examine how advanced L1 Swedish–L2 English learners processed English free combinations and collocations. Reaction times and error rates showed a processing cost for collocations compared to free combinations, due to the semantically semitransparent nature of collocations. In the same vein, Yamashita (Reference Yamashita2018) explored the potential contribution of semantic transparency in explaining the congruency effect, and found that congruent items were dominated by transparency while incongruent items were generally characterized by opacity, indicating a clear overlap between these variables.
A third determinant of the processing of lexical patterns is frequency. A study by Sonbul (Reference Sonbul2015) explored the sensitivity of native and nonnative speakers of English to the corpus-derived frequency of collocations using both off-line (typicality rating task) and online (eye movements) measures. There was a clear sensitivity to corpus-derived frequency among both natives and nonnatives in the off-line task. The frequency effect was also notable in the early stages of reading but disappeared later for both groups. Wolter and Gyllstad (Reference Wolter and Gyllstad2013) examined the influence of frequency on the processing of congruent and incongruent collocations. They found that advanced Swedish learners of English were highly sensitive to the frequency of collocations, regardless of whether or not the collocations had a congruent form in the L1. Likewise, Wolter and Yamashita (Reference Wolter and Yamashita2018) used an acceptability judgment task to examine the processing of collocations by intermediate and advanced Japanese speakers of English and native English speakers. They found effects of both word-level and collocation-level frequency among the three groups of participants. Such results support usage-based models of language acquisition, whereby experience with the language predicts language processing and acquisition (e.g., Bybee, Reference Bybee2006; Ellis, Reference Ellis2002).
In addition, L2 proficiency seems to influence the processing of lexical patterns. For example, Sonbul (Reference Sonbul2015) found that the effect of corpus-derived frequency on the processing of collocations was greater among nonnative speakers of English as their proficiency increased. Similarly, Ding and Reynolds (Reference Ding and Reynolds2019) found that the influence of congruency was clearer among highly proficient than less proficient Chinese EFL (English as a foreign language) learners. Sonbul and El-Dakhs (Reference Sonbul and El-Dakhs2020) showed that the estimated proficiency of Arab learners of English influenced the processing of collocations, with congruency effects slowly diminishing as proficiency increased (and see similar effects of increasing proficiency from Yamashita & Jiang, Reference Yamashita and Jiang2010).
The research reviewed thus far has mainly focused on the processing of collocations and idioms. To the best of our knowledge, only one study has examined the processing of binomials by nonnatives. Siyanova-Chanturia et al. (Reference Siyanova-Chanturia, Conklin and van Heuven2011) employed eye-tracking to examine how native and non-native English speakers, of varied levels of proficiency, processed binomials that differed in phrasal frequency. The participants read sentences containing binomials in their preferred, frequent, order (bride and groom) or their reversed, less frequent, form (groom and bride). The results showed that both natives and nonnatives were generally sensitive to the frequency of occurrence of binomials, but only natives and higher proficiency nonnatives also exhibited a sensitivity to the canonical configuration.
Incidental acquisition of L2 vocabulary
Most of the available evidence on the incidental acquisition of L2 vocabulary from context has focused on single words (e.g., Pellicer-Sánchez & Schmitt, Reference Pellicer-Sánchez and Schmitt2010; Pitts et al., Reference Pitts, White and Krashen1989; Waring & Takaki, Reference Waring and Takaki2003). These studies mainly relied on off-line tests to assess gains and showed that L2 speakers do retain vocabulary from exposure, but the rate might be relatively low. Only recently have studies used eye-tracking, which allows for the examination of on-line processing as it unfolds in real time. Among the earliest eye-tracking studies to examine the incidental acquisition of word knowledge is Godfroid et al. (Reference Godfroid, Boers and Housen2013). In their study, Godfroid et al. (Reference Godfroid, Boers and Housen2013) had advanced Dutch-speaking learners of English read short English extracts that contained target known words and unknown pseudowords. Their results showed that participants spent more time processing the unknown pseudowords than the known words, and that longer fixations to the pseudowords were associated with better scores in an unannounced vocabulary posttest. Similarly, Pellicer-Sánchez (Reference Pellicer-Sánchez2016) combined off-line (paper-and-pencil) and online (eye-tracking) measures to examine the incidental acquisition of unknown words by nonnative English learners. Notably, the participants read a full story, not simply short extracts. The story contained pseudowords, each repeated eight times. The reading time (RT) for pseudowords decreased significantly after three to four encounters, and pseudowords were read in a similar manner to known real words after eight encounters. The paper-and-pencil tests showed that incidental acquisition of unknown words is possible. However, the acquisition of word meaning lagged behind the acquisition of word form.
Godfroid et al. (Reference Godfroid, Ahn, Choi, Ballard, Cui, Johnston, Lee, Sarkar and Yoon2018) conducted a study that involved participants reading an authentic novel. The participants (native and nonnative English speakers) read five chapters of an English novel that included foreign Dari (Farsi) words ranging in frequency (1–23 occurrences). After reading, the participants were given a comprehension test and surprise vocabulary tests. Using growth curve analysis to model form knowledge development, the results showed that both the quantity (number of exposures) and the quality (total RT) of lexical processing facilitated incidental vocabulary acquisition. The results showed a nonlinear, S-shaped pattern of RTs for newly acquired words with an initial speed up (one to four exposures) followed by a plateau and a slight increasing trend (7 to 10 exposures) before further decreases in RTs (11 to 23 exposures). Posttest scores suggested that the frequency of occurrence of the new words and how long the participants read them at each encounter predicted the acquisition of knowledge.
The research reviewed in this section thus far has focused on the acquisition of individual words. In contrast to single words, lexical patterns are often claimed to be less noticed by nonnatives (e.g., Boers & Lindstromberg, Reference Boers and Lindstromberg2012; Christiansen & Arnon, Reference Christiansen and Arnon2017; Wray, Reference Wray2000). However, some studies have suggested that nonnative speakers are able to notice lexical patterns in context. For example, Durrant and Schmitt (Reference Durrant and Schmitt2010) assigned nonnative speakers of English to one of three training conditions: single exposure, verbatim repetition, and varied repetition, followed by a naming task. Participants in both repetition conditions recalled the target collocations better than those in the single exposure condition. The authors concluded that adult nonnative learners retain information about the lexical patterns they are exposed to in input, in line with usage-based models of language acquisition (Bybee, Reference Bybee2006).
Evidence for the incidental acquisition of lexical patterns has accrued over the past few years. Some studies have focused on incidental vocabulary acquisition from television/video viewing (e.g., Majuddin et al., Reference Majuddin, Siyanova-Chanturia and Boers2021; Puimѐge & Peters, Reference Puimège and Peters2020). More relevant to the present study is research investigating the incidental acquisition of lexical patterns from a reading context (Pellicer-Sánchez, Reference Pellicer-Sánchez2017; Webb et al., Reference Webb, Newton and Chang2013). Webb et al. (Reference Webb, Newton and Chang2013) investigated the effect of repetition on the incidental acquisition of collocations. Taiwanese EFL learners simultaneously read and listened to one of four versions of a modified graded reader that included target collocations, with 1, 5, 10, and 15 encounters. Immediate-posttest results indicated that encountering collocations repeatedly when reading while listening contributed to incidental acquisition of form and meaning, with collocation acquisition increasing as a factor of frequency. Pellicer-Sánchez (Reference Pellicer-Sánchez2017) examined the incidental acquisition of collocational knowledge, focusing on adjective-pseudoword collocations in reading. L2 learners read a story seeded with target collocations that were repeated either four or eight times. The scores on a 1-week delayed posttest lent support to the benefits of collocation acquisition from a reading context, even suggesting that ESL learners can incidentally develop collocational knowledge at a similar rate to individual words. However, there was not a significant effect of the frequency manipulation on collocation acquisition.
While most studies have focused on collocations, very few studies have examined the incidental acquisition of binomials. In one such study, Alotaibi et al. (Reference Alotaibi, Pellicer-Sánchez and Conklin2022) investigated the effect of input mode (reading-only, listening-only, and reading-while-listening) and frequency of exposure (two, four, five, and six occurrences) on declarative binomial knowledge. Based on performance on immediate paper-and-pencil tests, results indicated that it was possible for nonnative learners of English (L1 Arabic) to develop declarative knowledge of the preferred order of binomials from the various input modes; novel binomials encountered six times showed similar familiarity ratings as existing binomials.
While previous studies indicate that lexical patterns (including binomials) can be acquired incidentally in L2 speakers, they are limited in that they used post-hoc measures of knowledge and did not examine processing as it unfolds in real time. Only a few studies have employed eye-tracking to examine the online processing of lexical patterns (Alotaibi, Reference Alotaibi2020, Study 2; Choi, Reference Choi2017). The aim of Alotaibi’s (Reference Alotaibi2020) Study 2, for example, was to examine how nonnative learners process novel binomials in different input modes. The findings showed that repeated exposure to novel binomials led to fewer fixations and shorter RTs. Additionally, with increased exposure, the processing of novel binomials gradually became comparable to existing ones. It should be noted that since the focus of Alotaibi’s (Reference Alotaibi2020, Study 2) was on mode of processing rather than acquisition per se, she did not include a reversed form of the novel binomials. Including a reversed form can help address the special nature of binomials (see “Introduction”) that does not merely involve co-occurrence restrictions but also entails word order preferences.
The present study
As noted earlier, the current study is an extension of Conklin and Carrol (Reference Conklin and Carrol2021), who investigated the processing of novel binomials amongst native speakers, exploring sensitivity to co-occurrence information and canonical word order. They monitored the eye movements of 40 native English speakers while reading short stories that contained existing binomials in their common forward form (e.g., time and money), seen once, and novel binomials (e.g., wires and pipes), seen one to five times in their experimentally defined forward form. Then, the readers saw the existing and novel patterns in the reversed order (e.g., money and time, pipes and wires). The results revealed an initial co-occurrence memory effect for the components of novel binomials (i.e., “wires” and “pipes”) regardless of direction whereby the last “reversed” form was processed similar to or even significantly faster than the first forward occurrence. However, when frequency of encounter was considered, an advantage emerged for forward novel patterns over subsequently encountered reversed forms after four to five exposures, suggesting that natives could develop a sensitivity to the order of novel binomials rapidly from exposure.
The current study aims to examine whether the effect found for natives by Conklin and Carrol (Reference Conklin and Carrol2021) emerges for nonnative speakers. More specifically, we are interested in whether nonnative speakers rapidly develop a sensitivity to the preferred word order of novel binomials through natural reading. The current study addresses the following questions:
-
1. Is the language processing system sensitive to novel binomials in L2 input that simulates a real-world context?
-
2. What is the effect of frequency of exposure on nonnatives’ sensitivity to novel binomials in a real-world context?
Similar to Conklin and Carrol (Reference Conklin and Carrol2021), the current study presented existing binomials only once in their forward form followed by once in their reversed form. However, unlike Conklin and Carrol (Reference Conklin and Carrol2021) who included five frequency levels for novel binomials, the design of the present study included two frequency categories only (2-repetition vs. 4-repetition) to increase item power. In Conklin and Carrol (Reference Conklin and Carrol2021) as the number of repetitions increased, the number of items per frequency level decreased. For example, 25 items were read once, but only five items were read five times. Thus, there was much less item power at the fifth occurrence versus the first. By only looking at two frequency levels, we were able to include the same number of items for both categories.Footnote 2 We selected two versus four repetitions for our frequency categories based on Conklin and Carrol’s (Reference Conklin and Carrol2021) finding that natives showed a clear sensitivity to a given configuration after four exposures but not after two exposures. Thus, the novel binomials in the current study involved two main factors. The first factor was Category with two levels: 2-repetition versus 4-repetition, and the second factor was Iteration with three levels: first, last (i.e., second occurrence for 2-repetition items and fourth occurrence for 4-repetition items), and reversed.
To evaluate an emerging sensitivity to novel binomials (RQ1), we will compare RTs of existing and novel binomials in the forward (first) and reversed iterations only. When examining the effect of frequency on sensitivity to novel binomials (RQ2), we will include the first, last, and reversed forms of both frequency categories.
To establish any similarities or differences between native and nonnative processing of novel binomials, the current findings from nonnative speakers will be discussed in relation to those of the native speakers from Conklin and Carrol (Reference Conklin and Carrol2021). It is important to point out, that while we have introduced certain changes to the design (see preceding text), the experiments are very similar (i.e., use the same items and the same texts with minor modifications). Thus, comparing the findings of the current study to that of Conklin and Carrol (Reference Conklin and Carrol2021) should be justified.Footnote 3
We do not expect nonnatives in the present study to necessarily have a sensitivity to the canonical order of existing binomials across all proficiency levels. Similar to Siyanova-Chanturia et al. (Reference Siyanova-Chanturia, Conklin and van Heuven2011), we anticipate that sensitivity to binomials’ word order should only emerge for participants with high L2 proficiency. For the novel binomials, based on Alotaibi (Reference Alotaibi2020, Study 2), we expect our nonnatives to show online memory effects for the co-occurrence of novel binomials’ components, that is, shorter RTs for the last over the first encounter in the forward form. However, nonnatives might or might not develop sensitivity to the canonical order of novel binomials (wires and pipes vs. pipes and wires). If nonnatives follow the same pattern as natives (Conklin & Carrol, Reference Conklin and Carrol2021), they should show a processing advantage (i.e., shorter RTs) for novel binomials in the forward form over the backward form after four exposures, but likely not after as few as two encounters. If, however, nonnative speakers are not sensitive to word order differences in lexical patterns, our nonnative participants might not show such a processing advantage even after four exposures. Rather, they might treat the backward form as another co-occurrence of the components and overlook the direction preference. In that case, the final backward occurrence should demonstrate an additional processing advantage over the last encounter with the forward form.
Experiment
Methods
Participants
Initially, 40 participants took part in the experiment. One participant was excluded as her score in the V_YesNo vocabulary test was below 4,000 word families, suggesting that she might not know all words comprising the target binomials (see the following text for more details).
The final pool of 39 participants were all nonnative speakers of English who were academic and administrative staff at a university in Saudi Arabia (L1 Arabic; 30 females, average age = 34.39, SD = 10.61).Footnote 4 They started learning English at an average age of 6.5 years (M = 6.46; SD = 5.19). Their self-reported proficiency scores (on a scale from 1 = very poor to 5 = excellent) were: reading M = 4.63, SD = 0.55; writing M = 4.60, SD = 0.55; speaking M = 4.63, SD = 0.60; and listening M = 4.57, SD = 0.61.
To obtain a rough estimate of their proficiency in English, the participants completed the V_YesNo online vocabulary test (Meara & Miralpeix, Reference Meara and Miralpeix2017; maximum score = 10,000). The test presents participants with 200 items (half real words and half imaginary pseudowords) and instructs them to press “Yes” if they know the meaning of the presented form and “Next” if they do not. The score is adjusted downward based on guessing (i.e., pressing “Yes” for pseudowords) using an equation (see Meara & Miralpeix, Reference Meara and Miralpeix2017, p. 120). Uchihara and Clenton (Reference Uchihara and Clenton2020) found a significant association between the V_YesNo test scores and speaking ability. Moreover, previous versions of the Yes/No vocabulary test format demonstrated a medium to strong correlation with proficiency measures (e.g., Meara & Jones, Reference Meara, Jones and Grunwell1988; Miralpeix & Muñoz, Reference Miralpeix and Muñoz2018). The scores of our participants in the V_YesNo test ranged between 4,000 and 9,302 (M = 6712.05, SD = 1323.96), which roughly indicates a good to high level of proficiency (Meara & Miralpeix, Reference Meara and Miralpeix2017). The vocabulary test score was added as a covariate in all mixed-effects models (see “Analysis”) to control for the effect of proficiency on RTs.
Materials
Thirty-two “noun-and-noun” binomials were selected for the present study, taken from Conklin and Carrol (Reference Conklin and Carrol2021), to represent two categories of binomials: existing (n = 12) and novel (n = 20). The full list of 32 items and their features is presented in Appendix S1 (Online Supplementary Materials). All constituent words belonged to the most frequent 4,000 word families in English (BNC/COCA List with 25 1,000-word bands; Nation, 2012), meaning that our participants should be familiar with them.
The existing binomials were highly frequent phrases (BNC frequency per million) and had a conventionalized order: forward M = 351.58, SD = 305.74; reversed M = 23.25, SD = 27.00; t(11) = 3.71, 95% CI [144.59, 512.08], p < .001, d = 1.51. The novel binomials were infrequent phrases (1–11 occurrences in the BNC) constructed using two common nouns (most frequent 4,000 word families). They did not have a typical configuration (forward M = 3.90, SD = 2.75; reversed M = 3.35, SD = 2.91; t(19) = 0.82, 95% CI [−0.86, 1.96], p = .43, d = 0.19). More details on item selection and categorization can be found in Conklin and Carrol (Reference Conklin and Carrol2021). Appendix S2 (Online Supplementary Materials) presents characteristics of the target stimuli.
Because participants in the present study were nonnatives from the same L1 background (Arabic), we also considered L1-L2 congruency of the existing and novel binomials. We operationalized congruency in two steps: existence (exists as a common binomial in both language vs. only exists in one language) and configuration or direction (same in both languages vs. different in the two languages). The two steps are explained in detail in Appendix S3 (Online Supplementary Materials). Based on this operationalization, 18 out of the 20 novel English binomials were not common in Arabic (two existed in Arabic in the opposite direction) but only 6 out of the 12 existing English binomials had the same direction in Arabic (six had a different direction in Arabic). To check whether the pattern of results reported in the following text (see “Results”) was influenced by existence and direction in Arabic, we fit all models with and without the eight nonmatching items. The pattern of results remained the same.Footnote 5
We also considered how familiar our participants are likely to be with the English form of the binomials. To test this, a familiarity rating task was administered to 23 L1 Arabic–L2 English speakers who were comparable to our main participant pool.Footnote 6 They were instructed to rate both existing and novel binomials (intermixed in a list) for familiarity on a scale from 1 = very unfamiliar to 7 = very familiar. The results showed significantly higher familiarity ratings for existing binomials (M = 6.51, SD = 0.79) than novel binomials (M = 4.86, SD = 0.84; t (30) = 5.52, 95% CI [1.05, 2.27], p < .001, d = 2.03). To examine the potential effect of familiarity on the pattern of results, we fit all models (see “Results)” with the average familiarity rating score as a covariate and found no significant effect of rating scores. More importantly, including familiarity as a covariate in the analysis did not alter the pattern of results. It should be noted that the novel binomials were rated toward the middle of the scale. We will return to this point in “Discussion.”
Three stories of approximately 1,100 words each were adapted from Conklin and Carrol (Reference Conklin and Carrol2021) to include the 32 target items. The passages were simplified to ensure suitability for our nonnative participants (99% of words belonged to the most frequent 4,000 word families in English). All target “existing” and “novel” binomials were presented once in the forward form and once in the reversed form. Half the novel binomials (n = 10) were then presented one more time in the forward form to make a total of two exposures and the other half (n = 10) were presented three more times in the forward form to make a total of four exposures. The reversed form for both existing and novel binomials occurred once after all occurrences of the corresponding forward form. Conklin and Carrol (Reference Conklin and Carrol2021) conducted a predictability norming task with native speakers of English. We included their predictability scores as a potential covariate in the analysis. The passages, full data, and R codes are available through Open Science Framework at https://osf.io/kymsp/?view_only=ec6b2f7e9ac74f15be7afcd864373f07.
It is important to note here that the novel binomials in Conklin and Carrol (Reference Conklin and Carrol2021) were counterbalanced across two lists, such that one order (wires and pipes) was the forward direction on one list and the other (pipes and wires) was the forward direction on the other list. This was done to account for any inherent word order preferences in the novel items. However, no list differences were found, thus, in the current study, items appeared in a single list with one version designated as the forward version.
Procedure
Upon arrival at the lab, the participant signed a consent form and completed the vocabulary test, then the eye-tracking experiment started. Eye movements were recorded monocularly using an SR Research EyeLink 1000+ eye-tracker. A desk-mounted chinrest was used to minimize head movement. A 9-point grid calibration procedure was conducted before the experiment and each screen was preceded by a fixation point for drift correction. The eye-tracker was recalibrated before each story and whenever needed. The stories were presented in Courier New, 18-point font and were double-spaced. Participants were told to read the stories as naturally as possible for comprehension and to press the space bar to go to the next screen. In the texts, neither existing nor novel binomials appeared at the beginning or end of a line or across a line break. Each story was followed by five comprehension questions to ensure that participants attended to the text (average percentile score: 94.36%, SD = 6.41). Performance on the comprehension questions indicates that participants understood the stories.
Analysis
Data cleaning was done prior to the analysis according to the four-stage process in the DataViewer software. Single fixations shorter than 100 ms and longer than 800 ms were removed (5% of all fixations). Then, following Conklin and Carrol (Reference Conklin and Carrol2021), we excluded trials that were discontinued or where track loss was experienced. Any phrase that was completely skipped was also excluded from the analysis. In such cases, all subsequent occurrences of the item, including the reversed form, were also removed. This resulted in a loss of 1% of the data points in both analyses. We also conducted the analysis with the full data set and the resulting models were the same.
The analysis was conducted with R version 4.0.5. (R Core Team, 2021). Linear mixed-effects models were constructed and analyzed using the lme4 (version 1.1-26; Bates et al., Reference Bates, Mächler, Bolker and Walker2015) and lmerTest (version 3.1-3; Kuznetsova et al., Reference Kuznetsova, Brockhoff and Christensen2017) packages. Three interest areas were analysed for each model: the whole phrase, word 1, and word 3. The middle word “and” was skipped more than 45% of the time, so it was excluded from the analysis. Separate models were constructed for two eye-movement measures: first-pass RT and total RT. RTs were log-transformed to reduce skewness in the data. All analyses adopted the maximal random effects structure justified by the design (see the following text for more details). Final models were checked for collinearity, and no issues were observed (all VIFs < 7). Words and phrases that received no fixations during first-pass reading were excluded from subsequent analyses.
We conducted two separate analyses to answer the research questions. First, we compared the RTs of forward and reversed forms for both the existing and novel binomials (RQ1). Then, RTs for novel binomials only were compared across three iterations (first, last, reversed) for both the 2-repetition and 4-repetition binomial categories (RQ2).
Results
Table 1 presents mean RTs for both existing and novel binomials for all occurrences in both directions. Existing binomials showed no clear pattern, with slower RTs for the forward form under some measures and for the reversed form under other measures. For novel binomials, however, the pattern was much clearer. RTs were shorter (i.e., faster processing) with more exposure to target items (first vs. last vs. reversed) for both repetition categories (2 vs. 4). It should be noted that for the 4-repetition novel binomials, occurrences in the middle (second, third, last) did not always exhibit this processing advantage as exposure increased; the last exposure for 4-repetition items had a longer average RT than the third exposure under all measures. We will return to this in “Discussion.”
RQ1: Is the language processing system sensitive to novel binomials in L2 input that simulates a real-world context?
Table 1. Mean RTs in milliseconds with standard deviation in parentheses for existing binomials and novel binomials for first-pass and total RT
Notes: For word 1 and word 3 values, words that received no fixations are discounted; values for the whole phrase include trials where either word 1 or word 3 (but not both) was skipped.
To compare the mean RTs (first-pass and total) of existing and novel binomials, we constructed linear mixed-effects models for the whole phrase, word 1 and word 3. Following Conklin and Carrol (Reference Conklin and Carrol2021), we included Type (existing vs. novel) and Direction (forward vs. reverse) as fixed effects in all models. Word-level factors including length (in letters) and frequency (on the Zipf scale) were also included as fixed factors.Footnote 7 Additionally, number of repetitions (1, 2, 4) was added as a covariate in all models to control for the effect of variation in encounters on the subsequent reverse form of existing and novel binomials. All models included random intercepts for subjects and items. The optimx optimizer was used to avoid convergence errors when needed. Initially, by-subject random slopes for the effects of Type and Direction were included,Footnote 8 but this led to unavoidable convergence errors. As a result, only by-subject random slopes for Type were included in all models. Three other factors were then added stepwise one-by-one to each original model to examine their potential effect on RTs: phrase frequency on the Zipf scale, forward association strength, and cloze probability. Log-likelihood (X2) tests and AIC values were used to compare the resulting model with the original model, and only factors that significantly improved the model fit were kept. All models also included the log-transformed vocabulary test score as a proxy for L2 proficiency.
All resulting models are presented in Table 2. Vocabulary Score was a significant predictor in all models but not for the word 3 first-pass measure, pointing to shorter RTs as proficiency increased. The Type × Direction interaction was significant for all first-pass and total reading measures except for the word 1 first-pass analysis. The difflsmeans function in the lmerTest package was used to compute pairwise comparisons (see Appendix S4, Online Supplementary Materials). We calculated Cohen’s d for pairwise comparisons as a standardized measure of effect size based on the guidelines provided by Brysbaert and Stevens (Reference Brysbaert and Stevens2018). Results suggest that nonnatives were not sensitive to the established configuration of frequent existing binomials in English. There were no significant differences between the processing of forward and reversed forms for existing binomials (whole phrase) in first-pass RT (β = −0.01, t = −0.24, 95% CI [−0.08, 0.06], p = .81, d = −0.01) or total RT (β = −0.01, t = −0.39, 95% CI [−0.06, 0.04], p = .69, d = −0.02). These findings stand in sharp contrast with those reported in Conklin and Carrol (Reference Conklin and Carrol2021) for native speakers who showed a clear processing advantage for the forward form of existing binomials over their reversed forms. It seems that our nonnative speakers are not sensitive to the established word order of common binomials in English.
Table 2. Linear mixed-effects model for existing versus novel binomials in forward and reversed forms for whole phrase RTs, word 1 and word 3
Note: p-values are estimated using the lmerTest package in R (Kuznetsova et al. Reference Kuznetsova, Brockhoff and Christensen2017).
*p <0.05; **p <0.01; ***p <.001.
For the novel binomials, pairwise comparisons suggest significantly shorter RTs for the reversed forms in comparison to the forward forms for all measures with a small to medium effect. This is most clearly seen for the whole phrase, for both first-pass RT (β = 0.15, t = 5.11, 95% CI [0.09, 0.21], p < .001, d = 0.25) and total RT (β = 0.28, t = 13.95, 95% CI [0.24, 0.31], p < .001, d = 0.62). This result is similar to (but more robust than) the findings reported in Conklin and Carrol (Reference Conklin and Carrol2021) for native speakers, suggesting that just like natives, nonnatives seem to quickly develop a link between the two words in memory, exhibiting an advantage in processing, even when they appear in a different order than previous encounters.
As a final step in the analysis, we also tested for the contribution of the Type × Direction × Vocabulary Score interaction to the model fit.Footnote 9 This was intended to reveal any modulating effect of L2 proficiency, that is, to examine if nonnatives with higher vocabulary scores read existing and novel binomials in the forward and reverse direction differently. This three-way interaction did not significantly contribute to any of the models, suggesting similar patterns regardless of L2 proficiency.
One final notable finding in Table 2 is that number of repetitions was not significant in all models, suggesting no difference between items that were seen twice and those that were seen four times. The next research question aimed to examine the possibility that number of encounters might modulate this effect in more detail.
RQ2: What is the effect of frequency of exposure on nonnatives’ sensitivity to novel binomials in a real-world context?
The effect of frequency of exposure on RTs of novel binomials (word 1, word 3 and whole phrase) was explored in Analysis 2. This analysis included Category (2-repetition vs. 4-repetition) and Iteration (first, last, reversed) as fixed effects and by-subject random slopes for Category. All other fixed factors and covariates were the same as those included in Analysis 1. Additionally, we examined the three-way interaction between Category, Iteration, and Vocabulary Score, but it did not significantly improve any of the models.
The resulting models are presented in Table 3. Overall, there appears to be a main effect for Vocabulary Score (but see the total RT measure for Word 3) and a main effect for Iteration, but not Category (but see first-pass RT for the whole phrase). Pairwise comparisons across the three Iteration levels, regardless of repetition, are presented in Appendix S5 (see Online Supplementary Materials). These are computed using the difflsmeans function in the lmerTest package.
Table 3. Linear mixed–effects model for novel binomials in different iterations (first, last and reversed) for whole phrase RTs, word 1 and word 3
Note: p–values are estimated using the lmerTest package in R (Kuznetsova et al. Reference Kuznetsova, Brockhoff and Christensen2017).
*p <0.05; **p <0.01; ***p <.001.
The general pattern seems to suggest significantly different RTs across the three iterations with a small to medium effect (first > last > reversed). Thus, it appears that with more exposure to binomials, nonnatives developed a sensitivity to the co-occurrence of the content words, spending less time reading them each time they appeared together. As for the reversed form, which was always included after all occurrences of the forward form, the results suggest that nonnatives dealt with it as another exposure to the binomial, ignoring the configuration mismatch.
Discussion
Research examining how nonnative speakers process novel lexical patterns in context is fairly limited. The present study aims to fill this gap by recording the eye movement patterns of nonnative speakers of English (L1 Arabic) as they read stories seeded with novel binomials to address two research questions. First, we examined whether the nonnatives developed a sensitivity to the canonical order of novel binomials after exposure and compared their processing to existing binomials (Research Question 1). Second, we looked at the effect of frequency (two vs. four exposures) on the development of sensitivity to the novel binomials (Research Question 2).
In response to the first research question, results showed no processing advantage for existing, common, binomials (time and money) over their less frequent reversed forms (money and time). Thus, unlike natives in Conklin and Carrol (Reference Conklin and Carrol2021), nonnatives in the present study were generally not sensitive to binomials’ canonical word order. This result seems to support Siyanova-Chanturia et al.’s (Reference Siyanova-Chanturia, Conklin and van Heuven2011) finding for a limited nonnative sensitivity to word order preferences that emerged only as proficiency increased. In the present study, however, proficiency did not seem to modulate sensitivity to binomials’ configuration. While participants in Siyanova-Chanturia et al. (Reference Siyanova-Chanturia, Conklin and van Heuven2011) came from a variety of L1 backgrounds, we targeted a homogenous nonnative population (L1 Arabic–L2 English). Previous research on collocations and idioms often report congruency as an important factor in the nonnative processing of lexical patterns (e.g., Carrol et al., Reference Carrol, Conklin and Gyllstad2016; Sonbul & El-Dakhs, Reference Sonbul and El-Dakhs2020). A follow-up analysis that was conducted on a subset of binomials that matched in the two languages showed the same pattern of results, namely, no sensitivity to the canonical configuration (see “Materials” for details). The fact that the L1-L2 matched existing binomials in the present study comprised only six items might explain the lack of effect. Future research on nonnative binomial processing should address the congruency effect more directly with a larger set of congruent and incongruent items. Another factor to consider in future research is binomial familiarity. The norming study that we conducted with a group of L1 Arabic–L2 English speakers comparable to the main participant pool (see “Materials”) showed that familiarity with novel binomials elicited ratings toward the mid-point of the scale and was only slightly (though significantly) lower than the ratings for existing binomials. More research is needed in this area to tease apart off-line familiarity ratings and online real time performance.
For novel binomials, the results of Analysis 1 (initial forward exposure vs. reversed forms) showed a robust significant advantage (with a small/medium effect) for the reversed form over the forward form for all eye-movement measures (both early and late). As indicated in the preceding text, this result complements Conklin and Carrol’s (Reference Conklin and Carrol2021) finding for natives who initially exhibited sensitivity to the combination of single words (“wires” and “pipes”) regardless of direction. Thus, like natives in Conklin and Carrol’s (Reference Conklin and Carrol2021) study, nonnatives in the present study seem to keep a record of all occurrences of lexical patterns in the input; but unlike native speakers, they might not initially build sensitivity to the preferred word order (wires and pipes vs. pipes and wires). This seems to support Durrant and Schmitt’s (Reference Durrant and Schmitt2010) finding that nonnatives are able to extract co-occurrence restrictions from input, refuting traditional claims (e.g., Wray, Reference Wray2000), and backing up usage-based models of language processing (Bybee, Reference Bybee2006; Ellis, Reference Ellis2002). However, as noted earlier, binomials may be different from other forms of lexical patterns in that they involve co-occurrence and configuration restrictions. In a study on the processing of lexical bundles, Ellis et al. (Reference Ellis, Simpson-Vlach and Maynard2008) found that nonnative speakers are sensitive just to the frequency in the language, but native speakers seem to extract the nuance of the bundle’s association strength (i.e., how often two words tend to co-occur above chance). Similarly, one might claim that the nonnative participants in the present study were able to exhibit sensitivity to mere frequency but were not sensitive to higher-level restrictions on word order, at least not after one encounter as Analysis 1 seems to suggest.
The second research question (Analysis 2) examined the possible modulating effect of frequency of encounters (two vs. four) on the development of a sensitivity to the canonical order of binomials. Conklin and Carrol (Reference Conklin and Carrol2021) found that their native speakers processed the forward forms of novel binomials faster than their reversed forms, similar to existing binomials, after four to five exposures.Footnote 10 Crucially, exposure to the subsequent reversed form led to a cost (a marked rise in processing time) compared to the most recent encounter, despite this being faster than the first exposure. However, results of Analysis 2 failed to report similar effects for our nonnative participants even after four exposures. In line with the findings of Analysis 1, nonnatives in the present study processed the last reversed form (pipes and wires) significantly more quickly (with a small/medium effect) than the experimentally defined forward form (wires and pipes) regardless of how many times it was encountered. This finding is further supported by the raw RTs in Table 1, showing similar processing times for the third encounter in the 2-repetition category (backward) and the third encounter in the 4-repetition category (forward). Thus, it seems to be the case that whilst nonnatives did register co-occurrence restrictions in terms of which words go together, they did not register the configuration/order of the words. This lack of a configuration effect stands in contrast with findings of Alotaibi et al. (Reference Alotaibi, Pellicer-Sánchez and Conklin2022) who found that nonnative Arab learners of English were able to develop sensitivity to the preferred order of binomials (similar to existing phrases) after six exposures. It should be noted, however, that unlike the present study, Alotaibi et al. (Reference Alotaibi, Pellicer-Sánchez and Conklin2022) included higher frequency levels (up to six occurrences) than the present study (with a maximum of four encounters) and employed declarative post-treatment measures. Sonbul and Schmitt (Reference Sonbul and Schmitt2013) found a dissociation between gains in declarative (paper-and-pencil) tests and those reflecting online performance. In the present study, we did not employ any post-treatment declarative measure of sensitivity to target binomials’ word order. Future research can benefit from combining both online (eye-movement) measures and off-line (paper-and-pencil) tasks to compare findings at both processing levels. Moreover, more encounters can be included to allow participants to develop sensitivity to configuration restrictions. A relevant point, relating to Arabic speakers of English, may be that Arabic seems to be less fixed than English regarding the order of binomials’ components (see “Materials”). Thus, it can be speculated that, given the flexibility in their L1, Arabic speakers of L2 English might not develop sensitivity to binomial restrictions in context. Because research on the structure of binomials in Arabic is extremely limited (but see Kaye, Reference Kaye2009), this possibility can only be viewed as a hypothesis that needs to be explored by future research comparing L2 English speakers from a variety of L1 backgrounds. Another related issue is that, in contrast to English, the Arabic script is read from right to left. As the focus of the present study is on binomials’ word order in L2 English, Arabic native speakers might be disadvantaged (in comparison to L2 English speakers whose native language is read from left to right). This would be an interesting question to explore in future research.
The fact that nonnatives in the present study developed a sensitivity to one aspect of binomials (i.e., co-occurrence restrictions) but not another (i.e., configuration restrictions) is in line with eye-tracking evidence for the incidental acquisition of single words from context. As indicated earlier, the limited available research in this area (Pellicer-Sánchez, Reference Pellicer-Sánchez2016) seems to suggest that nonnative learners tend to develop different word knowledge aspects at different rates: While the form is acquired quickly, knowledge of meaning fails to develop even after repeated exposure. Along the same lines, it can be claimed that not all aspects of binomial knowledge are learned at the same pace. Online sensitivity to the co-occurrence restrictions of binomials can develop quickly from exposure, but sensitivity to the canonical order does not develop even after several exposures. Further eye-tracking research on the processing of novel binomials can include more encounters to arrive at an estimated frequency after which nonnatives develop a sensitivity to binomials’ preferred order.
Another parallel in eye-movement patterns between individual words and binomials is the effect of multiple encounters on processing. In their study on individual words, Godfroid et al. (Reference Godfroid, Ahn, Choi, Ballard, Cui, Johnston, Lee, Sarkar and Yoon2018) found a tendency for RTs of novel words to initially decrease but then increase around the seventh encounter, reflecting “increased cognitive effort and attempts on their [participants’] part to integrate the words into the sentence contexts and make form-meaning connections” (p. 575). Similarly, our findings showed a slight increasing trend at the fourth encounter for the 4-repetition novel binomials (see Table 1) which disappeared with the presentation of the reversed form. Thus, although we did not intend to examine the gradual effect of exposure on the processing of novel binomials in the present study, a tendency seems to emerge at the fourth encounter, similar to the effect reported by Godfroid et al. (Reference Godfroid, Ahn, Choi, Ballard, Cui, Johnston, Lee, Sarkar and Yoon2018) for single words. Building on predictions of the type of processing resource allocation (TOPRA) Model (Barcroft, Reference Barcroft2002), one might argue that in the first few encounters with a given binomial, the cognitive demands are high as language users are encoding co-occurrence restrictions (i.e., learning that ‘wires’ and ‘pipes’ occur together). Then, around the fourth encounter, more cognitive resources are freed and can thus be devoted to the specific configuration and the overall meaning that is denoted by the phrase. With more exposure, the forward form of novel binomials might show further decrease in RTs. However, given the limited number of encounters in the present study (maximum four), this interpretation is speculative. Future eye-tracking research on the online processing of novel binomials would do well to include more encounters with novel binomials, in line with Godfroid et al.’s (Reference Godfroid, Ahn, Choi, Ballard, Cui, Johnston, Lee, Sarkar and Yoon2018) design, to fully explore the possible S-shaped processing of novel lexical patterns in context.
Conclusion
The present study was intended to extend Conklin and Carrol’s (Reference Conklin and Carrol2021) findings for the native processing of novel binomials to a population of non-native English speakers (L1 Arabic). Results showed that nonnatives had limited sensitivity to the preferred order of existing binomials; they did not develop sensitivity to the experimentally defined configuration of novel binomials even after four encounters. These results seem to suggest that the nonnative participants were simply recording co-occurrence restrictions, disregarding direction preference, which may be a feature of language that is less salient and therefore requires more input to emerge. The study is limited, however, in that it did not include a balanced number of congruent/incongruent binomials to fully examine the congruency effect. Additionally, the study only included two frequency conditions (two vs. four) and did not include a post-exposure measure of declarative knowledge. Future research should explore the effect of increased frequency and L1-L2 congruency on the acquisition process (both online processing and off-line, post-exposure, gains). Despite the limitations, the present study can be viewed as an initial attempt to examine nonnatives’ processing of novel binomials in context. This line of research can further our understanding of the conditions that might help nonnatives develop sensitivity to lexical patterns like bread and butter (over butter and bread), enabling new and broader explanations of input-driven language development.
Acknowledgment
The researchers thank Prince Sultan University for funding this research project through the research lab [Applied Linguistics Research Lab - RL-CH-2019/9/1].
Supplementary Materials
To view supplementary material for this article, please visit http://doi.org/10.1017/S0272263122000237.
Data Availability Statement
The experiment in this article earned Open Materials and Open Materials badges for transparent practices. The materials and data are available at https://osf.io/kymsp/?view_only=ec6b2f7e9ac74f15be7afcd864373f07.
