Motivation for replication

The proposed replication is broadly motivated by the growing interest in the use of learner corpora to understand L2 development (McEnery et al., Reference McEnery, Brezina, Gablasova and Banerjee2019; Römer & Garner, Reference Römer, Garner, O’Keeffe and McCarthy2022). Lu (Reference Lu2023: 3) succinctly summarizes four major issues in corpus-based SLA research: (a) variation in L2 use, (b) factors influencing L2 processing and production, (c) group-level developmental trajectories, and (d) variability and variation in L2 development. The original study of R&B and this replication study contribute to the understanding of the third major issue in the field. A number of studies concerning this question have revealed a move toward the L1 usage norm (Römer, Reference Römer2019; Römer & Berger, Reference Römer and Berger2019), more productive usage of individual constructions (Liu & McManus, Reference Liu and McManus2020; Römer, Reference Römer2019; Römer & Berger, Reference Römer and Berger2019; Xu, Reference Xu2016), fewer erroneous cases (Römer & Berger, Reference Römer and Berger2019; Xu, Reference Xu2016), and more lower-frequency constructions (Ellis & Ferreira-Junior, Reference Ellis and Ferreira-Junior2009; Kyle, et al., Reference Kyle, Crossley and Verspoor2021) with increasing proficiency. For example, focusing on transitive and modal VACs, Römer (2019) found the correlations for verb-VAC combinations between German learners of English and L1 English users became stronger from A1 (via A2 and B1) to B2 level. Römer and Garner’s (Reference Römer and Berger2019) analysis of L2 spoken data showed a similar developmental pattern, that is, more proficient L2 speakers were more productive in verb-VAC combinations and closer to L1 English usage than less proficient speakers. Through a contrastive corpus-based analysis of give ditransitive constructions, Xu (Reference Xu2016) showed Chinese English as a foreign language (EFL) learners’ movement from less to more semantically varied give-ditransitives as well as a decline in unidiomatic usage. As L2 learners’ repertoire of constructions expands, the token frequency of already familiar constructions may decrease with L2 proficiency, reflecting a departure from the repetitive use of such familiar constructions (Biber et al., Reference Biber, Gray and Staples2016). Hwang and Kim (Reference Hwang and Kim2023) reported that while the frequency proportion of caused-motion, ditransitive, and passive constructions in 11 constructions increased with proficiency in L2 writing, that of attributive constructions (i.e., copular be + adjective) decreased with proficiency.

In addition to advancing our understanding of group-level L2 development trajectories, this replication study is specifically motivated by the need to grapple with data sparsity in corpus-based studies (McEnery et al., Reference McEnery, Brezina, Gablasova and Banerjee2019). The small scale of most existing learner corpora has frequently led to data sparsity issues; even with recent, larger learner corpora, the problem persists when researchers consider multiple factors in selecting target subcorpora for analysis. The issue of data sparsity also manifested in R&B’s correlation analyses of EFCAMDAT1 and the BNC, with a very large group of verb-VAC combinations in the BNC absent in their learner data. Such a data distribution can compromise the accuracy of Pearson correlation analysis, which assumes homoscedasticity. While R&B addressed this issue by log-transforming those counts, there exist several limitations for log-transformation, including (a) heteroscedasticity is oftentimes not addressed even after log-transformation, (b) the primary quantity of interest (i.e., the counts) is not modelled directly, and (c) log-transformation runs into issues for count data with many zeros (O’Hara & Kotze, Reference O’Hara and Kotze2010; Winter & Bürkner, Reference Winter and Bürkner2021), necessitating additional data processing procedures to avoid logging zeros. Commonly employed techniques include adding a small constant to all values before log-transforming them (e.g., Liu, Reference Liu2021) or replacing all zeros directly with a small constant without log-transformation (e.g., Ellis et al., Reference Ellis, Römer and O’Donnell2016; Römer & Berger, Reference Römer and Berger2019). However, the variation in data processing techniques and the choice of the specific constant value (e.g., –0.1, 0.1,0.01) can affect statistical inferences (e.g., Ekwaru & Veugelers, Reference Ekwaru and Veugelers2018). It will therefore be desirable to build a more robust statistical model that can well handle count data with many zeros to triangulate the results of correlation analyses in R&B. Considering the ubiquity of zero-abundant count data in corpus-based SLA research, this replication may hold methodological implications for future corpus-based research using count data with many zeros.

The replication

This study replicates the analyses performed for RQ3 in R&B, which constituted the bulk of the analysis in the original study. We only replicated RQ3 because this was the only question that pertained to the data sparsity problem, while the remaining three research questions did not face the same challenge, as they were based solely on the learner corpus dataset without comparison to a larger L1 reference corpus. We followed the same procedures as reported in the initial study unless otherwise noted. The main changes between the initial study and this replication study are summarized in Table 1. The first main part of our replication examines how R&B’s findings regarding the L2 development trajectory with EFCAMDAT1 extend to EFCAMDAT2. Specifically, the type-token frequency distribution of a set of VACs following the pattern of “VERB PREPOSITION NOUN” and their associated verbs were derived from relevant EFCAMDAT2 subcorpora and the BNC. The results were then compared against R&B’s results to determine whether their results hold with the larger corpus. This part of analysis is useful both for verifying the generalizability of R&B’s findings and for understanding the effect of learner corpus scale on corpus-based SLA research in general.

Table 1. Comparison between the initial study and the replication study

Second, zero-inflated negative binomial (ZINB) regression analyses were conducted to examine the role of L2 input and L2 proficiency in learners’ use of verb-VAC combinations. The ZINB regression model was chosen because of its suitability for modeling count data with overdispersion and excess zeros (O’Hara & Kotze, Reference O’Hara and Kotze2010; Winter et al., Reference Winter, Perlman and Majid2018). The datasets we used for comparing L1 speakers’ and L2 learners’ usage contained a large number of zeros, as many verb-VAC combinations in the BNC were not used by L2 learners. The results of ZINB models can help reveal whether using a more robust model for handling count data with overdispersion and excess zeros modifies the results.

Research questions

This replication study explores the same research question as RQ3 in the original study of R&B (p. 1092): “Do learners’ verb-VAC associations move closer to a native usage norm as language proficiency increases?” The investigation of the present RQ1 follows the same approach adopted in the original study, while that of the present RQ2 takes a different approach to further validate the results of RQ1.

RQ1: Based on correlation analyses, do learners’ verb-VAC associations move closer to an L1 usage norm as their language proficiency increases?

RQ2: Based on ZINB models, is learners’ usage of verb-VAC associations affected by the L1 usage norm and L2 proficiency? If so, in what ways?

Identification of VACs

To retrieve instances of VACs from the corpora, we used Stanford Tregex (Levy & Andrew, Reference Levy, Andrew, Calzolari, Choukri, Gangemi, Maegaard, Mariani, Odijk and Tapias2006) to traverse the syntactic parse trees and identify instances of VACs as defined by the Tregex search patterns. The Tregex patterns used in the present study targeted verb patterns consisting of a verb that immediately dominates a specific preposition. Through an iterative trial-and-error process, the original search patterns were further modified to avoid expected noises. For example, the following pattern was used for the retrieval of “V as n” constructions, in which “!<# VBN” excluded passive constructions (e.g., “the old logo was regarded as an old fashion”), “!<< (NP << (JJ|RB < much|little|few|many))” avoided cases containing “as much/little/few/many…as” (e.g., “to eat as much peanut butter as they would”), and “&!< (NP << (NN < result)))” removed cases of “as a result” (e.g., “the impressionist movement began as a result to [of] a very formal and rigid style of paintings”).

$$ \left(\mathrm{VP}!<\#\mathrm{VBN}\right)<{\displaystyle \begin{array}{l}2\Big(\mathrm{PP}<1\left(\mathrm{IN}<\mathrm{as}\right)!<<\left(\mathrm{NP}<<\left(\mathrm{JJ}\left|\mathrm{RB}<\mathrm{much}\right|\mathrm{little}\left|\mathrm{few}\right|\mathrm{many}\right)\right)\&!\\ {}<\left(\mathrm{NP}<<\left(\mathrm{NN}<\mathrm{result}\right)\right)\Big)\end{array}} $$

All Tregex patterns used in this study are presented in Appendix A in the supplementary material. For 18 of the 19 VACs, a single Tregex pattern was used to identify their instances, while the “V off n” construction involved two patterns, as off received two different tags, RP (participle) and IN (preposition), both of which were included in this study.

To check the accuracy of the automatic parsing and retrieval procedure in this study, we manually examined 100 randomly selected sentences for each VAC in the BNC and 50 for each in the L1 German and L1 Spanish subcorpora, respectively (i.e., 200 sentences per VAC total). Table 3 shows the accuracy for each VAC in each of the three datasets. For VACs occurring fewer than 100 or 50 times in the dataset, we reported the overall accuracy of all retrieved cases. We found comparatively low accuracy rates for certain VACs, but the accuracy rate reached 80% or above in all three datasets for the following nine VACs: “V about n,” “V across n,” “V against n,” “V between n,” “V into n,” “V off n,” “V through n,” “V toward n,” and “V with n”. We retrieved the head verbs and created the frequency-sorted verb lemma list for each VAC in each L1 group and L2 proficiency level. TagAnt (Anthony, Reference Anthony2015) was used for the lemmatization of the verb list. These frequency lists of verb lemmas served as the datasets for the statistical analyses in this study.

Table 3. Identification accuracy for VACs in each dataset

Note: Bolded rows represent VACs with an accuracy rate of 80% or more in all three datasets.

While we used the same procedure to automatically extract VACs from syntactically parsed versions of the BNC and the learner subcorpora, R&B used different procedures to extract VACs from them. For the BNC, they used a different parser, namely, the Robust Accurate Statistical Parsing System (Andersen et al., Reference Andersen, Nioche, Briscoe and Carroll2008), to parse the BNC files, and then relied on the same part-of-speech (POS) categories (i.e., verbs, prepositions, and nouns) and syntactic relationships (i.e., between a target preposition and a noun/verb) to extract VACs. Our BNC VAC files can be expected to be highly comparable to those obtained in R&B, with the caveat that some minor variation may have resulted from the disparities in the tagging and parsing accuracies between the two parsers. For the learner subcorpora, R&B first POS-tagged the files using TagAnt, then exhaustively retrieved all VAC candidates (i.e., sequences consisting of a verb followed by a target preposition) via concordance searches on the POS-tagged files, and finally manually filtered the candidates to identify true hits of each VAC. In this case, our automated procedure proved more efficient, but it came with limitations in terms of precision and recall that could affect the full comparability of our results and those of R&B.

Statistical analyses

In terms of statistical analyses, we first correlated the verb-VAC combination frequency in learner writing and the BNC to see whether R&B’s observation about learners’ move toward the L1 usage norm still holds in the expanded new version of the corpus. Following R&B, we used the log10 transformed frequencies of verb-VAC combinations in this analysis, with all non-zero values increased by . 01 before log-transformation and all zero values directly replaced by . 01 without log-transformationFootnote ³.

We then conducted additional ZINB regression analyses, in which L2 proficiency level and the verb-VAC combination frequency in the BNC were entered as explanatory variables, and the verb-VAC combination frequency in each learner subset served as the dependent variable. Given the varying sizes of the BNC and the learner subcorpora, the frequencies in each learner subcorpus and the BNC were normalized to 1,000,000 words. As the ZINB model only deals with integers (number of counts), the normalized frequencies were rounded to integers.

The pscl package in R was used for fitting ZINB models and the sjPlot package was used to obtain the incidence rate ratios (IRRs) for the ZINB models. The ggplot2 package was used for plotting, and cor.test, lm and r.squared functions were used to compute the Pearson r values, p values and R² values of correlation analyses. In addition to using normalized frequencies to account for the size differences among the corpora, we also fit ZINB models using raw frequencies with the log10 of the corpus size (i.e., number of word tokens in the corpus) as the offset variable. As detailed in Appendix B of the supplementary material, these models yielded comparable results as those obtained from the models with normalized frequencies.

Overall distribution of VACs

Following the original study, we computed the type and token frequencies of each VAC across proficiency levels. To better reveal the changes in construction diversity across proficiency levels, we additionally calculated the corrected type-token ratio for each VAC construction (VAC CTTR; Park, Reference Park2017), a transformed measure of type-token ratio ( $ VAC\; CTTR=\frac{types}{\sqrt{2\ast tokens}} $ ) that approximately addresses the impact of text length. Tables 4 and 5 present an overview of the frequencies of VACs in L1 German learners and L1 Spanish learners, respectively. A striking similarity in terms of construction frequency profile can be found between this replication and the original study. R&B reported particularly low type and token frequencies for “V across n,” “V off n” and “V toward n”, while much higher frequencies for “V about n” and “V with n”. Our analysis confirmed the same frequency profile of the above-mentioned VACs in the EFCAMDAT2 subsets, although more tokens of VACs were found in this expanded version of corpus. “V about n” and “V with n” were used most frequently by learners and “V with n” was the most productive one with the highest CTTRs among the nine VACs. Except for the three VACs with a small number of tokens (i.e., “V across n,” “V off n,” and “V toward n”), the other six VACs (i.e., “V about n,” “V against n,” “V between n,” “V into n,” “V through n,” and “V with n”) showed an increase with fluctuations in type frequency and CTTR from A1 to B2 proficiency levels, indicating learners’ more varied and productive usage and expanding knowledge repertoire of these VACs. A particularly interesting case is “V between n”. Despite a decline in its token frequency with proficiency, its type frequency and CTTR increased steadily with L2 proficiency in L1 German learners from A1 to B2 and in Spanish learners from A1 to B1. The observed decrease in the token frequency of “V between n” confirms previous findings that construction token frequency is not always positively correlated with L2 proficiency (Biber et al., Reference Biber, Gray and Staples2016; Hwang & Kim, Reference Hwang and Kim2023). On the other hand, the steady increase in type frequency and CTTR suggests that learners at higher proficiency possess a larger repertoire of verbs for this VAC despite the lower frequency in use.

Table 4. Overview of VACs across L1 German learner datasets

Note: Tps = types; Tks = tokens; CTTR = corrected type-token ratio; Tks 100K = tokens per 100K words.

Table 5. Overview of VACs across L1 Spanish learner datasets

Note: Tps = types; Tks = tokens; CTTR = corrected type-token ratio; Tks 100K = tokens per 100K words.

Results and discussion for RQ1

To address RQ1 (whether learners’ verb-VAC associations move closer to an L1 usage norm as their language proficiency increases), we will focus on the “V about n” VAC, the results of which were discussed in detail in R&B. The automatic identification procedure for “V about n” reached a high accuracy rate (see Table 3), and we further manually checked the list of lemmas against the retrieved sentences to ensure the reliability of the analysis. Table 6 compares the Pearson correlation r values reported by R&B and by the present study. Both studies showed a stronger correlation between learners’ and L1 English verb-VAC pairings at A2 than A1. While the correlation values remained at the same level from A2 to B2 in R&B’s dataset, this study demonstrates an increase from A2 to B2 in both L1 groups of learners. The lower correlations at the C1 level in R&B (particularly for L1 Spanish learners) and in this study can be due to the smaller size of the C1 subcorpus in both studies (as previously shown in Table 2). Overall, the correlations in Table 6, as argued by R&B, are non-trivial and provide empirical support for the important role of L2 input frequency in VAC acquisition.

Table 6. Correlations for verbs in “V about n” between L2 learners’ and L1 English usage in R&B and this replication study

To further examine the variations of verb-VAC combinations across different learner groups and proficiency levels, we created the same type of correlation plots as in R&B. Figure 1(a), for example, shows the correlations between the logarithmic frequencies of verb types in the “V about n” VAC used by A1 level L1 German learners (y-axis) and L1 English users (x-axis). The full statistics for all correlation analyses are reported in Tables 7 and 8. The black bar at y = .01, as all values were incremented by . 01, indicates a large number of verb types in the BNC that were not used by the learners. R&B reported two major findings about learners’ use of verbs in “V about n” across proficiency levels. First, learners produced a wider variety of verbs in “V about n” as their proficiency increased. Specifically, verbs became more populated in the plots for higher proficiency levels, and A2-level learners started to use legitimate verbs that were absent in A1-level writings (e.g., hear, like, report, laugh, and complain). Second, learners at B1 and B2 levels produced verbs that seldom co-occur with the “V about n” in L1 English usage. These verbs, nonetheless, are semantically related to the core verbs of the VAC, including the talk group, the think group, and the learn group as categorized by Francis et al. (Reference Francis, Hunston and Manning1996),Footnote ⁴ each named after the most frequent member in the respective meaning group. Such unidiomatic examples in learner writing, as R&B found, included consider in the think group and discuss and request in the talk group. The use of unidiomatic verb-VAC combinations, however, notably decreased at the C1 level.

Figure 1. Correlations of verbs in L1 German learners’ writing and the BNC for “V about n”.

Table 7. Statistics for the correlation analyses between the BNC and L1 German learners

Table 8. Statistics for the correlation analyses between the BNC and L1 Spanish learners

The first finding of R&B still holds in our dataset. As shown in Figures 1 and 2, the plots for higher proficiency levels are populated with a larger number of verb types. In line with R&B, we also noted that verbs such as laugh, report, and complain were used by learners at the A2 level and above but not by those at the A1 level. That is, learners seem to accumulate more knowledge of verb types legitimate for the target VAC with more L2 experience. In line with the second finding of R&B, the current analysis revealed fewer learner-specific, unidiomatic instances at the C1 level. Both studies found that (a) “discuss about n” was used by L1 German learners at the A2, B1, and B2 levels (e.g., “we will discuss about the topic in several teams”) and by L1 Spanish learners at the B2 level (e.g., “The purpose of this meeting is to discuss about company problems”) but were absent from the C1 learner dataset; and (b) “consider about n”

Figure 2. Correlations of verbs in L1 Spanish learners’ writing and the BNC for “V about n”.

(e.g., “we start to consider about a job change”) and “request about n” (e.g., “we request about a loan because we want to buy a house”) were used by B2-level L1 German learners but not by other groups of learners.

Two observations not discussed in the original study are worth noting. The first one is that L1 German and Spanish learners seemed to be able to use all three meaning groups of verbs (i.e., the talk group, the think group, and the learn group) for “V about n” as summarized by Francis et al. (Reference Francis, Hunston and Manning1996). Realizations such as “talk about n” and “think about n” remained as the top two in token frequency in the BNC and in 9 of the 10 learner datasets, with the exception of Spanish A2 where be ranked the second. While the use of “learn about n” was comparatively less frequent than the top two, both L1 German and Spanish learners started to use this expression at the beginner level (A1). L1 Spanish learners, in particular, used “learn about n” fairly frequently from A1 to B2 levels (see Figure 2). Second, plots for the B2 level in our study and in R&B both exhibited a high frequency of “enquire about n” at the B2 level, which ranked the fourth in token frequency after think, talk, and be in L1 German learners (Figure 1(d)) and the third after think and talk in L1 Spanish learners (Figure 2(d)). The frequent use of enquire may be attributed to the influence of writing topic. Among the writing tasks at the B2 level, there was a prompt asking learners to write a formal request letter for a bank loan, and the model answer to this task contained an instance of “enquire about n”, thus possibly promoting learners’ use of this expression. Topic effect, therefore, seems to be visible in the results of this study and R&B. Although few corpus-based studies to date have examined the effects of writing topic on L2 constructions, this is a critical issue to understand for making L2 written and/or spoken corpora more fruitful for L2 acquisition and writing research (Alexopoulou et al., Reference Alexopoulou, Michel, Murakami and Meurers2017), as different topics/tasks may create unequal opportunities of construction use (Caines & Buttery, Reference Caines, Buttery, Brezina and Flowerdew2017). R&B and Römer and Garner (Reference Römer and Garner2019) also acknowledge the potential task effect on L2 VACs and call for further investigation into the impact of writing topic on L2 construction use.

Results and discussion for RQ2

Tables 9 and 10 present the results of ZINB models for learners’ verb-VAC associations for “V about n” as a function of L1 usage norm and L2 proficiency level. The ZINB model consists of two parts: the count model that determines the count values (i.e., verb-VAC frequencies) and the zero-inflated model that predicts excess zeroes (i.e., verb-VAC combinations in the BNC that were not in the learner subset).

Table 9. Results of the ZINB model for Verb-VAC combinations in L1 German learners

Note: Freq = frequency; AIC = Akaike Information Criterion; IRR = incidence rate ratio; * p < .05; ** p < .01; *** p < .001.

Table 10. Results Of The ZINB Model For Verb-VAC Combinations In L1 Spanish Learners

Note: Freq = frequency; AIC = Akaike Information Criterion; IRR = incidence rate ratio; * p < .05; ** p < .01; *** p < .001.

The count models for L1 German and L1 Spanish learners consistently indicated the BNC verb-VAC frequency as a significant predictor of the verb-VAC frequency in L2 learners’ written production (estimate = .117, IRR = 1.124, z = 6.601, p < .001 for L1 German learners; estimate = .136, IRR = 1.145, z = 7.522, p < .001 for L1 Spanish learners). Nonetheless, the IRRs of this predictor for the two L1 groups of learners suggested an increase by an estimated factor of 1.124 and of 1.145 in learners’ verb-VAC frequency for each unit increase in the BNC. This small estimated increase in learners’ use of VAC instances with the BNC frequency can be partly attributed to the scope and the nature of writing topics in the learner corpus, which may limit the use of specific VAC instances despite their high frequencies in L1 usage. For example, set, grumble, shout, and mutter were used in “V about n” in the BNC, but such cases were not found in the learner corpus. “Set about n” means “to undertake a task” while the latter three verbs (i.e., grumble, shout, and mutter) entail specific ways of speaking. With very specific meanings, these VACs are arguably less likely to occur in argumentative writings than in narrative essays and novels. In addition to the lack of opportunities to use specific verbs arising from the limited range of topics, the absence of the above-mentioned verbs could also be partially attributed to the potential discrepancies between the L1 English corpus and the L2 input received by the learners (Kyle, et al., Reference Kyle, Crossley and Verspoor2021; Lu, Reference Lu2023: 85), which is particularly relevant for learners in EFL contexts, such as the majority of learners represented by EFCAMDAT.

In terms of the effects of L2 proficiency, Group [GB2] (where “G” denotes “L1 German” and “B2” denotes the proficiency level) and Group [GC1] of L1 German learners tended to have a significantly higher mean frequency of verb-VAC combinations than the reference A1 group (estimate = 1.011, IRR = 2.749, z = 2.878, p = .004 for Group [GB2]; estimate = 1.171, IRR = 3.225, z = 2.810, p = .005 for Group [GC1]). In a similar vein, for L1 Spanish learners, Group [SB1] (B1-level; estimate = 1.228, IRR = 3.414, z = 2.910, p = .004), Group [SB2] (estimate = 2.295, IRR = 9.922, z = 5.233, p < .001) and Group [SC1] (estimate = 2.764, IRR = 15.859, z = 5.774, p < .001) were positively related to a higher mean count of verb-VAC combinations.

The zero-inflated models for both learner groups showed that BNC frequency was negatively associated with excess zeros, with IRRs much lower than 1 (estimate = –1.696, IRR = .183, z = –4.706, p < .001 for L1 German learners; estimate = –1.921, IRR = .146, z = –5.089, p < .001 for L1 Spanish learners). That means the probability of having an excess zero decreased with every unit increase in the predictor BNC frequency, regardless of L1. In addition, as shown by the negative estimates and IRRs less than 1, higher proficiency levels, such as Group [GA2], Group [GB1], Group [GB2], Group [SA2], Group [SB1], and Group [SB2], were significantly negatively related to the odds of having an excess zero in comparison to the reference level of A1. In short, the results of the ZINB models corroborated our and R&B’s findings from the correlation analyses that BNC verb-VAC frequency was significantly positively associated with learners’ verb-VAC frequency as well as the findings regarding the increased overlap between the verb-VAC combinations used by L1 users and learners with higher proficiency. The estimated increase in verb-VAC associations in learners with BNC frequency, however, was relatively small, suggesting the existence of other important explanatory factors for the frequency of verb-VAC combinations in the learner corpus.

Future replication research

This replication study holds several implications for future research. First, the findings of this replication suggest the potential influence of writing topic on the use of verb-VAC associations in L2 writing. While corpus-based investigations into lexical and syntactic features of L2 writing (e.g., Alexopoulou et al., Reference Alexopoulou, Michel, Murakami and Meurers2017; Gablasova et al., Reference Gablasova, Brezina and McEnery2017; Hinkel, Reference Hinkel2009; Michel et al., Reference Michel, Murakami, Alexopoulou and Meurers2019; Yang et al., Reference Yang, Lu and Weigle2015; Yoon, Reference Yoon2021) have consistently supported the topic effect on L2 use, less attention has been paid to the topic effect on L2 form-meaning pairings at the lexis-grammar interface (e.g., VACs). Future replication studies are recommended to examine L2 constructions across proficiency levels with better control for writing/speech topics. Another possible approach to alleviate the potential topic effect is to incorporate data from psycholinguistic experiments, such as the free association tasks in Ellis et al. (Reference Ellis, Römer and O’Donnell2016), that provide frame prompts with minimum context information (e.g., she _____over the …). Integrating corpus investigations with free association tasks can also help address the data sparsity problem in learner corpus analysis, which persists even in large-scale learner corpora, such as EFCAMDAT1 and EFCAMDAT2, as demonstrated in R&B and in this replication, respectively.

Second, as a pioneering L1 English corpus compiled in the 1990s, the BNC may fall short in capturing recent language usage. Future research may explore the potential effect of using the BNC to derive the native usage norm in R&B and our study by considering a more up-to-date reference corpus as a proxy learner input corpus. An example of such a corpus may be the Corpus of Contemporary American English (Davies, Reference Davies2010), which encompasses language data from 1990 to 2019.

Third, constructing more representative L2 input corpora for learners in specific EFL contexts could constitute a valuable addition to usage-based SLA research. For instance, English textbooks, classroom interactions, and extracurricular readings recommended to or selected by learners may serve as important sources of English input for many EFL learners. A collection of these and other materials learners are actually exposed to may better represent their L2 input than general-purpose L1 English corpora such as the BNC. Compiling such a corpus of learner input, however, is no easy undertaking, as it necessitates recording and transcribing classroom interactions as well as tracking and collecting out-of-class materials that learners encounter over time. Additionally, the input corpus should best be compiled in tandem with a corpus of language output by the same cohort of L2 learners, as the language input to different cohorts of learners would likely vary. For this reason, it was not feasible for R&B or our study to construct such an input corpus for the learners represented in EFCAMDAT post hoc. Future endeavors that simultaneously build learner input and output corpora could test the value of such L2 input corpora for usage-based SLA research.

A final recommendation for future replication is the inclusion of VACs which are expected to decrease with L2 proficiency but with sufficient tokens for subsequent analysis. While the token frequency of certain construction (i.e., “V about n,” “V with n”) increased with L2 proficiency in the present learner corpus, some other constructions (e.g., “V that,” “be ADJ,” etc.) are expected to decrease in use with L2 proficiency (Biber et al. Reference Biber, Gray and Staples2016; Hwang & Kim, Reference Hwang and Kim2023). An examination of these types of constructions will enable a more thorough understanding of the L2 construction developmental trajectory.