Language learning motivation

Research on language learning motivation has a long history. In the current study, we adopt Dörnyei’s L2MSS as the main framework through which to explore motivational influences, but also draw on other constructs to provide what we believe to be a more complete picture of the different aspects of motivation. The L2MSS proposes that the decision to pursue and sustain language learning might be underlined both by a vision of what the individual wants to achieve, coined as the “ideal L2 self,” and individual’s belief of what one should aspire to, referred to as “ought-to L2 self” (Dörnyei, Reference Dörnyei2005). Another component that affects these decisions is “language learning experience” that concerns the effect of the immediate environment on language learning. The key motivational force in Dörnyei’s theory is driven by the gap between the future selves (ideal and/or ought-to) and current self-perception. The latter corresponds to English self-concept (Shavelson et al., Reference Yun, Hiver and Al-Hoorie1976), which reflects how the learner views their L2 self at the present moment. Other constructs drawn on in this study reflect learning goals. The most commonly researched language learning goals are “instrumentality,” which concerns the utilitarian reasons for learning a language (Gardner & Lambert, Reference Gardner, Tremblay and Masgoret1972) such as better job prospects, and “international orientation,” where English is learned to be able to communicate with other English speakers, and learn more about other cultures (Yashima, Reference Hessel2000).

Studies linking language learning motivation and language proficiency

Early studies of language learning motivation revealed a link between language learning motivation and proficiency, in particular motivation underlined by integrative motives, that is the willingness to be like a valued member of the target language community (Anisfield & Lambert, Reference Anisfield and Lambert1961; Gardner, Reference Gardner1960; Gardner & Lambert, Reference Gardner and Lambert1959, Reference Gardner, Tremblay and Masgoret1972; Peal & Lambert, Reference Ryan, Dörnyei and Ushioda1972). However, this body of research had substantial methodological limitations. Studies employed a mixture of measures, some of which were highly subjective or only tenuously related to language proficiency. Examples include: school grade reported by students; number of years speaking the language; average grade across a wide range of courses; and teachers’ rating of achievement (Lambert et al., Reference Gardner, Tremblay and Masgoret1972; Peal & Lambert, Reference Ryan, Dörnyei and Ushioda1972). A study in the 1990s by Gardner et al. (Reference Hancock, Mueller, Cudeck, du Toit and Sorbom1997) found motivation to be, in fact, more closely related to grades allocated by teachers than other more objective measures of achievement.

The line of research exploring the link between motivation and language proficiency appears to have been virtually abandoned, with few studies on the topic published after the initial wave in the 1960s and early 1970s. A number of studies since 2010 do, however, explore the relationship of L2MSS constructs with L2 proficiency (Dörnyei & Chan, Reference Dörnyei and Chan2013; T.-Y. Kim & Kim, Reference Kim and Kim2014; Y.-K. Kim & Kim, Reference Lamb2011; Lamb, Reference Lamb2012; Moskovsky et al., Reference Muthén, Hancock and Samuelson2016; Papi & Khajavy, Reference Papi and Teimouri2021; Saito et al., Reference Schwarz2019). Yet often the measures of proficiency are not robust, which makes comparisons difficult. Of these studies, in fact only a handful have the central aim to investigate the link between motivation and proficiency. These deserve some more attention here. Lamb’s study (2012) employed the C-test to measure Indonesian students’ proficiency level. Regression analysis revealed that only one motivational variable positively contributed to explaining variance in the test results, namely language learning experience at school. The role of the self-guides (ideal and ought-to) was, however, not clear. Moskovsky et al. (Reference Muthén, Hancock and Samuelson2016) used reading and writing tasks to measure the proficiency of English majors at a Saudi university. Correlation analysis showed the ideal L2 self to have a negative relationship with test results, with nonsignificant correlations for ought-to L2 self and language learning experience. The third study, which was longitudinal, focused on oral skills, and employed comprehensibility judgments by professional raters to measure proficiency (Saito et al., Reference Schwarz2019). Similar to the two previous studies, Saito et al.’s (Reference Schwarz2019) results pointed to a negligible role of the ought-to L2 self in explaining oral proficiency gains of Japanese learners of English but the ideal L2 self was found to be positively related to proficiency. Papi and Khajavy (Reference Papi and Teimouri2021) explored this further and showed that ideal L2 self and ought-to L2 self with different regulatory focus (own/other) lead to qualitative differences in motivated behavior that in turn affect achievement in different ways. Clearly there are nuances in the motivation-proficiency relationship that are not being captured by existing approaches to researching this area.

Al-Hoorie’s meta-analysis which incorporated 39 unique samples and 32,078 language learners (Al-Hoorie, Reference Shavelson, Hubner and Stanton2018) pointed to some of the conceptual difficulties in comparing studies. Such discrepancies are understood to have led to an overall unclear picture of relationships between L2 language proficiency and the motivational traits specified by contemporary theorists. There are also differences in the research context and type of analysis employed, as well as a focus on different language skills, and use of different measures of proficiency. It is clear that there is a need for further comprehensive studies that clarify the relationship between constructs in the L2MSS and L2 achievement.

Motivational profiling

The profiling approach can add to our understanding of the relationship between L2 motivation and L2 proficiency. Profiling is a person-centered methodology, which assumes heterogeneity in the population studied, bringing advantages for the level of analytic insight for a given dataset (Marcoulides & Heck, Reference Masyn and Little2013). Simply put, profiling is a means of grouping study participants based on their comparative levels across a series of metrics. For example, in terms of L2 skills, a profiling approach may show that one group of students performs comparatively better in productive over receptive skills, whilst the reverse may be true for a different group of students. In this case looking at learner profiles helps to qualitatively discriminate L2 ability for different groups of students. Such classification and understanding of the characteristics of students of different profiles opens up avenues for future interventions, changes in curriculum, or instructional modifications to provide bespoke support for different groups of learners.

Profiling has been used in SLA research to explore the interrelationships between motivational traits. Four key studies are relevant to mention here, three of which used cluster analysis to establish patterns of motivational trait levels: Csizér and Dörnyei (Reference Csizér and Dörnyei2005) studied motivation profiles of Hungarian eighth-grade language learners; Yun et al. (Reference O’Sullivan, Dunlea, Spiby, Westbrook and Dunn2018) worked with college students from South Korea; and Papi and Teimouri (Reference Peal, Lambert, Gardner and Lambert2014) Iranian secondary school students. A further study by Kangasvieri (Reference Kim, Kim, Csizér and Magid2017) employed latent profile analysis (LPA) to investigate motivational profiles of Finnish secondary school students learning a range of foreign languages. These four studies are not directly comparable to one another because they employed differing motivational variables and methodologies; there are, however, two clear parallels between their results. First, each study identified four or five distinct motivational profiles, with contrasting profiles of students with weak and strong motivation. Second, distinctions between groups were not always uniform across the traits, for example Kangasvieri (Reference Kim, Kim, Csizér and Magid2017) revealed self-concept in particular to vary independently of other motivational traits between the different profile groupings.

Of the four studies discussed here, only Yun et al. (Reference O’Sullivan, Dunlea, Spiby, Westbrook and Dunn2018) incorporated a measure of proficiency (Test of English Proficiency [TEPS], developed by Seoul National University), which enabled them to identify five distinct learner profiles: Thriver, Engaged, Striver, Dependent, and Disengaged. The possibilities engendered by integrating proficiency measures into motivation profiles is also emerging in other fields of educational assessment. A key example is from Michaelides et al. (Reference Moskovsky, Assulaimani, Racheva and Harkins2019) who use cluster analysis to explore the relationship between motivation and achievement in mathematics across a range of contexts. These researchers emphasize that a person-centered analytic approach enables motivational predictors of achievement to “interact in ways that will reveal stronger and weaker associations with achievement, compared to the average relationships observed in variable-centred approaches” (Michaelides et al., Reference Moskovsky, Assulaimani, Racheva and Harkins2019, p. 4). The study reported here is likewise premised on the possibility that the strength, or even the directionality, of motivation-proficiency relationships can vary between learners and learner groups. In taking a person-centered profiling approach in modeling L2 motivation and integrating robust measures of L2 proficiency, the study described in this article aims to set the groundwork for a more comprehensive and intricate understanding of the relationship between motivation and proficiency than has come before.

Rationale and research questions

This study takes steps toward addressing acknowledged gaps in our understanding of the relationship between L2 language learning motivation and proficiency. The review of literature suggests that multiple motivational subpopulations exist within a given group of teenage English language learners, each with a distinct profile. Notably, it has also been shown that integrating proficiency information into this profiling approach has the potential to add a valuable dimension to the findings. The current research progresses our understanding by integrating information from a validated language test into a rigorously executed and methodologically robust motivational profiling study. The following research questions are addressed:

To answer these questions, the data collected for the British Council’s English Impact Madrid study (Shepherd & Ainsworth, Reference Taguchi, Magid, Papi, Dörnyei and Ushioda2017) was revisited to establish the number and nature of subpopulations in this learner group with respect to motivation and proficiency. The data comprised of 15.5-year-old English language learners’ answers to a motivational questionnaire and the results of a multiskill English language test; all learners were currently attending state schools in the Madrid region of Spain. It is hoped that these results will be a basis for further research that will ultimately lead to a better understanding of the needs of different groups of learners and how they can be addressed by instructional intervention and curricular reform.

Sampling and participants

Data were collected as part of a wider study run by the British Council in 2017 (Shepherd and Ainsworth, Reference Taguchi, Magid, Papi, Dörnyei and Ushioda2017). Participants from within the Madrid state school system were selected using two-stage cluster sample methodology designed and executed by the Australian Council for Educational Research (ACER). First a sample of schools based on defined stratification variables (school type, geographical location, and bilingual status) was chosen, and then a random selection of 12 students from each school was sampled.

Participating students were from Compulsory Secondary Education (ESO) 4 and their mean age was 15.6. They were currently studying English as part of their studies at this grade level for at least 90 minutes per week. In total, 1,773 students from 169 schools participated, 50.9% of whom were female and 49.1% male. A total of 29.6% attended bilingual schools and a further 70.4% came from nonbilingual schools. The participation levels reached the minimum statistical required to be considered representative of this population of learners in the Madrid region.

Data-collection instruments

Procedure

The tests and questionnaires were delivered using an offline-enabled tablet in fully invigilated conditions during a period of one month in March 2017. Full involvement and approval of the Madrid Ministry of Education was obtained prior to conducting this study. Individual participation in the study was contingent on receiving written parental consent.

Analysis

The main data analysis took part in two stages using statistical software Mplus 8 (Muthén & Muthén, 1998–Reference Nagin2017). The initial Confirmatory Factor Analysis (CFA) used to confirm the measurement model is summarized in the results that follow and reported in full elsewhere ^{Footnote 2} (Shepherd & Ainsworth, Reference Taguchi, Magid, Papi, Dörnyei and Ushioda2017). The analysis novel to this study explored the motivation and proficiency profiles of the participants using LVMM, with the specific model applied being a Factor Mixture Model (FMM).

Latent Variable Mixture Modeling

The main analysis employed LVMM, a “person-orientated” approach to data modeling, which focuses on characteristics of the individuals from whom the measurements are taken (Bergman & Magnusson, Reference Bergman and Magnusson1997). This analysis identifies unobserved heterogeneity in a population and can be used to divide individuals into subtypes that, as a group, exhibit similar traits or patterns of behavior (Collins & Lanza, Reference Collins and Lanza2010).

A range of model structures can be described as LVMMs. The model applied here can be defined as a FMM that falls within category of “hybrid latent variable models” (Muthén, Reference Muthén and Asparouhov2007; Muthén & Asparouhov, Reference Muthén and Muthén2006) because it includes both continuous and categorical latent variables. Our model comprised eight continuous latent factors, each representing a motivational trait. These factors, alongside the five observed test score outcomes, acted as indicators for the latent categorical variable. Classifications could therefore vary with respect to the factor mean for each motivational trait, plus the observed means on the five language skill tests, meaning that each participant was allocated to a latent class in the model based on both motivation and proficiency indicators. A key point to note is that this approach does not assume that the high performers will have high motivation levels, and vice versa, but rather allows the relationship between motivation and proficiency, or indeed between specific measures within each of these, to vary between classes.

Figure 1 plots the relationships modeled, where: C = single latent categorical variable (comprising k categories); m _1-5 = measured language test scores; f _1-8 = latent factors representing the motivational traits; u _1-4 = observations from motivational questionnaire items; and factor loadings are assumed invariant across classes. Important to note here is that the latent class k is hypothesized to explain both the observed test scores (m_1—m₅) and the unobserved motivational traits (f₁–f₈). The results discussion will focus predominantly on how each latent class is characterized with respect to their levels across all these metrics.

Figure 1. Path diagram showing structure of relationships modeled in the current study.

The model comprised 37 dependent variables (each of the 5 language test scores, plus the 32 questionnaire responses); 8 continuous latent variables (representing the motivational traits); and 1 categorical latent variable (defining the classifications of interest). The model estimated loadings between each question response and its associated continuous latent variable (these were held constant across latent classes); latent means and thresholds for each continuous latent variable (these were allowed to vary between classes, with one class set for reference); and means for each observed test score (these were allowed to vary between classes). The full model specification in Mplus is given in Appendix D.

Analytic aims

This investigation first seeks to establish the number of distinct latent classes (k) in the current data with respect both to the participants’ English language proficiency and levels of language learning motivation. As with exploratory factor analysis, this is a heuristic technique, and it is not always the case that a definitive answer will arise from the data. In selecting the final number of classes there are statistical indicators that can be drawn upon to aid researchers; however, there is not one single indicator that is accepted for deciding on the number of classes (Nylund et al., Reference OʼSullivan, Weir and OʼSullivan2007). It is best therefore to examine a range of indicators (Masyn, Reference McLachlan and Peel2013; Nylund et al., Reference OʼSullivan, Weir and OʼSullivan2007). In the modeling exercise reported here, a series of successive models are estimated with the same overall structure (Figure 1) but with an incrementally increasing number of classes specified in the latent categorical variable (C). Model assessment took two stages: (a) examination of statistics to gauge relative model fit and (b) consideration of a range of metrics to judge the quality of class enumeration for the closest fitting models. The evidence accrued from this multistage statistical modeling exercise was then considered against the substantive nature of the class characteristics, as well as the theoretical backdrop provided by previous studies in this field, before establishing the final model. Note there was no explicit consideration of absolute model fit; the standard chi-square statistic can be oversensitive, especially in the case of large datasets, and unfortunately it was not possible to examine the residuals for each response pattern, as recommended by Masyn (Reference McLachlan and Peel2013), as Mplus did not report the output (citing: “the frequency table for the latent class indicator model is too large”). ^{Footnote 3}

Under (a), the following statistics provided information about relative fit: Bayesian Information Criterion (BIC; Schwarz, Reference Sclove1978); Consistent Akaike’s Information Criterion (CAIC; Bozdogan, Reference Bozdogan1987); Approximate Weight of Evidence Criterion (AWE; Banfield & Raftery, Reference Banfield and Raftery1993); and Sample size adjusted BIC (aBIC; Sclove, Reference Shepherd and Ainsworth1987). Each of these criteria take into account the fit of the model using log-likelihood values whilst penalizing for model complexity (Masyn, Reference McLachlan and Peel2013). Additionally, as the models with successive numbers of latent classes are nested, likelihood ratio tests can be used to compare two competing models. The LMR (Lo–Mendel–Rubin adjusted likelihood ratio test) compares the improvement in fit between models with adjacent numbers of classes (Lo et al., Reference Marcoulides, Heck and Teo2001). This measure provides a p-value, with a significant value suggesting that a greater number of classes are required in the latent variable.

Evaluations of class enumeration under (b) are only considered for models that have already been shown to reach a reasonable level of fit under (a); good class separation in itself does not indicate an overall well-fitting model (Masyn, Reference McLachlan and Peel2013). Average class probabilities (AvePP) provide a measure of the certainty with which model can predict membership of each class (Collins & Lanza, Reference Collins and Lanza2010). High classification certainty in the model, yields AvePP near 1 with values above 0.7 considered to indicate adequate precision in class assignment (Nagin, Reference Netemeyer2005). Additionally, as the accuracy of class allocation increases, so do the odds of correct classification ratio (OCC; Nagin, Reference Netemeyer2005). Values above 5 indicate good separation and class assignment (Nagin, Reference Netemeyer2005). Finally, entropy is a standardized index of model-based classification accuracy which ranges from 0 to 1, with values approaching 1 indicating favorable delineation of the classes (Celeux & Soromenho, Reference Celeux and Soromenho1996). The usefulness of entropy lies in highlighting problems with a model, rather than endorsing a particular model choice (Masyn, Reference McLachlan and Peel2013). It is reported in the current analysis as an indicator of the suitability of a solution, rather than a selection criterion.

Before moving on to discuss the results, a summary of the analytic steps is given in Table 1.

TABLE 1. Summary of analysis steps and aims