Titone and Tiv (Reference Titone and Tiv2022) propose a new application of Bronfenbrenner's socioecological theory to bilingualism, encouraging scholars to accept complexity. The work by Titone and Tiv energizes the area by broadening our understanding of “linguistic experience.” They are inspired by Bronfenbrenner's ecological systems framework, though it is not new to investigate bilingualism from this perspective (Green & Abutalebi, Reference Green and Abutalebi2013; Grosjean, Reference Grosjean1998; The Douglas Fir Group, 2016; Weisleder, Reference Weisleder2017). The systems framework has been previously proposed and applied to bilingualism, but it has largely been contained within the developmental literature. Thus, it is novel to consider applying this approach to the entire field of bilingualism and bilingual experience, to step back from the individuals and consider them in context. Another novel contribution of this work is applying tools and perspectives widely used by developmental researchers to bilingualism more broadly.
Introduction
Adopting Titone and Tiv's Systems Framework of Bilingualism is challenging the use of traditional methods (e.g., questionnaires or behavioral measures focused on individual and group differences). We can, however, use methodologies and concepts from other domains that have analogies to Titone and Tiv's “systems framework of bilingualism”, such as developmental research, network science (addressed in the keynote piece), and machine learning (see figure). Here, we address three methodological and analytical challenges and present possible solutions. The challenges are: 1) defending against simplistic stories; 2) collecting and curating complex data; and 3) analyzing complex datasets.
Challenge 1: Defending against simplistic stories
One challenge to complexity is simplicity. In their book “Made to Stick: Why Some Ideas Survive and Others Die”, Chip and Dan Heath use the SUCCESS acronym to argue that Simple, Unexpected, Concrete, Credible, Emotional, Stories are more likely to “stick.” In the case of the “bilingual advantage”, this is a story that has become too “sticky”, and it has taken off far beyond the interesting observations of Peal and Lambert. Peal and Lambert challenged the conventional wisdom of the time (that argued that bilingualism was disadvantageous for learning); they showed instead that students may have a more diversified set of mental abilities stemming from greater mental flexibility. Peal and Lambert presented a (seemingly) simple story of bilingualism that inverted the classic wisdom of the time, and the story stuck (Peal & Lambert, Reference Peal and Lambert1962). Under the surface though, Peal and Lambert engaged with many of the bioecological factors that Grosjean, Abutalebi and Green, and Titone and Tiv have recommended that researchers studying bilingualism should account for. For example, they stated that their findings applied to ten-year-old English–French Catholic pupils in Montreal's French school systems. They were also very careful about how they defined bilingualism, screening their participants four ways: a) through word association tests, b) through word detection tests, c) through the Peabody Picture Vocabulary Test, and finally, d) by a self-assessment of proficiency. The danger of a simple story is believing that the phenomenon is also simple. Bilingualism is a multifaceted and rich set of experiences, yet the simple story invites people to ask the simple question, “how do bilinguals and monolinguals differ on any given cognitive or neurological task?” Titone and Tiv's paper is a warning against collapsing across all of the complexity. All of the complexity should be accounted for, or we will likely yield nonsense.
Challenge 2: Collecting and curating complex data
Data management with group comparisons is simple and thus attractive to researchers, but the act of dichotomization has several drawbacks – particularly prior to data collection. One can't put the genii back in the bottle, so to speak, and infer complexity from oversimplified measures; though the reverse is true (if appropriate). There have been arguments in favor of using continuous data and sophisticated individual difference measures (e.g., language entropy) to capture and quantify bilingualism because “bilingualism is not a categorical variable” (Anderson, Hawrylewicz & Bialystok, Reference Anderson, Hawrylewicz and Bialystok2020; Anderson, Mak, Chahi & Bialystok, Reference Anderson, Mak, Chahi and Bialystok2018; Gullifer & Titone, Reference Gullifer and Titone2020a, Reference Gullifer and Titone2021; Luk & Bialystok, Reference Luk and Bialystok2013). Recent work shows continuous measures of bilingualism map onto brain and behavioral changes (Anderson et al., Reference Anderson, Mak, Chahi and Bialystok2018; Gullifer et al., Reference Gullifer, Chai, Whitford, Pivneva, Baum, Klein and Titone2018; Gullifer & Titone, Reference Gullifer and Titone2020b). A chief advantage of using continuous data is a boost in statistical power, since the middle of the distribution now contributes signal rather than noise (see Figure 1A).
Fig. 1. Practical considerations for a systems framework. Panel A) shows how moving from a binary classification of bilingualism to a spectrum allows more variance to be leveraged in linear and nonlinear models thus increasing power. Panel B) illustrates the systems framework, and panels C) through F) illustrate pragramatic approaches to testing within this framework.
The Bronfenbrenner paradigm does aid in highlighting relevant information researchers should collect (e.g., we need questions addressing individuals, interpersonal dynamics, local ecology, broader society, and time; Figure 1B). It may be helpful to use qualitative investigations using constructivist-interpretive methodologies within a structured interview framework before producing standardized questionnaires for this purpose to help explain the lived experience of bilinguals (Weiss, Reference Weiss1997). Using geotags with online data gathering to locate users is another method for capturing contextual elements of the data. Titone and Tiv also advocate a “many-labs” approach to collect richer datasets (and generalize beyond local samples; see Figure 1E). Additionally, legacy data from the LHQ and other questionnaires could be combined across labs to provide preliminary answers to some of the systems framework's queries (e.g., how robust are language measures across the globe and age ranges, etc.; Li, Zhang, Yu & Zhao, Reference Li, Zhang, Yu and Zhao2020). Well-formed social network questionnaires probing multiple levels of the systems framework also offer a solution: from the individual level, the interpersonal level, and possibly parts of the ecological level; and data from these levels can be joined together as a multilevel/hierarchical dataset (e.g., Collective, Reference Collective2016; Tiv et al., Reference Tiv, Kutlu, Gullifer, Feng, Doucerain and Titone2021; see Fig 1D).
Setting aside the additional time and expertise required to extract information about unique multilingual experiences and contexts, complex datasets also demand complex analyses.
Challenge 3: Analyzing complex datasets
Dataset complexity and the potential degrees of freedom increase as data becomes richer, and researchers strive to account for more aspects that better situate their participants and account for their lived experience. Collecting data becomes difficult unless a sample is constrained by targeting specific populations. We should not make broad statements about “bilinguals” writ large unless our findings are replicated in other samples and studies. We could, of course, tighten this more to ensure that more bioecological model experiences are comparable, but doing so would constrain the population to which our findings apply. As a result, genuine engagement with bioecological techniques will inevitably limit the breadth of the statements behavioral scientists are accustomed to making. This is not necessarily a bad thing, but it may take some getting used to: because most researchers are trained to talk about their research as highly significant, impactful, and having broad societal ramifications; therefore their assertions tend to stray beyond the boundaries of the facts. An alternative perspective is to consider the population as the background for the sample being gathered and collect data relevant to that larger context. The assertions would still need to be moderated in this case, but the data could be collapsed with other samples hierarchically to derive general truths about bilinguals within the population of interest. This viewpoint aligns with the earlier mentioned “many-labs” approach to data collection.
Research questions focusing on development necessarily consider change across time and contexts (e.g., language acquisition in schools/homes, across years; Carroll, Reference Carroll2017; Weisleder, Reference Weisleder2017). Developmental researchers are thus highly motivated to use hierarchical models and longitudinal approaches that situate the individual within their temporal and/or spatial contexts (see Figure 1C). These approaches address many of the complexities raised by the systems framework.
Beyond simple assessments of ability and usage, language entropy is a valuable technique for capturing unique information about which languages are utilized in each situation, and entropy can be computed readily at different levels of analysis. Multilevel/hierarchical methodologies will undoubtedly be used to analyze this type of data, which is fitting given the new focus on how the context impacts language speakers. Another viewpoint is that bilingual research should be reframed as an “exploratory mode” (Navarro-Torres, Beatty-Martínez, Kroll & Green, Reference Navarro-Torres, Beatty-Martínez, Kroll and Green2021). This aligns with social psychology phenomenological research and allows researchers to question whether it is even a reasonable a priori assumption to categorize people based on their linguistic experience.
A sophisticated, multidisciplinary theoretical framework requires collecting various measures and (preferably) participants. Many labs have pushed toward mass data collecting to make the most of their resources even without such a framework. On the other hand, analytical approaches have not kept up with the increase in data collecting. When presented with many measures (all of which could have projected importance), researcher degrees of freedom become a concern and can lead to p-hacking where significant results arise merely due to the number of tests conducted. The typical solution to this problem is to reduce complexity by assuming a simple theoretical model and limiting the number of variables (and thus hypothesis tests) to investigate a priori. However, this technique may lead to the current dilemma, in which a complex construct (in this case, bilingualism) is oversimplified.
Open practices such as study preregistration offer some new solutions to this challenge. However, we can also use solutions from machine learning and data science. Many of the growing pains that the social sciences are experiencing as a result of the data explosion were also experienced (and alleviated) during the “big data phenomena” in the early 2000s, and machine learning with cross-validation is a helpful complement to hypothesis testing. Multiverse analyses are a different approach to assess reliability. In this technique, researchers attempt to explicitly codify each of their choices to preprocess and analyze their data. The multiverse of analyses encompasses all possible combinations of researcher choices. It represents an explicit attempt to determine if the effect of interest is robust across all degrees of freedom for all researchers (Donnelly, Brooks & Homer, Reference Donnelly, Brooks and Homer2019).
Titone and Tiv's use of Bronfenbrenner's concept to call for complexity is innovative in its application to the broader topic of bilingualism beyond development and language acquisition. Researchers will surely benefit from grappling with this complexity and contemplating this type of paradigm as they encounter new and intriguing problems. As we have noted, there are hurdles in dealing with such complicated data. Still, the tools and skills to do so are readily available, and we simply need to accept the challenge and drive the field forward by using them.
Titone and Tiv (Reference Titone and Tiv2022) propose a new application of Bronfenbrenner's socioecological theory to bilingualism, encouraging scholars to accept complexity. The work by Titone and Tiv energizes the area by broadening our understanding of “linguistic experience.” They are inspired by Bronfenbrenner's ecological systems framework, though it is not new to investigate bilingualism from this perspective (Green & Abutalebi, Reference Green and Abutalebi2013; Grosjean, Reference Grosjean1998; The Douglas Fir Group, 2016; Weisleder, Reference Weisleder2017). The systems framework has been previously proposed and applied to bilingualism, but it has largely been contained within the developmental literature. Thus, it is novel to consider applying this approach to the entire field of bilingualism and bilingual experience, to step back from the individuals and consider them in context. Another novel contribution of this work is applying tools and perspectives widely used by developmental researchers to bilingualism more broadly.
Introduction
Adopting Titone and Tiv's Systems Framework of Bilingualism is challenging the use of traditional methods (e.g., questionnaires or behavioral measures focused on individual and group differences). We can, however, use methodologies and concepts from other domains that have analogies to Titone and Tiv's “systems framework of bilingualism”, such as developmental research, network science (addressed in the keynote piece), and machine learning (see figure). Here, we address three methodological and analytical challenges and present possible solutions. The challenges are: 1) defending against simplistic stories; 2) collecting and curating complex data; and 3) analyzing complex datasets.
Challenge 1: Defending against simplistic stories
One challenge to complexity is simplicity. In their book “Made to Stick: Why Some Ideas Survive and Others Die”, Chip and Dan Heath use the SUCCESS acronym to argue that Simple, Unexpected, Concrete, Credible, Emotional, Stories are more likely to “stick.” In the case of the “bilingual advantage”, this is a story that has become too “sticky”, and it has taken off far beyond the interesting observations of Peal and Lambert. Peal and Lambert challenged the conventional wisdom of the time (that argued that bilingualism was disadvantageous for learning); they showed instead that students may have a more diversified set of mental abilities stemming from greater mental flexibility. Peal and Lambert presented a (seemingly) simple story of bilingualism that inverted the classic wisdom of the time, and the story stuck (Peal & Lambert, Reference Peal and Lambert1962). Under the surface though, Peal and Lambert engaged with many of the bioecological factors that Grosjean, Abutalebi and Green, and Titone and Tiv have recommended that researchers studying bilingualism should account for. For example, they stated that their findings applied to ten-year-old English–French Catholic pupils in Montreal's French school systems. They were also very careful about how they defined bilingualism, screening their participants four ways: a) through word association tests, b) through word detection tests, c) through the Peabody Picture Vocabulary Test, and finally, d) by a self-assessment of proficiency. The danger of a simple story is believing that the phenomenon is also simple. Bilingualism is a multifaceted and rich set of experiences, yet the simple story invites people to ask the simple question, “how do bilinguals and monolinguals differ on any given cognitive or neurological task?” Titone and Tiv's paper is a warning against collapsing across all of the complexity. All of the complexity should be accounted for, or we will likely yield nonsense.
Challenge 2: Collecting and curating complex data
Data management with group comparisons is simple and thus attractive to researchers, but the act of dichotomization has several drawbacks – particularly prior to data collection. One can't put the genii back in the bottle, so to speak, and infer complexity from oversimplified measures; though the reverse is true (if appropriate). There have been arguments in favor of using continuous data and sophisticated individual difference measures (e.g., language entropy) to capture and quantify bilingualism because “bilingualism is not a categorical variable” (Anderson, Hawrylewicz & Bialystok, Reference Anderson, Hawrylewicz and Bialystok2020; Anderson, Mak, Chahi & Bialystok, Reference Anderson, Mak, Chahi and Bialystok2018; Gullifer & Titone, Reference Gullifer and Titone2020a, Reference Gullifer and Titone2021; Luk & Bialystok, Reference Luk and Bialystok2013). Recent work shows continuous measures of bilingualism map onto brain and behavioral changes (Anderson et al., Reference Anderson, Mak, Chahi and Bialystok2018; Gullifer et al., Reference Gullifer, Chai, Whitford, Pivneva, Baum, Klein and Titone2018; Gullifer & Titone, Reference Gullifer and Titone2020b). A chief advantage of using continuous data is a boost in statistical power, since the middle of the distribution now contributes signal rather than noise (see Figure 1A).
Fig. 1. Practical considerations for a systems framework. Panel A) shows how moving from a binary classification of bilingualism to a spectrum allows more variance to be leveraged in linear and nonlinear models thus increasing power. Panel B) illustrates the systems framework, and panels C) through F) illustrate pragramatic approaches to testing within this framework.
The Bronfenbrenner paradigm does aid in highlighting relevant information researchers should collect (e.g., we need questions addressing individuals, interpersonal dynamics, local ecology, broader society, and time; Figure 1B). It may be helpful to use qualitative investigations using constructivist-interpretive methodologies within a structured interview framework before producing standardized questionnaires for this purpose to help explain the lived experience of bilinguals (Weiss, Reference Weiss1997). Using geotags with online data gathering to locate users is another method for capturing contextual elements of the data. Titone and Tiv also advocate a “many-labs” approach to collect richer datasets (and generalize beyond local samples; see Figure 1E). Additionally, legacy data from the LHQ and other questionnaires could be combined across labs to provide preliminary answers to some of the systems framework's queries (e.g., how robust are language measures across the globe and age ranges, etc.; Li, Zhang, Yu & Zhao, Reference Li, Zhang, Yu and Zhao2020). Well-formed social network questionnaires probing multiple levels of the systems framework also offer a solution: from the individual level, the interpersonal level, and possibly parts of the ecological level; and data from these levels can be joined together as a multilevel/hierarchical dataset (e.g., Collective, Reference Collective2016; Tiv et al., Reference Tiv, Kutlu, Gullifer, Feng, Doucerain and Titone2021; see Fig 1D).
Setting aside the additional time and expertise required to extract information about unique multilingual experiences and contexts, complex datasets also demand complex analyses.
Challenge 3: Analyzing complex datasets
Dataset complexity and the potential degrees of freedom increase as data becomes richer, and researchers strive to account for more aspects that better situate their participants and account for their lived experience. Collecting data becomes difficult unless a sample is constrained by targeting specific populations. We should not make broad statements about “bilinguals” writ large unless our findings are replicated in other samples and studies. We could, of course, tighten this more to ensure that more bioecological model experiences are comparable, but doing so would constrain the population to which our findings apply. As a result, genuine engagement with bioecological techniques will inevitably limit the breadth of the statements behavioral scientists are accustomed to making. This is not necessarily a bad thing, but it may take some getting used to: because most researchers are trained to talk about their research as highly significant, impactful, and having broad societal ramifications; therefore their assertions tend to stray beyond the boundaries of the facts. An alternative perspective is to consider the population as the background for the sample being gathered and collect data relevant to that larger context. The assertions would still need to be moderated in this case, but the data could be collapsed with other samples hierarchically to derive general truths about bilinguals within the population of interest. This viewpoint aligns with the earlier mentioned “many-labs” approach to data collection.
Research questions focusing on development necessarily consider change across time and contexts (e.g., language acquisition in schools/homes, across years; Carroll, Reference Carroll2017; Weisleder, Reference Weisleder2017). Developmental researchers are thus highly motivated to use hierarchical models and longitudinal approaches that situate the individual within their temporal and/or spatial contexts (see Figure 1C). These approaches address many of the complexities raised by the systems framework.
Beyond simple assessments of ability and usage, language entropy is a valuable technique for capturing unique information about which languages are utilized in each situation, and entropy can be computed readily at different levels of analysis. Multilevel/hierarchical methodologies will undoubtedly be used to analyze this type of data, which is fitting given the new focus on how the context impacts language speakers. Another viewpoint is that bilingual research should be reframed as an “exploratory mode” (Navarro-Torres, Beatty-Martínez, Kroll & Green, Reference Navarro-Torres, Beatty-Martínez, Kroll and Green2021). This aligns with social psychology phenomenological research and allows researchers to question whether it is even a reasonable a priori assumption to categorize people based on their linguistic experience.
A sophisticated, multidisciplinary theoretical framework requires collecting various measures and (preferably) participants. Many labs have pushed toward mass data collecting to make the most of their resources even without such a framework. On the other hand, analytical approaches have not kept up with the increase in data collecting. When presented with many measures (all of which could have projected importance), researcher degrees of freedom become a concern and can lead to p-hacking where significant results arise merely due to the number of tests conducted. The typical solution to this problem is to reduce complexity by assuming a simple theoretical model and limiting the number of variables (and thus hypothesis tests) to investigate a priori. However, this technique may lead to the current dilemma, in which a complex construct (in this case, bilingualism) is oversimplified.
Open practices such as study preregistration offer some new solutions to this challenge. However, we can also use solutions from machine learning and data science. Many of the growing pains that the social sciences are experiencing as a result of the data explosion were also experienced (and alleviated) during the “big data phenomena” in the early 2000s, and machine learning with cross-validation is a helpful complement to hypothesis testing. Multiverse analyses are a different approach to assess reliability. In this technique, researchers attempt to explicitly codify each of their choices to preprocess and analyze their data. The multiverse of analyses encompasses all possible combinations of researcher choices. It represents an explicit attempt to determine if the effect of interest is robust across all degrees of freedom for all researchers (Donnelly, Brooks & Homer, Reference Donnelly, Brooks and Homer2019).
Titone and Tiv's use of Bronfenbrenner's concept to call for complexity is innovative in its application to the broader topic of bilingualism beyond development and language acquisition. Researchers will surely benefit from grappling with this complexity and contemplating this type of paradigm as they encounter new and intriguing problems. As we have noted, there are hurdles in dealing with such complicated data. Still, the tools and skills to do so are readily available, and we simply need to accept the challenge and drive the field forward by using them.
Acknowledgements
JAEA is supported by a Canada Research Chair II (CRC-2020-00174), and a Brain and Behavior Research Foundation Young Investigator's Grant (29306) JG is supported by the Social Sciences and Humanities Research Council (430-2019-00935). Both authors contributed equally