Research designs in CDST

Expanding our research agenda

CDST studies that have incorporated a relation-intensive element to their research design are far less common compared to the number of studies that have taken time-intensive approaches. Despite the fact that “complexity theorists are interested in understanding the relations [emphasis in original] that connect the components of a complex system” (Hiver & Larsen-Freeman, Reference Hiver, Larsen-Freeman, Al-Hoorie and MacIntyre2019, p. 287), to date there have been very few attempts to empirically model these relations. One potential reason behind this is relates to methodological challenges and the view that cross-sectional data, zero-order correlations and linear regression are ill-suited to studying complex systems (Al-Hoorie & Hiver, Reference Al-Hoorie, Hiver, Al-Hoorie and Szabó2022). Another reason relates to the theoretical challenges of conceptualizing abstract psychological constructs as complex systems. A number of individual differences constructs in language learning have been conceptualized as complex systems, such as motivation (Papi & Hiver, Reference Papi and Hiver2020), strategy development (Amerstorfer, Reference Amerstorfer2020), anxiety (Gregersen, Reference Gregersen2020), working memory (Jackson, Reference Jackson2020), and willingness to communicate (MacIntyre, Reference MacIntyre2020). To examine these constructs from a relation-intensive perspective, for example to model L2 motivation as a complex system, we must consider the components that form the system, and how these components align with our measurement instruments. Researchers must also confront “the boundary problem” (Larsen-Freeman, Reference Larsen-Freeman, Ortega and Han2017), accepting the theoretical impossibility of measuring a complex system in its entirety, whereby “the whole is greater than the sum of its parts” (Han, Reference Han and Han2019, p. 156). Consideration should also be given to the phenomenological validity of equating conceptual and theoretical concepts as systems, and the practical implications this has for a chosen methodology (Hiver & Al-Hoorie, Reference Hiver and Al-Hoorie2016). Mercer (Reference Mercer and Mercer2011b, p. 59) discusses these issues in relation her network-based model of the self-concept, acknowledging the theoretical and empirical difficulty of distinguishing the blurred boundaries between different self-constructs. Despite the challenges of exploring psychological constructs related to language learning from a relation-intensive approach, and the inevitable reductionism this entails, focusing on system structures can offer a perspective that is currently missing from CDST research in SLA.

Take the construct of L2 motivation, for example, which has been much discussed in CDST research (e.g., Dörnyei, Reference Dörnyei, Ortega and Han2017, Dörnyei et al., Reference Dörnyei, MacIntyre, Henry, Dörnyei, MacIntyre and Henry2015; Henry, Reference Henry, Dörnyei, MacIntyre and Henry2014, Reference Henry2017; Hiver & Papi, Reference Hiver, Papi, Lamb, Csizér, Henry and Ryan2019; Hiver & Larsen-Freeman, Reference Hiver, Larsen-Freeman, Al-Hoorie and MacIntyre2019; Papi & Hiver, Reference Papi and Hiver2020). Most CDST research on L2 motivation has been time-intensive with a focus on observing micro-level changes in a small number of variables over time. Very few CDST researchers have explored L2 motivation from a relation-intensive perspective, although there has been some theoretical discussion of how to conceptualize the structural relationships between motivational constructs as complex systems (Henry, Reference Henry, Dörnyei, MacIntyre and Henry2014, Reference Henry2017). The L2 Motivational Self System (L2MSS) is a theoretical paradigm that was developed by Dörnyei (Reference Dörnyei2005, Reference Dörnyei, Ellis and Larsen-Freeman2009) that conceptualizes L2 motivation from a self-perspective. The L2MSS is comprised of three phenomenologically constructed concepts, each theorized to be a primary source of motivation to learn an L2: the Ideal L2 Self, the Ought-to L2 Self, and L2 Learning Experience (Dörnyei Reference Dörnyei2005, Reference Dörnyei, Ellis and Larsen-Freeman2009). Although the L2MSS was not originally conceptualized as a complex system, it has been conceptually extended to a CDST paradigm (Henry, Reference Henry, Dörnyei, MacIntyre and Henry2014, Reference Henry2017). For example, Henry (Reference Henry2017, p. 551) has described the self-concept as a multifaced dynamic structure, which can be understood as “the product of constant interactions between different subsystems (such as, e.g., self-efficacy and self-esteem).”

Taking a relation-intensive approach to L2 motivation could provide insight into the structural relationships between components of the L2 motivational system, and if this were expanded to a time-relation intensive approach, could potentially identify attractor states. SLA researchers have speculated about how the L2 self-system, in particular the Ideal L2 self, can manifest as an attractor state (Henry, Reference Henry2017; Hiver, Reference Hiver, Dörnyei, MacIntyre and Henry2014; Waninge et al., Reference Waninge, Dörnyei and de Bot2014), whereby “changes in the vision of the Ideal L2 Self and changes in the distance between it and the actual self, can be conceptualized as changes in attractor state geometries” (Henry, Reference Henry, Dörnyei, MacIntyre and Henry2014, p. 87, emphasis in original). Although longitudinal data is needed to show system self-organization and the emergence of attractor states, cross-sectional data can provide a perspective that is currently missing from CDST research in SLA. As Mercer (Reference Mercer2011a) reflects in relation to her network-based model of the self-concept:

Taking a relation-intensive CDST approach to the study of individual differences in SLA can thus be viewed as complimentary of time-intensive approaches. Cross-sectional data can provide insight into the structure of relationships between system components, which, if combined with what we have learned from time-intensive CDST studies, could enrich our understanding of the complex interplay between individual differences and L2 development.

There is currently little guidance on how to analyze and model interactions between system components from a relation-intensive perspective. As previously mentioned, CDST researchers have questioned whether methods such as zero-order correlations and linear regression are suitable for examining dynamic changes and interconnected (Al-Hoorie & Hiver, Reference Al-Hoorie, Hiver, Al-Hoorie and Szabó2022). Although scholars have emphasized the potential of quantitative analyses for CDST research (Al-Hoorie & Hiver, Reference Al-Hoorie, Hiver, Al-Hoorie and Szabó2022) for example to identify network structure or nested phenomena, there appears to be an overall reluctance to use cross-sectional data, with most CDST researchers preferring longitudinal data. Until now, most studies that have taken a relation-intensive approach have analyzed relationships between variables by correlations and multiple regression analysis (Li et al., Reference Li, Dewaele and Jiang2020; Piniel & Csizér, Reference Piniel, Csizér, Dörnyei, MacIntyre and Henry2014; Saito et al., Reference Saito, Macmillan, Mai, Suzukida, Sun, Magne, Ilkan and Murakami2020; Serafini, Reference Serafini2017). However, new advancements in statistics software and data analysis techniques such as GAMM are enriching the CDST toolbox. Other techniques that have been proposed as appropriate methods to study complex systems with a relation-intensive element are latent growth curve modeling (LGCM) and multilevel modeling (MLM) (Hiver & Al-Hoorie, Reference Hiver and Al-Hoorie2020a; MacIntyre et al., Reference MacIntyre, MacKay, Ross, Abel, Ortega and Han2017). To expand our CDST toolbox of relation-intensive approaches, we could also utilize network analysis, an underexplored methodology in SLA research.

Network analysis

Network analysis has become a popular technique for studying complex systems in the field of psychology. Readers should be aware that there are many different types of network models; network analysis can be performed on cross-sectional data from a relation-intensive perspective (Epskamp & Fried, Reference Epskamp and Fried2019; Hevey, Reference Hevey2018), and also on longitudinal time-series data from a time- or time-relation intensive perspective (Bringmann et al., Reference Bringmann, Vissers, Wichers, Geschwind, Kuppens, Peeters and Tuerlinckx2013). Although we outline some other variants of network analysis later in the discussion section, it is beyond the scope of this article to describe each type of network analysis in detail. We have opted to focus on psychological networks with cross-sectional data for relation-intensive approaches, which is an underexplored dimension of CDST research in applied linguistics.

As readers may be more familiar with social network analysis, we would also like to briefly explain some differences between social networks and psychological networks. Social networks show patterns of relationships among individuals or groups, whereas psychological networks show patterns of relationships among variables (at item level or composite level). It is important to note that with social networks, the relationships between variables are known; social networks are created from an adjacency matrix, whereby the relationships between variables are directly observed (O’Malley & Onnela, Reference O’Malley, Onella, Levy, Goring, Gatsonis, Sobolev, van Ginneken and Busse2019). In contrast, with psychological networks, relationships between variables are not known but are estimated. Psychological networks are estimated from a variance-covariance matrix, based on the strength of partial correlations between variables (Epskamp & Fried, Reference Epskamp and Fried2019).

Psychological network analysis has been used to model constructs such as intelligence (van der Maas et al., Reference van der Maas, Dolan, Grasman, Wicherts, Huizenga and Raijmakers2006, Reference van der Maas, Kan, Marsman and Stevenson2017) cognitive development (Kievit, Reference Kievit2020), and mental disorders (Borsboom, Reference Borsboom, Fried, Epskamp, Waldorp, van Borkulo, van der Maas and Cramer2017) from a CDST perspective, and has also been applied to clinical research on psychological disorders such as depression and eating disorders (Elliott et al., Reference Elliott, Jones and Schmidt2020; Lutz et al., Reference Lutz, Schwartz, Hofmann, Fisher, Husen and Rubel2018). In network models, variables (also referred to as components) are represented as circles called nodes. In psychological networks, nodes represent elements of a construct or an entire construct, such as attitudes or symptoms of a mental disorder. Lines between nodes are called edges, which represent the direct association between a pair of nodes. The strength of association between nodes is called the edge weight; the thicker the edge, the stronger the association. Edges in psychological networks are typically undirected, which reflect the hypothesized multicausal relationships between system components. Positive relationships are typically denoted using blue edges, while red edges are used to indicate negative relationships. The layout of the network model can be selected by the researcher. Psychological networks are often plotted (by default) using the Fruchterman-Reingold algorithm (Fruchterman & Reingold, Reference Fruchterman and Reingold1991), which places nodes with stronger connections closer together, and nodes with weaker connections further apart. Besides visual inspection, network models can be analyzed on several different levels, depending on what the research questions are. For example, researchers typically analyze the network density if the overall interest is the network structure or focus on particular nodes and edges (Burger et al., Reference Burger, Isvoranu, Lunansky, Haslbeck, Epskamp, Hoekstra, Fried, Borsboom and Blanken2022).

The most common models used to estimate psychological networks are pairwise Markov random field (PMRF) models. Within PRMF models, Gaussian graphical models (GGM) are used with continuous multivariate data to estimate partial correlations between variables (Epskamp, Reference Epskamp2014). Partial correlation networks are undirected graphs, estimated by analyzing the strength of correlations between variables after controlling for the effect of other measured variables in the network (Hevey, Reference Hevey2018). As such, a psychological network can be viewed as a “nomological net, which functions as a specification of the phenomenological concepts or theoretical constructs of interest in a study, their observable manifestations, and the linkages between them” (Hiver & Al-Hoorie, Reference Hiver and Al-Hoorie2016, p. 747). Psychological networks created using cross-sectional data can therefore be viewed as a snapshot of the system at a given time.

Network analysis and CDST

Network analysis has some advantages over other relation-intensive methods used in CDST research. One advantage is that network analysis is more conceptually aligned with CDST compared to factor-based statistical techniques that are rooted in latent variable theory (Fried, Reference Fried2020). Originally developed by Spearman (Reference Spearman1904), factor models function under the theoretical assumption that a latent construct, such as intelligence or personality, can be measured through observable indicators (e.g., behavioral tests or questionnaire items). This means that there is a hypothesized unidirectional relationship from the latent construct to the observable indicator, whereby answers to questionnaire items or tests are thought to “reflect” the latent construct (Edward & Bagozzi, Reference Edwards and Bagozzi2000). In contrast, from a network perspective, psychological constructs “exist as systems where components mutually influence each other without the need to call on latent variables” (Guyon et al., Reference Guyon, Falissard and Kop2017, p. 2). Statistically, factor models and psychological networks are closely related, as both analyze the covariance between observed variables. The difference between each approach is their competing causal explanations (Fried, Reference Fried2020). As van Bork et al. (Reference van Bork, Rhemtulla, Waldorp, Kruis, Rezvanifar and Borsboom2019, p. 1) explain, “whereas latent variable approaches introduce unobserved common causes to explain the relations among observed variables, network approaches posit direct causal relations between observed variables.”

These two competing causal explanations are reflected in the choice of statistical model selected by the researcher. For example, factor-based techniques such as SEM or LGCM generate directed graphs, with edges from the latent construct to the observed indicators and/or between latent constructs, which are determined by the researcher a priori. Psychological network analysis is a more data-driven approach and produces an undirected graph with edges estimated between all nodes, better reflecting key CDST concepts such as multicausality and interconnectedness. This has already been noted in the field of psychology, where researchers working from a CDST perspective are using network analysis as an exploratory tool to better visualize the complex patterns of relations between variables of interest (Hilpert & Marchand, Reference Hilpert and Marchand2018; Sachisthal et al., Reference Sachisthal, Jansen, Peetsma, Dalege, van der Maas and Raijmakers2019; van der Maas et al., Reference van der Maas, Kan, Marsman and Stevenson2017).

In this article, we explore how network analysis could be used to model psychological constructs that influence language learning from a relation-intensive perspective. We provide two examples of psychological networks created using the datasets of existing studies that are publicly available online in support of Open Science practices. As the nested nature of educational phenomena can be analyzed at multiple levels (Marchand & Hilpert, Reference Marchand and Hilpert2018), our network models illustrate two different levels of analysis; with nodes at item level and composite level. The first example is a network model of L2 motivation made using the dataset from Hiver and Al-Hoorie’s (Reference Hiver and Al-Hoorie2020b) study on the role of vision in L2 motivation. This example explores how an individual difference construct such as L2 motivation can be modeled as a complex system, by analyzing relationships between the L2MSS at the item level. The second example is a network model of individual differences in native language ultimate attainment, made using the dataset from Dąbrowska’s (Reference Dąbrowska2018) study. The second example takes a wider relation-intensive perspective by analyzing interactions between multiple individual difference constructs at the composite level. Note that the authors of the original studies (Dąbrowska, Reference Dąbrowska2018; Hiver & Al-Hoorie, Reference Hiver and Al-Hoorie2020b) did not position their research within a CDST paradigm, and our reanalysis of their data is not a critique on their work.

We performed all statistical analyses using the open-source software R (R Core Team, 2020) and the R-packages qgraph (Epskamp et al., Reference Epskamp, Cramer, Waldorp, Schmittmann and Borsboom2012) and bootnet (Epskamp et al., Reference Epskamp, Borsboom and Fried2018a) in particular. The R code that we used to create these two examples is available in the online Supplementary Materials on our Open Science Framework (OSF) page. This article is not intended to serve as a tutorial in network analysis (for tutorials, we refer readers to Burger et al., Reference Burger, Isvoranu, Lunansky, Haslbeck, Epskamp, Hoekstra, Fried, Borsboom and Blanken2022; Epskamp et al., Reference Epskamp, Borsboom and Fried2018a; and Hevey, Reference Hevey2018). Rather, our overall aim is to raise awareness of this methodology and illustrate how it can be applied to model psychological constructs related to language learning from a relation-intensive CDST perspective. Within each example, we evaluate (a) the extent to which a network analysis of the datasets supports the same conclusions as the original authors and (b) whether network analysis can offer any additional insights to the original analyses.

Example 1

The first example was made using the dataset from Hiver and Al-Hoorie’s (Reference Hiver and Al-Hoorie2020b) study “Reexamining the Role of Cision in Second Language Motivation: A Preregistered Conceptual Replication of You, Dörnyei, and Csizér (Reference You, Dörnyei and Csizér2016).” Both Hiver and Al-Hoorie (Reference Hiver and Al-Hoorie2020b) and You et al. (Reference You, Dörnyei and Csizér2016) used SEM to explore interrelationships between components of the L2 Motivational Self System (L2MSS). The L2MSS is a theoretical paradigm that was developed by Dörnyei (Reference Dörnyei2005, Reference Dörnyei, Ellis and Larsen-Freeman2009) based on Possible Selves Theory (Markus & Nurius, Reference Markus and Nurius1986). The L2MSS is comprised of three components, each theorized to be a primary source of motivation to learn an L2: the Ideal L2 Self, the Ought-to L2 Self, and L2 Learning Experience. The ideal L2 self refers to learners’ internal desires and wishes to learn the L2, while the ought-to L2 self refers to learner’s perceived external duties and social pressures to learn the L2 (Dörnyei & Chan, Reference Dörnyei and Chan2013). L2 experience concerns learners’ attitudes toward learning, based on their experience of the learning process and environment. In addition to these three components, vision and imagery are also considered key aspects of the L2MSS, whereby motivation is viewed as “a function of the language learners’ vision of their desired future language selves” (Dörnyei & Chan, Reference Dörnyei and Chan2013, p. 437). Vision can be considered as a combination of imagery capacity and ideal selves and is typically measured by visual and auditory learning style preferences, and vividness of imagery capacity (You et al., Reference You, Dörnyei and Csizér2016). A number of studies have used SEM to explore the interrelationships between these motivational constructs and the extent to which the L2MSS can predict language learning or intended effort (Dörnyei & Chan, Reference Dörnyei and Chan2013; Hiver & Al-Hoorie, Reference Hiver and Al-Hoorie2020b; You et al., Reference You, Dörnyei and Csizér2016). However, as You et al. (Reference You, Dörnyei and Csizér2016, p. 97) have pointed out, “because the L2 Motivational Self System was originally proposed as a framework with no directional links among the three components, past empirical studies employing SEM have not been uniform in specifying these interrelationships.” For example, whereas some studies have presented a directed pathway from the ideal L2 self to L2 learning experience, other studies have reversed this relationship (for further details see You et al., Reference You, Dörnyei and Csizér2016).

Hiver and Al-Hoorie (Reference Hiver and Al-Hoorie2020b) conducted a conceptual replication and extension of You et al. (Reference You, Dörnyei and Csizér2016) to evaluate the role of vision in L2 motivation and to assess whether intended effort is an outcome or a predictor of motivation. They justified these aims in part due to the fact that You et al. did not test equivalent or competing models, which could be considered a form of confirmation bias. Hiver and Al-Hoorie also stressed the need for more robust research designs, and further replication of research on language motivation. Hiver and Al-Hoorie (Reference Hiver and Al-Hoorie2020b) collected data from 1297 L2 learners of English in secondary schools in South Korea. In addition to the same 10 scales of motivation and vision used by You et al., Hiver and Al-Hoorie also included two measures of L2 proficiency, midterm grades and final exam grades, which were analyzed as one variable called L2 achievement. To determine the number of underlying factors, they submitted the dataset to Mokken scaling analysis, confirmatory factor analysis, exploratory factor analysis, scree plot, optimal coordinates, and parallel analysis (Hiver & Al-Hoorie, Reference Hiver and Al-Hoorie2020b, p. 73). These analyses resulted in only four factors: visual style, ideal L2 self, ought-to L2 self, and intended effort. With these four factors and the measures of L2 achievement, Hiver and Al-Hoorie used SEM to test two competing causal models of vision and L2 motivation, where intended effort was either an antecedent or an outcome of motivation. Contrary to You et al. (Reference You, Dörnyei and Csizér2016), Hiver and Al-Hoorie hypothesized intended effort to be an antecedent of the ideal L2 self and the ought-to L2 self. In both competing models, vision (visual style) was considered a predictor of motivation, which was the same as You et al. (Reference You, Dörnyei and Csizér2016). Results showed that the model with intended effort as a predictor of motivation showed a better overall fit. Although this was contrary to You et al.’s model, Hiver and Al-Hoorie note that as their dataset and analyses differed greatly from the initial study, their model cannot be used to contradict You et al.’s model and call for further replication of research on the L2MSS.

In both studies (Hiver & Al-Hoorie, Reference Hiver and Al-Hoorie2020b; You et al., Reference You, Dörnyei and Csizér2016), the authors were interested in the relationships between the L2MSS, vision, and intended effort. By using SEM, they operationalized motivational constructs as latent variables, depicting hypothesized causal relationships between latent constructs with unidirectional arrows. However, in both studies, the authors note potential issues and limitations of using SEM to model interactions between motivational constructs. One issue relates to the theorized dynamic nature of the L2MSS and the multicausal relationships between motivational constructs. Possible Selves Theory was originally proposed to have dynamic qualities, whereby current and ideal selves are shaped by multiple ongoing processes (Henry, Reference Henry, Dörnyei, MacIntyre and Henry2014; Markus & Nurius, Reference Markus and Nurius1986). For example, Hiver and Al-Hoorie speculate that once an L2 learner puts in the effort and engages in the L2 learning process, “there will be a dynamic interaction between motivation … and task demands, leading to continuous recalibration of that motivational construct” (Reference Hiver and Al-Hoorie2020b, p. 86). One might question the extent to which SEM can effectively model these dynamic interactions, as SEM operationalizes motivational constructs as latent variables with a unidirectional causal relationship. In fact, both studies’ authors acknowledge that a further limitation of SEM is that it requires the researcher to specify the direction of the relationship between latent constructs. SEM can only test the theoretical model that is selected by the researcher, although equivalent or alternative models may likely exist. This issue was illustrated by Hiver and Al-Hoorie’s (Reference Hiver and Al-Hoorie2020b) two competing SEM models. As discussed earlier, there has already been discussion of how the L2MSS could be conceptualized as a complex system (Henry, Reference Henry, Dörnyei, MacIntyre and Henry2014, Reference Henry2017) and manifest as an attractor state (Henry, Reference Henry2017; Hiver, Reference Hiver, Dörnyei, MacIntyre and Henry2014; Waninge et al., Reference Waninge, Dörnyei and de Bot2014). From a CDST perspective, causal relationships between motivational constructs are not unidirectional, but reciprocal. To further investigate the relationship between motivation and intended effort, Hiver and Al-Hoorie (Reference Hiver and Al-Hoorie2020b) have encouraged researchers to consider using nonrecursive models where causality is reciprocal. In this first example, we illustrate how network analysis can be used to model the L2MSS as a complex system, with hypothesized reciprocal causation between motivational constructs with nodes at item level.

Network estimation and visualization

Figure 1 is a GGM of the L2MSS that we made with the dataset from Hiver and Al-Hoorie’s (Reference Hiver and Al-Hoorie2020b) study. In support of open science practices, they made their dataset and analyses publicly available through the OSF website. To allow for ease of comparison, we included the same variables in our network analysis as Hiver and Al-Hoorie’s SEM analyses, with the exception of visual style 2, which we explain in a later section. The network model in Figure 1 has nodes at item level, to better explore the interrelatedness of these motivational constructs, and the questionnaire items used to measure them. Table 1 contains information about which items correspond to each node.

Figure 1.

A network model of the L2MSS and L2 achievement.

Note: In this network model of the L2MSS, there are four motivational constructs: the ideal L2 self, the ought-to L2 self, intended effort, and visual style. Each node represents a questionnaire item. Ought-to L2 self has been measured with six questionnaire items, and the other motivational constructs with five questionnaire items. There are also two composite measures of L2 proficiency: L2_T1 (students’ mid-term grades) and L2_T2 (students’ final grades).

Table 1.

Legend of node labels

We chose GGM model selection (ggmModSelect function implemented in the bootnet R-package; Epskamp et al., Reference Epskamp, Borsboom and Fried2018a) as estimation method because of the large size of the dataset. Model search works by setting edges to zero and using a stepwise algorithm to continuously estimate the model until the optimal model is identified (Epskamp, Reference Epskamp2014). This technique uses Bayesian information criterion (BIC) obtained through estimating the maximum likelihood of sparsity.

In their original analyses, Hiver and Al-Hoorie (Reference Hiver and Al-Hoorie2020b) tested for normality and found that their data were not multivariate normal both in skewness and kurtosis. For this reason, the network was estimated using Spearman correlations (Epskamp, Reference Epskamp2014).

After estimating the model, we evaluated the stability of the network structure in terms of edge-weight accuracy using bootstrapping (see Epskamp et al., Reference Epskamp, Borsboom and Fried2018a for an in-depth explanation of bootstrapping in psychological networks). We used 5,000 samples of the nonparametric bootstrap to assess the variability of the edge-weights. This step should always be performed (Epskamp et al., Reference Epskamp, Borsboom and Fried2018a) as any interpretation of the network becomes limited if the network is unstable (Burger et al., Reference Burger, Isvoranu, Lunansky, Haslbeck, Epskamp, Hoekstra, Fried, Borsboom and Blanken2022). The results show a good overlap between the estimated model and the bootstrapped edge-weights, indicating that the network of Figure 1 is stable. The results of the nonparametric bootstrap can be viewed in the supplementary materials.

To assess the stability of the centrality coefficients, we again used bootstrapping. We used the case-dropping bootstrap, specifically developed to this aim (Epskamp et al., Reference Epskamp, Borsboom and Fried2018a). The case-dropping bootstrap assesses the stability of the order of centrality in subsets of the data, that is, after systematically dropping an increasing percentage of participants from the dataset. The centrality stability for “strength” centrality was estimated on a sample of 5,000 bootstraps, which resulted in a correlation stability coefficient (CS-coefficient) of 0.52 for the “strength” centrality. This is above the 0.5 (CS-coefficient) recommendation (Epskamp et al., Reference Epskamp, Borsboom and Fried2018a), which is why we conclude that the stability of node centrality in this network model is good. The results are presented in the supplementary materials.

Based on the centrality indices (see Figure 2), ought-to L2 self 2 is the most central component in the network model in Figure 1 in terms of node strength, followed by intended effort 5. The questionnaire items that correspond to these components are “Studying English is important to me to gain the approval of my peers” and “English would be still important to me in the future even if I failed in my English course.” This suggests that peer approval and perceived future importance of English play important roles in L2 motivation, as they are the strongest direct relationships with other motivational constructs in the system.

Figure 2.

Centrality plots for the L2MSS network.

Note: Centrality plots for the network model of the L2MSS. Centrality measures are shown as standardized z-scores. The raw centrality indices can be found in the online Supplementary Materials.

In addition to node strength, we also computed node centrality indices based on closeness and betweenness. The closeness index “indicates a short average distance of a specific node to all other nodes” (Hevey, Reference Hevey2018, p. 311). In the network model, the nodes with the highest closeness are the five intended effort nodes. This is an interesting find and indicates that intended effort may have an integral role in L2 motivation. Although the role of central components is not yet fully understood, it is thought that central nodes with high closeness are the most likely to both effect changes and be affected by changes in the system (Hevey, Reference Hevey2018). The third measure of centrality, betweenness, refers to how well one node connects other nodes together; nodes with high betweenness lie on the shortest path between pairs of nodes. As shown in Figure 2, the node with the highest betweenness is intended effort 5, followed by ought-to L2 self 5 and ideal L2 self 1.

Example 2

The second example illustrates how network analysis can be used to explore the relationships between multiple individual differences using the dataset from Dąbrowska’s (Reference Dąbrowska2018) study Experience, aptitude and individual differences in native language ultimate attainment. The dataset is publicly available online using the IRIS Database. The network model made from Dąbrowska’s (Reference Dąbrowska2018) dataset presents a different level of analysis from the previous example. In contrast to the network model in example 1, where each node represents a single questionnaire item, each node in the network model in Figure 3 represents a distinct variable measured by aggregated task scores. In the original study, Dąbrowska tested the assumption that adult native speakers tend to converge on the same grammar. She addressed this question by considering two opposing approaches to language acquisition: the usage-based perspective and the modular perspective. From a usage-based perspective, language abilities are thought to emerge out of interactions between general cognitive mechanisms and exposure to linguistic input (Ellis & Wulff, Reference Ellis, Wulff, Miller, Bayram, Rothman and Serratrice2018). From this perspective, “causal mechanisms interact iteratively to produce what appears to be structure” (Bybee & Beckner, Reference Bybee, Beckner, Heine and Narrog2009, p. 23). A usage-based approach is thus aligned with CDST, where linguistic knowledge emerges as a network of interrelated and interacting components. In contrast, from a modular perspective, language abilities are thought to stem from an innate universal grammar, whereby different types of language knowledge rely on autonomous modules within the mind (Tan & Shojamanesh, Reference Tan, Shojamanesh, Lutsenko and Lutsenko2019).

Figure 3.

A network model of individual differences in native language ultimate attainment.

Note: The nodes in this network are composite scores representing three measures of language proficiency and four individual differences. The three proficiency measures are receptive vocabulary, collocations, and grammatical comprehension. The four individual differences are nonverbal IQ, print exposure, language analytic ability, and years of education.

Dąbrowska (Reference Dąbrowska2018) discusses the plausibility of these two theories in connection with analyses of a dataset of 90 native English speakers’ performance on different linguistic and nonlinguistic tasks. She first analyzed the amount of individual variation on six tasks that measured grammatical comprehension, receptive vocabulary, collocations, nonverbal IQ, language analytic ability, and print exposure. Full details regarding which tests were used to measure each construct can be found in the original study. Dąbrowska then conducted Pearson correlations to explore interactions between the six aforementioned tasks as well as education (measured by number of years spent in education). This revealed several significant correlations between the measures of language knowledge as well as between other variables. To determine potential causes of individual differences in linguistic knowledge, Dąbrowska then conducted regression analyses with the four predictor variables (nonverbal IQ, language analytic ability, print exposure, and education) on each of the three measures of language knowledge. Overall, results showed that nonverbal IQ was strongly related to grammar and vocabulary, but not to collocations. Language analytic ability was also significantly related to grammar and vocabulary, as well as several other variables. Print exposure contributed more to vocabulary and collocations than to grammar, and education only weakly predicted each measure of language knowledge. Based on the significant correlations between the three measures of language knowledge and the fact that the same nonlinguistic variables predicted different areas of language knowledge, Dąbrowska concluded that these findings support a usage-based approach.

Adding age to the network model

To explore this idea, we expanded on the original study by adding the variable “age” to the network model. Participants’ ages were contained within the original dataset that is available online, but Dąbrowska did not include this variable in her analyses. It seemed particularly interesting to include this variable because Dąbrowska used the dataset to evaluate the usage-based approach and the modular approach to language acquisition. Age is an indirect measure of language experience. With first language development, it is logical to assume that the older a person is, the more exposure to linguistic input they have. Thus, from a usage-based perspective, we might hypothesize age to be significantly related to a number of other variables, including measures of language knowledge. The 90 participants in Dąbrowska’s study varied greatly in age, with a range of 17 to 65 and a mean age of 38. The network model in Figure 4 is a GGM of partial correlations between eight variables (the seven variables from the original analyses plus age). The model was made following the same procedures described for the network model in Figure 3. Similarly to the model without age, the bootstraps show wide 95% confidence intervals, meaning that the estimated network structure is also not very stable and the found links should be interpreted with care.

Figure 4.

A network model of individual differences in language knowledge, including age.

Note: In addition to the same variables as the network model in Figure 3, this model also has the variable age. Blue edges denote positive partial correlations and red edges denote negative partial correlations.

The network model in Figure 4 shows that age is partially correlated with all other variables. Out of the three measures of language knowledge (grammar, vocabulary, and collocations), age is most strongly linked to vocabulary (0.35), which is the strongest positive edge in the network.Footnote ¹ This is in line with previous studies which have shown that vocabulary is typically the only aspect of language knowledge that does not tend to decline with age (Reifegerste, Reference Reifegerste2021). As could be expected, age is also related to print exposure. Age has a negative association with nonverbal IQ and language analytic ability, which is consistent with previous research on cognitive decline and aging (Reifegerste, Reference Reifegerste2021). There is also a negative relationship between age and education, which is logical considering that the percentage of university attendance has increased over the years. Overall, the considerable effect that age has on the system provides tentative support for the usage-based approach. Using network analysis, we can visualize individual differences in language abilities as a complex system. While we cannot draw conclusions about emergent processes from cross-sectional data, based on this network model, we could speculate that vocabulary knowledge emerges out of interactions between cognitive abilities (nonverbal IQ) and other language experience (print exposure) throughout the lifespan (age).

In addition, controlling for age alters the partial correlations between other nodes. For example, in the model that includes age, vocabulary knowledge has a weak positive relationship with grammar (0.11) and language analytic ability (0.12), whereas in the model without age, these relationships are weaker (0.06 and 0.05). This indicates that age is a moderating variable. When controlling for age, the edge weight between grammar and language analytic ability is stronger (more positive) because age has negative partial correlations with grammar and language analytic ability. In a similar way, age also moderates the relationship between IQ and print exposure; these variables have an edge weight of –0.43 without age, and –0.26 when age is added to the model. In this case, the edge weight between IQ and print exposure is weaker (less positive) when controlling for age because age has negative partial correlations with IQ and print exposure.

Although the network models estimated in example 2 are not stable, and a larger sample size is necessary to draw any firm conclusions, our examples serve to illustrate how network analysis can be used to model multiple individual differences in language learning from a CDST perspective. The network analyses support the same conclusions as the original study (Dąbrowska, Reference Dąbrowska2018), but rather than analyzing unidirectional relationships between individual difference constructs, the undirected network models in example two depict hypothesized multicausal relationships between variables. By estimating partial correlations between all variables, network analyses also reveal a more complex network of relationships between variables than the original study’s regression analyses, offering additional insights into the data.