2.1 Interactional patterns in online CALL courses

Online courses in language-teacher training play a significant role in educating CALL practitioners in technology-enhanced language teaching (Canals & Al-Rawashdeh, Reference Canals and Al-Rawashdeh2019). Previous studies on CALL online courses have explored discussion forums to investigate student interaction patterns during course engagement. Interaction has been defined as students’ interaction with their peers through posting feedback messages or replies. For example, to gain a better understanding of users’ behavior, interaction, and academic performance in a language massive open online course (LMOOC), Martín-Monje, Castrillo and Mañana-Rodríguez (Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018) explored the connection between the patterns in students’ online interaction with peers and course engagement based on four descriptors of interaction (task submission, assessment submission, submission of peer assessment, and participation in the course forums). Their results indicated that the assessment and peer feedback submissions significantly differed across two groups. For instance, participants who completed the course were found to have submitted peer feedback more frequently than did those who did not complete the course. Regarding course success, the two groups displayed significant differences in all four descriptors. Findings from this study suggest that students’ interaction patterns in the LMOOC can be mapped by student course completion and success. That is, those who successfully completed the course were found to be more active in the online course.

Regarding participants’ engagement in online discussions, Sert and Aşık’s (Reference Sert and Aşık2020) corpus-based study investigated 111 preservice teachers’ engagement in online peer feedback in a blog. The participants posted their designed CALL materials and also evaluated each other’s materials on an online teacher education program in Turkey. Based on the blog posts, the authors compiled a corpus of online peer feedback on teaching and found that participants used constructive feedback with hedged evaluations to highlight particular features of CALL materials, such as learner interest and visual aspects. Their findings support Zhang, Liu, Chen, Wang and Huang’s (Reference Zhang, Liu, Chen, Wang and Huang2017) claim that community spaces such as Moodle and blogs enable teachers to learn from each other in an online community. Previous studies have also looked at computer-mediated communication (CMC) in online discussions. For example, Son (Reference Son2003) analyzed student–student interaction in an online discussion group and investigated students’ perception of online discussion to evaluate the use of group-based CMC discussions in an online discussion group established for an applied linguistic course. The results showed that students’ attitudes toward the CALL and CMC were positive, as they believed that such online discussion is a way of learning CALL and that CMC is a means of facilitating collaborative learning. Son further explained that language-teacher education needs to consider CMC because it provides teachers with practical experience in using CMC and collaborative communication for sharing ideas, resources, and questions.

2.2 Social network analysis

One way of identifying the interaction patterns of participants in an online discussion forum is through SNA. In SNA, the frequency of each participant’s online interactions with interlocutors is measured to examine and visualize the patterns of connectedness that arise between individuals (Prell, Reference Prell2012; Scott, Reference Scott1987). In other words, SNA allows us to understand how individuals are connected within a network by revealing who speaks to whom in an online discussion forum. Researchers interested in identifying central and peripheral participants in an asynchronous learning environment (e.g. Satar & Akcan, Reference Satar and Akcan2018; Shea et al., Reference Shea, Hayes, Uzuner-Smith, Gozza-Cohen, Vickers and Bidjerano2014; Wu, Gao & Zhang, Reference Wu, Gao and Zhang2014) have increasingly utilized SNA, especially with the growth in globalization due to technology.

SNA researchers primarily take two approaches: a whole-network/sociocentric network approach, which allows us to understand the overall level of relationships between participants in a group, and a personal network/egocentric network approach, which enables us to examine the amount of interaction between a participant and other individuals in a network (Prell, Reference Prell2012; Scott, Reference Scott1987). Specifically, in the personal network/egocentric network analysis, a centrality score is calculated to identify, for example, the central participant of a particular social network by measuring “in-degree” and “out-degree” values. In-degree centrality represents the number of replies individual participants receive for a post, which indicates popularity or prestige, whereas out-degree centrality represents the number of replies a participant has posted to other participants of the network, which indicates influence in a network (Prell, Reference Prell2012). In addition to in-degree and out-degree centrality, betweenness centrality scores can be obtained to identify participants who play a bridge role in the network. Using these measures in SNA, we can generate a variety of visuals, called sociograms, to illustrate the connectedness of a network in an online discussion forum. In this study, we took the personal network/egocentric network approach mainly to investigate the patterns of interaction among individual participants.

Satar and Akcan (Reference Satar and Akcan2018) employed SNA to examine Turkish preservice students’ interaction in a blended CALL course over two semesters. The results of their SNA showed that preservice participants were involved more in the spring than the fall semester, mainly due to the smaller group size, less teacher involvement in discussions, and the use of tasks and topics that were related to their discipline. The authors also suggested that the pattern of participation and engagement in online courses has increased over time. Similar to Satar and Akcan, Baek and Kim (Reference Baek and Kim2015) used SNA to explore the interaction patterns of more than 100,000 participants in two Korean online discussion communities over a 15-month period. They also reported that the nature of discussion topics affected participants’ interaction patterns. In line with these two studies, Wu et al. (Reference Wu, Gao and Zhang2014), examining the professional growth of three Chinese English teachers by using SNA to reveal their interactional patterns in an online community over 5 years, also found that time is a potentially significant indicator of improvement in the interactional patterns of teachers.

As Godwin-Jones (Reference Godwin-Jones2020) underlined, the widespread switch to distance learning is highly likely to be the new normal rather than a one-time occurrence. Therefore, investigating the nature of participants’ online engagement in distance learning is becoming more important. While the studies reviewed above have made invaluable contributions to this area, their community members lacked heterogeneity, which is crucial for the sustainable and healthy growth of an online professional development community (Wu et al., Reference Wu, Gao and Zhang2014). As Wu et al. (Reference Wu, Gao and Zhang2014) suggested, such heterogeneity might motivate participants to interact more actively because they want to share their ideas, feelings, and experiences with other community members who are from diverse contexts but have a common vision of professional pursuits. Thus, there is a need for investigation into the engagement patterns of English teachers with various teaching backgrounds from different parts of the world in an online professional development course. In this study, we address that need by examining the following research questions:

1. 1. What are the frequent patterns of engagement in the discussion posts, and how are they related to types of discussion prompts?

2. 2. What are the overall patterns of peer-to-peer interaction in the replies as revealed through SNA?

3. 3. What are the qualitative patterns of peer-to-peer interaction in the replies?

3.1 Context

3.1.4 Selection of focal participants

Because of the highest engagement profile observed in Winter 2020 Section 2, we further investigated nine focal participants’ engagement profiles in more depth in this particular section. The nine participants represent the following regions: Asia (n = 4), Europe (n = 3), and Latin America (n = 2). Table 1 lists the countries of and other information about these participants, who represent a variety of teaching contexts.

Table 1.

Focal participants

^a Number of hours a participant teaches in a week.

^b Average number of students in a class.

3.2 Data collection and analysis

Based on our analysis of participants’ online contributions, we used SNA to quantitatively examine the interaction patterns of the participants by calculating centrality scores for the personal/egocentric network. In this analysis, we used NodeXL, an extendible toolkit for network overview, discovery, and exploration implemented as an add-in to the Microsoft Excel 2010 spreadsheet software (Smith et al., Reference Smith, Shneiderman, Milic-Frayling, Rodrigues, Barash, Dunne, Capone, Perer and Gleave2009). In preparing the data for SNA, we entered in a spreadsheet a data set that contains only participant replies. We excluded from this analysis the original post entries because these posts addressed the prompt in each discussion thread rather than another individual, which is not considered as an interaction.

After the SNA, we also qualitatively analyzed the patterns of engagement in participants’ original posts and replies using the inductive thematic analysis method. The thematic analysis involves inductive identification of recurring themes and patterns, which follows the grounded theory approach that allows themes to emerge from the data (Strauss & Corbin, Reference Strauss, Corbin, Denzin and Lincoln1994). The coders thematically coded both original posts and replies retrieved from the online discussion forums. We adopted an idea (i.e. a complete thought identified in the response) as the unit of analysis. The coding was conducted in NVivo 12, following the four steps.

We conducted the initial coding based on the preliminary coding scheme developed by Kochem et al. (Reference Kochem, Muhammad, Karatay, Jin, Hegelheimer, Kruk and Peterson2020). Then we used an iterative process to empirically refine the existing coding scheme by adding new categories as they emerged from the data. The final coding scheme consisted of 15 primary patterns and 28 sub-patterns.

Three coders coded the original posts (4,265 words) and replies (2,933 words) retrieved from the Winter 2020 Section 2 course by assigning each coding unit all relevant codes in the coding scheme. The coders started by having a training session on the coding scheme. Then, to establish the reliability of coding, three coders initially coded one fourth of the data independently. The results of this pilot coding were used to calculate the intercoder reliability. The intraclass correlation coefficient for these data was .87 (p < .001), indicating a high level of agreement (Shrout & Fleiss, Reference Shrout and Fleiss1979). Then each of the three coders coded one third of the remaining discussions independently. Once the data were coded, the frequency of each primary and sub-pattern of engagement was calculated and analyzed to identify the recurring trends present in the data.

Next, to probe into how participant engagement profiles aligned with the primary goals of individual discussions, we categorized the eight discussions into different types based on a qualitative analysis of the prompts. These types (called prompt types) were determined according to the specific goals that are common across multiple discussions (e.g. Type 1 discussions: sharing teaching practices). Finally, we analyzed the discussion posts, identifying the most frequent engagement patterns for each prompt type.

4.1 Patterns of engagement and types of discussion prompts (RQ1)

4.1.1 Engagement patterns in the original posts

This section explores overall engagement patterns in the original posts. Table 2 shows the total frequencies of the engagement patterns, along with a description of each category.

Table 2.

Engagement patterns in the original posts

As Table 2 shows, the most frequent pattern across the eight discussions was sharing teaching practices, followed by making connections with lecture. The former pattern indicates cases when the participants shared their practices, approaches, and/or strategies of teaching a target skill, whereas the latter pattern indicates cases when the participants made connections to the course in general and/or to the specific concepts and technology tools covered in the lecture. The fact that those were the two most frequent engagement patterns identified in the original posts indicates that the participants actively engaged with the course content and brought in relevant practices from their professional contexts. These patterns align well with the primary goal of the discussion assignments in this GOC course: to “discuss the local feasibility and pedagogical uses of technology tools in the participants’ teaching contexts (GOC syllabus).” This finding clearly shows that, overall, the participants have achieved the main goal of the discussions.

4.1.2 Engagement patterns and types of discussion prompts

The analysis of the original posts across different types of discussions uncovers a more precise picture of how the participants’ engagement patterns align with the specific goals of individual discussions. The eight discussions were categorized into three prompt types based on the specific goals that are common across multiple discussions (see Supplementary Material B).

Figure 1 illustrates the frequencies of engagement patterns across the three types of prompts.

Figure 1.

Engagement patterns by prompt types.

Looking at the most frequent engagement patterns for each prompt type separately, we first note that these patterns from the participants’ posts align with the primary goals of each prompt type. For instance, Type 1 discussions tended to elicit sharing teaching practices, which is a pattern highly relevant to the main goal of this type. Similarly, the most common patterns in Type 2 discussions were making connections with lecture and plan for integrating tools in future classes, which are directly connected to the objectives of this type. Type 3 discussions, however, mostly engaged the participants with reporting results (e.g. a portfolio of projects in the Module 8 discussion). We also note that making connections with lecture is a relatively common pattern in all three types. This finding indicates that the participants consistently commented on the course content when they responded to the discussion prompts, regardless of the specific goals of individual discussions. This finding is clearly a desired outcome because we expect participants to reflect on the course content and connect it with their previous and future teaching practices.

Another interesting trend is that Module 4 and Module 8 discussions elicited more occurrences of the interacting with other participants pattern than did other discussions (56% of total instances were elicited in Modules 4 and 8). This pattern reflects cases in which participants interacted with their peer participants by asking questions or providing suggestions (e.g. “What online resource or tools do you use in your teaching?”). This finding could be attributed to the specific directions about commenting on peer participants’ posts within the prompt for these two discussions. For example, the following prompt for the Module 8 discussion includes an explicit statement about how to interact with other participants:

Please comment on at least one portfolio by replying to that student and sharing a compliment or a constructive suggestion for improvement. (Module 8)

This statement is comparable to a more general statement presented in all other discussion prompts (i.e. “Please comment on at least one classmate’s post”). This finding suggests that providing a specific guideline within a discussion prompt could elicit a higher level of engagement from the participants.

4.2 SNA analysis of interaction patterns (RQ2)

To answer the second research question, we used SNA to identify both statistically and visually the interaction patterns of the course participants. Table 3 presents the centrality measures – degree, out-degree (number of replies a participant posted), in-degree (number of replies a participant received), and betweenness – that give a general description of the GOC discussion network.

Table 3.

SNA centrality measures

The degree, out-degree, and in-degree centrality measures indicate that the most active participants in the discussion forum were participant09 and participant01. Degree centrality is a simple count of the total number of connections linked to a participant, which can also be viewed as an overall popularity measure (Hansen, Shneiderman & Smith, Reference Hansen, Shneiderman and Smith2010). In this sense, participant09 and participant01, with degrees of 15, were the most popular participants because they were directly connected to 15 of the 22 course participants. In comparison, participant05 was the least popular participant because s/he had a direct connection with only eight other participants throughout the 8-week period. Also, participant09 and participant01, who each had an out-degree of 15, were the most influential course participants, and participant09, who had an in-degree of 9, was the most popular one. The mean degree, out-degree, and in-degree measures for the nine participants in the course were 11.11, 8.33, and 7.01, respectively. Out of these nine participants, three had an above-average degree, four had above-average out-degrees, and five had in-degree scores above the mean in-degree scores.

Although popularity is significant, betweenness centrality is a measure that captures a completely different type of importance (Hansen et al., Reference Hansen, Shneiderman and Smith2010), revealing individuals who play a bridge role in a network. Betweenness centrality scores can also give us an idea about how much removing a participant would affect the interaction between other participants in a network. Table 3 indicates that participant09, with a betweenness centrality score of 33.9, played a bridge role for those 15 participants s/he interacted with in the course. Himelboim, Golan, Moon and Suto (Reference Himelboim, Golan, Moon and Suto2014) defined participants with high betweenness and high in-degree centrality values as social mediators, who “mediate the relations between an organization and its publics through social media” (p. 361). In this regard, we can consider participant09 as the primary social mediator of this course. The following examples illustrate how participant09 played a bridging role among the group members by helping participantX and participant01 connect with each other. First, participant09 posted the following comment under participant01’s original post:

Then, participant09 received a suggestion to his/her post-listening activity suggestion from participantX:

Finally, participant01 thanks both participant09 and participantX for their contributions to his/her post:

As seen in these examples, participant09 initiated another discussion under someone else’s post by suggesting a post-listening activity, which attracted participantX’s attention and established a bridge between these two participants.

Illustrating the trend manifest in Table 3, Figure 2 is a sociogram of the interactional patterns between the participants in the eight discussions throughout the course. In this figure, circle width represents the amount of interaction a participant had with other participants. The arrows show the direction of the posts. As Figure 2 shows, participant09 and participant01 were highly active and central in the course interactions, whereas participant05 and participant08 were much less interactive (as represented by a more outward appearance in the network).

Figure 2.

Sociogram of the GOC discussions over an 8-week period. Unspecified data points represent the remaining course participants

Figure 3 and Figure 4 compare the egocentric networks of the most interactive (participant09) and the least interactive (participant05) participants throughout the course, further revealing the interactional patterns of these two participants. The thickness of the arrow between two participants represents the level of interaction. For example, the thickest arrow in Figure 3 shows that participant09 interacted the most with participant01 in Modules 2, 6, and 8. In contrast, the thin arrows in Figure 4 show that participant05 had only a low level of interaction with just eight different participants throughout the course.

Figure 3.

Egocentric network of participant09 throughout the course

Figure 4.

Egocentric network of particpant05 throughout the course

Although SNA elicits some interactional patterns in the discussion forum, it does not by itself provide a full picture of how and why we see these patterns. Therefore, along with these findings, we conducted a content analysis, foregrounded by a case study design (Yin, Reference Yin2004), of the replies in the GOC discussions to better understand how and why we see these patterns.

4.3 Qualitative content analysis of interaction patterns (RQ3)

To answer the third research question, we conducted thematic content analysis of the replies to examine the qualitative patterns of interaction among the nine participants. Table 4 summarizes the total frequencies of the themes of peer-to-peer interaction, along with a brief description of each category (see Supplementary Material C for sample responses).

Table 4.

Peer-to-peer interaction in the replies

The most frequent theme was appreciation – cases in which the participants expressed appreciation to their peer participants. In most cases, the participants appreciated others for sharing new ideas and resources. Another frequent pattern was when the participants complimented their peers’ ideas by providing specific feedback based on their own evaluations and perspectives. Participants also tended to express appreciation by posting expressions of encouragement (e.g. “you are really an early bird!”), which were mostly accompanied by the use of positive evaluative markers such as “amazing,” “helpful,” “interesting,” or “impressed.”

These patterns of appreciation indicate that the participants were actively engaged in providing positive feedback to support each other and help improve each other’s ideas. This is a pattern that is frequently observed in previous studies on peer feedback in online discussions (Farr & Riordan, Reference Farr and Riordan2012; Sert & Aşık, Reference Sert and Aşık2020). Such expressions of encouragement were most frequently observed in participant09, whom our SNA identified as the most interactive participant. Based on having the highest betweenness and degree centrality values (see Table 3), participant09 was identified as the primary social mediator of the course, mediating the relations and interaction between the course participants (Himelboim et al., Reference Himelboim, Golan, Moon and Suto2014). A closer analysis indicated that participant09 started almost all of his/her replies by sharing expressions of encouragement such as this:

In contrast, participant05, whom our SNA identified as the least active participant, rarely used explicit expressions of appreciation.

The second most frequent theme was initiating engagements – when participants initiated engagement with other participants by asking direct or indirect questions. The analysis of these cases further revealed four functions of initiating engagements: (a) ask for clarification, (b) request additional information, (c) ask probing questions, and (d) prompt actions. Participants frequently asked for clarification or requested additional information. Such questions urged participants to clarify confusion in their posts or to share more details on a particular issue. Some participants asked probing questions (Garrison, Anderson & Archer, Reference Garrison, Anderson and Archer2001), namely, rhetorical questions intended to help the writer of the original post to think more deeply about the issue at hand. For example, in the following example, participant01 asks a series of probing questions in the reply to raise potential issues in using ESL lab for lower-level learners:

In another example, playing the social mediator role in the group, participant09 encourages the writer of the original post to think more deeply about how to further apply the course knowledge (i.e. technological tools learned from the lecture) to his/her own teaching context:

Finally, some participants tended to ask questions to prompt the writer of the original post to take specific actions (e.g. share a supplemental link). For instance, in the following example, the participant is prompting the writer to share additional material (i.e. link to the audio) so other course participants can engage in a deeper analysis and evaluation of his/her activity:

These instances of asking probing and prompting questions suggest that participants did not just provide positive feedback, as displayed in the pattern of appreciation, but also engaged in an advanced level of critical thinking through their continued interaction with peer participants in the replies. Such an advanced level of interaction was a feature that distinguished participant09 from less interactive participants. This finding provides additional qualitative evidence that participant09 served as an important mediator in the peer-to-peer interaction, promoting their peers’ cognitive development by asking questions to prompt a deeper level of critical thinking and actions.

The third most frequent theme, showing empathy, indicates cases in which the participants empathized with other participants’ experiences, thoughts, or feelings. In the majority of these cases, the participants expressed empathy with other participants’ frustration, sharing specific challenges or constraints in implementing technology tools into their teaching contexts. In other cases, participants showed empathy for others’ teaching methods and perspectives. We observed that several participants, especially participant09 as in the following example, established rapport and felt connected with their peer participants (“we all,” “we, together”):

Such feelings of group commitment have been found to promote participation and collaboration in ACMC (Zhao, Sullivan & Mellenius, Reference Zhao, Sullivan and Mellenius2014). We could see evidence that the explicit expression of a sense of community and connectedness promotes interaction among our course participants.

Finally, the least frequent theme that we found, response to a question, indicates the cases in which participants responded to other participants’ replies to their own posts (i.e. response to a follow-up question). The fact that this pattern occurred less frequently in the participants’ replies than the other patterns did (see Table 4) is surprising, given the high frequency of the initiating engagement theme. We observed that three of the six responses to a question were from participant09, as in the following example:

This is another example that shows that participant09 played an important bridge role in the interaction among course participants by encouraging continued communication with other participants. Except for participant09 and participant01, other participants rarely showed such a pattern, while many participants brought up new questions requesting a further response. Such a discrepancy could have been an effect of the discussion prompt. That is, since the participants were asked to reply to peer participants’ posts at least once, they tended not to go back to check the follow-up questions once they met the minimum requirement. The closing date for discussions could have been another reason for the lack of follow-up discussions. Thus, the discussion prompts might need to be refined in order to facilitate participants’ engagement in a continued discussion with peer participants.