Hostname: page-component-586b7cd67f-rdxmf Total loading time: 0 Render date: 2024-11-25T23:11:11.991Z Has data issue: false hasContentIssue false

Exploring global online course participants’ interactions: Value of high-level engagement

Published online by Cambridge University Press:  14 February 2022

Haeyun Jin
Affiliation:
Iowa State University, United States ([email protected])
Yasin Karatay
Affiliation:
Iowa State University, United States ([email protected])
Fatemeh Bordbarjavidi
Affiliation:
Iowa State University, United States ([email protected])
Junghun Yang
Affiliation:
Iowa State University, United States ([email protected])
Timothy Kochem
Affiliation:
Iowa State University, United States ([email protected])
Ananda Astrini Muhammad
Affiliation:
Iowa State University, United States ([email protected])
Volker Hegelheimer
Affiliation:
Iowa State University, United States ([email protected])
Rights & Permissions [Opens in a new window]

Abstract

Participation in online courses has become essential for training language professionals in under-resourced contexts with skills in computer-assisted language learning (CALL) (Godwin-Jones, 2014). Most online CALL courses use asynchronous computer-mediated communication (ACMC) to facilitate meaningful learning for participants. Although participants’ sustained engagement with ACMC is the target, global realities of participants interfere with their participation levels. This article investigates participants’ engagement profiles in asynchronous online discussions in an 8-week CALL-based global online course developed and implemented by a team at Iowa State University. Using a case study approach, nine focal participants’ engagement profiles have been analyzed in terms of identifying patterns of engagement in the discussion posts and their relation to the types of discussion prompts. Then, social network analysis (SNA) and thematic analysis were employed to investigate patterns of interaction among the participants in the replies. The results indicated that engagement patterns observed in discussion posts overall aligned with the primary goals of prompt types. SNA further identified two participants as social mediators to connect participants with each other. These findings are significant in that they suggest the effectiveness of using ACMC to promote co-construction of knowledge for a global audience. This article also provides implications regarding the design of discussion prompts to help maximize participant engagement with course content.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of European Association for Computer Assisted Language Learning

1. Introduction

The proliferation of online courses has had a profound global impact on the remote delivery of learning and teaching. Webinars, massive open online courses (MOOCs), and other delivery models have provided access to high-quality content across the globe (Godwin-Jones, Reference Godwin-Jones2014). This development extends to the field of computer-assisted language learning (CALL), in which specialized training has been a focal point for decades (e.g. Guichon & Hauck, Reference Guichon and Hauck2011; Hong, Reference Hong2010; Hubbard, Reference Hubbard2008; Torsani, Reference Torsani2016). Even still, professional development opportunities to learn skills in CALL remain relatively limited (Kessler & Hubbard, Reference Kessler, Hubbard, Chapelle and Sauro2017). One such opportunity to develop these skills is a global online course (GOC) provided by the Online Professional English Network (OPEN; https://exchanges.state.gov/non-us/program/OPEN-Program/details), formerly known as the American English E-Teacher Program. The GOC is designed for language teachers around the world with limited access to professional development opportunities and provides them with the opportunity to engage in higher-order processing of information by contributing to online discussions. As with most online learning environments, this GOC uses asynchronous computer-mediated communication (ACMC) to ensure meaningful learning for participants.

Researchers have agreed on the benefit of using ACMC to facilitate higher levels of learning in online courses (e.g. LaPointe & Gunawardena, Reference LaPointe and Gunawardena2004). These virtual environments provide meaningful learning opportunities because participants can interact not only with the course content but also with other colleagues who bring different levels of knowledge and expertise to the course (Coleman, Hampel, Hauck & Stickler, Reference Coleman, Hampel, Hauck, Stickler, Levine, Phipps and Blythe2012; McCrory, Putnam & Jansen, Reference McCrory, Putnam and Jansen2008). Using the ACMC learning environment also provides a social benefit to participants because they can receive emotional support and motivation by communicating with colleagues who face similar issues or challenges of their own (Mason & Weller, Reference Mason and Weller2000). Although extensive research has been conducted to demonstrate such benefits, it is less clear whether language professionals’ engagement in ACMC leads to effective and meaningful learning in professional development courses in CALL, such as the GOC. A handful of studies have investigated participants’ engagement in online teacher training courses to inform the improvement of course design (e.g. Nami, Marandi & Sotoudehnama, Reference Nami, Marandi and Sotoudehnama2018). These studies, however, mostly targeted teacher trainees and preservice teachers from similar backgrounds (e.g. same nationality or institution), so their findings are harder to generalize to online courses with a more heterogeneous participant group. Less is known about ACMC between professionals who are from different countries and have varying degrees of expertise and resources.

To address such gaps in research, this study investigates participants’ engagement profiles in asynchronous online discussions in a GOC, Using Educational Technology in the English Language Classroom. Although this course targets sustained high levels of engagement, the global realities of participants, who also have a job or two and a family to support, interfere with these engagement levels. Thus, we aimed to explore the effectiveness of the professional development opportunities provided in our GOC course by analyzing participants’ patterns of engagement with the course content and peer-to-peer interaction. First, we analyzed frequent patterns of engagement in the discussion posts to explore participants’ demonstration of the course content knowledge and how they are related to different types of discussion prompts. Then, we used social network analysis (SNA) and qualitative thematic analysis to investigate patterns of interaction among participants in the replies. Given that this study targets language professionals representing a mix of low-, medium-, and high-resource environments (i.e. from three different continents), our results provide valuable insights about the effectiveness of using ACMC to promote professional development for a global audience. Our study also provides implications regarding the design of discussion prompts in online CALL courses to help maximize student engagement with the course content.

2. Literature review

2.1 Interactional patterns in online CALL courses

Online courses in language-teacher training play a significant role in educating CALL practitioners in technology-enhanced language teaching (Canals & Al-Rawashdeh, Reference Canals and Al-Rawashdeh2019). Previous studies on CALL online courses have explored discussion forums to investigate student interaction patterns during course engagement. Interaction has been defined as students’ interaction with their peers through posting feedback messages or replies. For example, to gain a better understanding of users’ behavior, interaction, and academic performance in a language massive open online course (LMOOC), Martín-Monje, Castrillo and Mañana-Rodríguez (Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018) explored the connection between the patterns in students’ online interaction with peers and course engagement based on four descriptors of interaction (task submission, assessment submission, submission of peer assessment, and participation in the course forums). Their results indicated that the assessment and peer feedback submissions significantly differed across two groups. For instance, participants who completed the course were found to have submitted peer feedback more frequently than did those who did not complete the course. Regarding course success, the two groups displayed significant differences in all four descriptors. Findings from this study suggest that students’ interaction patterns in the LMOOC can be mapped by student course completion and success. That is, those who successfully completed the course were found to be more active in the online course.

Regarding participants’ engagement in online discussions, Sert and Aşık’s (Reference Sert and Aşık2020) corpus-based study investigated 111 preservice teachers’ engagement in online peer feedback in a blog. The participants posted their designed CALL materials and also evaluated each other’s materials on an online teacher education program in Turkey. Based on the blog posts, the authors compiled a corpus of online peer feedback on teaching and found that participants used constructive feedback with hedged evaluations to highlight particular features of CALL materials, such as learner interest and visual aspects. Their findings support Zhang, Liu, Chen, Wang and Huang’s (Reference Zhang, Liu, Chen, Wang and Huang2017) claim that community spaces such as Moodle and blogs enable teachers to learn from each other in an online community. Previous studies have also looked at computer-mediated communication (CMC) in online discussions. For example, Son (Reference Son2003) analyzed student–student interaction in an online discussion group and investigated students’ perception of online discussion to evaluate the use of group-based CMC discussions in an online discussion group established for an applied linguistic course. The results showed that students’ attitudes toward the CALL and CMC were positive, as they believed that such online discussion is a way of learning CALL and that CMC is a means of facilitating collaborative learning. Son further explained that language-teacher education needs to consider CMC because it provides teachers with practical experience in using CMC and collaborative communication for sharing ideas, resources, and questions.

2.2 Social network analysis

One way of identifying the interaction patterns of participants in an online discussion forum is through SNA. In SNA, the frequency of each participant’s online interactions with interlocutors is measured to examine and visualize the patterns of connectedness that arise between individuals (Prell, Reference Prell2012; Scott, Reference Scott1987). In other words, SNA allows us to understand how individuals are connected within a network by revealing who speaks to whom in an online discussion forum. Researchers interested in identifying central and peripheral participants in an asynchronous learning environment (e.g. Satar & Akcan, Reference Satar and Akcan2018; Shea et al., Reference Shea, Hayes, Uzuner-Smith, Gozza-Cohen, Vickers and Bidjerano2014; Wu, Gao & Zhang, Reference Wu, Gao and Zhang2014) have increasingly utilized SNA, especially with the growth in globalization due to technology.

SNA researchers primarily take two approaches: a whole-network/sociocentric network approach, which allows us to understand the overall level of relationships between participants in a group, and a personal network/egocentric network approach, which enables us to examine the amount of interaction between a participant and other individuals in a network (Prell, Reference Prell2012; Scott, Reference Scott1987). Specifically, in the personal network/egocentric network analysis, a centrality score is calculated to identify, for example, the central participant of a particular social network by measuring “in-degree” and “out-degree” values. In-degree centrality represents the number of replies individual participants receive for a post, which indicates popularity or prestige, whereas out-degree centrality represents the number of replies a participant has posted to other participants of the network, which indicates influence in a network (Prell, Reference Prell2012). In addition to in-degree and out-degree centrality, betweenness centrality scores can be obtained to identify participants who play a bridge role in the network. Using these measures in SNA, we can generate a variety of visuals, called sociograms, to illustrate the connectedness of a network in an online discussion forum. In this study, we took the personal network/egocentric network approach mainly to investigate the patterns of interaction among individual participants.

Satar and Akcan (Reference Satar and Akcan2018) employed SNA to examine Turkish preservice students’ interaction in a blended CALL course over two semesters. The results of their SNA showed that preservice participants were involved more in the spring than the fall semester, mainly due to the smaller group size, less teacher involvement in discussions, and the use of tasks and topics that were related to their discipline. The authors also suggested that the pattern of participation and engagement in online courses has increased over time. Similar to Satar and Akcan, Baek and Kim (Reference Baek and Kim2015) used SNA to explore the interaction patterns of more than 100,000 participants in two Korean online discussion communities over a 15-month period. They also reported that the nature of discussion topics affected participants’ interaction patterns. In line with these two studies, Wu et al. (Reference Wu, Gao and Zhang2014), examining the professional growth of three Chinese English teachers by using SNA to reveal their interactional patterns in an online community over 5 years, also found that time is a potentially significant indicator of improvement in the interactional patterns of teachers.

As Godwin-Jones (Reference Godwin-Jones2020) underlined, the widespread switch to distance learning is highly likely to be the new normal rather than a one-time occurrence. Therefore, investigating the nature of participants’ online engagement in distance learning is becoming more important. While the studies reviewed above have made invaluable contributions to this area, their community members lacked heterogeneity, which is crucial for the sustainable and healthy growth of an online professional development community (Wu et al., Reference Wu, Gao and Zhang2014). As Wu et al. (Reference Wu, Gao and Zhang2014) suggested, such heterogeneity might motivate participants to interact more actively because they want to share their ideas, feelings, and experiences with other community members who are from diverse contexts but have a common vision of professional pursuits. Thus, there is a need for investigation into the engagement patterns of English teachers with various teaching backgrounds from different parts of the world in an online professional development course. In this study, we address that need by examining the following research questions:

  1. 1. What are the frequent patterns of engagement in the discussion posts, and how are they related to types of discussion prompts?

  2. 2. What are the overall patterns of peer-to-peer interaction in the replies as revealed through SNA?

  3. 3. What are the qualitative patterns of peer-to-peer interaction in the replies?

3. Methodology

This study employs a descriptive case study approach with embedded quantitative data in order to describe a phenomenon and the real-life context in which it occurred using a variety of data sources (Yin, Reference Yin2004, Reference Yin2014). For this study, we initially collected and analyzed quantitative data to investigate interaction patterns in the form of discussion posts and replies. Then we conducted a qualitative data analysis to explain, validate, and provide a more in-depth perspective and converged the results of both the quantitative and qualitative data analyses in an attempt to understand the overall case. The nature of our case study methodology enabled us to look at the case through various lenses by employing SNA and thematic analysis in order to describe the overall course engagement profile of our participants.

3.1 Context

3.1.1 Global online course

In partnership with the U.S. Department of State and FHI 360, a team of faculty members and graduate students at Iowa State University developed and implemented an 8-week online CALL course titled Using Educational Technology in the English Language Classroom. This GOC, implemented in the Canvas by Instructure learning management system, is designed for English language professionals and educators around the world. Participants are English language teachers from different countries with varying backgrounds and levels of proficiency in the English language and in technology. They teach in secondary schools, primary schools, universities, supplementary English programs, teacher training institutions, or adult vocational schools.

This course covers six skills of English language learning: vocabulary, grammar, reading, speaking, listening, and writing. In each of these areas, the course aims to acquaint participants with technology tools and help them learn how to integrate pedagogical knowledge and skills with such tools in order to enhance the learning and teaching of English. Another objective is to help participants engage in professional communication through interacting with their peer participants, educational and technology experts, teaching assistants, and mentors of the program. For further information on the course and course content, see Kochem, Muhammad, Karatay, Jin and Hegelheimer (Reference Kochem, Muhammad, Karatay, Jin, Hegelheimer, Kruk and Peterson2020).

3.1.2 Participants

Yin (Reference Yin2014) suggested that a large number of eligible participants in a descriptive case study warrants a two-stage screening procedure: The first stage is to collect relevant quantitative data about the entire sample, whereas the second stage is to limit the number of research participants to a reasonable number for an in-depth investigation. In this sense, using the descriptive approach allowed us to reveal an overall pattern of the course engagement profile across all sections and then explain how and why there was a specific participant engagement pattern.

3.1.3 Initial analysis of the entire sample

Initially, we looked at the written records of 225 English language teachers’ asynchronous online discussions over the 8-week courses in summer 2019, fall 2019, and winter 2020. We used a Python script to extract the posts and replies from the Canvas learning management system, which allowed us to save the contents of each post in text files, along with useful metadata, such as word count, type (original post or reply), discussion forum, and course section. Then we calculated the total number of entries and word counts of the discussion posts in a total of nine sections over three semesters in order to analyze participants’ online contributions. This initial analysis indicated that participants in the winter 2020 course, especially Section 2, engaged in discussions more than participants did in the other courses in terms of the length of original posts and replies and the average number of original posts and replies per discussion (see Supplementary Material A).

3.1.4 Selection of focal participants

Because of the highest engagement profile observed in Winter 2020 Section 2, we further investigated nine focal participants’ engagement profiles in more depth in this particular section. The nine participants represent the following regions: Asia (n = 4), Europe (n = 3), and Latin America (n = 2). Table 1 lists the countries of and other information about these participants, who represent a variety of teaching contexts.

Table 1. Focal participants

a Number of hours a participant teaches in a week.

b Average number of students in a class.

3.2 Data collection and analysis

Based on our analysis of participants’ online contributions, we used SNA to quantitatively examine the interaction patterns of the participants by calculating centrality scores for the personal/egocentric network. In this analysis, we used NodeXL, an extendible toolkit for network overview, discovery, and exploration implemented as an add-in to the Microsoft Excel 2010 spreadsheet software (Smith et al., Reference Smith, Shneiderman, Milic-Frayling, Rodrigues, Barash, Dunne, Capone, Perer and Gleave2009). In preparing the data for SNA, we entered in a spreadsheet a data set that contains only participant replies. We excluded from this analysis the original post entries because these posts addressed the prompt in each discussion thread rather than another individual, which is not considered as an interaction.

After the SNA, we also qualitatively analyzed the patterns of engagement in participants’ original posts and replies using the inductive thematic analysis method. The thematic analysis involves inductive identification of recurring themes and patterns, which follows the grounded theory approach that allows themes to emerge from the data (Strauss & Corbin, Reference Strauss, Corbin, Denzin and Lincoln1994). The coders thematically coded both original posts and replies retrieved from the online discussion forums. We adopted an idea (i.e. a complete thought identified in the response) as the unit of analysis. The coding was conducted in NVivo 12, following the four steps.

We conducted the initial coding based on the preliminary coding scheme developed by Kochem et al. (Reference Kochem, Muhammad, Karatay, Jin, Hegelheimer, Kruk and Peterson2020). Then we used an iterative process to empirically refine the existing coding scheme by adding new categories as they emerged from the data. The final coding scheme consisted of 15 primary patterns and 28 sub-patterns.

Three coders coded the original posts (4,265 words) and replies (2,933 words) retrieved from the Winter 2020 Section 2 course by assigning each coding unit all relevant codes in the coding scheme. The coders started by having a training session on the coding scheme. Then, to establish the reliability of coding, three coders initially coded one fourth of the data independently. The results of this pilot coding were used to calculate the intercoder reliability. The intraclass correlation coefficient for these data was .87 (p < .001), indicating a high level of agreement (Shrout & Fleiss, Reference Shrout and Fleiss1979). Then each of the three coders coded one third of the remaining discussions independently. Once the data were coded, the frequency of each primary and sub-pattern of engagement was calculated and analyzed to identify the recurring trends present in the data.

Next, to probe into how participant engagement profiles aligned with the primary goals of individual discussions, we categorized the eight discussions into different types based on a qualitative analysis of the prompts. These types (called prompt types) were determined according to the specific goals that are common across multiple discussions (e.g. Type 1 discussions: sharing teaching practices). Finally, we analyzed the discussion posts, identifying the most frequent engagement patterns for each prompt type.

4. Results

4.1 Patterns of engagement and types of discussion prompts (RQ1)

4.1.1 Engagement patterns in the original posts

This section explores overall engagement patterns in the original posts. Table 2 shows the total frequencies of the engagement patterns, along with a description of each category.

Table 2. Engagement patterns in the original posts

As Table 2 shows, the most frequent pattern across the eight discussions was sharing teaching practices, followed by making connections with lecture. The former pattern indicates cases when the participants shared their practices, approaches, and/or strategies of teaching a target skill, whereas the latter pattern indicates cases when the participants made connections to the course in general and/or to the specific concepts and technology tools covered in the lecture. The fact that those were the two most frequent engagement patterns identified in the original posts indicates that the participants actively engaged with the course content and brought in relevant practices from their professional contexts. These patterns align well with the primary goal of the discussion assignments in this GOC course: to “discuss the local feasibility and pedagogical uses of technology tools in the participants’ teaching contexts (GOC syllabus).” This finding clearly shows that, overall, the participants have achieved the main goal of the discussions.

4.1.2 Engagement patterns and types of discussion prompts

The analysis of the original posts across different types of discussions uncovers a more precise picture of how the participants’ engagement patterns align with the specific goals of individual discussions. The eight discussions were categorized into three prompt types based on the specific goals that are common across multiple discussions (see Supplementary Material B).

Figure 1 illustrates the frequencies of engagement patterns across the three types of prompts.

Figure 1. Engagement patterns by prompt types.

Looking at the most frequent engagement patterns for each prompt type separately, we first note that these patterns from the participants’ posts align with the primary goals of each prompt type. For instance, Type 1 discussions tended to elicit sharing teaching practices, which is a pattern highly relevant to the main goal of this type. Similarly, the most common patterns in Type 2 discussions were making connections with lecture and plan for integrating tools in future classes, which are directly connected to the objectives of this type. Type 3 discussions, however, mostly engaged the participants with reporting results (e.g. a portfolio of projects in the Module 8 discussion). We also note that making connections with lecture is a relatively common pattern in all three types. This finding indicates that the participants consistently commented on the course content when they responded to the discussion prompts, regardless of the specific goals of individual discussions. This finding is clearly a desired outcome because we expect participants to reflect on the course content and connect it with their previous and future teaching practices.

Another interesting trend is that Module 4 and Module 8 discussions elicited more occurrences of the interacting with other participants pattern than did other discussions (56% of total instances were elicited in Modules 4 and 8). This pattern reflects cases in which participants interacted with their peer participants by asking questions or providing suggestions (e.g. “What online resource or tools do you use in your teaching?”). This finding could be attributed to the specific directions about commenting on peer participants’ posts within the prompt for these two discussions. For example, the following prompt for the Module 8 discussion includes an explicit statement about how to interact with other participants:

Please comment on at least one portfolio by replying to that student and sharing a compliment or a constructive suggestion for improvement. (Module 8)

This statement is comparable to a more general statement presented in all other discussion prompts (i.e. “Please comment on at least one classmate’s post”). This finding suggests that providing a specific guideline within a discussion prompt could elicit a higher level of engagement from the participants.

4.2 SNA analysis of interaction patterns (RQ2)

To answer the second research question, we used SNA to identify both statistically and visually the interaction patterns of the course participants. Table 3 presents the centrality measures – degree, out-degree (number of replies a participant posted), in-degree (number of replies a participant received), and betweenness – that give a general description of the GOC discussion network.

Table 3. SNA centrality measures

The degree, out-degree, and in-degree centrality measures indicate that the most active participants in the discussion forum were participant09 and participant01. Degree centrality is a simple count of the total number of connections linked to a participant, which can also be viewed as an overall popularity measure (Hansen, Shneiderman & Smith, Reference Hansen, Shneiderman and Smith2010). In this sense, participant09 and participant01, with degrees of 15, were the most popular participants because they were directly connected to 15 of the 22 course participants. In comparison, participant05 was the least popular participant because s/he had a direct connection with only eight other participants throughout the 8-week period. Also, participant09 and participant01, who each had an out-degree of 15, were the most influential course participants, and participant09, who had an in-degree of 9, was the most popular one. The mean degree, out-degree, and in-degree measures for the nine participants in the course were 11.11, 8.33, and 7.01, respectively. Out of these nine participants, three had an above-average degree, four had above-average out-degrees, and five had in-degree scores above the mean in-degree scores.

Although popularity is significant, betweenness centrality is a measure that captures a completely different type of importance (Hansen et al., Reference Hansen, Shneiderman and Smith2010), revealing individuals who play a bridge role in a network. Betweenness centrality scores can also give us an idea about how much removing a participant would affect the interaction between other participants in a network. Table 3 indicates that participant09, with a betweenness centrality score of 33.9, played a bridge role for those 15 participants s/he interacted with in the course. Himelboim, Golan, Moon and Suto (Reference Himelboim, Golan, Moon and Suto2014) defined participants with high betweenness and high in-degree centrality values as social mediators, who “mediate the relations between an organization and its publics through social media” (p. 361). In this regard, we can consider participant09 as the primary social mediator of this course. The following examples illustrate how participant09 played a bridging role among the group members by helping participantX and participant01 connect with each other. First, participant09 posted the following comment under participant01’s original post:

I have one optional suggestion for the post-listening activity. We can ask students to work in groups and design the poster about environment protection or increasing people’s awareness of recycling, etc. This activity will make the class fun and interesting. Students can do it at home if they don’t have enough time.

Then, participant09 received a suggestion to his/her post-listening activity suggestion from participantX:

Hi participant09. Also, if they are listening to a story, you can ask them in the post phase to change the ending or to rewrite the story in pictures.

Finally, participant01 thanks both participant09 and participantX for their contributions to his/her post:

Your suggestions are also very interesting and fun, especially for teenagers. But for my students it will not suit as they are very shy to talk about it among other students. Anyway, we can use it with older students. Thanks a lot.

As seen in these examples, participant09 initiated another discussion under someone else’s post by suggesting a post-listening activity, which attracted participantX’s attention and established a bridge between these two participants.

Illustrating the trend manifest in Table 3, Figure 2 is a sociogram of the interactional patterns between the participants in the eight discussions throughout the course. In this figure, circle width represents the amount of interaction a participant had with other participants. The arrows show the direction of the posts. As Figure 2 shows, participant09 and participant01 were highly active and central in the course interactions, whereas participant05 and participant08 were much less interactive (as represented by a more outward appearance in the network).

Figure 2. Sociogram of the GOC discussions over an 8-week period. Unspecified data points represent the remaining course participants

Figure 3 and Figure 4 compare the egocentric networks of the most interactive (participant09) and the least interactive (participant05) participants throughout the course, further revealing the interactional patterns of these two participants. The thickness of the arrow between two participants represents the level of interaction. For example, the thickest arrow in Figure 3 shows that participant09 interacted the most with participant01 in Modules 2, 6, and 8. In contrast, the thin arrows in Figure 4 show that participant05 had only a low level of interaction with just eight different participants throughout the course.

Figure 3. Egocentric network of participant09 throughout the course

Figure 4. Egocentric network of particpant05 throughout the course

Although SNA elicits some interactional patterns in the discussion forum, it does not by itself provide a full picture of how and why we see these patterns. Therefore, along with these findings, we conducted a content analysis, foregrounded by a case study design (Yin, Reference Yin2004), of the replies in the GOC discussions to better understand how and why we see these patterns.

4.3 Qualitative content analysis of interaction patterns (RQ3)

To answer the third research question, we conducted thematic content analysis of the replies to examine the qualitative patterns of interaction among the nine participants. Table 4 summarizes the total frequencies of the themes of peer-to-peer interaction, along with a brief description of each category (see Supplementary Material C for sample responses).

Table 4. Peer-to-peer interaction in the replies

The most frequent theme was appreciation – cases in which the participants expressed appreciation to their peer participants. In most cases, the participants appreciated others for sharing new ideas and resources. Another frequent pattern was when the participants complimented their peers’ ideas by providing specific feedback based on their own evaluations and perspectives. Participants also tended to express appreciation by posting expressions of encouragement (e.g. “you are really an early bird!”), which were mostly accompanied by the use of positive evaluative markers such as “amazing,” “helpful,” “interesting,” or “impressed.”

These patterns of appreciation indicate that the participants were actively engaged in providing positive feedback to support each other and help improve each other’s ideas. This is a pattern that is frequently observed in previous studies on peer feedback in online discussions (Farr & Riordan, Reference Farr and Riordan2012; Sert & Aşık, Reference Sert and Aşık2020). Such expressions of encouragement were most frequently observed in participant09, whom our SNA identified as the most interactive participant. Based on having the highest betweenness and degree centrality values (see Table 3), participant09 was identified as the primary social mediator of the course, mediating the relations and interaction between the course participants (Himelboim et al., Reference Himelboim, Golan, Moon and Suto2014). A closer analysis indicated that participant09 started almost all of his/her replies by sharing expressions of encouragement such as this:

I’m really impressed with your ideas, energy and time you devote to teaching reading.

In contrast, participant05, whom our SNA identified as the least active participant, rarely used explicit expressions of appreciation.

The second most frequent theme was initiating engagements – when participants initiated engagement with other participants by asking direct or indirect questions. The analysis of these cases further revealed four functions of initiating engagements: (a) ask for clarification, (b) request additional information, (c) ask probing questions, and (d) prompt actions. Participants frequently asked for clarification or requested additional information. Such questions urged participants to clarify confusion in their posts or to share more details on a particular issue. Some participants asked probing questions (Garrison, Anderson & Archer, Reference Garrison, Anderson and Archer2001), namely, rhetorical questions intended to help the writer of the original post to think more deeply about the issue at hand. For example, in the following example, participant01 asks a series of probing questions in the reply to raise potential issues in using ESL lab for lower-level learners:

May I know what class, grade, or level, your students are? … Do you think this topic will be applicable when teaching adults? Additionally, would you discuss the idioms mentioned in the ESL-lab? …

In another example, playing the social mediator role in the group, participant09 encourages the writer of the original post to think more deeply about how to further apply the course knowledge (i.e. technological tools learned from the lecture) to his/her own teaching context:

Notebook exchange is a good way for students to learn from each other. But with technology, we can do it much better, right?

Finally, some participants tended to ask questions to prompt the writer of the original post to take specific actions (e.g. share a supplemental link). For instance, in the following example, the participant is prompting the writer to share additional material (i.e. link to the audio) so other course participants can engage in a deeper analysis and evaluation of his/her activity:

I wonder if you could add the link so we could all see what audio you are working with and learn from it. Thanks!

These instances of asking probing and prompting questions suggest that participants did not just provide positive feedback, as displayed in the pattern of appreciation, but also engaged in an advanced level of critical thinking through their continued interaction with peer participants in the replies. Such an advanced level of interaction was a feature that distinguished participant09 from less interactive participants. This finding provides additional qualitative evidence that participant09 served as an important mediator in the peer-to-peer interaction, promoting their peers’ cognitive development by asking questions to prompt a deeper level of critical thinking and actions.

The third most frequent theme, showing empathy, indicates cases in which the participants empathized with other participants’ experiences, thoughts, or feelings. In the majority of these cases, the participants expressed empathy with other participants’ frustration, sharing specific challenges or constraints in implementing technology tools into their teaching contexts. In other cases, participants showed empathy for others’ teaching methods and perspectives. We observed that several participants, especially participant09 as in the following example, established rapport and felt connected with their peer participants (“we all,” “we, together”):

Thank you for sharing your ideas. I found it interesting to read your discussion because it’s nearly the same as mine … I myself find it difficult to teach one point of grammar like tenses. It’s really overwhelming even to teachers or advanced students. However, it’s surely a good resource for us to apply in teaching. We, together, will find the appropriate ways, I hope. Best.

Such feelings of group commitment have been found to promote participation and collaboration in ACMC (Zhao, Sullivan & Mellenius, Reference Zhao, Sullivan and Mellenius2014). We could see evidence that the explicit expression of a sense of community and connectedness promotes interaction among our course participants.

Finally, the least frequent theme that we found, response to a question, indicates the cases in which participants responded to other participants’ replies to their own posts (i.e. response to a follow-up question). The fact that this pattern occurred less frequently in the participants’ replies than the other patterns did (see Table 4) is surprising, given the high frequency of the initiating engagement theme. We observed that three of the six responses to a question were from participant09, as in the following example:

Thank you for reading my discussion. Here is the link of the definition of the Physical environment from Merriam-Webster. I hope to give you more information about it and hear from you all [reply by participant09 to participantX under their own post].

This is another example that shows that participant09 played an important bridge role in the interaction among course participants by encouraging continued communication with other participants. Except for participant09 and participant01, other participants rarely showed such a pattern, while many participants brought up new questions requesting a further response. Such a discrepancy could have been an effect of the discussion prompt. That is, since the participants were asked to reply to peer participants’ posts at least once, they tended not to go back to check the follow-up questions once they met the minimum requirement. The closing date for discussions could have been another reason for the lack of follow-up discussions. Thus, the discussion prompts might need to be refined in order to facilitate participants’ engagement in a continued discussion with peer participants.

5. Discussion

This research explored participant engagement in discussion activities in a GOC. Through quantitative and qualitative analyses of the engagement patterns in participants’ discussion posts and replies, we answered three research questions. The first research question aimed at identifying the frequent patterns of engagement in the discussion posts and how these patterns relate to the types of discussion prompts. The thematic analysis of participants’ original posts identified 11 engagement patterns, with sharing teaching practices and making connections with lecture the top two most frequent patterns. Taking the type of discussion prompt into account, we found that the engagement patterns observed in participants’ posts aligned with the primary goals of the prompt types. The second research question aimed at representing the interaction patterns of the GOC participants statistically and visually through SNA. We found that of the nine participants analyzed, two were particularly active and played a vital role in the discussions, serving as a bridge between other participants. That is, they acted as social mediators (Himelboim et al., Reference Himelboim, Golan, Moon and Suto2014) who, through their interactions, connected participants with each other. Finally, our third research question aimed at describing the SNA findings using qualitative data. Our thematic analysis of the participants’ replies identified seven themes. Of these themes, we most frequently observed appreciation, initiating engagement, and showing empathy and found that the initiating engagement theme seemed to help distinguish between high- and low-level interactants.

These findings suggest that our discussion assignments (especially the reply portion of it) enabled the participants’ co-construction of knowledge and cognitive development through their exchanging of constructive feedback in replies, which is in line with findings in previous studies (Farr & Riordan, Reference Farr and Riordan2012; Sert & Aşık, Reference Sert and Aşık2020; Wilkins, Shin & Ainsworth, Reference Wilkins, Shin and Ainsworth2009; Zhang et al., Reference Zhang, Liu, Chen, Wang and Huang2017). In addition, we found that participants built a community of practice by exchanging supportive comments and encouragements, or explicitly expressing a sense of community. Our discussion boards seemed to serve as a platform to build a sense of belonging and commitment among peer participants. Such feelings of group commitment or community feeling have been found to promote participation and collaboration in ACMC (Zhao et al., Reference Zhao, Sullivan and Mellenius2014).

Our findings also highlight several important elements of discussion prompts that could elicit higher levels of interaction and thus higher levels of participant engagement with the course. For example, discussion prompts could be designed to elicit higher levels of interaction and cognitive engagement. Ideally, all interactions between participants should be interconnected. Zhu (Reference Zhu2006) characterized interconnected interactions as having higher centrality measures, which means that participants are interacting with multiple people within the course. Having this type of interaction occur within different discussion activities will likely motivate students “to exchange, elaborate on, and challenge each other’s ideas” (p. 470). Subsequently, it might help students to better understand and remember the learning content. To elicit such high levels of interaction between participants, several strategies could be embedded within the discussion posts. As we noted in our Results section, providing specific guidelines about how to interact with peer participants helped to promote peer-to-peer interaction. In addition, higher centrality measures could be achieved through providing instructions such as “Comment on at least three different participants’ posts. Additional replies will be awarded extra credit.” Such instructions might help motivate students to interact with different participants and encourage them to respond to follow-up questions posed by their peers, an issue that we observed in our findings.

In terms of cognitive engagement, we expect discussion assignments to be higher-level cognitive activities that require higher levels of thinking, which is believed to produce useful learning (Stoney & Oliver, Reference Stoney and Oliver1999). For meaningful learning to occur, discussions must be carefully planned so that they can elicit higher-level critical thinking. Findings from previous research (Garrison et al., Reference Garrison, Anderson and Archer2001; Kanuka & Anderson, Reference Kanuka and Anderson1998) found that the purpose of most of the online discussions was to simply share and compare information. But for an online course geared toward CALL practitioners, such as the GOC, the goal of most discussion activities is to elicit a high level of cognitive engagement through knowledge construction. This goal can be achieved through deeper levels of information processing associated with elaborating concepts (Gagné, Reference Gagné1985), such as providing explanation, analysis, and evaluation and debating or negotiating meaning. Such deeper-level information processing is more likely to lead to the desired construction of knowledge.

In our discussion activities, such high level of cognitive engagement has been achieved especially through the presence of social mediators. Social mediators displayed an advanced level of interaction in that they initiated engagement by asking probing questions or prompting actions as opposed to merely asking simple clarification questions. Asking questions and prompting actions tended to result in deeper analysis of the learning materials within the module, thereby promoting higher levels of cognitive engagement. To encourage more social mediators within an online CALL course, participants should be oriented toward using probing questions or prompting actions, perhaps through providing explicit examples of good original posts and replies that help promote higher levels of cognitive engagement.

6. Conclusion

This study has explored the interactions between course participants in a GOC focused on the effective integration of technology into a language classroom. By generating centrality scores and analyzing patterns of engagement, we were able to identify several themes that separated those with a high level of influence from those with a low level of influence. Despite the small-scale case study nature of this study, our results have implications for prompt development, CALL instruction for teacher training, and effective online instruction. For example, this study shows that the connection between prompts and participant interaction should be taken into account when developing an online course. We also found that participants engaged differently in the discussion forums, with some acting as social mediators and others having fewer interactions. Future research could use an increased number of participants from various backgrounds (nationality, education, L1s, etc.) to determine if there are observable patterns to how members of particular backgrounds interact within ACMC environments in an online CALL course. By understanding the engagement patterns of participants from different backgrounds, researchers could structure discussion prompts in a way that allows participants to contribute equally.

Moving forward, we can undoubtedly state that technology’s role within the language classroom will continue to grow. Therefore, creating the most effective means of CALL instruction for language teachers everywhere should be a top priority. One important aspect of this instruction is that language teachers must be able to share and co-construct knowledge together because there is not a one-size-fits-all approach to incorporating technology into the language classroom.

Supplementary Materials

To view supplementary material for this article, please visit https://doi.org/10.1017/S0958344021000331

Acknowledgements

This publication was prepared under a grant funded by Family Health International under Cooperative Agreement/Grant No. S-ECAGD-15-CA-1095 funded by the U.S. Department of State, Bureau of Educational and Cultural Affairs (ECA/A/L). The content of this publication does not necessarily reflect the views, analysis, or policies of FHI 360 or the U.S. Department of State, nor does any mention of trade names, commercial products, or organizations imply endorsement by FHI 360 or the U.S. Department of State.

Ethical statement and competing interests

The authors of this paper assure that this study has been rigorously reviewed and approved by the institutional review board and that appropriate informed consent has been obtained from all participants. The authors declare that they have no conflict of interest to disclose.

About the authors

Haeyun Jin is a Fulbright scholar and a PhD candidate in the Applied Linguistics and Technology program at Iowa State University (ISU). She works as a research assistant for the US OPEN Program. Her research interests include computer-assisted language learning and assessment, integrated writing assessment, and corpus linguistics.

Yasin Karatay is a PhD student and a research assistant in the Applied Linguistics and Technology program at ISU. His research interests include computer-based speaking assessment, CALL use in materials development and assessment, English for specific purposes, and teacher training.

Fatemeh Bordbarjavidi is a PhD student in Applied Linguistics and Technology at ISU. Her research interests are corpus linguistics, academic writing, and CALL. Currently, she works for Using Educational Technology in the English Language Classroom Global Online Course sponsored by the U.S. Department of State and provided by ISU.

Junghun Yang is a PhD student in Applied Linguistics and Technology at ISU. His research interests include corpus linguistics and technology-infused language learning. He is a research assistant for the global online course provided by ISU and an editorial assistant for Journal of Supply Chain Management.

Timothy Kochem is a PhD candidate in Applied Linguistics and Technology at ISU. He currently works as an instructor of English for teaching purposes and also as an English speaking and writing consultant. His research focuses on the intersection of L2 pronunciation, teaching training, and distance education.

Ananda Astrini Muhammad is a PhD student in the Applied Linguistics and Technology program at ISU. Her research interests include second language pragmatics in language teaching/learning and assessment, as well as teacher professional development.

Volker Hegelheimer is a professor of applied linguistics in the Department of English at ISU. He teaches courses on technology in language teaching and research, language assessment, and research methodology. His research interests include applications of technology in language learning and language testing.

References

Baek, S. I. & Kim, Y. M. (2015) Longitudinal analysis of online community dynamics. Industrial Management & Data Systems, 115(4): 661677. https://doi.org/10.1108/IMDS-09-2014-0266 CrossRefGoogle Scholar
Canals, L. & Al-Rawashdeh, A. (2019) Teacher training and teachers’ attitudes towards educational technology in the deployment of online English language courses in Jordan. Computer Assisted Language Learning, 32(7): 639664. https://doi.org/10.1080/09588221.2018.1531033 CrossRefGoogle Scholar
Coleman, J. A., Hampel, R., Hauck, M. & Stickler, U. (2012) Collaboration and interaction: The keys to distance and computer-supported language learning. In Levine, G. S., Phipps, A. & Blythe, C. (eds.), Critical and intercultural theory and language pedagogy. Cengage Learning, 161–180.Google Scholar
Farr, F. & Riordan, E. (2012) Students’ engagement in reflective tasks: An investigation of interactive and non-interactive discourse corpora. Classroom Discourse, 3(2): 129146. https://doi.org/10.1080/19463014.2012.716622 CrossRefGoogle Scholar
Gagné, R. (1985) The conditions of learning and theory of instruction (4th ed.). Holt, Rinehart and Winston.Google Scholar
Garrison, D. R., Anderson, T. & Archer, W. (2001) Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1): 723. https://doi.org/10.1080/08923640109527071 CrossRefGoogle Scholar
Godwin-Jones, R. (2014) Global reach and local practice: The promise of MOOCS. Language Learning & Technology, 18(3): 515.Google Scholar
Godwin-Jones, R. (2020) Building the porous classroom: An expanded model for blended language learning. Language Learning & Technology, 24(3): 118.Google Scholar
Guichon, N. & Hauck, M. (2011) Editorial: Teacher education research in CALL and CMC: More in demand than ever. ReCALL, 23(3): 187199. https://doi.org/10.1017/S0958344011000139 CrossRefGoogle Scholar
Hansen, D. L., Shneiderman, B. & Smith, M. A. (2010) Analyzing social media networks with NodeXL: Insights from a connected world. Burlington: Morgan Kaufmann. https://doi.org/10.1016/B978-0-12-382229-1.00002-3 Google Scholar
Himelboim, I., Golan, G. J., Moon, B. B. & Suto, R. J. (2014) A social networks approach to public relations on Twitter: Social mediators and mediated public relations, Journal of Public Relations Research, 26(4): 359379. https://doi.org/10.1080/1062726X.2014.908724 CrossRefGoogle Scholar
Hong, K. H. (2010) CALL teacher education as an impetus for L2 teachers in integrating technology. ReCALL, 22(1): 5369. https://doi.org/10.1017/S095834400999019X CrossRefGoogle Scholar
Hubbard, P. (2008) CALL and the future of language teacher education. CALICO Journal, 25(2): 175188.CrossRefGoogle Scholar
Kanuka, H. & Anderson, T. (1998) Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13(1): 5774.Google Scholar
Kessler, G. & Hubbard, P. (2017) Language teacher education and technology. In Chapelle, C. A. & Sauro, S. (eds.), The handbook of technology and second language teaching and learning. Hoboken: Wiley Blackwell, 278292. https://doi.org/10.1002/9781118914069.ch19 CrossRefGoogle Scholar
Kochem, T., Muhammad, A. A., Karatay, Y., Jin, H. & Hegelheimer, V. (2020) Considerations for future technology development based on EFL teachers’ integration of technology. In Kruk, M. & Peterson, M. (eds.), New technological applications for foreign and second language learning and teaching. Hershey: IGI Global, 239260. https://doi.org/10.4018/978-1-7998-2591-3.ch012 CrossRefGoogle Scholar
LaPointe, D. K. & Gunawardena, C. N. (2004) Developing, testing and refining of a model to understand the relationship between peer interaction and learning outcomes in computer-mediated conferencing. Distance Education, 25(1): 83106. https://doi.org/10.1080/0158791042000212477 CrossRefGoogle Scholar
Martín-Monje, E., Castrillo, M. D. & Mañana-Rodríguez, J. (2018) Understanding online interaction in language MOOCs through learning analytics. Computer Assisted Language Learning, 31(3): 251272. https://doi.org/10.1080/09588221.2017.1378237 CrossRefGoogle Scholar
Mason, R. & Weller, M. (2000) Factors affecting students’ satisfaction on a web course. Australasian Journal of Educational Technology, 16(2): 173200. https://doi.org/10.14742/ajet.1830 CrossRefGoogle Scholar
McCrory, R., Putnam, R. & Jansen, A. (2008) Interaction in online courses for teacher education: Subject matter and pedagogy. Journal of Technology and Teacher Education, 16(2): 155180.Google Scholar
Nami, F., Marandi, S. S. & Sotoudehnama, E. (2018) Interaction in a discussion list: An exploration of cognitive, social, and teaching presence in teachers’ online collaborations. ReCALL, 30(3): 375398. https://doi.org/10.1017/S0958344017000349 CrossRefGoogle Scholar
Prell, C. (2012) Social network analysis: History, theory and methodology. London: SAGE Publications.Google Scholar
Satar, H. M. & Akcan, S. (2018) Pre-service EFL teachers’ online participation, interaction, and social presence. Language Learning & Technology, 22(1): 157183.Google Scholar
Scott, J. (1987) Social network analysis: A handbook. Thousand Oaks: Sage Publications.Google Scholar
Sert, O. & Aşık, A. (2020) A corpus linguistic investigation into online peer feedback practices in CALL teacher education. Applied Linguistics Review, 11(1): 5578. https://doi.org/10.1515/applirev-2017-0054 CrossRefGoogle Scholar
Shea, P., Hayes, S., Uzuner-Smith, S., Gozza-Cohen, M., Vickers, J. & Bidjerano, T. (2014) Reconceptualizing the community of inquiry framework: An exploratory analysis. The Internet and Higher Education, 23: 917. https://doi.org/10.1016/j.iheduc.2014.05.002 CrossRefGoogle Scholar
Shrout, P. E. & Fleiss, J. L. (1979) Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2): 420428. https://doi.org/10.1037/0033-2909.86.2.420 CrossRefGoogle ScholarPubMed
Smith, M. A., Shneiderman, B., Milic-Frayling, N., Rodrigues, E. M., Barash, V., Dunne, C., Capone, T., Perer, A. & Gleave, E. (2009) Analyzing (social media) networks with NodeXL. In Carroll, J. M. (Chair), Proceedings of the Fourth International Conference on Communities and Technologies. New York: Association for Computing Machinery, 255264. https://doi.org/10.1145/1556460.1556497 CrossRefGoogle Scholar
Son, J.-B. (2003) Online discussion in a CALL course for distance language teachers. CALICO Journal, 20(1): 127144. https://doi.org/10.1558/cj.v20i1.127-144 CrossRefGoogle Scholar
Stoney, S. & Oliver, R. (1999) Can higher order thinking and cognitive engagement be enhanced with multimedia? Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1(2). http://imej.wfu.edu/articles/1999/2/07/index.asp Google Scholar
Strauss, A. & Corbin, J. (1994) Grounded theory methodology: An overview. In Denzin, N. K. & Lincoln, Y. S. (eds.), Handbook of qualitative research. Thousand Oaks: SAGE Publications, 273285.Google Scholar
Torsani, S. (2016) CALL teacher education: Language teachers and technology integration. Rotterdam: Sense Publishers. https://doi.org/10.1007/978-94-6300-477-0 CrossRefGoogle Scholar
Wilkins, E. A., Shin, E.-K. & Ainsworth, J. (2009) The effects of peer feedback practices with elementary education teacher candidates. Teacher Education Quarterly, 36(2): 7993.Google Scholar
Wu, H., Gao, J. & Zhang, W. (2014) Chinese EFL teachers’ social interaction and socio-cognitive presence in synchronous computer-mediated communication. Language Learning & Technology, 18(3): 228254.Google Scholar
Yin, R. K. (ed.) (2004) The case study anthology. Thousand Oaks: SAGE Publications.Google Scholar
Yin, R. K. (2014) Case study research: Design and methods (5th ed.). Thousand Oaks: SAGE Publications.Google Scholar
Zhang, S., Liu, Q., Chen, W., Wang, Q. & Huang, Z. (2017) Interactive networks and social knowledge construction behavioral patterns in primary school teachers’ online collaborative learning activities. Computers & Education, 104: 117. https://doi.org/10.1016/j.compedu.2016.10.011 CrossRefGoogle Scholar
Zhao, H., Sullivan, K. P. H. & Mellenius, I. (2014) Participation, interaction and social presence: An exploratory study of collaboration in online peer review groups. British Journal of Educational Technology, 45(5): 807819. https://doi.org/10.1111/bjet.12094 CrossRefGoogle Scholar
Zhu, E. (2006) Interaction and cognitive engagement: An analysis of four asynchronous online discussions. Instructional Science, 34(6): 451480. https://doi.org/10.1007/s11251-006-0004-0 CrossRefGoogle Scholar
Figure 0

Table 1. Focal participants

Figure 1

Table 2. Engagement patterns in the original posts

Figure 2

Figure 1. Engagement patterns by prompt types.

Figure 3

Table 3. SNA centrality measures

Figure 4

Figure 2. Sociogram of the GOC discussions over an 8-week period. Unspecified data points represent the remaining course participants

Figure 5

Figure 3. Egocentric network of participant09 throughout the course

Figure 6

Figure 4. Egocentric network of particpant05 throughout the course

Figure 7

Table 4. Peer-to-peer interaction in the replies

Supplementary material: File

Jin et al. supplementary material

Jin et al. supplementary material

Download Jin et al. supplementary material(File)
File 22 KB