Effectiveness of the KiVa Antibullying Program with and without the Online Game in Chile: a Three-Arm Cluster Randomized Controlled Trial

Bullying is a major health problem. The KiVa antibullying program has been evaluated in Finland and other European countries, showing preventive effects on self-reported bullying victimization and perpetration. No evaluations of this program have been conducted in Latin America. A cluster randomized controlled trial was conducted at socially vulnerable schools in Santiago, Chile, to assess the effectiveness of the KiVa antibullying program in grades 5 and 6 (aged 10–12 years). Schools were randomly assigned (1:1:1) to three groups: the full KiVa group (including the online game), the partial KiVa group (did not include the online game), and the control group in which the regular school curriculum was implemented. The primary outcome was self-reported bullying victimization, assessed before the intervention (baseline) at the end of the academic year (November 2016) and post-intervention, 12 months after the baseline assessment (November 2017). This trial is registered with ClinicalTrials.gov, number NCT02898324. A total of 39 schools (13 in each group) were included; no schools withdrew. The baseline survey included 5923 participants, and the endpoint survey included 3968 participants. Participants in the partial KiVa group had lower bullying victimization at the endpoint survey than those in the control group (OBVQ-R adjusted mean difference − 0.14; 95% CI, − 0.26 to − 0.01; effect size − 0.13, 95% CI − 0.24 to − 0.01, p = 0.035). There was no effect of the full KiVa group for bullying victimization compared with the control and partial KiVa groups. Compared to the control group, participants in the partial KiVa group had lower witnessing bullying at school (adjusted mean difference = − 0.25; 95% CI − 0.45 to − 0.05; effect size − 0.18, 95% CI, − 0.32 to − 0.04, p = 0.013). No effects were found for other secondary outcomes, including bullying perpetration in any comparisons between arms. The implementation of the KiVa antibullying program had mixed results in Chile. There was only a small effect on bullying victimization and witnessing when KiVa was delivered without the online game.


Introduction
School bullying is aggressive behavior intended to cause harm to another person, characterized by an imbalance of power, and repeatedly occurring over time (Menesini & Salmivalli, 2017). Aggressive behavior can be physical (hitting, pushing), verbal (insulting, nicknaming), indirect/ relational (social exclusion), or through digital devices (cyberbullying). Bullying may be based on the targeted student's characteristics, such as ethnicity, socioeconomic background, sexual orientation, or without apparent reason. A systematic review estimated that globally, one third of children and adolescents are involved in bullying situations, as either bullies or victims (Zych et al., 2015).
In Latin America, children and adolescents are exposed to high levels of violence within their families, neighborhoods, and schools (Truco & Inostroza, 2017). Various social factors, such as high levels of inequality, drug trafficking, and low-quality education, further contribute to youth aggression (Atienzo et al., 2017). According to a study from 15 countries in Latin America, the prevalence of bullying victimization ranged from 17 to 39% (McClanahan et al., 2015), similar to what is found in Chile (Prevention & Organization, 2013).
Bullying has been associated with many negative consequences in the short and long-term, especially for victims (Moore et al., 2017), such as internalizing problems (e.g., depressive, anxiety, and somatic symptoms), problems with peers (e.g., rejection, isolation), and risk of suicidal ideation (Klomek et al., 2015). On the other hand, there is evidence that some bullies also have problems controlling anger and are more involved in substance use and delinquent behavior (Ttofi et al. 2012). Moreover, recent studies have indicated that even bystanders, merely witnessing bullying, can show adverse outcomes compared to students who have not been exposed to bullying at school (Gaete et al., 2017a).
Several school-based antibullying programs have been developed and scientifically evaluated. A recent metaanalysis found that these programs reduced bullying perpetration in a range of 19-20% and bullying victimization between 15 and 16% (Gaffney et al., 2019). However, no antibullying program has been tested with a RCT design in Latin America (Atienzo et al., 2017).
One of the most evaluated antibullying programs worldwide is KiVa, developed in Finland. KiVa was evaluated in a large RCT and during its roll-out across Finland (Kärnä et al., 2011), showing positive effects on reducing bullying perpetration and victimization, depression, anxiety, and increased well-being at school and academic motivation. In the Netherlands and Italy, KiVa implementation had good results in changes in bullying experience for students (Huitsing et al., 2020;Nocentini & Menesini, 2016). However, a recent study in the UK showed no evidence of positive effects of the program (Axford et al., 2020).
The theory of change of the KiVa program is mainly based on promoting activities to generate a positive change in the behavior of bystanders, to reduce the rewards gained by bullies with their behavior, and consequently to reduce their motivation to continue bullying others (Nocentini & Menesini, 2016). KiVa program has universal and indicated actions. The universal actions include ten 90-min students' lessons, kick-off meetings for the school community, posters displayed on classrooms and corridors, vests for teachers supervising recess time, information for parents, and an online game (Kärnä et al., 2011). In the online KiVa game, students are introduced to a virtual school populated by animated characters who assume the roles that children may take in bullying situations, aiming to develop coping strategies to deal with bullying in real life (Poskiparta et al., 2012). The indicated actions are conducted by the KiVa Team, targeted at victims and perpetrators (Kärnä et al., 2011). The KiVa team is composed of 2-4 school staff, and they were responsible for leading individual counseling of victims and perpetrators.
The evaluation of the effectiveness KiVa program faces several challenges, especially regarding its adaptation to countries different than Finland. First, it is unknown how much each of the components of the program contributes to the relative effectiveness of the KiVa program (Menesini & Salmivalli, 2017). Specifically, the added value of the online game has not yet been evaluated. Second, as KiVa program was developed in Finland, a country highly recognized as having one of the best education systems in the world and with high expenditure in education (The World Bank, 2021), cultural adaptation to Latin American countries with more vulnerable conditions may be difficult (Atienzo et al., 2018). The results of exploring the effectiveness of the KiVa program in highly vulnerable schools in a Latin American country and testing the added value of the online game component will contribute to filling the knowledge gap of how this program may work in a different context compared to where it has been previously studied (Axford et al., 2020;Huitsing et al., 2020;Kärnä et al., 2011;Nocentini & Menesini, 2016). Additionally, these results may inform and help other researchers improve the cultural adaptation process of other preventive programs in school contexts.
In this study, we have tested the effectiveness of a culturally adapted version of the KiVa antibullying program, specifically with students attending grades 5th and 6th (10 to 12 years old) in socioeconomically vulnerable schools in Santiago, Chile. As we wanted to assess the added value of the online game, we used a three-arm cluster RCT to compare the effectiveness of the KiVa program with the online game (full KiVa schools), the KiVa program without the online game (partial KiVa schools), and treatment as usual (control schools). We hypothesized that at the end of the intervention, fewer students would identify themselves as victims, perpetrators, or bystanders of bullying in KiVa schools than in control schools. Furthermore, we expected the program effects to be stronger when the online game is implemented together with the other components. A systematic review supports that using Information and Communication Technologies (ICTs) tools may enhance the preventive effects of bullying interventions for children and youth by affecting the motivational system and cognitive and emotional learning process mediated by activities where there is less social pressure compared to face-to-face experiences (Nocentini et al., 2015). Finally, we hypothesized that students' psychological adjustment, psychological sense of school membership, and academic performance would be higher in schools implementing the KiVa program than in control schools.
The RCT protocol for this study was published before starting the implementation of the KiVa antibullying program (Gaeteet al., 2017b).

Study Design and Participants
We did a three-arm cluster RCT study in schools (clusters) within Santiago, Chile, with schools as the unit of allocation and individual participants as the unit of analysis. Schools were randomly allocated to any of three arms: (1) full KiVa group, (2) partial KiVa group, and (3) control group. We selected Santiago since it is the most populated area in Chile (40.5% of the total population of the country), and we only included schools from socioeconomically vulnerable areas because there is a higher prevalence of bullying and less access to preventive programs.
Our sample frame comprised all mixed-sex primary education schools located in Santiago with two or three classes in grades 4th and 5th and with high levels of socioeconomic vulnerability (IVE-SINAE ≥ 75%; School Vulnerability Index -National System of Equity Allocation). According to the funding, the educational system in Chile is structured into three types schools: (1) public fully state-funded schools (44.6% of students attend these schools); (2) subsidized schools, which are administered by private nonprofit organizations, which also receive state funds (49.5%); (3) and private schools, which are administered by private organizations (either nonprofit or for-profit), and they do not receive state funds (5.9%) (Centro de Estudios, 2019). On the other hand, school vulnerability is based on the IVE-SINAE, which assesses a range of variables, including health, family income, housing conditions, state benefits, and other factors. The IVE-SINAE estimates the proportion of students in a school who are in most need, ranging from 0% (no students at risk) to 100% (all students at risk) (Agencia de Calidad de Educación, 2013).
Schools that were implementing other manualized antibullying programs were not included. All eligible schools were invited to participate, and meetings were coordinated with school authorities to explain the study. The school principal signed a consent if the school agreed to participate in the study. Parents/caregivers signed an informed and written consent if they wanted their children to participate in the RCT; parents could request the withdrawal of their children from the study assessments at baseline or at any other time throughout the project. Students also signed an assent agreeing to participate or not in the study.

Randomization and Masking
An independent statistician performed the randomization (1:1:1) to assign schools to either full KiVa group, partial KiVa group, or control group using a computer-generated randomization sequence. Randomization took place after the baseline assessment to balance the study arms concerning the size of the school (number of students per school), number of classes per grade, and the level of self-reported bullying victimization. We used a computergenerated list of random numbers to select one allocation sequence from the 1000 sequences with the most desirable balance properties. The assigned study arm was informed after all schools had been recruited into the trial, which took place after all student's baseline measurements had been completed.
Research assistants, blind to allocation, administered the assessments. They received a full day of training to ensure a fully standardized data collection. When hiring and training research assistants, the research team was conscientious not to reveal assigned study arms to the school to keep them blind. However, due to the nature of the intervention, which included several visible school actions, the research team could not assure complete blindness to the assigned study arm to research assistants.
We have followed a similar cultural adaptation procedure like the one followed by the research team in Italy (Nocentini & Menesini, 2016). We distinguished between deep structure and surface structure of interventions (Sundell et al., 2014). The deep structure of the program, such as the theory of change, the universal lessons delivered by the facilitator, and the indicated actions conducted by the KiVa team in the school, were unchanged because they were essential to lead the mechanisms to reduce bullying (Saarento et al., 2015). On the other hand, the intervention surface structure consists of language, graphic material, culturally appropriate strategies of teachings, and channels of communication (Herkama & Salmivalli, 2013). We worked together with the original developers, and it was decided to produce a minimally adapted edition of the KiVa program with only minor surface structure changes (Sundell et al., 2014).
To start the adaptation process, two members of the Chilean team took a 1-week training course at the University of Turku, Finland, with the original developers and became certified KiVa trainers. During 2016, the research team worked on the translation and adaptation of the material for the intervention, and a pilot study of the students' lessons was conducted in a school with 5th and 6th graders, similar to those participating in the RCT. This pilot study included consultation with teachers and students, who proposed minor changes.
Further modifications were made in the implementation model due to differences between the Chilean and Finnish school systems. First, the content of the students' lessons in the Finnish version was originally organized in 10 double lessons (2 × 45 min), which included 14 practical lessons, five online game lessons, and one final lesson where students evaluate the program. Conducting two lessons together on the same day was not suitable for Chile. Schools have only a 45-min "orientation class" each week where the Ministry of Education of Chile has proposed that health promotion and prevention topics be implemented. Therefore, as part of the cultural adaptation, the research team decided to implement the lessons in the "orientation class" Second, the content of the last two original practical lessons could be implemented in only one lesson. Therefore, 13 45-min weekly lessons were included in the KiVa program in Chile. Third, the original KiVa program included short films portraying different bullying situations in Finland, which were used to promote a discussion among students in the class. Due to the limited resources of the study to adapt and film culturally appropriate videos in Chile, these materials were not included. However, the research team selected appropriate videos showing similar content from YouTube. Fourth, the workload of school teachers in Chile is very high (Cabezas et al., 2017), and when conducting the pilot study, it was clear that teachers were reluctant to implement the lessons; therefore, it was decided to hire facilitators to deliver the students' lessons. Most of the professionals working as facilitators in this study were primary education teachers, and a minority were educational psychologists. On the other hand, the school class teacher was part of the school personnel, who had the following functions: (a) collaborate with the discipline of the students during the lessons; (b) provide knowledge to the facilitators of the students' personalities and psychological features to assure the psychological security and protection of students who participated in the activities of the lessons; and (c) help in some of the activities (e.g., leading the work of group activities of students, helping with some explanations of the content of the program to be adapted to their school context).

KiVa Program Training
Before implementing, certified KiVa trainers delivered a 3-day training to all facilitators. These facilitators were also supervised during the whole period of the lesson implementation. Additionally, certified KiVa trainers gave a 3-day training to school class teachers and school authorities of participating schools. The research team invited all school personnel of participating schools to attend the training; however, not all of the people invited could participate (see Appendix, Suppl. 1). Particular emphasis in the training was given on how school teachers can help with the students' lessons, delivered by the facilitators, and how to use the referring protocols to manage victims and bullies. Finally, the research team provided all the material to the schools free of charge.

Universal Actions: KiVa Lessons
The students' lessons were interactive sessions where students learned the definition of bullying, how to recognize it, and the role of the group in maintaining bullying. Using role-play, group games, and solving quizzes, students practice coping strategies to deal with bullying and learn how to support victims. Thirteen 45-min weekly lessons were implemented between March and October 2017. The lessons were delivered by an external facilitator, assisted by the school class teacher. Facilitators followed the KiVa teacher's manual to give the lessons.

Other Universal Actions
KiVa intervention was widely advertised within schools with posters encouraging students to support victims and behave constructively when witnessing bullying. A school member was present in all school breaks and at lunchtime wearing a visible KiVa vest or any other cloth distinctive article. Parents were informed about bullying, how to identify it in their children, and how to manage it and also about the KiVa program through a website, letters, meetings, and a parent's guide. At the beginning of the implementation, kick-off meetings with school staff, parents/caregivers, and students were organized to inform them about the KiVa program and answer any doubts.

Kiva Game (Only for Arm Full KiVa Schools)
In the online game, students are introduced to a virtual school populated by animated characters who assume the roles children take while bullying situations occur (victims, perpetrators, or bystanders) and learn coping strategies to deal with bullying in real life. The game has five stage levels, and each of them has three components named the following: "I know," "I can," and "I do." Specifically, students acquire new information and test their existing knowledge about bullying, play quizzes about the contents ("I know"), learn new skills to act in appropriate ways in bullying situations, play with a character that has to choose how to behave from different options available ("I can"), and are encouraged to make use of their knowledge and skills in real-life situations, reporting in the game which KiVa rules they have succeeded in putting into practice ("I do"). The Kiva game has five 45-min weekly online game lessons. Four of the online game lessons were played in the schools using tablets brought by the research team (as sometimes socioeconomically vulnerable schools do not have enough computers for students) or at the school computer lab when available. Each student played the game individually, and the activity was supervised by a school teacher and the facilitator to answer students' questions regarding the game or the program. The students played the last online game lesson at their homes.

Indicated Actions
The school authorities designated the KiVa team integrated by at least two school staff: a psychologist and any other professional assigned by school authorities such as a teacher or a social worker). The members of this KiVa team were trained in the 3-day training at the beginning of the academic year before the implementation of the intervention. In the training, there was a chapter focused on how to conduct individual counseling with victims and perpetrators (victims and perpetrators always participated at different meetings, in a private place, with two members of the KiVa team) and how to fill out the protocols to assess the bullying experiences.

Intervention Adherence and Fidelity Strategies
The research team coached and supported the intervention schools throughout the implementation. Each facilitator was assigned to a supervisor (psychologist) to discuss the implementation. The quality of the implementation of the intervention was assured through the following procedures: (1) Several Student's lessons were observed by the supervisor in vivo; (2) facilitators had to fill out a report form each time they delivered a lesson; (3) weekly meetings between the supervisor and the facilitators were conducted to discuss the lesson report, teaching strategies, and contingencies; and (4) monthly meeting between the research team and facilitator.
In addition, KiVa teams and a member of the research team met twice a year (less or more, depending on their needs), to supervise universal actions at school and provide guidance on the management of bullying cases in the indicated actions.

Control Group
Control schools implemented their normal teaching activities during "orientation class." Usually, students receive teaching on diverse health promotion topics in this class. Schools in Chile do not have government guidelines on what to teach during this class, but it may include some lessons about bullying; however, this is not known.

Outcomes
The primary outcome variable was self-reported school bullying victimization at baseline compared to endpoint assessment. It was measured by the 40-item Revised Olweus Bully/Victim Questionnaire (R-OBVQ). The OBVQ-R is a self-report questionnaire that assesses bullying experience (i.e., bullying victimization and bullying perpetration). This questionnaire has been validated in Chile among 4th to 8th graders (Gaete et al., 2021). To measure bullying victimization, we used the global item from the questionnaire: "How often have you been bullied at school in the last couple of months?" Students answered on a 5-point scale (0 = not at all to 4 = several times a week). Among secondary outcomes, we included self-reported bullying perpetration and self-reported witnessing. Perpetration was assessed with the global item: "How often have you bullied others at school in the last couple of months?." The witnessing item was added by the research team using the same format as the bullying victimization and bullying perpetration items: "How often have you witnessed bullying situations at school in the last couple of months?".
Other secondary outcomes were psychological adjustment, psychological sense of school membership, and academic performance. The psychosocial adjustment was assessed with a self-report questionnaire, the Strengths and Difficulties Questionnaire (SDQ), which evaluates 25 attributes from five subscales (five items each): (1) emotional symptoms, (2) conduct problems, (3) hyperactivity/inattention, (4) peer relationship problems, and (5) prosocial behavior. The first four subscales generate a total score of difficulties (20 items), and the prosocial behavior scale is considered to reflect the personal strengths. The research team validated this questionnaire in Chile (Gaete et al., 2018). The Psychological Sense of School Membership (PSSM) scale is a self-reported instrument developed to assess the sense of school belonging. The items are related to students' perceptions of being "accepted, respected, included and supported by others in the school social environment." This instrument was validated for the Chilean population, and the original scale with 18 items was shortened to 13 items (Gaete et al., 2016). Each statement may be answered using a scale from 1 "Not at all true" to 5 "Completely true." Each student's responses to the 13 items are averaged together to create their total score. A higher score indicates a higher level of school membership.
Academic performance was assessed with the grade point average (GPA), which in Chile goes from 1.0 (lowest) to 7.0 (highest). Permission was granted to have access to the registry of GPA.
All the variables mentioned above will be analyzed as continuous in the statistical analyses.

Other Measures
Lost at Follow-Up This variable was included in a sensitivity analysis to explore which outcomes were related to this condition. The variable was categorized as 0 (not lost at the follow-up survey) and 1 (lost at the follow-up survey).

Data Collection Dates
Using paper-based questionnaires, baseline assessments were carried out at the end of the 2016 academic year when students were in 4th and 5th grade (between 9 and 11 years old). The endpoint assessment was done in November 2017 (12 months after baseline).

Statistical Analyses
The sample size was calculated to obtain a significant mean difference between groups at the school level. We used the results of a previous study (Kärnä et al., 2011) to estimate the number of clusters in two arms (full KiVa versus control; and partial KiVa versus control) using an intraclass correlation of 0.02 for self-reported bullying victimization. The geometric average of students per cluster was estimated at 115, resulting in 1495 eligible students per arm, so we needed at least 39 schools with 13 schools per arm. Further details of the sample size calculation can be found in the published study protocol (Gaete et al., 2017b).
The data analyses followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines for cluster RCTs (Schulz et al., 2010). Descriptive statistics were used to compare the three arms at baseline. Statistical analyses were done using Stata 14.2 and on an intentionto-treat basis. Due to the hierarchical nature of the data, we used mixed linear effects models to compare the mean of bullying victimization at endpoint assessment (12 months after baseline) between the full KiVa group, partial KiVa group, and control group. For bullying victimization, we first used a model with adjustment for baseline scores of bullying victimization and a second model with adjustment for baseline scores of bullying victimization, sex, and age. For exploratory purposes, we additionally conducted a secondary analysis for bullying perpetration, witnessing bullying, psychological adjustment, psychological sense of school membership, and academic performance, comparing the three arms of the trial with adjustment for baseline scores, sex, and age. In every model, we included a random effect for school and fixed effects to account for baseline scores, sex, age, and group effect (intervention arms vs control). The outcomes were continuous, so the intervention effect was reported as adjusted mean difference and effect sizes (standardized mean difference, Cohen's d) with 95% CIs. As pre-planned, we assessed differential effects of the intervention on bullying victimization, bullying perpetration, and witnessing bullying, using subgroup interaction terms in the regression models by school grade.
We conducted sensitivity analyses to assess the potential effects of missing data. We performed logistic regression models, using the variable "Lost at follow-up" as a dependent variable and variables such as bullying experience, psychological functioning (SDQ), or sense of school membership (PSSM) as independent variables to explore the variables to be included in the imputation models. All results of this sensitivity analysis were reported as odds ratio with their 95% CIs (Appendix, Suppl. 2).
With multiple imputation methods, we created 20 datasets and showed that non-imputed data and imputed data for bullying victimization were virtually the same (see Tables 2 and  3, and Appendix, Suppl. 3 and 4). The main results presented in this paper are from imputed data.
The trial is registered with ClinicalTrials.gov, number NCT02898324.

Results
There were 39 schools participating in the RCT. Of the 5923 students eligible to participate in the trial, 12% were not included at baseline (168 did not have parental consent, 65 did not assent, 477 were absent). Figure 1 shows the flow of schools and students in the study. There were a comparable number of students allocated in each arm. No schools dropped from the study during the RCT. At endpoint assessment, data were available for 67% of the students (n = 3968) assessed at baseline; completion was lower in the partial   KiVa group (61.2%) than in the control group (71.9%) and the full KiVa group (68.4%). The sensitivity analysis showed some differences at baseline between the students lost at follow-up survey in the three groups, which were considered in the imputation analyses (See Appendix, Suppl. 2 for further details). Baseline school characteristics were similar by group, although the partial KiVa group had more subsidized schools and fewer public schools than the other groups. The control group tended to be slightly smaller than the other two (Table 1). Participants' characteristics were generally similar between groups (Table 1). More boys than girls participated, and the mean age for the whole sample at baseline was 10.3 years old, with 72.5% aged 10 or 11 years. There were no differences in baseline scores for primary and secondary outcomes by group (Table 1). Appendix Suppl. 5 and Suppl. 6 show baseline and endpoint means (SD) for primary and secondary outcomes. At baseline, the ICC for self-reported victimization for the 39 schools was 0.02 (95% CI 0.01 to 0.03) (Appendix, Suppl. 7 shows ICCs for secondary outcomes).
Based on the research team records, similar coverage of the KiVa program activities and students' lessons was recorded for both intervention groups (see Appendix, Suppl. 3.9 (0.8) 3.9 (0.9) 3.9 (0.8) School performance (GPA), mean (SD) 5.7 (0.5) 5.5 (0.5) 5.6 (0.5) 1). However, more schools from the partial KiVa group attended the three-day training (at least one person), and more classrooms completed the 13 practical students' lessons than the full KiVa group. More schools from the full KiVa schools exhibited the KiVa posters than partial KiVa schools.
Bullying victimization was lower at endpoint (12 months after baseline) for participants in the partial KiVa group compared to the control group (mean 1.63 [SD 1.10] vs 1.78 [SD 1.23]; model 1: adjusted mean difference − 0.14, 95% CI, − 0.27 to − 0.01, p = 0.028; model 2: adjusted mean difference − 0.14; 95% CI, − 0.27 to − 0.01; effect size − 0.13, 95% CI − 0.25 to − 0.01, p = 0.035) ( Table 2). Among the secondary outcomes considered, there was a clear reduction of witnessing bullying for participants in the partial KiVa group compared with the control group (mean 2.06 [SD 1.34] vs 2.36 [SD 1.52]; adjusted mean difference = − 0.23; 95% CI − 0.44 to − 0.03; effect size − 0.17, 95% CI, − 0.32 to − 0.02, p = 0.013), but there was no evidence of a difference between the partial KiVa group and the control group on the other secondary outcomes (Table 3). Most findings were consistent between fifth and sixth graders, with a difference in fifth grade where the partial KiVa group had lower bullying victimization (p = 0.024) and witnessing bullying (p < 0.0001) than the control group (Appendix, Suppl. 8). There was strong evidence of effect modification by school grade, indicating that the intervention effects were stronger in fifth graders in the partial KiVa group for bullying victimization (p = 0.040), witnessing bullying (p < 0.0001), and psychological difficulties (p = 0.027) (Appendix, Suppl. 9).
By contrast, there was no evidence of an intervention effect of the full KiVa group versus the control group on the primary outcome (mean 1.73 [SD 1.20] vs 1.78 [SD 1.23]; model 1: adjusted mean difference − 0.07, 95% CI − 0.20 to     Table 2). There was no evidence of a difference between the full KiVa group and the control group on secondary outcomes (Table 3). These findings were consistent for both school grades (Appendix, Suppl. 8). There was evidence of an effect modification by school grade, indicating a stronger effect in sixth graders from the full KiVa group for bullying perpetration (p = 0.042) (Appendix, Suppl. 9). There was no evidence of a reduction in the primary outcome in the full KiVa group compared to the partial KiVa group (mean 1.73 [SD 1.20] vs 1.63 [SD 1.10]; model 1: adjusted mean difference 0.07, 95% CI − 0.06 to 0.20, p = 0.21; model 2: adjusted mean difference 0.07, 95% CI − 0.06 to 0.20, p = 0.21; effect size 0.06, 95% CI − 0.06 to 0.18) ( Table 2). Participants in the full KiVa group had worse outcomes than the partial KiVa group for psychological difficulties (mean 14.0 [SD 6.54] vs 13.38 [SD 6.50]; adjusted mean difference 0.79, 95% CI 0.13 to 1.45; effect size 0.14, 95% CI 0.02 to 0.25; p = 0.037). There was no evidence of a difference between the full KiVa group and the partial KiVa group on other secondary outcomes (Table 3). Most findings were consistent between fifth and sixth graders, with a difference in fifth grade where the full KiVa group had higher witnessing bullying (p < 0.0001) and psychological difficulties (p = 0.024) than the partial KiVa group (Appendix, Suppl. 8). There was strong evidence of effect modification by school grade, indicating that the intervention effects were stronger in sixth graders in the full KiVa group for bullying victimization (p = 0.018) and witnessing bullying (p = 0.001) (Appendix, Suppl. 9).

Discussion
This is the first large cluster randomized controlled trial (RCT) evaluating the effects of the antibullying KiVa program in Latin America and the first aiming to explore the added value of the online game component to the effectiveness of KiVa worldwide. Overall, the results of this study indicate that the cultural adaptation of the KiVa program had mixed results in Chile. On the one hand, partial KiVa schools had lower levels of bullying victimization than control schools. In contrast, students in full KiVa schools reported similar levels of bullying victimization to students in control schools. There was no clear superiority between KiVa schools and control schools regarding bullying perpetration. And for witnessing bullying, partial KiVa schools performed better than full KiVa schools than control schools. Therefore, the online game did not provide any additional effect to KiVa. We did not find evidence of an intervention effect of the KiVa program in other secondary outcomes such as students' mental health, school membership, or academic performance.
Comparing the evaluation of the KiVa Program in Chile with other studies exploring the effectiveness of full KiVa program, the results found in this study share some similarities and differences. The effect size for self-reported bullying victimization in Finland (Cohen's d = 0.17) (Kärnä et al., 2011) was small and similar to the one we observed in our study, but in the partial KiVa group (Cohen's d = 0.14). In addition, the RCT conducted in The Netherlands also found small effect sizes, especially during the first year of KiVa implementation (Huitsing et al., 2020), and there was a larger reduction in bullying victimization than in bullying perpetration. Finally, another study, conducted in Wales (UK), showed no evidence for reducing bullying victimization or perpetration (Axford et al., 2020), similar to the results for the full KiVa implementation in Chile.
To our knowledge, the only other study exploring the effect of partial KiVa (without the online game) was conducted in Italy. The effect size of the program was larger (Cohen's d = 0.38) (Nocentini & Menesini, 2016) than the one we found. Additionally, our study showed evidence of better effects of the KiVa Program for younger students (5th grade) than for older students (6th grade), in the partial KiVa group. These results are consistent with the Italian RCT, where they also found better effects for younger students (fourth grade) than older ones (sixth grade) (Nocentini & Menesini, 2016).
The results found in the full KiVa arm may differ from the original KiVa effectiveness study (Kärnä et al., 2011) for several reasons. First, Finland is highly recognized as having one of the best education systems in the world, where the government expenditures on education are higher than the Latin America and Caribbean region (The World Bank, 2021). In addition, the KiVa in Finland included comprehensive schools, not making differences regarding vulnerability indexes of the schools for inclusion criteria as we did in the present study. Changing aggressive patterns in a vulnerable socioeconomically context in Latin America can be more complicated than in other contexts (Atienzo et al., 2018). Second, the fact that the main facilitators of the intervention were teachers external to schools may have had a role in the small effect found in our study because school teachers know how to approach their students better when compared to external facilitators (Atienzo et al., 2018).
Regarding the different effects of the two intervention arms, the results were different from what we expected, as we had a priori hypothesized that schools implementing the full KiVa Program (with the online game) would show a greater reduction in bullying experience (i.e., bullying victimization, bullying perpetration and witnessing bullying at school) than the partial KiVa Program (without the online game). However, the results were better for the partial KiVa Program group. Exploring potential explanations, we found some differences between these two groups. First, in terms of the coverage of the program activities, more schools (teachers and authorities) from the partial KiVa group participated during the training (3-day training) than the full KiVa group. This may suggest a better program implementation during the year, as these schools were supposedly more prepared and trained by certified KiVa trainers. Higher training participation may have reflected a greater commitment and motivation from these schools' authorities and teachers. However, the real impact of this involvement is not clear because we do not have data regarding the actual performance of teachers and authorities and if the participation in the training was translated in more collaboration of teachers, during the students' lessons, and authorities, supporting the KiVa Team.
Additionally, even though more classrooms from the partial KiVa group completed all 13 practical lessons (91%) than the full KiVa group (35.6%) and only 10.2% of classes completed just 11 lessons in the Full Kiva, it is unclear if these differences are sufficient to explain the results. Practical lessons of the KiVa program are essential to generate changes in students, as these lessons include learning social and emotional skills such as communication skills and social problem solving, and knowledge about bullying, its consequences, and how to stop it. The practical lessons are designed for practicing the skills in a safe environment individually and as a group. The difference in the implementation highlighted here may suggest that schools in both groups differed in variables not measured in our study, such as school organization and school culture, which seem to impact the implementation of programs such as KiVa (Durlak & DuPre, 2008).
Other differences between the full and partial KiVa groups were the effects across school grades. The study showed evidence of a reduction in bullying victimization and witnessing bullying at school for fifth grades in the partial KiVa group, but the full KiVa group did not show evidence of a decrease in bullying experience in any school grades. The explorative subgroup analysis by school grade provided more information about these differences. It seems that the partial KiVa group had better results for fifth graders. In contrast, in the full KiVa group, there was an effect modification of school grade, indicating that it worked better for sixth graders. All the bullying experience outcomes showed a trend in terms of their reduction.
Another explanation for these findings may be related to a potential adverse effect of the game component. On the one hand, the feedback and rewards provided in the virtual environment of the game may have reduced the inherent motivation of the students (Yuichi & Insoo, 2020). Additionally, the KiVa game features may not have been considered fun or appealing to students, reducing game engagement (Yuichi & Insoo, 2020). On the other hand, the game cannot replace face-to-face education, which has been proven to be the best channel to enhance motivation among students (Yuichi & Insoo, 2020). Additionally, playing the online game in the Full KiVa group may have reduced the opportunities to continue working on antibullying preventive topics during "orientation class" (outside the program), which may have increased the differences with the partial KiVa group. Sadly, the research team did not collect data about how much work was developed and how much practice was performed by teachers and students during "orientation class" (outside the program) and in other classes; so, we do not have evidence to support this hypothesis.
Among the limitations of the present study, we can mention the lack of data collection regarding the implementation process. First, we did not gather information about differences in frequency or quality of the intervention across schools regarding practical lessons. Second, regarding the game lessons, we did not collect data about the acceptability and adherence of the online game. We do not have systematic data about students' perceptions of the game or which aspects could be improved. Third, regarding the indicated actions of KiVa, the registry of KiVa team meetings held with students did not follow a systematic approach or were unreliable and scarce. Another limitation was that we used self-report questionnaires to assess most outcomes, except for academic performance. Students can answer surveys to maintain social desirability, and the information is potentially biased, since only one informant is included (Bouman et al., 2012). Finally, the attrition was, on average, 31% of the students at outcome assessment. The sensitivity analyses showed that students lost at the endpoint survey were students with more problems, such as worst academic performance, more psychological difficulties, and reported more bullying experience. Although there was some differential attrition across arms in the final assessment, results remained unaltered when imputing missing values for the primary outcome.
We suggest that antibullying research continues with the adaptation and evaluation of antibullying programs for future studies. Additionally, it is important to explore the effect of these programs on other outcomes, such as mental health or school membership, over a longer period and in different contexts. Education policymakers and schools need to make research-informed decisions, consider all the evidence available in their context, and implement evidence-based interventions.
Funding This project (AKA-EDU/15) was funded by an international collaboration grant between the Academy of Finland (AKA) and the National Agency of Research and Development, Chile (ANID); and by ANID -Millennium Science Initiative Program -NCS2021_081.

Declarations
Ethics Approval The study was approved by the Ethics Research Committee of Universidad de los Andes, Chile (January 18, 2016), and it was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Disclaimer The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Conflict of Interest
Christina Salmivalli and her team at University of Turku developed the KiVa antibullying program. All other authors declare no competing interests.