Introduction
Gamification is the introduction of game elements into non-game educational contexts that have gained significant attention in health-based programmes and have produced a significant number of publications in recent years. Reference Yang and Li1–Reference Gentry, Gauthier and L’Estrade Ehrstrom3 Gamification is considered to be a goal-orientated activity, and two common defining qualities of gamification are a goal that players are trying to achieve and rules that give structure relating to how players can achieve the goal. Reference Warsinsky, Schmidt-Kraepelin, Rank, Thiebes and Sunyaev4 According to the goal setting theory, developed by Locke in the 80s, goals should be specific, measurable and challenging. Specific, as this provides clarity to the person what is being evaluated and has a clearer aim of what is being measured. Challenging, as more difficult goals result in better performance and higher achievement satisfaction, with most studies showing a linear relationship between the degree of goal difficulty and performance. Reference Latham and Locke5
Gamification can be set up in a number of different ways by presenting the learners with different social interactions. Cooperation allows students to work as a team, collaborating with each other in order to reach the goal. This form of gamification encourages interaction between the students, whereas in conflict players play directly against other player(s) in order to be first to meet the challenge by constraining/impeding/being better than the other player(s). The third type of gamification is that of competition. This is different from conflict as players individually play against the computer. As a result, players cannot impede each other, and the focus is on optimising their own performance that can later be compared to others.
Within health-based professions training, particularly therapeutic radiography, there is a need to instil the demands for communication, working together and coordinating effort in order to meet the requirements of the role and profession. The very nature of the role is teamwork, yet elements of competition inherently exist. Acceptance into university itself is a competitive process, and students are actively competing for a good degree and comparing their grades to that of their peers and considering what career a good degree can provide them.
Although gamification is predominantly utilised using digital technology, it is not limited to this alone. Reference Deterding, Khaled, Nacke and Dixon6 Virtual environment of a radiotherapy treatment room (VERT) is a virtual reality training system, comprised of software and hardware that are focused towards therapeutic radiography students, but can be used to educate patients and other healthcare professional students and staff, on the use of radiotherapy. Reference Jimenez and Lewis7 The VERT system affords students the opportunity to experience and learn on the simulator and visualise theoretical concepts of radiotherapy treatment away from the clinical department. This offers the educators and clinical staff the potential to further enhance education provision for students. Benefits of such systems are the high degree of standardisation when simulating radiotherapy treatments, with no risk to service users in a real clinical environment. It also allows an objective rather than subjective evaluation of procedural skills, providing a standardised platform for assessment. Reference Haycock8 The benefits offered by VERT also lend the system to competitive gamification, with students being able to pit themselves against the computer simulation. Competition in education has been shown to have both positive and negative aspects associated with it. Positives include increased enjoyment in the task and increased productivity, whereas negatives include reduced intrinsic motivation and in certain cases aggression between users. Reference Witoszek-Kubicka9 It is not possible to accurately predict how participants will react. Reference Featherstone, Geril and King10
The aim of this study was to evaluate the use of a timed gamified simulation via VERT using competitive task elements in order to see how acceptable competitive simulation was to the student body and to discover what the participants considered to be the perceived benefits and weaknesses of such a simulation.
Method
The study took place on the VERT system at City, University of London. The study was submitted to the School’s Proportionate Review Committee and was classed as service evaluation. All participants were second-year therapeutic radiography students each of who already had a total of 9 hours of experience using the VERT system as part of their undergraduate training. The study ran for 4 years and included four different cohorts, the first cohort acting as the pilot group and the other three the study population.
The foundation of the task was based on the TV series Top Gear which featured the segment ‘Star in a reasonably priced car’. The simulation utilised was a simulated treatment set-up of a 10 × 7 cm electron field to the breast. The format of the session was explained to the participant, and a demonstration was undertaken led by the radiotherapy lecturer. The participant was then given the opportunity to undertake the simulation task with guidance and advice from the facilitator until they reached the end point of the task, and the participant was satisfied with their attempt. The participant was then debriefed and given feedback, and then they progressed to the competitive element of the task under timed conditions. Participants were then asked to undertake a set-up with the facilitator present, but offering no advice. They were instructed to undertake the task safely and obtain a clinically treatable patient set-up with accurate treatment distance and minimum variation in stand-off. When the set-up was completed to their satisfaction, they informed the facilitator.
The students were then invited to the scoreboard, and their ‘scores’ added to the board and feedback were given about the approach taken to the set-up. The ‘score’ constituted the mean stand-off of the finished set-up and the time taken to complete it. This information was written on a magnetic strip and added to the leaderboard in the appropriate place. Participants were also informed that a small prize (chocolates) would be available to the winner of the simulation. The small prize was a prerequisite of the competition but was deliberately kept unnamed during the competition to help ensure that their focus remained on the simulation, not on the prize. Reference Cantador and Conde11
Survey instrument
Following the simulation, the participants were asked to complete a short questionnaire based on their experience. The questionnaire consisted of four questions on the simulation design and four on competitive element of the simulation. Two questions were reverse-scored, and all were based on a seven-point Likert scale. Validated tools for looking at competitiveness exist, such as the Competitiveness Index, Reference Harris and Houston12 Competitive Attitude Scale Reference Menesini, Tassi and Nocentini13 and Co-operative/Competitive Strategy Scale; Reference Simmons, Wehner, Tucker and King14 however, the study was not looking at participant competitiveness per se, but the acceptability of the simulation and if they enjoyed the competitive element of the simulation. It was also important to maximise the completion of the questionnaire which was why a very short questionnaire based on a seven-point Likert scale was used. There were also two open questions about the simulation allowing participants to openly comment on their experience and voice of any additional thoughts on improving aspects of the simulation. A further two questions looked at how they perceived the simulation and the use of the leaderboard. The questionnaire was scrutinised by the radiotherapy academic staff within the institution and piloted on a cohort of 17 participants to establish the face validity of the questionnaire. This revealed that the open questions predominantly elicited feedback regarding VERT itself and its strengths and weaknesses. The open questions were therefore adapted to produce feedback more directly related to the gamification/competitive elements of the simulation and again scrutinised by staff knowing the weakness in the existing questions raised by the pilot study. All questionnaires were anonymous.
Statistical analysis was undertaken with MedCalc® Statistical Software version 20.211 using non-parametric tests. A significance value of 0.05 was assumed throughout the study.
Results
Three cohorts of therapeutic radiography students were asked to complete the survey following the simulation between 2020 and 2022. Seventy-eight students returned a survey out of a possible 99, a completion rate of approximately 79%. Of the 78 participants, 65 (83.3%) identified themselves as female, 12 (15.4%) male and 1 (1.3%) left the question blank.
Overall, the feedback from the four questions regarding the elements of the VERT session were very positive, the median scores all being 6 or above in each of the four items. No gender differences in the scores were seen (Figure 1).
The four items focusing on the competitive element were scored less positively, particularly with respect to enjoyment from trying to outperform other students. However, all the scores were positive with respect to the competitive elements, as shown in Figure 2. Male participants tended to enjoy all the competitive aspects more compared to the female participants; however, only satisfaction from the competition and encouraged performance were significantly higher scores. Enjoyment from trying to outperform others was the least liked aspect of the competition questions. For this particular question, 47 (61%) of participants responded favourably with a score of over 4, 16 (20.8%) neutrally and 14 (18.2%) unfavourably with a score of less than 4.
Feedback on the simulation.
Feedback on gamification/competition elements.
A comparative analysis across the three different cohorts of how they perceived the simulation revealed no significant difference in any of the eight items. Values ranged from H = 4.11, p = 0.08 for transferability of skills to H = 0.17, p = 0.91 for feeling more confident in clinical after the simulation.
The simulation set-up was both gamified and competitive. In order to examine how the students perceived the simulation, they were asked to consider if the simulation felt more like a competition, a game or both equally. 49.1% of students felt that the simulation felt more like a game, 18.9% of students felt that the simulation felt more like a competition and 32.1% of students felt that there was an equal balance between game and competition. Kruskal–Wallis tests revealed that the students’ perception of simulation had no significant effect on the scores of the eight items of the questionnaire, values ranged between H = 3.2, p = 0.24 for transferability of skills and H = 0.10, p = 0.99 for enjoyment from trying to outperform other students.
The use of the leaderboard could be considered problematic as students could see where they were on the leaderboard compared to their peers. Of the students providing a response, 76.8% stated that they were in favour of the leaderboard, 12.5% were unsure and 10.7% did not like the leaderboard. Participants who liked the leaderboard tended to score higher on the competitive items (Figure 3) in the questionnaire, and this difference was significant for the three items shown in Figure 3. Post hoc tests revealed a significant difference only existed between the groups that liked the leaderboard compared to those that did not like the leaderboard.
Median scores as a function of opinion to leaderboard.
Discussion incorporating qualitative feedback
The quantitative feedback from the first four items on the questionnaire were very positive. The participants appreciated the verbal feedback provided both during the practice elements and after the competitive element of the simulation. The majority of the participants also considered that the experience was transferable to the clinical setting, and it would positively impact their confidence in the clinical setting. No gender differences were observed for any of the questions. The most common word used to describe the simulation from the qualitative comments was ‘fun’ with a number of participants requesting in their feedback to do more simulations of this nature. A small number also acknowledged that the gamification element provided more motivation and encouraged them to perform better, ‘It provided motivation to pay attention and apply the skills…… It also encourages more involvement and participation’.
Gender differences were observed, however, on two of the questions that focused on the gamification/competitive aspects of the simulation with male participants seeing the competitive elements more positively than female participants. Male participants derived significantly higher levels of satisfaction from competing U = 329.5, p = 0.02. They also felt that the competitive element of the simulation encouraged their performance during the simulation more so than female participants, responding better to the external motivation of the competition. A number of participants commented positively on the incentive of a prize at the end of the simulation, and this was generally seen as a positive element of the simulation by supporting the extrinsic motivation of the competition and adding encouragement to perform better. Conversely, two participants commented that they did not like not knowing what the prize was when undertaking the simulation.
This gender bias finding is important considering the demographic make-up of therapeutic radiography students being largely female. 15 Females are reportedly more reluctant to compete than men, Reference Niederle and Vesterlund16,Reference Sutter and Glätzle-Rützler17 with it being suggested that this might be because females tend to be less confident than males about their relative ability and are less risk-averse in nature. Reference García-Gallego, Georgantzís and Jaramillo-Gutiérrez18 Denden Reference Denden, Tlili, Essalmi, Jemni, Chen and Burgos19 further suggests that personality affects the students’ perceptions of gamification, and that this in itself is further modified by a gender interaction. It is important to note here that the study is noting differences in satisfaction and encouragement within the gamified simulation, not in the performance itself which was not measured.
Participants also acknowledged that the competition element produced stress/pressure in the simulation. Some saw this as a good thing encouraging them to perform better ‘stressed in a good way’, ‘think fast like you have to in clinical’ and ‘I liked that I was quite fast and was able to make decisions quickly’, whereas a few participants saw this as negative, ‘Can be nerve racking seeing your results compared to others, and that others could see your score’ and ‘Pressure does not make me perform better’. Stress that is seen as positive is called ‘eustress’ where the student has an increased concentration level, mental agility and enhanced motor coordination. Reference Pinheiro, Patta and Zaggia20 Some participants from their comments experienced this during the gamified simulation, whilst others derived no benefit from this aspect of the simulation. No comments were made about stress being a negative aspect, for example, anxiety within the simulation. The competition element was important and commented on both positively and negatively, despite approximately half the participants (49.1%) feeling that the simulation ‘felt’ more game-like than competition.
76.8% of participants enjoyed the leaderboard which was very positive; however, the qualitative comments revealed mixed opinions about this aspect. Some thought it as a good idea ‘students took their practice attempts more seriously and tried their best to win’ and ‘Liked leaderboard because you could see how you compared to other people’. Others though felt that because other students could see it, it added undue pressure or embarrassment, ‘Did not like pressure of leaderboard’ and ‘People who have a low score could feel embarrassed’. Leaderboards that display all the participants and their scores (absolute leaderboards) can give the students with better scores a broader sense of accomplishment than students lower on the leaderboard; Reference Ortiz-Rojas, Chiluiza and Valcke21 however, students who have performed less well feel weakened compared to better placed students and suffer stress as a result of this. Reference Schlömmer, Spieß and Schlögl22 This issue might also be related to about outperforming other students which received the lowest overall score of the questions.
The leaderboard in this instance was based on the Top Gear segment, ‘Star in a reasonably priced car’ which is an absolute leaderboard; however, its structure could be improved moving forward. A relative leaderboard could be used where the students know their rank is a possibility, where no scoring information is placed on the board and students are only aware of their relative performance to others. Bai Reference Bai, Hew, Sailer and Jia23 found that as with this study, most students reported a positive attitude towards the use of leaderboards, and that on a relative leaderboard students preferred public comparison compared to absolute leaderboard where lower ranked students preferred anonymous comparison compared to public comparison. Other possibilities exist such as the use of badges and digital credentials acknowledging a skills level instead of a leaderboard. One student from the pilot group suggested that they would prefer that students were informed how they performed in comparison to the average score from previous year, and another student suggested that only the top students were put on the board.
Other issues raised by participants were that some students had an advantage based on their clinical placements ahead of the simulation, and the order students took the simulation potentially may affect the score, ‘Making the score visible gave those that came after a reference, a number to beat, putting those who go first at a disadvantage’. The simulation was however completed visually, that is, students did not know the stand-off measurement until the task was complete. The perception of an advantage was mentioned by some participants, and this could have been mitigated by having the scores from the previous year’s cohort visible as a target for the first students to complete the simulation.
Some participants requested more competitive gamification with the programme. Currently, the simulation being reported on is the only competitive element in the curriculum, and there is room to expand this form of activity. However, Koivisto Reference Kovisto and Hamari24 showed that perceived enjoyment and usefulness of the gamification declines with use, and many of the comments and scores might be positive because of the novelty of the gamified simulation.
One limitation of this study is the low number of males. Data were collected over three cohorts in an attempt to increase the number of males included in the study, but the final study only contained 12 male respondents compared to 65 females, a ratio of over 5:1. The sample collected reflects the student population being studied and so there is no bias present, but unbalanced sample sizes can lead to a reduced power of the tests.
Conclusion
Overall, the simulation was viewed very positively by students. The competitive aspect was also viewed positively by most students, although females derived less satisfaction from the competitive element in the simulation. Although most students felt that the competitive element encouraged their performance in the simulation, this was significantly lower for females. Despite this gender difference, the simulation was well received by the students as a whole. Regardless of the emphasis on teamwork within the therapeutic radiography profession, the competitive gamification simulation does appear to have a role within the curriculum if carried out correctly and not used too frequently so as not to demotivate students who do not like competition. The major lesson learnt from the study was that the leaderboard can be viewed negatively by some students. Moving forward only the top scores will be displayed (to avoid having students at the bottom), we plan to implement this for future iterations and provide every student with their score in comparison to the average in lieu of listing every score. The inclusion of this competitive element within the curriculum remains in the newly validated course starting this year as it is a key clinical skill that can be a problem providing consistently across our clinical sites. Students taking part now are briefed that the purpose is to improve their clinical skills and knowledge, and the competitive element is just for fun. It was recognised that some students felt disadvantaged that others had real clinical experiences of electron set-ups beforehand. As this tutorial is not summative, we feel that this situation does not impact on overall student experience.
Study limitations
During the simulation because the participants were aware they were being observed by a facilitator (lecturer), they might have modified their behaviour. This bias which is sometimes referred to as the Hawthorne effect means the participant’s behaviour might be different to that in clinical. Admittedly in clinical students are always observed, but the power imbalance between a clinical member of staff and lecturer may be different, and this may affect a student’s behaviour. Further studies should be conducted using validated scales looking more closely at the use of competition in therapeutic radiography to build on this initial work.
Acknowledgements
The authors wish to acknowledge the invaluable assistance of the participants for their willing engagement in the study.
Financial support
None.
Competing interests
The authors declare none.
Section/Category
Education and training.
Ethics
The study was reviewed by the Maternal and Child Health Proportionate Review Committee, School of Health Sciences at City, University of London, and was classed as service evaluation.
Open access
