Introduction
Radiotherapy uses precisely targeted high-energy beams of X-rays to destroy cancer cells and is an effective curative treatment for many common cancers. There is growing interest, however, in using high-energy proton beams for these treatments. Protons are charged subatomic particles and are capable of delivering ionising dose to targets in a more conformal manner and hence reducing long-term side-effects. One of the key attributes of protons is their unique dose deposition. They typically deposit low amounts of ionising energy along their entry track, causing most ionising damage when they slow down. They deposit most of their energy at the Bragg Peak at the end of their useful range in tissue, causing minimal exit doseReference Hu, Jiang, Cui, Zhang and Yu1 and thus increasing conformity of dose deposition.
Protons are indicated for a wide range of tumour sites, including paediatric,Reference Leroy, Benahmed, Hulstaert, Van Damme and De Ruysscher2 head and neck,Reference Blanchard, Gunn, Lin, Foote, Lee and Frank3 craniospinal axisReference Indelicato4 and reirradiation.Reference Verma, Rwigema, Malyapa, Regine and Simone5 Accordingly, there is increased demand for proton therapy internationally with 99 centres worldwide and 67 new facilities being constructed or planned.6 In many countries, proton beam therapy is still a relatively new radiotherapy modality so the opening of new centres presents challenges in relation to staff training. In many cases, there are few trained staff who have used protons before and, although proton radiotherapy is generally taught in pre-registration courses, there exists a skills gap in the current workforce that needs to be addressed, as identified in a 2014 paperReference Winey, Shih, Sahoo, Lee, Vapiwala and Both7 outlining core competencies.
The Virtual Environment for Radiotherapy Training (VERT) is a virtual reality simulation of a radiotherapy treatment roomReference Bridge, Appleyard, Ward, Phillips and Beavis8 where undergraduates as well as staff members can explore and learn the physics and operation of a radiotherapy treatment machine. Using VERT brings many benefits to training studentsReference Kane9 or staff.Reference James and Dumbleton10 In addition to technical process training, it can aid visualisation of 3D structures, dose distribution and treatment plans. Students can explore patient radiographic anatomy, practise decision-making skills and gain practical skills such as treatment set-upReference Green and Appleyard11 and image matchingReference Chamunyonga, Burbery, Caldwell, Rutledge, Fielding and Crowe12 in a safe non-pressured environment. Training with VERT has shown that students feel more confident about working clinically after virtual experience with treatments.Reference Kane9 Student satisfaction is also high and the software has been widely used to teach radiotherapy, physics and treatment planning principles.Reference Bridge, Giles, Williams, Boejen, Appleyard and Kirby13,Reference Kirby14 Recent development of a proton module within VERT now brings the possibility of using VERT to provide training to prepare staff for the transition from photons to protons easily, safely and effectively.
With proton radiotherapy becoming an increasingly important aspect of clinical practice, more evidence is urgently needed to support training and education of the workforce. Proton beam radiotherapy is relatively new to the UK, and opportunities for training are currently restricted to those facilities and staff that have the expensive clinical hardware in place. This project aimed to identify the current and future role of virtual reality-based resources in proton therapy training. Virtual reality simulation could offer a useful means of preparing the UK workforce for the expanding clinical use of proton radiotherapy but as yet has not been formally evaluated. Accordingly, the primary aim of this study was to identify the impact VERT has on proton radiotherapy understanding and confidence. Additional aims were to identify those aspects of VERT which students find most useful and to devise a template for future study days utilising the simulation resource. The final aim was to gather data relating to delegate requirements with regard to future training and preparation for clinical use of proton radiotherapy. Overall, the project aimed to provide key evidence relating to training and ongoing education of the radiotherapy workforce that will best prepare them for the future use of protons.
Methods and Materials
Two groups comprising 38 Second Year radiotherapy students from the BSc Radiotherapy and the PgDip Radiotherapy Courses were invited to participate in an optional formative study day in October 2019. All participants were provided with written information about the study day and were asked to provide written consent for their feedback and comments to be used for project data analysis. Students were advised that provision of feedback and data was voluntary, and that all data were anonymous. Ethical approval for the study was provided by the University Human Research Ethics Committee.
Each of the study days comprised a morning lecture and tutorial to deliver proton radiotherapy knowledge followed by a practical session using the VERT proton module in conjunction with a dedicated workbook as per Nisbet’sReference Nisbet and Matthews15 recommendations. The workbook contained different tasks for them to complete relating to dose distribution, dose delivery and proton beam physics. The VERT module comprised a realistic 3D model of a proton therapy machine along with visualisation of patient, CT and dose deposition as seen in Figure 1.
Screenshot from the VERT proton module.
Quantitative evaluation of the impact of the VERT practical session on learning utilised a survey method with a pre-post questionnaire design. Participants completed a short multiple choice questionnaire before and after undertaking the VERT practical workshop. Example questions can be seen in Figure 2. Anonymity was preserved by asking participants to code each questionnaire with a secret word or phrase that they created. This enabled matching of paired questionnaires in order to compare scores for each participant and therefore identify whether knowledge had been improved. Additional evaluation data were gathered in the second questionnaire using Likert scales as seen in Table 1 to harvest user feedback relating to overall experience and the potential future role of this initiative. Finally, a series of open questions, as seen in Table 2, asked participants to rate their satisfaction and the perceived value of VERT for this purpose.
Example knowledge-based questions.
Likert questions and responses
Abbreviations: VSD, very strong disagree; VD, strong disagree; D, disagree; N, neither agree nor disagree; A, agree; SA, strong agree; VSA, very strong agree; VERT, Virtual Environment for Radiotherapy Training.
Open questions and main themes arising
Quantitative data from the questionnaires were transferred to an Excel Spreadsheet, and final scores were compared between the two questionnaires using a two-tailed paired student t-test. Qualitative data from the open questions were collated, and thematic analysis of content was performed adopting an established process adapted from Giorgi.Reference Giorgi16 The full dataset was read initially in order to make sense of the participant responses. After this, the data were interrogated more closely to identify codes from common words, phrases or sentiments. Codes were then arranged into broader categories in order to better guide interpretation. Blind coding was performed by two independent researchers before themes were agreed. In order to eliminate bias and aid in the objective interpretation of the responses, the authors used a reflexive approach guided by the seven key questions proposed by Weis and Fine.Reference Weis and Fine17
Results
A total of 38 students participated in the study day and evaluation; of these, 21 were Bachelor of Science students and 17 were Postgraduate Diploma students. There was little difference between responses from the two cohorts, and results for the study were combined from both cohorts as seen in Table 1. Overall, there were high levels of satisfaction and enjoyment with 35 participants reporting enjoyment and 36 indicating that the event be repeated in the future. These triangulated well with the identified themes from the open questions as seen in Table 2. The comparison of knowledge before and after the VERT session did not identify any significant increase in understanding as seen in Table 3.
Cohort knowledge test comparison
Discussion
Impact on knowledge and understanding
The results in Table 3 do suggest a slight increase in overall understanding, albeit below the level of statistical significance. This triangulates well with student feedback in Table 2 indicating that VERT had improved understanding of proton beam therapy and dose distribution.
“Visually seeing the beams and energies helped understanding of Bragg peak” (P13)
“Seeing the patient helps visualise dose deposition, helped me understand beam arrangements in different techniques” (P31)
It is likely that this was due to students being able to apply the theoretical content to practical scenarios and thus consolidate their understanding. This finding was also affirmed through high levels of disagreement with the two negatively phrased Likert stems concerning a lack of learning. This apparent disconnect between student feedback and measured knowledge gain perhaps suggests that the assessment tool was too short and lacked discrimination between levels of knowledge. A more robust assessment tool, while presenting participants with a more onerous evaluation activity, would perhaps yield more reliable data concerning knowledge gain.
Perceived benefits
All participants agreed that VERT had provided improved understanding of proton dose deposition and this was supported well by the open question responses. Most of the qualitative responses related to the perceived benefits of visualisation and participants expressed the value of this across a range of themes including beams, dose deposition and relevant structures. Most participants agreed that the simulation was realistic and that it had improved their understanding. There was a clear focus on the value of the interactive nature of the session and how this had reinforced learning.
“It was interactive and we could see what was explained in the previous lecture” (P4)
Realism
Participants were asked about the perceived realism of the simulation and although most participants thought the simulation was realistic, some comments contradicted this stating that the simulation was unrealistic but that they found it helpful. This finding echoes the results of a recent paperReference MacLean, Geddes, Kelly and Della18 concluding that perceived realism is highly individual. The realism rating here was, perhaps, flawed as the cohort had not undertaken any clinical experience with a real proton centre but it does raise a pertinent question. Is realism important for knowledge gain within VERT or does the unrealistic animation and visualisation enable in-depth knowledge gain? Findings from behavioural simulationReference Warwick and Rodgers19 strongly suggest that there is a finite level of realism necessary for skills transfer and learning. Additionally, comparison with other published findings concerning levels of fidelity within healthcare simulationReference Jimenez, Thwaites, Juneja and Lewis20,Reference Ketterer, Callender and Warren21 suggests that absolute realism within the software is not as important for knowledge gain and that realism can be injected into process simulation through use of actors and scenarios.
Challenges and limitations
Feedback from multiple participants (11/38) suggested that the session would have benefitted from a smaller group format; this is a common requirement for VERT teaching.Reference Bridge, Giles, Williams, Boejen, Appleyard and Kirby13 They also expressed a wish for more patient cases, more time and additional supporting resources. Additional clinical datasets would be a valuable addition to the VERT resource and would enable a wider range of techniques to be showcased and used for teaching. Timetable constraints restricted the time available for the session reported here. An additional limitation is that the participants in this study were student radiographers with limited clinical experience and no experience of real proton therapy.
Future use
The findings from this evaluation support future use of the VERT proton module in teaching student radiographers. Future teaching will need to utilise small groups to maintain high levels of interaction and will also benefit from use of a wider range of clinical cases. Extension of this teaching to other professional groups such as medics and medical physicists is likely to be well received and plans are underway to use this as the basis for interprofessional scenario-based learning.Reference Jimenez, Thwaites, Juneja and Lewis20 The Likert data provided suggested that VERT would be useful preparation for qualified staff who are intending to work with protons. It would be interesting to test this hypothesis by repeating the Likert measure after participants had gained experience in a real proton centre and by gathering reflective data evaluating the value of the VERT-based preparation.
Conclusions
A proton simulation module has been shown to be an enjoyable teaching tool that improves students’ confidence in their knowledge of the underpinning theory and clinical usage of the modality. The visualisation of beams and dose deposition was reported as being particularly useful. Although the teaching did not result in quantifiable knowledge gain, users reported that it had improved their understanding and that they felt better prepared to encounter protons in clinical practice. Future work will build on these findings using smaller group work and a more robust assessment tool to identify long-term impact of the training.
Acknowledgements
The authors gratefully acknowledge the support of Virtual Inc and the loan of the VERT proton module. The work was also supported through the ERASMUS exchange programme.
Open access
