Introduction
YouTube is the most popular video sharing website and ranks second in terms of global internet engagement (i.e. daily page views and time spent on the website).1 It is increasingly used to circulate healthcare information, with both lay persons and professional users uploading content targeted at a range of audiences. It is an accessible and engaging source of medical information that has the potential to improve patients’ understanding of their own conditions and treatment options.
Previous work has demonstrated that, in addition to direct contact with professionals, a significant number of patients utilise internet resources to supplement their understanding.Reference Tassone, Georgalas, Patel, Appleby and Kotecha2 However, there are concerns regarding the quality of these videos, which are challenging to regulate.Reference Danino, Muzaffar, Mitchell-Innes, Howard and Coulson3
Cochlear implant surgery is a specialist procedure performed in just 23 centres across the UK. This can result in limited access to cochlear implant surgeons and knowledgeable professionals. The difficulty of navigating the internet to obtain accurate information about cochlear implant surgery has previously been highlighted.Reference Zaidman-Zait and Jamieson4 Accurate information about the risks as well as the benefits is vital for patients to provide informed consent.
Given the importance of reliable, unbiased information, and the known variability of online resources, this study aimed to evaluate the value of YouTube as a source of information about cochlear implant surgery.
Materials and methods
Using the search term ‘cochlear implant’ and the default setting on YouTube, which automatically sorts by ‘most relevant’, the first 60 results were included. Videos were excluded from analysis if they were not in the English language, not relevant to cochlear implants or if they were aimed at healthcare professionals.
Each video was reviewed by two independent observers, who collated the information into a proforma on Microsoft Excel® spreadsheet software. Where there was disagreement between reviewers, the points of contention were discussed, and where an agreement was not made, a third independent reviewer made the final decision.
In addition to a qualitative assessment, video metric data were collected. This included the number of views, ‘likes’ and ‘dislikes’, the length of the video, whether the video used sign language or subtitles, and the account that had uploaded the video.
The Discern instrumentReference Charnock, Shepperd, Needham and Gann5 (Institute of Health Sciences, University of Oxford, UK) is a validated tool used to assess the reliability and quality of published medical information with regard to treatment choice. Comprised of 16 points, questions 1–8 address the reliability of the publication, questions 9–15 focus on specific details of the treatment choice, and question 16 provides an overall quality score for the video.
Minor modifications were made to the second part of the Discern instrument so that sections pertaining to specific parts of the procedure (e.g. benefits and risks) contained the relevant information for cochlear implants. This information, considered important for patients to know, came from a combination of the National Institute for Health and Care Excellence Technology appraisal guidance (‘TA566’), and National Health Service hospital cochlear implant patient information websites and leaflets.6–8 The modified Discern instrument is shown in Table 1.
Modified Discern instrument
Section 2 questions are adapted from the original Discern toolReference Charnock, Shepperd, Needham and Gann5 to specifically address cochlear implant patients. *n = 47; †n = 37; ‡n = 6. **p < 0.05 (Mann–Whitney U test). SD = standard deviation
The Discern instrument uses a 5-point Likert scale, where 1 is ‘no’ (the quality criterion has not been fulfilled), 2–4 means ‘partially’ (the quality criterion has been fulfilled to some extent) and 5 is ‘yes’ (the quality criterion has been completely fulfilled).
Correlations between video rank, ‘likes’ and overall Discern score were assessed using Spearman's rank test. Videos were categorised into their country of origin, and differences were examined using a Mann–Whitney U test; p-values of 0.05 or lower were considered significant.
Results
The video search was performed on 24 March 2020. Of the first 60 videos, 11 were excluded as they were aimed at healthcare professionals, 1 was excluded as the main language was Spanish and 1 was excluded as it was not relevant to cochlear implants. Forty-seven videos were included in the data extraction and analysis.
The total number of views was 4 738 772 (range of 278 to 1 702 914). Average video length was 2 minutes and 40 seconds; the shortest video was 54 seconds, and the longest was 29 minutes and 53 seconds. The included videos were produced between 2011 and 2020, with the majority of videos being released after 2016 (n = 33).
Videos were produced by YouTube accounts that could be described as the following: hospital (n = 39), charity (n = 1), news station (n = 2), individual (n = 3), science channel (n = 1) and technology company (n = 1). Five of the videos had subtitles included, and none used sign language.
The mean overall Discern score was 2.0 (standard deviation (SD) = 0.77) and the mean total Discern points scored was 36.8 (SD = 6.6, max score of 95). Table 1 shows the mean scores and SDs for all the modified Discern instrument domains. Figure 1 shows a heat map charting the distribution of video scores, with the darker shade indicating more videos achieving that score. The videos scored best in terms of the relevance and benefits of cochlear implant surgery, with mean Discern scores of 3.5 (SD = 0.91) and 3.4 (SD = 1.39) respectively. The videos scored most poorly in terms of: describing their sources of information, referencing when the information used in the video was produced, communicating the contraindications for cochlear implant surgery, stating the risks of cochlear implant surgery, and providing support for shared decision-making (means of 1.1, 1.0, 1.1, 1.3 and 1.2 respectively).
A heat map charting the distribution of video scores by Discern domain, with darker shades indicating more videos achieving that score. CI = cochlear implant; pre-op = pre-operatively
Correlations between where the video ranked in the search results and views were compared with ratio of ‘likes’ to ‘dislikes’ and overall modified Discern score, assessed using Spearman's rank test. Higher YouTube ranking of videos was found to weakly but significantly correlate with higher overall modified Discern score (Rs = −0.31, p = 0.0364). No other significant correlation was found.
Videos originated from the USA (n = 37), UK (n = 6), India (n = 2) and Canada (n = 2). When comparing the country of origin of the videos, significant differences were noted in the modified Discern domain scores. When comparing videos from the USA and UK, videos from the UK scored better in terms of ‘relevance’, ‘aims’ and ‘areas of uncertainty’, whereas videos from the USA scored better in terms of addressing ‘patient suitability’ (Table 1). No other statistically significant differences were noted.
Discussion
This is the first paper to objectively assess the quality of information available on YouTube regarding cochlear implants. Our results reveal the high volume of individuals that utilise YouTube as a source of information on cochlear implants, with the 47 videos included in our study receiving a total of 4 738 772 views between 2011 and 2020.
Despite most of these videos being made for people seeking information about cochlear implants, only four had subtitles and none made use of sign language, making them less accessible to those with a hearing impairment. YouTube has recently introduced a free automated subtitle feature. Although its accuracy is poor, the uploader is able to edit the transcription. The majority of videos in this study would have been published before this service was available, but it is a useful feature for future uploaders who may not have the resources to subtitle their videos. For this reason, accurate subtitles should be considered a standard feature of YouTube videos aimed at the hearing-impaired population.
Recent publications examining medical information on YouTube have suggested that videos fall below acceptable quality standards for medical information, and, importantly, they contain misleading or even harmful information.Reference Haymes and Harries9,Reference Mueller, Hongler, Jungo, Cajacob, Schwegler and Steveling10 The mean overall modified Discern score in this study was 2.0 (suggesting that the videos had important shortcomings); this indicates that the majority of YouTube videos did not meet an acceptable standard of information. Reassuringly, no harmful information was encountered in the videos; however, the risks of cochlear implant surgery were poorly addressed, with a mean modified Discern score of 1.3. It is concerning that 40 videos (85 per cent) offered no information on the risks of surgery. Given that there are serious risks involved in cochlear implant surgery, such as the risk of meningitis, cerebrospinal fluid leak, facial nerve damage and implant failure, it is essential that these are mentioned when giving information about the operation.
The videos also performed poorly (mean score of less than 2) in the following domains: demonstrating clear aims of the video; referencing where the information in their video was from and when the video was produced; providing details for patients to find more information or resources; areas of uncertainty such as success of the cochlear implant in improving hearing; describing who cochlear implants are unsuitable for; describing risks of cochlear implant surgery; and support for shared decision-making, for example encouraging patients to discuss the procedure with family, friends or healthcare professionals.
These serious shortcomings may be because many of the videos advertise cochlear implants, rather than providing balanced information about the operation and what it entails. In keeping with this, videos scored highly (mean score of 3.4) when describing the benefits of cochlear implant surgery, and only 28 videos (60 per cent) mentioned alternatives to cochlear implant surgery (mean score of 2.1).
Our results have shown that cochlear implant videos on YouTube have a bias towards discussing the positives of cochlear implant, with little or no mention of the risks or the uncertainty involved. This echoes the findings of previous work by Zaidman-Zait and Jamieson,Reference Zaidman-Zait and Jamieson4 who reported that increasing numbers of the parents of children receiving cochlear implants were using the internet to learn more about cochlear implants; however, the information on the internet offered almost no criticism of cochlear implants.
Interestingly, the video that scored most highly was published by a science channel on YouTube, with an overall score of 4. This video described and met clear aims, referenced its sources, and clearly described what a cochlear implant is and who it is suitable for. However, like most of the others, this video did not address the risks involved in the surgery.
• YouTube is increasingly used as a source of healthcare information
• No published papers to date have objectively assessed the quality of information available on YouTube about cochlear implants
• The quality of information regarding cochlear implants on YouTube is highly variable
• The results demonstrate a bias towards the positive attributes of cochlear implants, with little or no mention of the risks and uncertainty involved
• YouTube videos may be useful as supplementary information; however, critical elements required to make an informed decision are lacking
There was no correlation found between ‘likes’ or views and overall modified Discern score. This is consistent with previous studies,Reference Haymes and Harries9,Reference Mueller, Hongler, Jungo, Cajacob, Schwegler and Steveling10 which also demonstrate that numbers of views and ‘likes’ do not reflect the quality or reliability of videos. A higher ranking in the search results was found to weakly but significantly correlate with a higher overall modified Discern score (Rs = −0.31, p = 0.0364). Given this study's findings that overall views, ‘likes’ and ‘dislikes’ did not correlate with the Discern score, this result is curious and likely artefactual; although it is not clear exactly how YouTube rank their ‘most relevant’ videos, it is not likely to be based on the quality of the content.
Conclusion
YouTube videos on cochlear implants vary greatly in quality. Whilst many videos do clearly explain what a cochlear implant is and the benefits of an implant, the majority fail to explain the risks involved in cochlear implant surgery and any negative impact on quality of life that the surgery may have. The Royal College of Surgeons of England states that all patients considering surgery should receive balanced information, including an awareness of the complications of surgery,11 which the videos in our sample did not adequately provide. Medical practitioners should not recommend YouTube as the sole source of information about cochlear implants, although it may be a useful supplement.
Competing interests
None declared
Open access
