Introduction
Head and neck cancer accounts for 3 per cent of all cancer diagnoses in the UK and approximately 2 per cent of all cancer-related deaths.1 Early diagnosis is well recognised as a key principle in improving outcomes in cancer care. As a result, various ‘fast track’ referral pathways have been devised to get patients with potential malignancy into specialist services as soon as possible. Between 1999 and 2000, the Department of Health developed a series of National Guidelines that implemented a fast-track referral pathway that recommended all suspected cancer patients in England should be seen by a hospital specialist within 14 days.2,3 Similarly, the Scottish Referral Guidelines for Suspected Cancer were first published in 2002 and required a diagnosis of cancer within 31 days and treatment to commence within 62 days of referral.4
Several guidelines containing important red-flag features for head and neck cancer, such as persistent unexplained lumps, oral ulceration, hoarseness or odynophagia, have been published to help assist general practitioners utilise urgent referral pathways for head and neck cancer.4–6 However, despite these developments, several studies have indicated that a low percentage of those referred via the fast-track pathway are diagnosed with head and neck cancer and that many new patients present outwith the cancer pathway.Reference Li, Tikka, Morton, Chan, Hayois and Henderson7–Reference McKie, Ahmad, Fellows, Meikle, Stafford and Thomson11 This may be due to the low incidence of head and neck cancer compared with other cancers, limiting general practitioners’ familiarity with these red-flag symptoms and patients’ reluctance to seek medical attention.Reference Nieminen, Aro, Mäkitie, Harlin, Kainulainen and Jouhi12 It also may be attributed to the fact many head and neck cancer symptoms are non-specific and accompany common benign conditions, such as upper respiratory tract infections.Reference Nieminen, Aro, Mäkitie, Harlin, Kainulainen and Jouhi12 One effort to address this discrepancy in service delivery and increase the efficacy of cancer detection was the development of a symptom-based head and neck cancer risk calculator.
The head and neck cancer risk calculator is a statistical model designed to help predict an individual's risk of head and neck cancer by generating a percentage risk score based on symptoms, demographics and lifestyle factors. The score can be used to assist allocation to appropriate referral pathways, for example urgent suspicion of cancer more than 7.1 per cent predicted risk, urgent 7.1–2.2 per cent, routine less than 2.2 per cent. Clinic appointments offered with urgent referral pathway have a target a waiting time of 4 weeks, while routine referral pathway has a waiting time of 12 weeks. This can help to target patients appropriately to specialist assessment, rapid diagnosis and early management.Reference Tikka, Kavanagh, Lowit, Jiafeng, Burns and Nixon8 Whilst such calculators are well established in many common cancers, statistical model of risk stratification (Head and Neck Cancer Risk Calculator) was first designed in 2016 in two tertiary head and neck cancer centres in England, where it demonstrated high specificity and sensitivity.Reference Li, Tikka, Morton, Chan, Hayois and Henderson7,Reference Tikka, Kavanagh, Lowit, Jiafeng, Burns and Nixon8,Reference Tikka, Pracy and Paleri13 The calculator was later externally validated in a Scottish population, where it demonstrated a predictive power of 77 per cent.Reference Tikka, Kavanagh, Lowit, Jiafeng, Burns and Nixon8 The model was subsequently refined in 2019 (Head and Neck Cancer Risk Calculator version 2) to include social history, further head and neck cancer symptoms, such as unintentional weight loss, as well as symptom duration and laterality.Reference Tikka, Kavanagh, Lowit, Jiafeng, Burns and Nixon8 This more recent version of the calculator was shown to have better predictive power, sensitivity and specificity than the original model.Reference Tikka, Pracy and Paleri13 The risk calculator has proven useful when applied by secondary care surgeons, with several studies reflecting its use in reducing inappropriate referrals and facilitating urgent investigations in high-risk patients.Reference Li, Tikka, Morton, Chan, Hayois and Henderson7,Reference Banerjee, Voliotidis, Parvin and Rama14–Reference Kaddour, Jama, Stagnell, Kaddour, Guner and Kumar16
In July 2020, in an attempt to improve referral pathways, the Head and Neck Cancer Risk Calculator version 2 was introduced as a triage aid to primary care in NHS Lothian. The impact of the calculator in primary care triage was assessed in a study by Li et al.Reference Li, Tikka, Morton, Chan, Hayois and Henderson7 This study analysed the risk calculator score, referral urgency category and cancer yield from a sample of 1110 referrals from the pre-calculator period and 913 referrals from the post-calculator period. This preliminary analysis proved the calculator had a high sensitivity and showed a correlation between predicted and actual risk of cancer in those patients with a calculated risk of more than 70 per cent. However, this preliminary study also demonstrated the calculator typically inflated the predicted risk of cancer, which was thought to encourage primary care physicians to upgrade the urgency of referral. This resulted in a 30 per cent increase in patients referred with suspected cancer, which has the potential to overwhelm secondary care services.Reference Li, Tikka, Morton, Chan, Hayois and Henderson7
The conclusions from the results of the study by Li et al. were limited by the impact the coronavirus disease 2019 (Covid-19) pandemic had on primary care and cancer service delivery of discouraging patients from accessing medical assessment and skewing secondary care services towards urgent appointments. The primary aim of this study was to re-analyse the impact and efficacy of the Head and Neck Cancer Risk Calculator version 2 and to assess its utility in primary care triage in the post-Covid-19 period where services are less restricted than during the pandemic. The secondary aim of this study was to collect general practitioners’ feedback regarding the risk calculator after two years of its implementation.
Methods
The current primary care throat referral pathway within NHS Lothian is shown in Figure 1. At the point of referral, general practitioners are prompted to complete the risk calculator via an external website. Based on clinical judgement and the risk calculator score, which is entered on the e-referral, general practitioners are asked to select the appropriate referral category (routine or urgent or urgent suspicion of cancer). Head and neck consultants then evaluate all the referrals and re-triage them accordingly.
Figure 1. NHS Lothian head and neck referral pathway. GP = general practitioner
Data from patients who were seen following referral between January 2021 and August 2022 (post-calculator period) were collected using the NHS Lothian clinical electronic system. The data collected were referrer's urgency, re-triage category, Head and Neck Cancer Risk Calculator version 2 score, cancer diagnosis, any further investigation and types of cancer. Data were compared with the results from the pre-calculator period data (April–August 2019 and April–July 2020). Statistical analysis was performed using the chi-square test to compare categorical variables. Results were considered statistically significant if the p value was less than 0.05. Sensitivity, specificity, positive predictive value and negative predictive value were calculated by re-categorising data into two-by-two tables based on the authors’ recommended cut-off score as urgent suspicion of cancer or non-cancer and cancer diagnosed versus no cancer diagnosed.
Only patients referred by primary care were included in the study. Exclusion criteria were non-primary care referrals, patients with missing records, referrals without an exact risk calculator score and consultation of patients carried out based on referrals from the pre-calculator cohort (Figure 2).
Figure 2. Data exclusion pathway. GP = general practitioner
General practitioner feedback
An online questionnaire was made available to all general practitioners in NHS Lothian. The questionnaire consisted of five questions to investigate the usefulness of the risk calculator, the influence of the risk calculator in decision making, and the impact of the risk calculator score on anxiety and reassurance.
Results
A total of 1110 and 1559 referrals from the pre- and post-Head and Neck Cancer Risk Calculator version 2 periods, respectively, were analysed. The age and sex distributions were comparable between both cohorts (age, p = 0.10; sex, p = 0.42) (Table 1).
Table 1. Comparison of age and sex in the pre- and post-Head and Neck Cancer Risk Calculator version 2 cohorts
*One patient from the pre-calculator cohort did not specify sex
Overall, there was a statistically significant increase in urgent suspicion of cancer referrals in the post-calculator period from 51.1 per cent to 64.0 per cent (p < 0.01) (Table 2). Urgent suspicion of cancer referrals further increased by 5.9 per cent after being re-triaged by secondary care specialists, with all cancers diagnosed were in the post-triaged urgent suspicion of cancer category (Table 3). The rate of cancer diagnosis was slightly increased in the post-calculator cohorts, but this failed to reach statistical significance (2.7 per cent vs 3.3 per cent, p = 0.41). There was no significant difference in the rate of cancer diagnosis between the pre- and post-calculator urgent suspicion of cancer referrals (4.1 per cent vs 4.6 per cent, p = 0.62) (Table 2).
Table 2. Comparison of referral categories and cancer diagnoses between pre- and post-Head and Neck Cancer Risk Calculator version 2 cohorts
Table 3. Referrals re-triaged by head and neck consultants in secondary care
The Head and Neck Cancer Risk Calculator version 2 usage rate in the post-calculator period was 89.7 per cent (1398 cases), with 10.3 per cent of referrals (161 cases) lacking a score in the period after the calculator was introduced. There was no difference between the rate of cancer diagnosed with or without the calculator being used (3.3 per cent vs 3.1 per cent). However, there was a significant difference in the rate of cancer diagnosis within the urgent suspicion of cancer referrals category with and without using the calculator (4.3 per cent vs 15.2 per cent, p = 0.008) (Table 4).
Table 4. Referral categories and head and neck cancer diagnosis between referrals in post-Head and Neck Cancer Risk Calculator version 2 cohort with and without risk calculator score
All post-calculator data were re-categorised using the cut-off scores proposed by the authors (urgent cut-off, ≥2.2 per cent, urgent suspicion of cancer cut-off, >7.1 per cent). It was found that 63.7 per cent of referrals were within the urgent suspicion of cancer category, with 96 per cent of all cancers diagnosed in this category (Table 5). There was also a positive correlation between a higher risk calculator score and a higher likelihood of head and neck cancer diagnosis. This was especially true for risk calculator scores of more than 70 per cent (Figures 3 and 4). However, across all deciles of cancer risk predicted, the calculator overestimated the risk of malignancy significantly.
Table 5. Referral categories for the post-Head and Neck Cancer Risk Calculator version 2 cohort based on cut-offs proposed by the authors
Figure 3. Distribution of the post-Head and Neck Cancer Risk Calculator version 2 score based on author suggested cut-off scores.
Figure 4. Head and neck cancer diagnosis distribution by Head and Neck Cancer Risk Calculator version 2 score.
When referrals were stratified using the risk score to over and under 7.1 per cent (urgent suspicion of cancer threshold recommended) and by the presence or absence of cancer following review, the sensitivity of the risk calculator was 95.7 per cent and specificity was 37.4 per cent. The positive predictive value and negative predictive value were 4.9 per cent and 99.6 per cent, respectively (Table 6).
Table 6. Sensitivity, specificity, positive predictive value and negative predictive value based on cut-offs proposed by authors
USOC = urgent suspicion of cancer
The risk calculator received positive feedback from primary care providers. The response rate to the survey was low. There was a total of 38 responses out of about 1130 general practitioners working in NHS Lothian. Overall, 82 per cent of responding general practitioners agreed that the risk calculator was a useful tool, with 63 per cent stating that the risk calculator had had an influence on their decision making and 53 per cent reporting avoidance of referrals with low-risk calculator scores. In addition, 43 per cent of general practitioners fed back that a high-risk calculator score did not impact patients’ anxiety levels (Figure 5).
Figure 5. General practitioner feedback for the Head and Neck Cancer Risk Calculator version 2: (a) usefulness, (b) influence on decision making, (c) avoid referral with low score and (d) rising anxiety with high score.
Discussion
Since the development of the Head and Neck Cancer Risk Calculator in 2016, studies have shown that it has proved useful in re-triaging referrals at a secondary care level.Reference Banerjee, Voliotidis, Parvin and Rama14–Reference Paleri, Hardman, Tikka, Bradley, Pracy and Kerawala17 This was especially true during the Covid-19 pandemic period, which significantly restricted consultations in secondary care. Multiple groups have suggested incorporating the Head and Neck Cancer Risk Calculator in the secondary care cancer triaging pathway,Reference Banerjee, Voliotidis, Parvin and Rama14–Reference Paleri, Hardman, Tikka, Bradley, Pracy and Kerawala17 but our project is the first to investigate the performance of the Head and Neck Cancer Risk Calculator version 2 in the primary care setting.Reference Li, Tikka, Morton, Chan, Hayois and Henderson7 Interpretation of our initial findings was hampered by the impact of the pandemic, which prompted this updated analysis in the post-pandemic era.
The age and sex distributions, and head and neck cancer diagnosis rates post introduction of Head and Neck Cancer Risk Calculator version 2 were comparable to the data obtained from the pre-calculator cohort, suggesting overall similarity between the two cohorts. However, a significant increase in the urgent suspicion of cancer referral rate was observed following the introduction of the Head and Neck Cancer Risk Calculator version 2 (51.1 per cent vs 64.0 per cent, p < 0.01), with a further 5.9 per cent increase in urgent suspicion of cancer referrals post-triage by secondary care specialists. This could be explained by significant overestimation of cancer risk.
Five per cent of patients with a score over 7.1 were ultimately diagnosed with cancer, slightly higher than the score recommended in the National Institute for Health and Care Excellence and Scottish guidelines, which recommend a 3 per cent threshold on cancer pathways.6 Indeed, the rate of cancer in the cohort did not exceed 10 per cent until the calculator score predicted a risk of over 80 per cent. Inflated risk prediction seems likely to encourage general practitioners to refer patients as urgent suspicion of cancer, increasing the burden on secondary care to meet government timelines. This result is similar to that found in our previous study, which was conducted during the Covid-19 pandemic.Reference Li, Tikka, Morton, Chan, Hayois and Henderson7
All studies of the risk calculator to date have involved secondary care physicians. These studies confirm the benefits of the calculator in screening patients and avoiding unnecessary appointments.Reference Banerjee, Voliotidis, Parvin and Rama14–Reference Paleri, Hardman, Tikka, Bradley, Pracy and Kerawala17 Our group is the first to report its use in the primary care setting.
Interestingly, in our user survey, general practitioners reported avoidance of referrals with low-risk calculator scores. From our data, it may seem unlikely that a significant number of low-risk patients were not referred as an explanation for the higher percentage of urgent suspicion of cancer referrals because the number of referrals per month was higher in the post-calculator cohort (pre-calculator, 123 referrals per month vs post-calculator, 187 referrals per month). However, it is difficult to draw a conclusion from the available data as there are no formal data on the number of patients who were presented with relevant symptoms and were not referred by primary care.
From our experience, there was an overall significant increase in urgent suspicion of cancer referrals for all cancer types compared with the pre-Covid-19 period. It could also be possible that the primary care clinicians who completed the questionnaire are not representative of referring clinicians. A further potential reason to explain the increase in urgent referrals is that general practitioners were reviewing more patients with Covid-19-related symptoms, which may overlap with red-flag symptoms for head and neck cancer.Reference Aiyegbusi, Hughes, Turner, Rivera, McMullan and Chandan18,19
Despite the overestimation of risk, the Head and Neck Cancer Risk Calculator version 2 demonstrated a good diagnostic efficacy profile when recommended cut-offs are applied: high sensitivity (95.7 per cent) and negative predictive value (99.6 per cent). This is comparable to the sensitivity and negative predictive value when the calculator was used in secondary care triage.Reference Tikka, Kavanagh, Lowit, Jiafeng, Burns and Nixon8,Reference Tikka, Pracy and Paleri13,Reference Tikka, Paleri and MacKenzie20 Hence, when a patient has a risk calculator score of less than or equal to 7.1, general practitioners should be able to safely refer patients on a non-urgent suspicion of cancer pathway, unless there are obvious clinical concerns. Our study showed that only 0.4 per cent of patients with a risk calculator score less than or equal to 7.1 were diagnosed with cancer as compared with a 5 per cent actual cancer risk when the score exceeds 7.1 per cent.
Our study also suggests that the Head and Neck Cancer Risk Calculator version 2 does have efficacy in predicting head and neck cancer risk. There was a positive correlation between predicted and actual risk of cancer, particularly after the 70 per cent predictive risk level, where the actual cancer detection rate increased from 5 per cent to 50 per cent. This trend is similar to that found in our previous study,Reference Li, Tikka, Morton, Chan, Hayois and Henderson7 although in both studies risk was overestimated. We therefore suggest that there may still be a role for the Head and Neck Cancer Risk Calculator version 2 in a primary care head and neck cancer referral pathway, since it performs differently from its application in the secondary care setting. Not only is there some correlation between predicted and actual risk, but in addition the tool encourages general practitioners to take a targeted clinical history. This is especially crucial if general practitioners are less familiar with head and neck cancer-related symptoms than head and neck specialists in secondary care due to the relative rarity of head and neck cancer.1,3
Our study demonstrated that there was a high cancer diagnosis rate (15 per cent) in the cohort where the referrals did not include the calculator score. The reason for this requires further study, but it is likely to be because this cohort included patients who were considered to have an obvious malignancy and as a result the referring clinician felt no risk calculator score would influence referral urgency. This indicates the importance of the clinical judgement made from correlation with the clinical picture rather than full reliance on the risk calculator score.
Overall, the Head and Neck Cancer Risk Calculator version 2 was well received by the surveyed general practitioners and the majority reported the calculator was a useful tool. However, the general practitioner sample size was small and thus subject to sampling bias. It was also noted in the feedback survey that some general practitioners were not aware of the calculator. More education sessions about the Head and Neck Cancer Risk Calculator version 2 could therefore be useful in the primary care setting.
Limitations
Despite including over 2500 patients in this study, the number of cancers (n = 81) was low. This limits the conclusions that can be drawn. In addition, because of local limitations related to automated referral systems, we were unable to capture specific symptomatology but simply recorded an overall risk score for each patient. We are therefore unable to refine the risk calculator based on specific symptoms at the current time.
Future directions
Given the repeated evidence that primary care application of the Head and Neck Cancer Risk Calculator version 2 leads to an overestimation of cancer risk and is associated with an increase in the number of patients referred on the urgent suspicion of cancer pathway, we now plan to move away from requiring general practitioners to provide a score. Although we plan to signpost general practitioners to the tool, we also plan to capture symptoms from the Head and Neck Cancer Risk Calculator version 2 electronically within the referral template to try and optimise its use in primary care referral.
• Although it may not be the only contributing factor, the integration of the Head and Neck Cancer Risk Calculator version 2 in the primary care referral pathway for head and neck cancers appeared to increase the referrals of urgent suspicion of cancer after the coronavirus disease 19 pandemic
• The use of the Head and Neck Cancer Risk Calculator version 2 in primary care shows similar high sensitivity and negative predictive values to those found for its use in secondary care, indicating the possibility of using it in primary care in a wider setting
• The Head and Neck Cancer Risk Calculator version 2 overestimates the actual risk of cancer
• There is a positive correlation between predicted and actual cancer risk, particularly after the 70 per cent predictive risk.
• More work is required to optimise the gap between predictive and actual risk before the Head and Neck Cancer Risk Calculator version 2 can be adopted more widely in primary care referral pathways
Conclusion
This study demonstrates that the introduction of Head and Neck Cancer Risk Calculator version 2 into primary care referral pathways leads to a significant overestimation of the risk of cancer, which may be associated with an increased percentage of urgent suspicion of cancer referrals in the post-pandemic era. The positive correlation of predictive and actual cancer risk shown by our study suggests there may be a future role for the calculator in the primary referral pathway. However, further work is required to improve the gap between predictive and actual cancer risk before full adoption of the calculator in the primary care setting. Although it is apparent the calculator was well received by general practitioners, more work is needed to refine this tool with the aim of improving its performance at the interface between primary and secondary care.
Competing interests
None declared
