Introduction
In many healthcare systems, including Ireland and the United Kingdom, GPs are the first point of contact for patients with the health system and the majority of medical problems are subsequently managed in primary care (O’Donnell, Reference O’Donnell2000). A key role of the GP is to act as a gatekeeper for access to secondary services, with one systematic review showing an inverse association between good quality primary care and avoidable hospitalisation (Rosano et al., Reference Rosano, Loha, Falvo, Van der Zee, Ricciardi, Guasticchi and De Belvis2012). Good gate keeping in general practice is dependent on a strong doctor–patient relationship, understanding of the bio-psychosocial model as well as effective diagnostic and referral-making skills (Mathers and Mitchell, Reference Mathers and Mitchell2010). Optimal communication at the primary–secondary care interface is necessary to prevent delays in care, patient frustration and inaccurate information (Sampson et al., Reference Sampson, Cooper, Barbour, Polson and Wilson2015) and the importance of high-quality referral letters has been recognised (Ramanayake, Reference Ramanayake2013).
Previous studies of referral letters have found content deficits in the documentation of: medications (Toleman and Barras, Reference Toleman and Barras2007); prior investigations (Culshaw et al., Reference Culshaw, Clafferty and Brown2008); presenting symptoms (Su et al., Reference Su, Cheang and Khalil2013); and appropriateness, particularly regarding stated level of urgency (Blundell et al., Reference Blundell, Clarke and Mays2010). One study reported that completeness of documentation could have an important impact on how and when the patient is managed by specialists (Jiwa et al., Reference Jiwa, Walters and Cooper2002). Recent qualitative research involving patient participants stated that gaps in their care were often due to problems in the ‘coordination of management’ (Tarrant et al., Reference Tarrant, Windridge, Baker, Freeman and Boulton2015). Furthermore, hospital physicians in Norway considered only 15.6% of referrals from general practice to be of good quality (Martinussen, Reference Martinussen2013). A report commissioned by the King’s Fund found that the quality of ‘a substantial minority’ of referral letters could be improved (Foot et al., Reference Foot, Naylor and Imison2010).
Attempts to improve referral letter quality have therefore been the subject of research for some years but neither a Cochrane review (Akbari et al., Reference Akbari, Mayhew, Al-Alawi, Grimshaw, Winkens, Glidewell, Pritchard, Thomas and Fraser2008) or a previous systematic review (Faulkner et al., Reference Faulkner, Mills, Bainton, Baxter, Kinnersley, Peters and Sharp2003) showed evidence of improvement by interventions. This study aims to review the current literature pertaining to interventions that are designed to improve referral letter quality.
Methods
The authors believed that a narrative review would best facilitate focussed analysis of the literature. A search strategy was designed using a Problem, Intervention, Comparator, Outcome (PICO) model (see Table 1). The databases used in the study included: PubMed, EMBASE, Web of Science, PsycINFO, Cochrane and CRD. The search used the following key words and MeSH terms for example ‘referral’/exp OR referral AND (‘letter’/exp OR letter) AND gp AND (‘secondary’/exp OR secondary) AND (‘care’/exp OR care) AND (‘quality’/exp OR quality) AND improvement; innovation* AND patient AND information AND referral* AND primary AND care*.
Problem, Intervention, Comparator, Outcome strategy
PCP=primary care provider.
Inclusion and exclusion criteria
Research papers published in peer-reviewed journals between January 2007 and 31 July 2017 that were written in the English language were included in the search. The start date was chosen to overlap with the previous Cochrane review. Only papers that focused specifically on interventions to improve the quality of patient information conveyed in primary to secondary care referrals of health systems in developed countries were considered. The exclusion criteria were papers not written in the English language, studies that did not evaluate interventions on letter quality, studies of referrals to non-secondary care destinations. Papers that focused on cost effectiveness were not included in order to focus on the referral letter quality rather than financial implications.
Screening
Literature was reviewed by two researchers using the inclusion and exclusion criteria outlined above, and queries on the suitability of individual studies were discussed and decided upon by a third assessor. Bibliographies of selected publications were screened for any more potentially relevant resources. Previous research (Frye and Hemmer, Reference Frye and Hemmer2012; Kvan, Reference Kvan2013; Lewis et al., Reference Lewis, Edwards, Whiting and Donnelly2017) in the fields of medical and inter-professional education have applied Kirkpatrick’s levels as a model for evaluating learning and training outcomes (Kirkpatrick, Reference Kirkpatrick1967). The approach involves categorising the outcomes of an intervention into one of four levels: (1) the level of attitude or reaction; (2) whether learning has occurred in terms of knowledge or skills; (3) to what extent has the skills or knowledge been applied in practice; (4) an impact on the health system or patients (Lewis et al., Reference Lewis, Edwards, Whiting and Donnelly2017). While the outcome measurements from each level are not hierarchical, they are considered a useful starting point for comprehensive evaluation (Lewis et al., Reference Lewis, Edwards, Whiting and Donnelly2017) and go beyond the level of learner satisfaction (Frye and Hemmer, Reference Frye and Hemmer2012).
Results
The initial search yielded 291 papers after the removal of duplicates. Full details of the searches are included in Figure 1. Selected papers were screened by title and subsequently screened on their abstract or full manuscript. A total of 18 studies were selected for review based on the set criteria. Papers were mainly excluded that did not pertain to referral letters, were not directed to secondary care and did not evaluate an intervention to promote quality. The publications were assessed thematically, and their results presented by intervention type.
Search string. * indicates a truncation search term that matches any string and can be used anywhere in the search string.
Four of the studies demonstrated some degree of impact on the health system (Evans, Reference Evans2009; Kim et al., Reference Kim, Chen, Keith, Yee and Kushel2009; Rokstad et al., Reference Rokstad, Rokstad, Holmen, Lehmann and Assmus2013; Wright et al., Reference Wright, Hagmayer and Grayson2015). Table 2 describes each of the studies and Table 3 outlines the main findings of each intervention.
Description of studies
Study outcomes
ER=electronic referral; PCP=primary care provider.
Impact of electronic referrals (ERs)
Shaw and de Berker (Reference Shaw and de Berker2007) reviewed electronic and paper referrals written and found that ERs were more effective at containing demographic data when compared with manual referring but less effective at clinical data that would lead to a diagnosis. The authors cautioned against prioritising the ER process over the clinical context of the patients’ problems. This was a small, descriptive study, involving retrospective data analysis. ERs were superior at recording prescription lists and patients’ demographics compared with paper referrals but difficulties cited with free text may reflect an inherent problem with the design of the proforma itself. Nash et al. (Reference Nash, Hespe and Chalkley2016) found that ERs were of better quality than handwritten, providing more information on medication and medical history. In contrast, a survey of 298 primary care providers (PCPs), (Kim et al., Reference Kim, Chen, Keith, Yee and Kushel2009), found that the majority believe that ERs promoted better quality of care. This was a self-report design and consequently, the results are subject to recall bias. The survey was conducted online and it is possible that participants that were more IT savvy would be more inclined to respond to the web-based questionnaires.
In a qualitative study of the ER system (Hysong et al., Reference Hysong, Esquivel, Sittig, Paul, Espadas, Singh and Singh2011) primary and secondary care physicians agreed that ERs could enhance the referral system but that key systems coordination principles needed to be in place in order for an ER system to function. These included clarity of roles, standardisation of practises and adequate resourcing. This qualitative study was limited to participants from a single health network, which may limit transferability to other health centres. Zuchowski et al. (Reference Zuchowski, Rose, Hamilton, Stockdale, Meredith, Yano, Rubenstein and Cordasco2015) found that the capability of the ER system to improve communication with secondary care specialists varied between specialties. A recurring theme in relation to ER systems was that of ‘rigid informational requirements’, with many GPs resorting to telephone and email use to communicate with those specialists ‘with whom they had established relationships’. This study was confined to one regional network, which limits the transferability of results. Only PCPs were interviewed for this study. Involvement of the specialists who received the referral letters would had been useful for triangulation.
Impact of peer feedback
A year-long intervention (Evans, Reference Evans2009) which provided GPs with protected and resourced time for peer-review and regular meetings with hospital specialists reported substantial improvement in letter quality. Referrals were rated for their content and in two of the three participating practices the content improved. This was a one year pilot study but it was limited to one region and the authors suggest that the intervention may not be suited to other regions. Xiang et al. (Reference Xiang, Smith, Hine, Mason, Lanza, Cave, Sergeant, Nicholson and Devlin2013) retrospectively analysed GP referrals before and after the introduction of a system that provided GPs with peer feedback for seven months. They found significant improvements in documentation of past medical history and prescribed medication; however, no significant increase in the relevant clinical information or clarity of reason for referral was detected. Both internal and external validity were strong in this study as the design involved a large number of referral letters from a setting with a diverse population. However, the hospital specialist was not involved in assessing referral letters. There was a follow-up with only seven months between baseline and assessment periods. An uncontrolled study of GP referrals to endoscopy units (Elwyn et al., Reference Elwyn, Owen, Roberts, Wareham, Duane, Allison and Sykes2007), referrals were analysed by two GPs to evaluate their adherence to NICE guidelines. Same day written feedback was provided to those whose letters did not comply, outlining their deficits. The mean adherence to guidelines improved from 55% before the intervention to 75% afterwards. This study involved a wider timeframe – five months pre and six months post-intervention data, which did not include a control group. Authors stated that they received several letters of complaint from clinicians voicing concerns that the system would erode clinical freedom.
Impact of templates
A study of referrals using templates from nine primary care practices to nephrology clinics reported a significant increase in the level of documentation of relevant clinical information from pre- to post-intervention (Haley et al., Reference Haley, Beckrich, Sayre, McNeil, Fumo, Rao and Lerma2015). Furthermore, in post-intervention interviews, PCPs said that the intervention helped to increase awareness of risk factors and management guidelines in chronic kidney disease. Familiarity with interviewees may have introduced bias and skewed the results. The specific patient group attending PCPs and nephrology practices in two locations are not reflective of the wider healthcare system. Practices were recruited on a voluntary basis so volunteer bias was a factor in this study. A study of referrals from general practice to lung specialists (Rokstad et al., Reference Rokstad, Rokstad, Holmen, Lehmann and Assmus2013) investigated an optional electronic guideline incorporated in the practice software. Lung specialists, who were blinded as to whether the referrers were using the intervention or not, used an evaluation form to score the referral and reported improved quality of referrals and time saving. Both the GP and hospital specialist were interviewed about the referral tool, which facilitates a wider range of perspectives. There were problems with the implementation of the intervention as many GPs who agreed to use the template did not continue to do so, which may reflect a problem with usability of the template.
Wahlberg et al. (Reference Wahlberg, Valle, Malm and Broderstad2015) conducted a randomised cluster trial using templates for four commonly encountered, potentially serious presenting complaints across 14 practices in Norway. Statistically significant improvements in quality of referral letters were associated with three of the four templates were reported. The randomised cluster design of this study led to a number of problems. First, there is possible bias whereby more proactive GPs may be inclined to use the referral templates and thereby skew results. Second, adherence to the referral template may be variable depending on workload and time constraints. A second analysis published one year later (Wahlberg et al., Reference Wahlberg, Braaten and Broderstad2016) investigated the impact on patient experience of the care process using self-report questionnaires and found no significant improvement in patient experience. This paper had a high response rate (82%) but the use of a short-form questionnaire limited the depth of data that was collected. The authors conceded that the study lacked a solid analytical framework. A final analysis (Wahlberg et al., Reference Wahlberg, Valle, Malm, Hovde and Broderstad2017) investigated the impact of the referral template on the quality of care received in the hospital and, similarly, no significant improvement in hospital care was observed. A large number of assessors were involved in grading the quality of referrals, which may have implications for reproducibility of the findings. The authors acknowledged that because of the retrospective nature of the design, that they can only assess actions recorded and that there may have been actions performed and not recorded.
Eskeland et al. (Reference Eskeland, Brunborg, Rueegg, Aabakken and de Lange2017) asked GPs to read gastroenterology-related clinical vignettes and write clinical referral letters based on the information. GPs were randomised to a control or an intervention, which was a set of diagnosis-specific checklists. A consistent improvement in referral quality was observed in the intervention group. Clinical vignettes were used instead of real-life consultations in order to standardise the setting but the findings are therefore not reflective of the interpersonal interactions of which general practice consultations consist. The system did not record all aspects of the referral and this may affect the validity of the findings. Jiwa and Dhaliwal (Reference Jiwa and Dhaliwal2012) introduced templates for referring to six hospital disciplines. They compared 56 referral letters from seven GPs (pre-intervention) to 48 ERs four months after and found that the amount of referral information and the confidence of the clinician receiving the referral in their ability to make a decision based on the referral increased. Of the 10 GPs who commenced the study, only seven completed the intervention, which may reflect usability problems with the referral software. The mean number of patients per practice was given but not the total number of patients involved in the study. Jiwa et al. (Reference Jiwa, Meng, O’Shea, Magin, Dadich and Pillai2014), in a non-randomised controlled trial asked GPs in both control and intervention groups to read clinical vignettes and make referral decisions based on what they had read. The quantity of clinical information in the letter improved but this did not result in a significant change in appointment scheduling. The design of this study involved actors playing a part in a simulated consultation and would not reflect the reality of the interaction of a real doctor–patient interaction and it is likely therefore that the referrals suggested by the participating GPs would be different from real-life situations. This paper took into account that there was no doctor–patient interaction as actors are used to play the role of the patient. In phase one, GPs were shown vignettes of an actor-patient performing a monologue and phase two, the intervention group used the referral software and the control group did not. GPs withdrew after phase one in the control and intervention groups for reasons that were not explained which resulted in lower numbers in phase two.
Impact of mixed interventions
A pilot study of 13 practices in the United Kingdom (Wright et al., Reference Wright, Hagmayer and Grayson2015) used a service combining referral guidelines, templates and feedback from those who triage referrals. In the intervention group, fewer referrals were challenged for incompleteness or insufficiency of information and the number of referrals decreased. Interviews with practice staff and patients found a high degree of satisfaction with the system. Practices were recruited on a voluntary basis so volunteer bias was a factor in this study. In a small-scale study (Corwin and Bolter, Reference Corwin and Bolter2014), GPs were initially given written feedback on their letters from hospital colleagues and a comparison was made between the letter quality before and five months post this intervention. Second, ERs were introduced and a comparison was again made between referrals before and five months after. Feedback improved the referral quality and ERs did not. The sample size was small but quality was assessed at five months and again at 10 months after baseline. Quality of referrals was measured using only a single tool; a nine-point checklist, with some letters scoring high because they contained a lot of information despite being difficult to follow and sometimes incoherent.
Discussion
Our results have shown that several interventions have had moderate success in improving referral letter quality. Some studies claim to have had an additional impact on the health system and have been initially categorised as a Kirkpatrick level 4. However, a deeper analysis contests this assertion. Kim et al. (Reference Kim, Chen, Keith, Yee and Kushel2009) were relying on the perceptions of physicians and not on an objective measure of systems improvement. Rokstad et al. (Reference Rokstad, Rokstad, Holmen, Lehmann and Assmus2013) found that specialists could afford to spend less time reviewing letters done using templates but this time saving does not necessarily translate into a positive impact for the system or the patient. Both Evans (Reference Evans2009) and Wright et al. (Reference Wright, Hagmayer and Grayson2015) report a reduction in referrals as a result of their intentions but the use of referral counts as a proxy for improvements in health systems has been contested (Foot et al., Reference Foot, Naylor and Imison2010). Higher or lower referral rates do not translate to good quality practice or referral writing (Knottnerus et al., Reference Knottnerus, Joosten and Daams1990).
In all, 12 of the interventions scored a Kirkpatrick level of three but the outcomes based focus of system can give an impression of high impact, while missing out in the processes involved the associated intricacies. One such feature in the case of templates is that, in many instances, GPs preferred to use free text rather than the ‘tick-box’ approach provided by the template, which was interpreted as a preference among GPs for including the patient narrative (Jiwa et al., Reference Jiwa, Meng, O’Shea, Magin, Dadich and Pillai2014). Similarly, Zuchowski et al. (Reference Zuchowski, Rose, Hamilton, Stockdale, Meredith, Yano, Rubenstein and Cordasco2015) commented on the rigidity of ERs and that inter-clinician communication was an essential component of referrals. More robust methodology is also needed, including follow-up assessments at six and 12 months post-intervention; longer duration of interventions and involvement of GPs at the design of any intervention that involves them. We suggest that a needs assessment of GPS be conducted and described in any future paper involving interventions that involve them.
Perceptions about quality differ between GPs and hospital specialists. In a large survey of American physicians (O’Malley and Reschovsky, Reference O’Malley and Reschovsky2011) 69.3% of GPs believed that they usually included relevant clinical details in referral letters whereas only 34.8% of consultants said that they received those details. Our study has reviewed investigations that were designed to improve referral letter quality but this question must be considered in the context of how quality is assessed. Furthermore, long standing concerns over a lack of consensus among practising GPs about what constitutes a good quality referral letter have been expressed (Jiwa and Burr, Reference Jiwa and Burr2002).
The more favourable interventions reviewed in this paper involved a combination of peer feedback with a software intervention (Corwin and Bolter, Reference Corwin and Bolter2014; Wright et al., Reference Wright, Hagmayer and Grayson2015). This finding has been noted in research (Bennett et al., Reference Bennett, Haggard, Churchill and Wood2001) showing that ear, nose and throat referrals from primary care were improved by combining a basic template with an educational video. GPs have expressed preference to learn about how best to make a referral and various clinical conditions through engagement with consultant colleagues (Eaton, Reference Eaton2008). Interestingly, a recent meta-analysis showed that there was a role for ‘interactive communication’ to improve ‘the effectiveness of primary care-specialist collaboration’ (Foy et al., Reference Foy, Hempel, Rubenstein, Suttorp, Seelig, Shanman and Shekelle2010). A prior review of healthcare communication called for an increase in feedback between GPs and specialists to improve the quality of referral letters (Vermeir et al., Reference Vermeir, Vandijck, Degroote, Peleman, Verhaeghe, Mortier, Hallaert, Van Dael, Buylaert and Vogelaers2015). Furthermore, a recent qualitative study with newly qualified GPs proposed integration of training across different specialties to help future GPs and consultants to ‘work collaboratively across the organisational boundaries’ at the primary–secondary care interface (Sabey and Hardy, Reference Sabey and Hardy2015).
Jiwa and Dadich (Reference Jiwa and Dadich2013) systematically analysed the literature around communication and reported overall poor quality of communication leading to compromised patient outcomes. The question of how to improve quality has eluded previous systematic reviews. Its complexity is that it is interlinked with several other factors relating to the health system, clinician capacity, attitudes and experiences, as well as the complexity of the clinical problem. A systematic review, restricted to protocol, cannot peel away the layers of contextual variables. Indeed, an analysis by Pawson et al. (Reference Pawson, Greenhalgh, Brennan and Glidewell2014) of the lack of success of reviews of healthcare studies stated that ‘multiple lessons’ are often missed because of their failure to ‘address the wider scenario’. Previous research on peer feedback (Jiwa et al., Reference Jiwa, Meng, O’Shea, Magin, Dadich and Pillai2014) concurs with studies included in this review (Evans, Reference Evans2009; Haley et al., Reference Haley, Beckrich, Sayre, McNeil, Fumo, Rao and Lerma2015) showing GPs welcome feedback but, that as a stand-alone measure, it does not significantly improve quality of referrals.
Studies varied in methodologies: 12 studies were quantitative, four were qualitative and two studies used a mixed methods approach. Study limitations included having a small sample size (Jiwa and Dhaliwal, Reference Jiwa and Dhaliwal2012; Corwin and Bolter, Reference Corwin and Bolter2014), and being limited to a single region or health service network (Shaw and de Berker, Reference Shaw and de Berker2007; Evans, Reference Evans2009; Hysong et al., Reference Hysong, Esquivel, Sittig, Paul, Espadas, Singh and Singh2011; Rokstad et al., Reference Rokstad, Rokstad, Holmen, Lehmann and Assmus2013; Wahlberg et al., Reference Wahlberg, Valle, Malm and Broderstad2015; Reference Wahlberg, Braaten and Broderstad2016; Reference Wahlberg, Valle, Malm, Hovde and Broderstad2017; Zuchowski et al., Reference Zuchowski, Rose, Hamilton, Stockdale, Meredith, Yano, Rubenstein and Cordasco2015; Nash et al., Reference Nash, Hespe and Chalkley2016), and consequently, the findings may not be generalisable and relevant to other health systems. Some of the studies involved only PCPs as participants, whereas the involvement of specialists would have been useful for triangulation (Xiang et al., Reference Xiang, Smith, Hine, Mason, Lanza, Cave, Sergeant, Nicholson and Devlin2013; Eskeland et al., Reference Eskeland, Brunborg, Rueegg, Aabakken and de Lange2017). Many of the quantitative studies had pre and post-intervention data analysis but no longer term follow-up after one year (Elwyn et al., Reference Elwyn, Owen, Roberts, Wareham, Duane, Allison and Sykes2007; Xiang et al., Reference Xiang, Smith, Hine, Mason, Lanza, Cave, Sergeant, Nicholson and Devlin2013; Haley et al., Reference Haley, Beckrich, Sayre, McNeil, Fumo, Rao and Lerma2015). Some of the studies involved voluntary participation with associated volunteer bias (Haley et al., Reference Haley, Beckrich, Sayre, McNeil, Fumo, Rao and Lerma2015; Wright et al., Reference Wright, Hagmayer and Grayson2015) and one of the studies used a self-report design with the potential for recall bias (Kim et al., Reference Kim, Chen, Keith, Yee and Kushel2009). Therefore, it is likely that there is insufficient rigour in the studies analysed to make strong conclusions and recommendations.
Limitations only papers published in the English language were reviewed and there is a possibility that publications were missed. There is also a risk of publication bias in that studies that reported negative findings from interventions were not published. Future research should include objective assessments of clinical care quality measures to investigate more rigorously if referral letter improvements can improve the care the patient receives. Also, studies that evaluate the processes involved in the referral including the patient experience are needed as well as evaluations of the implementation of quality improvement interventions. Research on the sustainability of ongoing peer feedback (between GPs) and inter-professional communication involving clinicians who write and receive referral letters with long term follow-up data is needed.
Conclusion
This review has summarised and categorised interventions for quality improvement in GP referral letters over the past 10 years. Our analysis demonstrates that a combination of interventions, introduced as part of a joint package and involving peer feedback can improve both letter quality and, in a small number of instances, the healthcare system. Inter-clinician collaboration is most likely the single most important factor.
Acknowledgements
The authors would like to acknowledge the help of Mr. Fintan Bracken and Ms. Liz Dore; librarians at the University of Limerick.
Conflicts of Interest
None.
