Dementia is increasingly prevalent with age, affecting an estimated 17% of those aged over 89 (Matthews Reference Matthews, Arthur and Barnes2013). In England, only one-third of those estimated to have dementia (aged 65 or over) have a coded diagnosis in their medical records (NHS Digital 2022), which suggests that cases of dementia are being missed and appropriate treatment and support is not being initiated. Barriers to accessing and utilising dementia care are multifactorial. For instance, there may be delays in seeking help due to poor recognition and understanding of symptoms as well as a reluctance to seek help because of stigma (National Collaborating Centre for Mental Health 2018). Primary care services are often the first point of contact for people with dementia and therefore form an important interface in determining which patients require referral to specialist services for further assessment (Pentzek Reference Pentzek, Wagner and Abholz2019; Creavin Reference Creavin, Noel-Storr and Langdon2022). In fact, previous studies have found that a general practitioner's (GP's) clinical judgement is a known added predictor for identification of individuals at risk of dementia (Box 1) (Pentzek Reference Pentzek, Wagner and Abholz2019).
What is a predictor variable?
A predictor variable is an independent variable that can be used to assess whether there is a relationship between a specific outcome (dependent variable) and the predictor variable, independent of other variables considered in the model.
For instance, Pentzek et al (Reference Pentzek, Wagner and Abholz2019) found that the GP's clinical judgement added specific information related to the prediction of time to dementia onset in a 12-year period, independent of other variables used in their model.
Diagnosing dementia is challenging for a multitude of reasons. For example, it can be difficult to differentiate between cognitive decline associated with ageing and cognitive decline due to pathological processes assumed to underlie dementia (Slavin Reference Slavin, Brodaty and Sachdev2013). Dementia can also mimic other conditions, such as depression, compounding diagnostic uncertainty (Dungen Reference Dungen, Warwijk and Horst2011). There is no single diagnostic tool to diagnose dementia. In the primary care setting, the National Institute for Health and Care Excellence (NICE) recommends that primary care physicians initially take a thorough history, including a collateral history (NICE 2018: section 1.2). If a diagnosis of dementia is suspected, NICE advises GPs to conduct a physical examination and blood and urine tests to exclude reversible causes of cognitive decline, as well as administering a validated brief structured cognitive instrument such as the Mini-Cog. It is important to note that a normal score on a cognitive instrument does not rule out dementia (NICE 2018). Therefore, in the primary care setting, the clinical judgement of primary care physicians plays an important role in formulating a suspected diagnosis and identifying which patients should be referred to specialist services, such as memory clinics and community old age psychiatry services, for further investigations.
The Cochrane Review
This month's Cochrane Review (Creavin Reference Creavin, Noel-Storr and Langdon2022) aimed to assess the clinical accuracy of GPs in diagnosing two target conditions: dementia and mild cognitive impairment. This article will focus on the primary outcome analysed, namely, the clinical accuracy of GPs in diagnosing dementia, as defined by sensitivity and specificity (Box 2).
What is the difference between sensitivity and specificity?
Specificity refers to the ability of a test to correctly identify those who do not have a particular disease. Sensitivity refers to the ability of a test to correctly identify those who do have the disease.
A test that is 100% specific means that there are no false-positive results, i.e. all individuals who do not have the disease have a negative test result.
Conversely, a test that is 100% sensitive means that there are no false-negative results, i.e. all individuals who have the disease have a positive test result.
In the context of this article, sensitivity refers to the ability of GPs to correctly identify patients with a diagnosis of dementia in the studied population. Specificity is the ability of GPs to correctly identify those without a diagnosis of dementia in the studied population.
Defining clinical judgement and selecting a reference standard
This review included 4287 adults presenting to primary care with symptoms of dementia or cognitive impairment, without a prior formal diagnosis of either target condition. The index test was defined as the clinical judgement of a GP, unaided by any additional test beyond that which is immediately available to the clinician. Studies that defined clinical judgement as the impression formed by the clinician following a patient consultation (prospective approach) were included, as well as studies that defined clinical judgement as an impression based on prior knowledge of the patient and review of the medical notes (retrospective approach).
The reference standard used was a recognised system for diagnosing dementia as defined by the individual studies. For instance, using the clinical opinion of a clinician who specialises in diagnosing and managing dementia in secondary care or the Cambridge Mental Disorders of the Elderly Examination (CAMDEX). The included studies must have utilised recognised criteria for diagnosis, such as DSM-III, DSM-IV-TR or ICD-10.
The studies included in the review had a maximum of 6 months between the GP's clinical judgement and the reference test. This reduces risk of misclassification of disease due to deterioration of the condition between the index test and reference standard.
Assessing and avoiding bias
The review utilised valid and comprehensive methods, such as a thorough literature search, obtaining details of unpublished studies and implementing clear definitions of the outcome conditions (dementia and mild cognitive impairment). For instance, if mild cognitive impairment was diagnosed but then found to be related to a neoplasm or head injury, it was counted as a false positive.
Overall, the review was limited by a small sample size, with only eight studies included in the meta-analysis considering dementia. Appropriate steps were taken to assess for risk of bias in eligible studies. For example, studies in which the diagnosis had been written in the patient records prior to the GP reviewing the patient notes were excluded. This reduced systematic bias should GPs have seen dementia coded in the patient record before seeing the patient.
Additionally, the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool was utilised to assess for bias (Box 3). Studies that had more than one QUADAS-2 domain at high risk of bias were excluded from the main meta-analysis. The authors conducted sensitivity analyses, which resulted in two studies being excluded owing to high risk of bias, which is in keeping with a suggested Cochrane analysis strategy (Boutron Reference Boutron I, Page and Higgins2023).
The QUADAS-2 tool:
This is a tool to assess study bias. It assesses four domains that can introduce bias:
(a) how patients in the study were selected
(b) the index test
(c) the reference standards used
(d) the flow and timing of the studies in question.
Flow and timing look at when the index test and reference standard are administered and how this may introduce potential bias. For instance, having a long time period between the index test and reference standard may increase risk of misclassification of disease owing to deterioration of disease between the two time points. Furthermore, there is risk of partial verification bias if the result of the index test influences the result of the reference standard.
A summary receiver operating characteristic (ROC) curve (Box 4) was utilised and a random effects meta-analysis was performed.
What is a ROC plot?
A ROC (receiver operating characteristic curve) plot is a graph that shows the relationship between sensitivity and specificity. It can be utilised to evaluate a diagnostic test, in this case, the clinical judgement of GPs in diagnosing dementia.
Simplified representation of a ROC curve.
The light red line indicates the 95% confidence interval around the summary point containing the ‘true value’ within that region 95% of the time. The dark red line indicates the 95% predictive region (containing the results 95% of the time should new future study data be utilised). The black dot indicates the summary point estimate of diagnostic accuracy.
A ‘perfect’ test, one that has no false positives or false negatives, will run through the upper left corner of the graph. Therefore, the closer the curve is to the upper left-hand corner, the higher the overall accuracy of the test.
Study outcomes
The included studies demonstrated a large variation in results. Sensitivity of diagnosing dementia and cognitive impairment varied from 34 to 91% and specificity ranged between 58 and 99%. One study reported a sensitivity and specificity of 100%. This was excluded from the meta-analysis as it was found to be at high risk of bias. A meta-analysis conducted for dementia alone as the target condition found that the diagnostic accuracy of GPs’ clinical judgement was 58% sensitive and 89% specific. This suggests that GPs are more accurate in excluding a diagnosis of dementia as opposed to diagnosing dementia. The ROC curve demonstrated that only four studies were within the 95% confidence interval. This suggests that there was significant heterogeneity in the data.
Discussion
A significant limitation of the study was that it assumed low inter-observer variability among GPs. Given the element of subjectivity in forming a suspected diagnosis of dementia in primary care, different GPs may have differing clinical judgements when assessing the same patient. This limits the generalisability of the findings to larger populations (Cerullo Reference Cerullo, Quinn and McCleery2020).
Additionally, the study did not comment on the clinical judgement of GPs dependent on the severity of disease. The accuracy of clinical judgement may be related to the stage of the disease, which was not within the scope of the review. Although the review acknowledges the challenges in assessing this, the impact of disease severity on clinical judgement is an important consideration since a timely diagnosis of dementia is crucial. An early diagnosis enables patients to make advanced directives, identify their care needs and, ultimately, improves patients’ quality of life (Hout Reference Hout, Vernooij-Dassen and Stalman2007).
What are the clinical implications of the findings?
This review suggests that GPs’ clinical judgement is more specific than sensitive in diagnosing dementia, which is in keeping with previous literature (Dungen Reference Dungen, Warwijk and Horst2011). However, the meta-analysis is limited by small sample size and significant heterogeneity in the data. The paper highlights the proportion of false-negative results, i.e. patients who may be diagnosed as not having dementia by a GP but who do have a diagnosis of dementia. This introduces the possibility that subjective measures may have a low sensitivity in dementia diagnosis and highlights the importance of assessing the clinical accuracy of objective measures such as structured cognitive assessments like the Mini-Cog, which could be used in such cases. To understand how timely diagnosis can be expedited it would be useful to investigate whether the clinical judgement of GPs is affected by disease severity and analyse at what stage of disease progression GPs most commonly form a suspected diagnosis and refer to secondary care. Given the shift towards telehealth in general practice, it would also be useful to compare diagnostic accuracy following virtual consultations as opposed to conventional face-to-face consultations (McCleery Reference McCleery, Laverty and Quinn2021).
Data availability
Data availability is not applicable to this article as no new data were created or analysed in this work.
Author contributions
A.R. is responsible for the ideation and design of the manuscript. A.R. and G.S. contributed to the interpretation and analysis of data for the work. A.R. and G.S. were involved in drafting the work and revising it critically.
Funding
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Declaration of interest
None.
eLetters
No eLetters have been published for this article.