We read with interest Roberts et al's paper on dopaminergic imaging as a biomarker for mild cognitive impairment with Lewy bodies (MCI-LB).^{Reference Roberts, Donaghy, Lloyd, Durcan, Petrides and Colloby1} However, we believe that the authors’ conclusion that dopamine imaging is useful when Lewy body disease is suspected in a patient with MCI is not supported by the data presented. This otherwise well-designed and conducted study has issues that we commonly see in our Cochrane Dementia reviews of dementia tests. We highlight the potential biases here to raise the visibility of these issues in the dementia test accuracy community.

First, unquantifiable bias may be introduced by the reference standard (probable MCI-LB assessed using recent research criteria^{Reference McKeith, Ferman, Thomas, Blanc, Boeye and Fujishiro2}). Imperfect reference standards are common in dementia research.^{Reference Cerrullo, Quinn, McCleery, Cooper and Sutton3} However, for Lewy body disease, in particular clinical diagnosis (that is not informed by imaging) is problematic as there are studies of dopamine imaging versus neuropathology that suggest the imaging is more accurate than clinical diagnosis.^{Reference Thomas, Attems, Colloby, O'Brien, McKeith and Walker4,Reference Walker, Jaros, Walker, Lee, Costa and Livingston5} It is axiomatic that a test cannot be assessed against a reference standard that is less accurate than the new test itself.

Another common issue in dementia test accuracy is around the generalisability of the populations studied. Here we agree with the authors that the place of dopaminergic imaging in the clinical pathway is when Lewy body disease is clinically suspected but uncertain. The study population seems appropriate, although 36/144 patients already met criteria for probable Lewy body disease when recruited. It is difficult to see how imaging adds value to the assessment of these patients. In the primary analysis, the authors exclude the 26 patients with the most uncertain diagnoses at follow-up. This approach risks introducing spectrum bias and inflating accuracy estimates. If we recalculate test accuracy limited to the 108 patients for whom the diagnosis of Lewy body disease was clinically uncertain at baseline, we find sensitivity of 60% and specificity of 83% for dopaminergic imaging. If we include the 36 patients meeting criteria for probable Lewy body disease at baseline, on the grounds that at least some of them might not have been identified in less specialist services, sensitivity is increased to the authors’ estimate of 66% but specificity remains lower at 83%.

Sensitivity and specificity metrics can seem abstract and clinicians will want to know whether the tests have value in their practice. The accuracy figures that would demonstrate the dopamine imaging to be clinically ‘useful’ are not prespecified, but both our figures and the author's own accuracy estimates suggest substantial potential for misclassification. The relatively low sensitivity means that negative dopamine imaging in a patient with MCI and suspicion of Lewy body disease cannot be relied upon to exclude the diagnosis for the purposes of treatment (for example with antipsychotics for psychiatric symptoms) or for research inclusion (for example Alzheimer's disease therapy trials); as many as half of negative results could be wrong.

We believe that imaging and other biomarkers have a useful role in dementia diagnosis. However, for this study, because of the biases described, the recommendations around adopting dopamine imaging in clinical practice are not supported.