Integration of mental health services into nonspecialist settings is expanding in low and middle income countries (LMICs). Among many factors required for success, such programs require reliable administration of mental health screening tools. While several tools have been validated in carefully conducted research studies, few studies have assessed how reliably nonspecialist clinicians administer these tools to low-literacy LMIC populations in routine care.

Ninety-seven patients accessing human immunodeficiency virus primary care in Malawi who completed Patient Health Questionnaire (PHQ)-9 depression screening with their clinician then completed a second PHQ-9 with a trained research assistant (RA) blinded to the first result.

Compared to clinicians, RAs identified more patients with any depressive symptoms (PHQ-9 score ⩾5: 38% v. 32%), moderate/severe symptoms (PHQ-9 ⩾ 10: 14% v. 6%), any suicidality (14% v. 4%), and active suicidality (3% v. 2%). Across these indicators, clinician and RA ratings had strong overall agreement (81–97%) but low corrected Kappa agreement (31–59%). Treating RA results as the reference standard of a carefully supervised research administration of the PHQ-9, clinician administration had high specificity (90–99%) but low sensitivity (23–68%) for these indicators.

In routine care in LMICs, clinicians may administer validated mental health screening tools with varying quality. To ensure quality, integration programs must incorporate appropriate and ongoing training, support, supervision, and monitoring.