Routine patient care data are increasingly used for biomedical research, but such “secondary use” data have known limitations, including their quality. When leveraging routine care data for observational research, developing audit protocols that can maximize informational return and minimize costs is paramount.

For more than a decade, the Latin America and East Africa regions of the International epidemiology Databases to Evaluate AIDS (IeDEA) consortium have been auditing the observational data drawn from participating human immunodeficiency virus clinics. Since our earliest audits, where external auditors used paper forms to record audit findings from paper medical records, we have streamlined our protocols to obtain more efficient and informative audits that keep up with advancing technology while reducing travel obligations and associated costs.

We present five key lessons learned from conducting data audits of secondary-use data from resource-limited settings for more than 10 years and share eight recommendations for other consortia looking to implement data quality initiatives.

After completing multiple audit cycles in both the Latin America and East Africa regions of the IeDEA consortium, we have established a rich reference for data quality in our cohorts, as well as large, audited analytical datasets that can be used to answer important clinical questions with confidence. By sharing our audit processes and how they have been adapted over time, we hope that others can develop protocols informed by our lessons learned from more than a decade of experience in these large, diverse cohorts.

In 2018, Minneapolis began phased implementation of an ordinance to increase the local minimum wage to $15/h. We sought to determine whether the first phase of implementation was associated with changes in frequency of consumption of fruits and vegetables (F&V), whole-grain-rich foods, and foods high in added sugars among low-wage workers.

Natural experiment.

The Wages Study is a prospective cohort study of 974 low-wage workers followed throughout the phased implementation of the ordinance (2018–2022). We used difference-in-difference analysis to compare outcomes among workers in Minneapolis, Minnesota, to those in a comparison city (Raleigh, North Carolina). We assessed wages using participants’ pay stubs and dietary intake using the National Cancer Institute Dietary Screener Questionnaire.

Analyses use the first two waves of Wages data (2018 (baseline), 2019) and includes 267 and 336 low-wage workers in Minneapolis and Raleigh, respectively.

After the first phase of implementation, wages increased in both cities, but the increase was $0·84 greater in Minneapolis (P = 0·02). However, the first phase of the policy’s implementation was not associated with changes in daily frequency of consumption of F&V (IRR = 1·03, 95 % CI: 0·86, 1·24, P = 0·73), whole-grain-rich foods (IRR = 1·23, 95 % CI: 0·89, 1·70, P = 0·20), or foods high in added sugars (IRR = 1·13, 95 % CI: 0·86, 1·47, P = 0·38) among workers in Minneapolis compared to Raleigh.

The first phase of implementation of the Minneapolis minimum wage policy was associated with increased wages, but not with changes in dietary intake. Future research should examine whether full implementation is associated dietary changes.

Audits play a critical role in maintaining the integrity of observational cohort data. While previous work has validated the audit process, sending trained auditors to sites (“travel-audits”) can be costly. We investigate the efficacy of training sites to conduct “self-audits.”

In 2017, eight research groups in the Caribbean, Central, and South America network for HIV Epidemiology each audited a subset of their patient records randomly selected by the data coordinating center at Vanderbilt. Designated investigators at each site compared abstracted research data to the original clinical source documents and captured audit findings electronically. Additionally, two Vanderbilt investigators performed on-site travel-audits at three randomly selected sites (one adult and two pediatric) in late summer 2017.

Self- and travel-auditors, respectively, reported that 93% and 92% of 8919 data entries, captured across 28 unique clinical variables on 65 patients, were entered correctly. Across all entries, 8409 (94%) received the same assessment from self- and travel-auditors (7988 correct and 421 incorrect). Of 421 entries mutually assessed as “incorrect,” 304 (82%) were corrected by both self- and travel-auditors and 250 of these (72%) received the same corrections. Reason for changing antiretroviral therapy (ART) regimen, ART end date, viral load value, CD4%, and HIV diagnosis date had the most mismatched corrections.

With similar overall error rates, findings suggest that data audits conducted by trained local investigators could provide an alternative to on-site audits by external auditors to ensure continued data quality. However, discrepancies observed between corrections illustrate challenges in determining correct values even with audits.

To evaluate the association between three behavioural economics ‘nudges’ and store sales of promoted healthier foods.

Multiple interrupted time series.

Two predominantly rural counties in central North Carolina, USA.

Aggregated store transaction data from two grocery stores (one intervention, one control) and two convenience stores (one intervention, one control) were analysed using ANOVA to examine the association between three ‘nudges’ and store sales of promoted items. The nudges included: a ‘cognitive fatigue’ experiment, in which floor arrows guided customers to the produce sections; a ‘scarcity’ experiment, in which one sign in one area of the produce section portrayed a ‘limited amount’ message; and a ‘product placement’ experiment, where granola bars were moved into the candy bar aisle.

In convenience stores, there were no significant differences between sales of the promoted items during the intervention period for any of the nudges when implemented individually. However, compared with baseline sales, implementation of all three nudges simultaneously was associated with an increase in sales during the intervention period based on proportional computations (P = 0·001), whereas no significant changes in sales were observed in the control convenience store. Among the grocery stores, there were no significant differences in sales during the intervention period for any of the nudges or the combined intervention compared with baseline sales.

Implementing three nudges concurrently in a convenience store setting may increase sales of promoted items. However, before stores consider implementing these nudges to increase sales of nutritious foods, additional research is warranted.

Item 9 of the Patient Health Questionnaire-9 (PHQ-9) queries about thoughts of death and self-harm, but not suicidality. Although it is sometimes used to assess suicide risk, most positive responses are not associated with suicidality. The PHQ-8, which omits Item 9, is thus increasingly used in research. We assessed equivalency of total score correlations and the diagnostic accuracy to detect major depression of the PHQ-8 and PHQ-9.

We conducted an individual patient data meta-analysis. We fit bivariate random-effects models to assess diagnostic accuracy.

16 742 participants (2097 major depression cases) from 54 studies were included. The correlation between PHQ-8 and PHQ-9 scores was 0.996 (95% confidence interval 0.996 to 0.996). The standard cutoff score of 10 for the PHQ-9 maximized sensitivity + specificity for the PHQ-8 among studies that used a semi-structured diagnostic interview reference standard (N = 27). At cutoff 10, the PHQ-8 was less sensitive by 0.02 (−0.06 to 0.00) and more specific by 0.01 (0.00 to 0.01) among those studies (N = 27), with similar results for studies that used other types of interviews (N = 27). For all 54 primary studies combined, across all cutoffs, the PHQ-8 was less sensitive than the PHQ-9 by 0.00 to 0.05 (0.03 at cutoff 10), and specificity was within 0.01 for all cutoffs (0.00 to 0.01).

PHQ-8 and PHQ-9 total scores were similar. Sensitivity may be minimally reduced with the PHQ-8, but specificity is similar.

Different diagnostic interviews are used as reference standards for major depression classification in research. Semi-structured interviews involve clinical judgement, whereas fully structured interviews are completely scripted. The Mini International Neuropsychiatric Interview (MINI), a brief fully structured interview, is also sometimes used. It is not known whether interview method is associated with probability of major depression classification.

To evaluate the association between interview method and odds of major depression classification, controlling for depressive symptom scores and participant characteristics.

Data collected for an individual participant data meta-analysis of Patient Health Questionnaire-9 (PHQ-9) diagnostic accuracy were analysed and binomial generalised linear mixed models were fit.

A total of 17 158 participants (2287 with major depression) from 57 primary studies were analysed. Among fully structured interviews, odds of major depression were higher for the MINI compared with the Composite International Diagnostic Interview (CIDI) (odds ratio (OR) = 2.10; 95% CI = 1.15–3.87). Compared with semi-structured interviews, fully structured interviews (MINI excluded) were non-significantly more likely to classify participants with low-level depressive symptoms (PHQ-9 scores ≤6) as having major depression (OR = 3.13; 95% CI = 0.98–10.00), similarly likely for moderate-level symptoms (PHQ-9 scores 7–15) (OR = 0.96; 95% CI = 0.56–1.66) and significantly less likely for high-level symptoms (PHQ-9 scores ≥16) (OR = 0.50; 95% CI = 0.26–0.97).

The MINI may identify more people as depressed than the CIDI, and semi-structured and fully structured interviews may not be interchangeable methods, but these results should be replicated.

Drs Jetté and Patten declare that they received a grant, outside the submitted work, from the Hotchkiss Brain Institute, which was jointly funded by the Institute and Pfizer. Pfizer was the original sponsor of the development of the PHQ-9, which is now in the public domain. Dr Chan is a steering committee member or consultant of Astra Zeneca, Bayer, Lilly, MSD and Pfizer. She has received sponsorships and honorarium for giving lectures and providing consultancy and her affiliated institution has received research grants from these companies. Dr Hegerl declares that within the past 3 years, he was an advisory board member for Lundbeck, Servier and Otsuka Pharma; a consultant for Bayer Pharma; and a speaker for Medice Arzneimittel, Novartis, and Roche Pharma, all outside the submitted work. Dr Inagaki declares that he has received grants from Novartis Pharma, lecture fees from Pfizer, Mochida, Shionogi, Sumitomo Dainippon Pharma, Daiichi-Sankyo, Meiji Seika and Takeda, and royalties from Nippon Hyoron Sha, Nanzando, Seiwa Shoten, Igaku-shoin and Technomics, all outside of the submitted work. Dr Yamada reports personal fees from Meiji Seika Pharma Co., Ltd., MSD K.K., Asahi Kasei Pharma Corporation, Seishin Shobo, Seiwa Shoten Co., Ltd., Igaku-shoin Ltd., Chugai Igakusha and Sentan Igakusha, all outside the submitted work. All other authors declare no competing interests. No funder had any role in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review or approval of the manuscript; and decision to submit the manuscript for publication.

To meta-analytically summarize lamotrigine’s effectiveness and safety in unipolar and bipolar depression.

We conducted systematic PubMed and SCOPUS reviews (last search =10/01/2015) of randomized controlled trials comparing lamotrigine to placebo or other agents with antidepressant activity in unipolar or bipolar depression. We performed a random-effects meta-analysis of depression ratings, response, remission, and adverse effects calculating standardized mean difference (SMD) and risk ratio (RR) ±95% confidence intervals (CIs).

Eighteen studies (n=2152, duration=9.83 weeks) in patients with unipolar depression (studies=4, n=187; monotherapy vs lithium=1, augmentation of antidepressants vs placebo=3) or bipolar depression (studies=14, n=1965; monotherapy vs placebo=5, monotherapy vs lithium or olanzapine+fluoxetine=2, augmentation of antidepressants vs placebo=1, augmentation of mood stabilizers vs placebo=3, augmentation of mood stabilizers vs trancylpromine, citalopram, or inositol=3) were meta-analyzed. Lamotrigine’s efficacy for depressive symptoms did not differ significantly in monotherapy vs augmentation studies (vs. placebo: p=0.98, I2=0%; vs active agents: p=0.48, I2=0%) or in unipolar vs bipolar patients (vs placebo: p=0.60, I2=0%), allowing pooling of each placebo-controlled and active-controlled trials. Lamotrigine outperformed placebo regarding depressive symptoms (studies=11, n=713 vs n=696; SMD=–0.15, 95% CI=–0.27, –0.02, p=0.02, heterogeneity: p=0.24) and response (after removing one extreme outlier; RR=1.42, 95% CI=1.13–1.78; p=0.003, heterogeneity: p=0.08). Conversely, lamotrigine did not differ regarding efficacy on depressive symptoms, response, or remission from lithium, olanzapine+fluoxetine, citalopram, or inositol (studies=6, n=306 vs n=318, p-values=0.85–0.92). Adverse effects and all-cause/specific-cause discontinuation were similar across all comparisons.

Lamotrigine was superior to placebo in improving unipolar and bipolar depressive symptoms, without causing more frequent adverse effects/discontinuations. Lamotrigine did not differ from lithium, olanzapine+fluoxetine, citalopram, or inositol.