Based on an efficacious face-to-face theory-driven psychological therapy for persecutory delusions in the context of psychosis, we set out to develop a scalable guided 6-month online program. The aim was an intervention that patients can easily access and use, produces large clinical effects, and can be supported by a range of mental health professionals in less contact time than face-to-face therapy. We report here the proof-of-concept testing. At least moderate-sized clinical effects were required to progress to a randomized controlled trial (RCT).

In the 6-month Feeling Safer online program, a certified medical device, patients complete a brief assessment and then are provided with up to 10 modules that match their difficulties. Regular remote meetings with a mental health professional also take place. These may be supplemented by in-person visits. A pre- to post-treatment cohort trial was conducted with 14 patients with persistent persecutory delusions. The primary outcome was the Psychotic Symptoms Rating Scale (PSYRATS)-Delusions.

Satisfaction and usability ratings of the program were high. Very large reductions in persecutory delusions were observed (PSYRATS mean reduction = 7.1, 95% C.I. = 3.4, 10.8, n = 13, Cohen’s d = 3.0). There were large improvements in paranoia, anxiety, depression, agoraphobic distress, psychological wellbeing, meaningful activity, personal recovery, recovering quality of life, and moderate improvements in insomnia, agoraphobic avoidance, and quality of life.

The clinical effects associated with Feeling Safer were very high, comparable to those seen in the evaluations of the face-to-face therapy, and enable progression to an RCT.

There is an increasing number of policy and guidance documents on the use and acceptability of real-world evidence (RWE) to support regulatory and health technology assessment (HTA) decision-making. The Innovative Health Initiative Integration of Heterogeneous Data and Evidence towards Regulatory and HTA Acceptance (IDERHA) partnership is undertaking a global landscape review of these documents to understand where there is consensus and divergence, and where further policy development is needed.

A literature search of the MEDLINE and Embase databases was performed, in addition to handsearching the websites of specific HTA and regulatory organizations. All policies, standards, frameworks, and guidance documents on requirements for acceptable RWE data use published from 2017 were included. Two reviewers independently extracted data using a standard data extraction form that was pilot tested before use. Any discrepancies between the reviewers were resolved by consensus. Extracted data are currently being analyzed by researchers with regulatory or HTA expertise. A workshop held in October 2023 sought input from experts on analysis plans.

The initial literature search yielded 3,184 results. After screening against the inclusion criteria, a total of 87 documents were selected for full-text review (21 HTA and 62 regulatory documents). Of these, 32 were identified as key documents and prioritized for initial review. Key themes in the documents, including transparency, data collection, study design, and data quality, were identified and validated in a workshop with five regulatory or HTA experts. Data extraction is ongoing for the remaining documents and any further themes identified will be added. Any gaps and areas of divergence will be identified, so they can be addressed by future IDERHA work.

This review assessed the increasingly complex global landscape of regulatory and HTA policies and guidance on the use of RWE. Through the exploration of similarities, differences, and gaps in these policies, this work will extend the current understanding of best practice and identify areas that need development of further guidance.

Health apps are software programs that are designed to prevent, diagnose, monitor, or manage conditions. Inconsistent terminology for apps is used in research literature and bibliographic database subject headings. It can therefore be challenging to retrieve evidence about them in literature searches. Information specialists at the United Kingdom's National Institute for Health and Care Excellence (NICE) have developed novel validated search filters to retrieve evidence about apps from MEDLINE and Embase (Ovid).

A selection of medical informatics journals was hand searched to identify a “gold standard” (GS) set of references about apps. The GS set was divided into a development and validation set. The filters’ search terms were derived from and tested against the development set. An external development set containing app references from published NICE products was also used to inform the development of the filters. The filters were then validated using the validation set. Target recall was >90 percent. The filters’ overall recall, specificity, and precision were calculated using all the references identified from the hand search.

Both filters achieved 98.6 percent recall against their validation sets. Overall, the MEDLINE filter had 98.8 percent recall, 71.3 percent specificity, and 22.6 percent precision. The Embase filter had 98.6 percent recall, 74.9 percent specificity, and 24.5 percent precision.

The NICE health apps search filters retrieve evidence about apps from MEDLINE and Embase with high recall. They can be applied to literature searches to retrieve evidence about the interventions by information professionals, researchers, and clinicians.

Implementation scientists increasingly recognize that the process of implementation is dynamic, leading to ad hoc modifications that may challenge fidelity in protocol-driven interventions. However, limited attention to ad hoc modifications impairs investigators’ ability to develop evidence-based hypotheses about how such modifications may impact intervention effectiveness and cost. We propose a multi-method process map methodology to facilitate the systematic data collection necessary to characterize ad hoc modifications that may impact primary intervention outcomes.

We employ process maps (drawn from systems science), as well as focus groups and semi-structured interviews (drawn from social sciences) to investigate ad hoc modifications. Focus groups are conducted with the protocol’s developers and/or planners (the implementation team) to characterize the protocol “as envisioned,” while interviews conducted with frontline administrators characterize the process “as realized in practice.” Process maps with both samples are used to identify when modifications occurred across a protocol-driven intervention. A case study investigating a multistage screening protocol for autism spectrum disorders (ASD) is presented to illustrate application and utility of the multi-method process maps.

In this case study, frontline administrators reported ad hoc modifications that potentially influenced the primary study outcome (e.g., time to ASD diagnosis). Ad hoc modifications occurred to accommodate (1) whether providers and/or parents were concerned about ASD, (2) perceptions of parental readiness to discuss ASD, and (3) perceptions of family service delivery needs and priorities.

Investigation of ad hoc modifications on primary outcomes offers new opportunities to develop empirically based adaptive interventions. Routine reporting standards are critical to provide full transparency when studying ad hoc modifications.

Item 9 of the Patient Health Questionnaire-9 (PHQ-9) queries about thoughts of death and self-harm, but not suicidality. Although it is sometimes used to assess suicide risk, most positive responses are not associated with suicidality. The PHQ-8, which omits Item 9, is thus increasingly used in research. We assessed equivalency of total score correlations and the diagnostic accuracy to detect major depression of the PHQ-8 and PHQ-9.

We conducted an individual patient data meta-analysis. We fit bivariate random-effects models to assess diagnostic accuracy.

16 742 participants (2097 major depression cases) from 54 studies were included. The correlation between PHQ-8 and PHQ-9 scores was 0.996 (95% confidence interval 0.996 to 0.996). The standard cutoff score of 10 for the PHQ-9 maximized sensitivity + specificity for the PHQ-8 among studies that used a semi-structured diagnostic interview reference standard (N = 27). At cutoff 10, the PHQ-8 was less sensitive by 0.02 (−0.06 to 0.00) and more specific by 0.01 (0.00 to 0.01) among those studies (N = 27), with similar results for studies that used other types of interviews (N = 27). For all 54 primary studies combined, across all cutoffs, the PHQ-8 was less sensitive than the PHQ-9 by 0.00 to 0.05 (0.03 at cutoff 10), and specificity was within 0.01 for all cutoffs (0.00 to 0.01).

PHQ-8 and PHQ-9 total scores were similar. Sensitivity may be minimally reduced with the PHQ-8, but specificity is similar.

Different diagnostic interviews are used as reference standards for major depression classification in research. Semi-structured interviews involve clinical judgement, whereas fully structured interviews are completely scripted. The Mini International Neuropsychiatric Interview (MINI), a brief fully structured interview, is also sometimes used. It is not known whether interview method is associated with probability of major depression classification.

To evaluate the association between interview method and odds of major depression classification, controlling for depressive symptom scores and participant characteristics.

Data collected for an individual participant data meta-analysis of Patient Health Questionnaire-9 (PHQ-9) diagnostic accuracy were analysed and binomial generalised linear mixed models were fit.

A total of 17 158 participants (2287 with major depression) from 57 primary studies were analysed. Among fully structured interviews, odds of major depression were higher for the MINI compared with the Composite International Diagnostic Interview (CIDI) (odds ratio (OR) = 2.10; 95% CI = 1.15–3.87). Compared with semi-structured interviews, fully structured interviews (MINI excluded) were non-significantly more likely to classify participants with low-level depressive symptoms (PHQ-9 scores ≤6) as having major depression (OR = 3.13; 95% CI = 0.98–10.00), similarly likely for moderate-level symptoms (PHQ-9 scores 7–15) (OR = 0.96; 95% CI = 0.56–1.66) and significantly less likely for high-level symptoms (PHQ-9 scores ≥16) (OR = 0.50; 95% CI = 0.26–0.97).

The MINI may identify more people as depressed than the CIDI, and semi-structured and fully structured interviews may not be interchangeable methods, but these results should be replicated.

Drs Jetté and Patten declare that they received a grant, outside the submitted work, from the Hotchkiss Brain Institute, which was jointly funded by the Institute and Pfizer. Pfizer was the original sponsor of the development of the PHQ-9, which is now in the public domain. Dr Chan is a steering committee member or consultant of Astra Zeneca, Bayer, Lilly, MSD and Pfizer. She has received sponsorships and honorarium for giving lectures and providing consultancy and her affiliated institution has received research grants from these companies. Dr Hegerl declares that within the past 3 years, he was an advisory board member for Lundbeck, Servier and Otsuka Pharma; a consultant for Bayer Pharma; and a speaker for Medice Arzneimittel, Novartis, and Roche Pharma, all outside the submitted work. Dr Inagaki declares that he has received grants from Novartis Pharma, lecture fees from Pfizer, Mochida, Shionogi, Sumitomo Dainippon Pharma, Daiichi-Sankyo, Meiji Seika and Takeda, and royalties from Nippon Hyoron Sha, Nanzando, Seiwa Shoten, Igaku-shoin and Technomics, all outside of the submitted work. Dr Yamada reports personal fees from Meiji Seika Pharma Co., Ltd., MSD K.K., Asahi Kasei Pharma Corporation, Seishin Shobo, Seiwa Shoten Co., Ltd., Igaku-shoin Ltd., Chugai Igakusha and Sentan Igakusha, all outside the submitted work. All other authors declare no competing interests. No funder had any role in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review or approval of the manuscript; and decision to submit the manuscript for publication.