Computerized Clinical Decision Support Systems (cCDSS) are promising digital health tools whose development and use are increasing. The Agency for Health Quality and Assessment of Catalonia (AQuAS) received a request from the Spanish Ministry of Health, through the Spanish Network of Health Technology Assessment (HTA) Agencies (RedETS), to perform an HTA of cCDSS for cancer. We present the challenges that arose during the evaluation process.

We evaluated the safety, effectiveness and cost-effectiveness through a systematic literature review. We involved two clinicians and a technology expert to gain insight into the pathology and technology, respectively. To identify cCDSS used in Spain, we consulted with the Spanish regulatory agency (AEMPS) and the Spanish Federation of Healthcare Technology Companies (FENIN), plus did a survey among Spanish hospitals. To understand applicable regulations, we reviewed the European regulation (MDR) and consulted the Medical Device Coordination Group from the European Commission, AEMPS, and a regional regulatory expert.

The scientific literature revealed large heterogeneity in the definition of cCDSS (e.g., from simple online prognostic calculators to complex commercial software using machine learning), making the literature search and screening arduous. Many articles dealt with cCDSS that do not qualify as medical devices, are not in the market anymore, are currently used as newer versions or are developed in-house. Next, we faced the difficulty to attain a comprehensive overview of the tools in use in the country. In terms of legislation, we observed that similar tools might receive different classifications in different jurisdictions, and the complexity of the MDR might lead to the need for a case-by-case discussion at a National level.

We identified many challenges in the HTA of cCDSS. The first step for proper assessment is a clear definition of the device and version to be evaluated. Multiple stakeholders must be involved, and alignment between regulatory and HTA agencies is key. We expect that the European Database on Medical Devices (EUDAMED) will help in the identification of existing cCDSS and hence ease their assessment.