COVIDIAGNOSTIX: health technology assessment of serological tests for SARS-CoV-2 infection

Abstract Objective In vitro diagnostic tests for SARS-COV-2, also known as serological tests, have rapidly spread. However, to date, mostly single-center technical and diagnostic performance's assessments have been carried out without an intralaboratory validation process and a health technology assessment (HTA) systematic approach. Therefore, the rapid HTA for evaluating antibody tests for SARS-COV-2 was applied. Methods The use of rapid HTA is an opportunity to test innovative technology. Unlike traditional HTA (which evaluates the benefits of new technologies after being tested in clinical trials or have been applied in practice for some time), the rapid HTA is performed during the early stages of developing new technology. A multidisciplinary team conducted the rapid HTA following the HTA Core Model® (version 3.0) developed by the European Network for Health Technology Assessment. Results The three methodological and analytical steps used in the HTA applied to the evaluation of antibody tests for SARS-COV-2 are reported: the selection of the tests to be evaluated; the research and collection of information to support the adoption and appropriateness of the technology; and the preparation of the final reports and their dissemination. Finally, the rapid HTA of serological tests for SARS-CoV-2 is summarized in a report that allows its dissemination and communication. Conclusions The rapid-HTA evaluation method, in addition to highlighting the characteristics that differentiate the tests from each other, guarantees a timely and appropriate evaluation, becoming a tool to create a direct link between science and health management.


Introduction
COVID-19 disease, caused by the coronavirus SARS-CoV-2, has gradually spread around the world since the last months of 2019 (1). Because the disease is asymptomatic in most cases, several diagnostic strategies are available to discriminate infected from noninfected people and define which people have developed antibodies to the infection (2). The first step in diagnosing SARS-CoV-2 infection is RT-PCR, which detects viral nucleic acids present in nasopharyngeal fluids, allowing viral detection in asymptomatic infected individuals (3). In vitro diagnostic tests, also known as serological tests, detect the presence of antibodies produced in response to SARS-CoV-2 infection and reveal past infections by identifying antibodies. The synthesis of antibodies is a primary immune response to infections (4): immunoglobulin A (IgA) responses appear between 4 and 10 days after infection; immunoglobulin M (IgM) levels rise during the first week after SARS-CoV-2 infection, peak after 2 weeks, and then return to near-background levels in most individuals; immunoglobulin G (IgG) is detectable after 1 week and is kept at a high level for an extended period.
Serological tests can help assess protection against subsequent viral exposure and/or for contact tracing purposes; therefore, these tests are essential for epidemiological assessments and forecasting estimates of global therapeutic needs (5;6).
In vitro diagnostic serological tests have rapidly spread in traditional, automated, and point-of-care forms; however, mostly single-center evaluations of these tests have been performed, without any systematic validation of the tests or health technology assessment (HTA) approach (7-9).
In the last months of 2020, Covidiagnostix, a multicenter national project funded by the Italian Ministry of Health, was launched with the aim of evaluating and comparing some of the serological tests available on the market for SARS-CoV-2 through the HTA approach.
In this work, by retracing the methodological (related to the stepwise method used in HTA) and analytical (related to the systematic revision of the literature) phases, we report the rapid HTA related to the serological tests of SARS-CoV-2 evaluated and validated in the Covidiagnostix project.

Materials and Methods
Five Italian Scientific Institute for Research, Hospitalization and Healthcare (IRCCS) and one Cooperative were involved in the study: IRCCS Ospedale San Raffaele Hospital (OSR) and IRCCS Orthopedic Institute Galeazzi (IOG) (Milano, Italy); IRCCS Casa Sollievo della Sofferenza (CSS) (San Giovanni Rotondo, FG, Italy); Fondazione IRCCS Policlinico San Matteo (Pavia, Italy); IRCCS Ospedale Pediatrico Bambino Gesù; and OSA Cooperative a r.l. (Roma, Italy). The participants are involved with the COVID-19 epidemic, in terms of both assistance and scientific research, providing a network of skills ranging from analytical evaluation to the management of processes related to laboratory methods.
To yield a timely and appropriate evaluation, we focused our study on safety, clinical efficacy, and organizational and economic impacts. The report was developed using the EUnetHTA (European Network for Health Technology Assessment) Core Model® version 3.0 (https://eunethta.eu/hta-core-model/) as the reference method.
From the entire list of assessment elements, only those relating to the diagnostic test were selected. Each selected assessment element was then evaluated considering evidence gathered through literature search.
The research and collection of the information were carried out relating to each aspect of assessment. In particular, for the description of the technologies and technical characteristics related to each test, both the information contained in the EUA Authorized Serology Test Performance issued by the Food and Drug Administration (FDA) and those provided by each manufacturer in the technical data sheets (method manuals, protocol, illustrative sheet) were used. The technical data sheets also allowed us to make conclusions regarding safety and organizational aspects.
Information on clinical effectiveness was obtained through a systematic review of the scientific literature by consulting the international database MedLine, through PubMed, and using the following keywords: antibody tests for SARS-CoV-2, serological tests for SARS-CoV-2, and commercial tests for SARS-CoV-2. The systematic review included all types of methodological studies (systematic reviews, overview, case studies, qualitative studies, methodological documents), published before 31 December 2020, which met the following criteria: methodological documents strictly related to antibody tests to SARS-CoV-2; studies in which a description or a comparison of the methods used for the antibody determination for SARS-CoV-2 and their features were provided. Studies in which antibody tests for SARS-CoV-2 were only mentioned without a description and those written in a language other than English were not considered. The studies identified by the research strategies were managed and shared using the Zotero program (version 5.0.95); each bibliographic reference was made identifiable by the surname of the first author and the year of publication.
The assessment of the economic aspect was conducted through direct evidence research based on the analysis/estimate of the costs: each participant routinely feeds an internal database with data relating to testing cost, batch size, cost of personnel involved, consumables, and waste disposal. Such data were shared with the study group to derive an estimate of the full cost for a single test.
Finally, based on all the information collected, the final report relating to each test was drawn up.

Results
The domain "Description and characteristics of the test" (Figure 1) reported the test characteristics that impact the purposes of use, level of health care, and administration methods. In particular, none of the six analyzed tests or the competitors can be administered by the patients themselves or their caregivers. Still, at all stages (from the blood sample to the report), the intervention of health personnel is required. The patient can decide for him/herself to undergo the serological test; however, it is important to know that when used, these tests in an individual are more sensitive as time passes from the time of exposure to infection or the appearance of symptoms.
Serological tests aiming to detect the immune response to SARS-CoV-2 rather than detecting the virus itself should not be used as the sole basis for diagnosing COVID-19; they could play a role in the fight against COVID-19 by helping healthcare professionals identify people who may have developed an immune response.
The information in the "Test safety" domain reports any unwanted or harmful effects caused by the use of the test; issues of safety for patients (Figure 2, panel A), healthcare professionals, and the environment (Figure 2, panel B) were also addressed.
In particular, in laboratory tests, indirect damage, that is, false negative or false-positive results, is taken into account (Figure 2, panel A). In the case of false-positive results, the risk is that the test leads to the need to isolate and monitor individual people, his/her family, and the people he/she has been in close contact with, limiting their possibility to work. Risks associated with a 2 Rossella Tomaiuolo et al.
false negative result include a lack of monitoring for him/her, his/ her family, or people with whom he/she has been in close contact and consequently lead to an increased risk of spreading COVID-19 within the community. The occurrence of adverse events is not expected for any of the tests analyzed; however, only a low possibility should be considered in the preanalytical phase due to possible inconveniences or complications during sample collection (venous blood collection). Furthermore, the technology is based on a fully automated platform, so there is a low risk of operator-related test failure. Also, the reagents are supplied in closed containers, which must be loaded onto the system, thus minimizing the risk of errors related to reconstitution, dosage, administration, or storage.
All six tests require the handling of human blood, so both human source materials and all consumables contaminated with human source materials should be considered potentially infectious and should be handled following applicable and appropriate biosecurity practices. Accidental contact with the reagents can cause an allergic skin reaction. Incidental contact between reagents and acids could release toxic gases. Finally, all tests produce biohazardous waste that must be disposed of according to local regulations; other reagents must not be released into the environment and/or disposed of following local regulations.
In the domain "Clinical effectiveness of the test" (Figure 3, panel A), the evaluation of health benefits through clinically significant end points such as mortality, morbidity, and quality of life was evaluated. An analysis of the literature revealed no differences between the various tests evaluated. As for their impact on the testtreatment chain, serological tests do not affect the treatment of the disease due to their low sensitivity during the first 14 days after the infection. Serological tests could help estimate the mortality rate. Also, all serological tests identify individuals who may have contracted the virus and developed an immune response to SARS-CoV-2 ( Figure 3, panel B). Finally, this domain contains the parameters that allow a comparison of the test accuracy of each of the tests (Figure 3, panel C), estimated from the scientific literature, as previously described. In particular, a systematic review of the scientific literature identified 117 scientific papers published in international journals with peer review and an average 3.99 impact factor (IF). In 54 percent of the cases, the selected paper analyzed and described the antibody tests for SARS-CoV-2; the other article elaborated comparison models between tests.
The "Organizational aspects for the execution of the test" domain ( Figure 4), which concerns the ways in which different types of resources must be mobilized and organized when implementing a technology, and the consequences they may additionally  International Journal of Technology Assessment in Health Care produce in the organization and the healthcare system as a whole, underlines when and if the presence of a trained operator is necessary or if the test can be managed independently by the individual. In particular, no preparation is needed (strict fasting, rest) before taking the sample, and no involvement should be mobilized for patients and/or caregivers. For all six tests, regarding the analytical phase, laboratory technicians receive adequate education and training on the automated technological platform used for the analysis and the method is described in the attached sheet supplied with each kit; the quality control and monitoring system of the test is overseen for each new analytical session undertaken through the quality controls provided by the kit.
The evaluation of the economic domain has been first searched in scientific literature. However, as available publications did not report any data on the economic or financial aspect, direct queries on internal databases have been performed by institutions that participated in the project.
Before economic computation, a process analysis was developed to map the overall test delivery and identify the main (differential) cost drivers. Such analysis highlighted two major findings: (i) no difference was found for the COVID test process with respect to other standard serological tests performed in each center and (ii) no major difference was found among different centers involved in the present study.
Concerning infrastructural and technological facilities, the initial cost of the test equipment/point-of-care test equipment was not included in the economic analysis, because the centers involved in the study are all major hospitals, thus already equipped with the required technical and structural means. The same consideration applied to human resources' endowment. Moreover, as each center was asked to comply with a Turn Around Time of 24 h, no other resources (technical or human) were needed. The only difference seen was relative to the cost of the test, which ranged from 1.4 to 12.5€.   Rossella Tomaiuolo et al.

Discussion
In Laboratory Medicine, the evolution of knowledge and the innovation of technologies are the basis of diagnostic progress. To guarantee the general optimization of the process and the organizational requirements of testing, technological progress must be supported by effective tests and by constantly updated operators. The large number of new technologies developed in a short period entails the risk of overlap between the traditional technology already in use and the innovative one, which, if prolonged outside the validation period, leads to organizational irrationality and an unjustified increase in costs. Evaluation tools, such as the rapid-HTA method, which can adequately manage and tackle uncertainty, while guaranteeing timely and adequate evaluation, are still helpful when it is more critical to provide decision makers with an indication based on incomplete yet timely evidence, rather than waiting for the collection of complete evidence but with such a delay that could make the decision vain (10).
In the specific case of antibody diagnostic tests for SARS-CoV-2, the simultaneous placing on the market of trials with similar characteristics makes their choice particularly difficult. However, having detailed information is essential as serological tests are strategic in detecting the actual prevalence of the disease in the general population, as they also consider asymptomatic or paucisymptomatic individuals. It is known that there is a part of the population infected but not tested, as they did not have any typical symptoms of COVID-19, such as cough, fever, and breathing difficulties.
For the Covidiagnostix project, developed according to the multidisciplinary nature required by the EuNetHTA model, health professionals and researchers with different backgrounds (biomedical, economic, financial, managerial) collaborated with each other (11).
The results show very few differences among tests from different manufacturers, namely test costs and the possibility of being used in vaccination triage. With vaccine triage, the authors intended the opportunity to optimize (in particular due to the scarcity of vaccine doses and to logistic constraints) the vaccination campaign by defining vaccine inoculation priorities as a function of antibodies titer (i.e., postponing the vaccination of natural seropositive individuals or the second dose of higherresponsive individuals to the first dose of vaccine).
Conversely, a common feature, the very low sensitivity in the first days of infection, makes them quite useless (at least if not adequately supported with other types of tests) to diagnose the disease and control the pandemic evolution. This is the reason for the current indication of molecular testing and self-isolation after contact with a confirmed COVID-19 positive individual.
Finally, the rapid-HTA evaluation of serological tests for SARS-CoV-2 has been summarized in a report that allows its dissemination and communication, thus becoming a tool to support managerial decisions to promote high-value-added tests and reduce the variability of the diagnostic test offered (12).
The EuNetHTA model is a flexible and adaptative method to evaluate various aspects of a medical procedure. In the case of serological tests for SARS-CoV-2, we completed the evaluation on the medical literature, which refers to the period when serological tests were used to indirectly identify individuals who came into contact with the virus and who had developed an immune response.
Then, the rapid-HTA report resulting from Covidiagnostix, the first project granted by the Italian Ministry of Health on the HTA approach, could be the basis to choose the tests to be used in the evaluation of the antibody titer developed by each individual against SARS-CoV-2 and of the efficiency of the vaccination campaign.

Conclusion
The HTA is confirmed as a valid tool to support managerial choices to promote high-value-added tests and reduce the variability of the diagnostic offer.
In particular, through Covidiagnostix, the importance of the rapid-HTA method is outlined, which, by adequately supporting the decision-making process, makes it possible to make the allocation of resources more efficient even in the event of an emergency.