Most prevention programs for colorectal cancer (CRC) are aimed at medium-risk populations and are included in population-based prevention strategies. The main inclusion criteria in this type of program is age (aged 50 and older without personal or family history of colorectal adenomas or CRC) (Reference Ribes, Navarro and Cleries1,2). There is another risk group, made up generally of individuals at increased risk of developing CRC, which includes the following criteria: (i) individuals who carry mutations in certain genes, (ii) individuals with a family history of CRC, and (iii) individuals with a history of high-risk colorectal adenomatous polyps. Individuals in the group of carriers of mutations in specific genes represent between 3 percent and 5 percent of all patients diagnosed with CRC.
In 2006, the High-Risk Clinic for Colorectal Cancer (HRC-CRC) (Reference Balaguer, Ocaña, Garrell and Ferrández3;4), a new healthcare model for the prevention of CRC in a high-risk population, was created at the Hospital Clinic of Barcelona under the coordination of the Institute of Digestive and Metabolic Disorders. The Hospital Clinic of Barcelona is a reference, community, level 1, public teaching hospital in Catalonia with renowned capabilities, both nationally and internationally, in healthcare, research, innovation, and quality teaching. Catalonia is one of the seventeen autonomous regions in Spain with a regional administration and autonomous government and parliament. The HRC-CRC was funded by the Catalan Department of Health with the aim of implementing prevention strategies in populations at greater risk of CRC (individuals with a hereditary or individual predisposition to colorectal cancer such as Lynch syndrome, familial adenomatous polyposis, or advanced colorectal adenomas). The HRC-CRC is staffed with health professionals from the fields of gastroenterology, biochemistry and molecular genetics, pathology, clinical psychology, nursing, gastrointestinal surgery, oncology, and family and community medicine. The roles of this new healthcare model/program included the following main areas of healthcare: (i) to identify individuals at increased risk of developing CRC (mainly individuals referred from primary health care according to evidence-based criteria in relation to CRC risk factors of increased risk or other specialized services from their own hospital or from other hospitals; patients and users also access the program on their own initiative; (ii) to establish the risk of CRC in users with personal risk factors and/or a family history of CRC; (iii) to propose the most appropriate screening and surveillance prevention strategies; (iv) to carry out genetic counseling in hereditary forms of CRC, genetic testing, and psychological assessment; (Reference Castells, Sala and Ascunce5) to carry out additional endoscopic and radiological procedures for prevention, diagnostic; and/or therapeutic aims.
In a first phase of implementation of the HRC-CRC, the Hospital Clinic of Barcelona, pending the incorporation of a secondary hospital, assumed the role of secondary and tertiary care provider, and was linked to two primary care centers, which took on the primary level functions of this new organizational model for the prevention of CRC. Tertiary level functions of this model consist mainly in the implementation of measures for genetic counseling and testing, and the performance of complex surgical or endoscopic procedures (e.g., Lynch syndrome). Secondary level functions include the management of high-risk forms of CRC requiring colonoscopies with a periodicity of less than 5 years (e.g., familial CRC) and the performance of conventional and therapeutic endoscopic procedures. Finally, functions at the primary care level consist in identifying populations at greater risk of CRC and the follow-up of users on the prevention program with intervals of over 5 years between colonoscopies (e.g., patients with non-advanced adenomas).
Although the activities (clinical practice) of the HRC-CRC are evidence-based, to the best of our knowledge, no assessment of the quality of care and the benefits/risk of the prevention program on the user's health have been carried out (4;Reference Castells, Sala and Ascunce5). To date, no published study has been identified that evaluates the quality of care of a CRC prevention program in a high-risk population taking into account all the healthcare processes involved. The primary objective of this study was to implement a set of indicators to assess the quality of care of a new healthcare model for the prevention of colorectal cancer in a high-risk population (at baseline and independently). Furthermore, as a secondary objective, factors related to adherence to screening and surveillance prevention strategies in this model were analyzed. Several studies have evidenced that demographic factors such as user gender, age, and health status are related to their adherence to screening and surveillance prevention strategies (Reference Griffin, Burgues and Vernon6;Reference Denberg, Melhado and Coombes7). The assessment of factors relating to adherence to this type of strategies in the HRC-CRC was considered relevant to assess the construct validity of three proposed indicators measuring patient-centered care.
METHODS
A retrospective observational and descriptive study was carried out. The study population included all users assessed at the HRC-CRC between January 1, 2006, and May 12, 2010. Patients with advanced colorectal adenoma, hereditary polyposis syndromes, CRC, and healthy users at increased risk (family members of patients with CRC or hereditary syndromes) aged 18 and older were included in this study (Reference Balaguer, Ocaña, Garrell and Ferrández3;4).
Development of Quality Indicators
In a previous study, thirty indicators were developed to assess the quality of care of the HRC-CRC or other prevention programs in Catalonia/Spain (Reference Serra-Sutton, Barrionuevo and Herdman8). A conceptual framework for the assessment of these programs and the definition of indicators was proposed. This framework included the main activities (process of care) of the new prevention program from the perspective of Donabedian for quality assessment (including the measure of the improvement of health services results and also the structure and process of healthcare). Five domains were also proposed in the framework to measure the quality of healthcare (clinical effectiveness, continuity of health care, access, safety, and patient-centered care).
A literature review was performed to identify published studies that assessed CRC prevention programs in a high risk population using quality indicators and their conceptual frameworks. Two consensus meetings were held with twelve experts from the Hospital Clinic of Barcelona and the Hospital Costa del Sol in Malaga (including the fields of gastroenterology, nursing, primary care medicine, molecular genetics, pathology, clinical psychology, medical oncology, management, and health services research). Experts were asked to propose indicators following the above-mentioned conceptual framework. Indicators were arranged in panels by the moderator of the consensus meeting (metaplan technique) according to the defined framework. All proposed indicators were voted by the experts in a final round. Stickers were placed on the indicators by each expert when deemed important or feasible. Two focus groups that included users from the HRC-CRC were also held to obtain additional indicators that had not been mentioned by the experts. The research team developed cards for each indicator, including the title of the indicator, formula, description and justification, sources of information to obtain the indicators, and quality standard.
Final consensus of indicators (cards) was defined by applying the Delphi technique, with the participation of fifty-three experts from different centers, regions in the Spanish Health System, and the same multidisciplinary profiles included in consensus meetings. Experts were asked to vote on the importance and feasibility of (on a scale of 1 to 10, where 1 was minimum and 10 was maximum) as well as their agreement with each proposed indicator (fully agree, agree with modifications, or fully disagree) (Reference Serra-Sutton, Barrionuevo and Herdman8). Experts in this final consensus were asked to propose a quality standard for each indicator (the standard was calculated using the mean value obtained from all the experts).
Indicators, Variables, and Sources of Information in the Phase of Implementation
Before collecting the data to implement the indicators, the research team reviewed each indicator taking into account a preliminary revision of available data, the feasibility to calculate each indicator in the study schedule, simplicity of the formula computation, and those indicators that applied to the majority of the study population. Of the thirty proposed indicators, twenty were selected for implementation and one additional indicator was proposed that included a composite of two previous indicators (global adherence to the prevention program-screening and surveillance colonoscopies); see Supplementary Material, which can be viewed online at www.journals.cambridge.org/thc2013089 (Reference Griffin, Burgues and Vernon6).
The necessary information for calculating each indicator, together with user demographic and clinical variables at the HRC-CRC, were collected, including gender; age (<45, 48–59, >59); type of prevention strategy (a) screening colonoscopy, which suggests an examination of the colon, following evidence-based clinical criteria; if abnormal areas are detected, tissue can be removed and examined to determine if there is presence of disease; (b) surveillance colonoscopy, which suggests a periodical examination of the colon, following evidence-based clinical criteria, after the detection or removal of disease in the colon)Footnote 1; (c) personal risk at the moment of inclusion in the study (advanced colorectal adenoma, polyposis syndrome, CRC, healthy high-risk relative); and site of referral (other hospitals, other intra-hospital services, primary care, and user's own initiative).
The primary source of information for calculating the process and outcome indicators was the information-based clinical records of the HRC-CRC (Progeny). In the case of structure indicators, clinical and management documentation was consulted. Additional information was collected through questionnaires sent by email to professionals or collected in face-to-face interviews at the HRC-CRC by the research team.
Statistical Analysis
Twenty-one indicators were computed according to a previously defined formula, and results obtained were compared with defined quality standards (0–100 percent) (Reference Serra-Sutton, Barrionuevo and Herdman8). For some composite indicators, global and partial compliance was described. Star graphs were computed to present a visual differentiation between the defined quality standard and the level of compliance.
Donabedian and other authors consider that outcomes of health services are mainly related to the structure and process of healthcare as well as to patient/user characteristics (Reference Donabedian9;Reference Sánchez del Río, Baudet, Naranjo and Campo10). To apply this perspective, the research team revised the proposed indicators and selected adherence to the screening and surveillance prevention program as an outcome indicator for multivariate analysis. The reasons for selecting this indicator were mainly due to the fact that it applied to the majority of the patients/users assessed in the new organizational model (ensuring an effective number of cases in the numerator and denominator of the formula to compute adherence). In addition, this indicator was also considered an intermediate-term outcome (in contrast with the global effectiveness of the program [diagnosis of colorectal cancer in early stages], which represented a small number of cases and needed a longer follow-up period to obtain a robust measurement). Most of the variables related to adherence to screening and surveillance prevention strategies in colorectal cancer used in the present study have been previously reported in other published studies.
To identify factors related to global adherence to screening and surveillance prevention strategies, multiple regression models were computed using global adherence as the dependent variable and user demographic, health status, and healthcare process factors that showed statistical significance on bivariate analysis (p < .05) or those that were hypothesized to be conceptually relevant (as described previously in the variables section) as independent variables. Each logistic model was adjusted by gender and age differences. The SPSS package was used for the statistical analysis.
RESULTS
Characteristics of Patients and Users and Some Aspects of the Healthcare Process
The initial database contained information on 1,607 users; 311 users that had been included in the HRC-CRC before January 1, 2006 (data at the start of the new healthcare model), 4 users that were under the age of 18, and 17 users without high risk of CRC in the database, were excluded from the study. Of the total sample, 54 percent were women, and 34 percent were over the age of 59 (Table 1). In addition, 63.1 percent of users were seen due to a family history of CRC (screening prevention strategy), 46.2 percent presented a high-risk colorectal adenoma at the time of the study, 63.2 percent had had at least one colonoscopy, and 32.4 percent had had two colonoscopies (at baseline and at follow-up). A total of twelve professionals provided information on some indicators related to the quality of care in the HRC-CRC.
Demographic, Health Status, and Process Characteristics of Patients in the HRC-CRC in the Period from 2006 to 2010 (n = 1,275)
aLesions ≥ 10 mm, with villous component or high grade dysplasia.
bMutations in certain genes.
cHealthy relative at risk: the user has been included in the HRC-CC but a polyposis syndrome or high-risk colorectal adenoma or CRC have not been identified.
HRC-CC, high-risk clinic for colorectal cancer.
Compliance of Indicators with Quality Standards
Structure indicators that reached the quality standard were: availability of information-based clinical records in the program (100 percent), access to an accredited and certified molecular genetics laboratory, and availability of prevention campaigns aimed at users and patients in the HRC-CRC (100 percent) (Figure 1a). Even though an evidence-based protocol and a clinical practice guideline were available at the HRC-CRC, the indicator that assessed the existence of a clinical protocol reached an overall compliance of 73.9 percent. The specific aspects of the protocol that did not meet quality standards were the failure to include: all activities with scientific evidence, algorithms (referral to clinical psychology service or molecular genetics), and the collaboration of all professional profiles in the development of such protocol. The level of implementation of the clinical protocol by professionals reached a compliance level of 83.3 percent.
Level of compliance of indicators with defined quality standards. a: Level of compliance of structure indicators. 1. Availability of a multidisciplinary clinical evidence-based protocol; 1a. Protocol with recommendations, decision tools; 1b. Protocol with evidence- based activities; 1c. Multidisciplinary profile of authors of the protocol; 2. Access to a certified and accredited molecular genetic testing laboratory; 3. Availability of a side-viewing duodenoscopy for screening of patients with polyposis syndrome; 4. Availability of prevention strategies of colorectal cancer in a high-risk population for users and patients; 5. Existence of information-based clinical records. b: Level of compliance of process indicators. 6. Availability of a colonoscopy quality program; 7. Rate of indication of dietetic and nutritional needs assessment; 8. Level of implementation of the clinical protocol; 9. Presymptomatic diagnosis of users with hereditary colorectal cancer with genetic testing; 10. Administration of a comprehension questionnaire to users; 11. Rate of indication of psychological assessment of users in the program. c: Level of compliance of outcome indicators. 12. Adherence to surveillance preventive strategies in patients with colorectal adenomas; 13. Global adherence to screening and surveillance preventive strategies in users at high risk of colorectal cancer; 14. Effectiveness of the program (diagnosis of early stage CRC); 15. Adherence to screening preventive strategies in users at high risk of colorectal cancer; 16. Administration of a satisfaction questionnaire to users; 17. Administration of a questionnaire to users to measure the impact of the program on their physical and emotional well-being.
Figure 1b shows compliance levels achieved by the process indicators. The rate of users referred to the program was 43.0 percent from primary care and 28.5 percent from other in-patient departments or services (data not shown). On the other hand, 1.9 percent of users with a possible polyposis syndrome needing molecular genetic testing were referred to clinical psychology. In families with hereditary CRC with an associated known mutation, a presymptomatic diagnosis through genetic testing was carried out in 56 percent of relatives at risk. In relation to the waiting time to obtain the results of the genetic test, 33 percent of users who had undergone this test obtained the results 12 months after the time of request. Finally, regarding the comprehension of the information given to users in the program, only 19 percent had completed a comprehension questionnaire and these corresponded to users who had been visited by the nurse in the program.
Regarding outcome indicators, the rate of complications during colonoscopies was 0.6 percent (data not shown), while the rate of early-stage CRC diagnosis was 55.6 percent (as a proxy of the global effectiveness of the program). Adherence to screening and surveillance prevention strategies achieved the agreed standard for the surveillance strategy (mainly applied to patients with advanced colorectal adenomas; 91.2 percent), whereas for the screening strategy, the level of compliance was 51.7 percent, below the agreed quality standard (80 percent) (Figure 1c). The compliance to global adherence was 67.3 percent. User satisfaction with the care provided was measured in 19 percent of the 599 users seen between 2008 and 2010 (period in which the collection of information started).
Overall, eight of seventeen indicators reached the previously defined quality standard (80 percent in the case of structure, 50 percent of process, and 17 percent of outcome indicators). Moreover, 4 indicators were computed but did not have a previously defined standard. When taking into account the quality domain measured, 40 percent of the indicators that measured clinical effectiveness achieved the standard (2/5), 100 percent of the indicators that measured accessibility achieved the standard (2/2), 50 percent of the indicators that measured continuity of healthcare achieved the standard (2/4), 28.6 percent of the indicators that measured patient-centered care achieved the standard (2/7), and 100 percent of the indicators that measured safety reached the standard (1/1). Two indicators measuring continuity of health care did not have a previous reference standard.
Demographic and Clinical Factors Related to Global Adherence to Screening and Prevention Strategies in Users in the HRC-CRC, 2006–2010: Logistic Regression Analysis of Adherence to Colonoscopies (n = 613)a
aAdherence has been computed as the date differences between baseline and follow-up colonoscopies; HRC-CC: High-risk clinic for colorectal cancer. N = 613 correspond to patients/users with two colonoscopies (baseline and follow-up). Nonadherence means that the patient had a baseline colonoscopy but did not come for the scheduled follow-up colonoscopy. Users and patients who did not have a second colonoscopy because it did not apply and those who did not have a baseline colonoscopy were excluded from the analysis.
bOR adjusted by age and gender.
cReference category.
HRC-CC, high-risk clinic for colorectal cancer; OR, odds ratio.
Adherence to the Screening and Surveillance Prevention Strategies and Related Factors
Users aged 59 and older had almost a two times greater probability (95 percent confidence interval [CI], 1.3–3.1) of adhering to screening and surveillance strategies than the younger age group (<45 years) (Table 2). Users with a polyposis syndrome presented a 14.4 times (95 percent CI, 6.3–32.9) greater probability of complying with scheduled follow-up examinations than those relatives with a high-risk profile for CRC. Users referred by other hospitals were 2 times more likely to adhere to the program than those who came to the HRC-CRC on their own initiative, although differences were not statistically significant. Finally, users who had undergone surveillance colonoscopies were 7.4 (95 percent CI, 4.6–11.7) more likely to adhere to the program than those who had had screening colonoscopies.
DISCUSSION
To the best of our knowledge, this study is the first to evaluate the quality of care and the areas of improvement of a new organizational model for the prevention of CRC in an increased-risk population, taking into account the entire care process and a multidisciplinary perspective. Some publications have assessed the quality of care of CRC prevention programs; however, most of these studies were conducted in medium-risk populations and focused only on one or two aspects of the healthcare process. For example, one study evaluated the quality of colonoscopies in eight endoscopic services and included indicators such as the existence of informed consent, medication before colonoscopy, the availability of complete colonoscopy, or complications (Reference Gagliardi, Simunovic and Langer11). Another study developed fifteen indicators of quality of care to assess colon surgery in patients with CRC (e.g., the appropriateness of adjuvant therapy and postoperative surveillance strategies) but only assessed two specific aspects of the organizational model of the HRC-CRC (colonoscopies and surgery) (Reference Sánchez del Río, Campo and Llach12). Finally, some studies that included patient-centered indicators such as satisfaction with prevention strategies and adherence to screening colonoscopies of users and their relatives were also carried out in medium-risk populations (users aged 50 and older without individual or family increased risk) (Reference Bujanda, Catells and Llor13;Reference Crispin, Birkner and Munte14).
Regarding the program's patient safety, the rate of complications during colonoscopies was relatively low (0.6 percent) but greater than the published standards in medium-risk populations (0.1 to 0.3 percent) (Reference Singh, Penfold and De Coster15;Reference Payne, Nicholls and McAllister16). Nevertheless, greater-risk and medium-risk populations are not strictly comparable in terms of clinical characteristics and type of complications. It is possible that in high-risk populations, the incidence of complications is slightly greater due to the need for therapeutic maneuvers that are associated with increased risk of complications. It should be mentioned that an increased waiting time for genetic testing results in patients with a probable hereditary syndrome may have an impact on their psychological well-being. This impact was mentioned in a qualitative study on users of the HRC-CRC carried out in a previous phase of the project for the definition of indicators as well as in other published studies on perceived outcome measures in genetic counseling and testing (Reference Serra-Sutton, Barrionuevo and Herdman8;Reference Watkins, Way and Fiander17;Reference Mack, Cook and Temple18). In future studies, complications from all the prevention interventions of a given program should be described, especially those that may be more difficult to measure such as the psychological or physical discomfort caused by the program (colonoscopies, genetic and molecular testing, preventive surgery, etc.).
Regarding adherence to screening and surveillance prevention strategies, the quality standard was met in the surveillance strategy (patients with high-risk colorectal adenomas), but not in the screening strategy. This lower adherence in healthy relatives could be due possibly to a lower user perception of CRC risk than users with a diagnosis of polyposis syndrome, advanced colorectal adenoma or CRC. Other published studies show similar results regarding factors, such as gender, age, and risk of CRC, explaining differences in the adherence to screening and surveillance prevention strategies 6;7;14;19;20). The lower adherence in healthy relatives with increased risk of CRC might also be due to multiple factors such as individual psychosocial characteristics, relationship with the doctor, family and social environment, and lower perceived risk (21–23).
More and more often, current medical actions are aimed at prevention, as part of community programs for secondary prevention but also as part of many clinical activities, being the balance between the benefits and risks of these activities in many cases uncertain (Reference Gérvas, Starfield and Heath24). In recent publications, several authors have criticized the heavy reliance on health prevention activities (25–27). The present study failed to demonstrate strictly the benefits of the prevention program in terms of health gains for the population, for instance the impact of the new organizational model (the HRC-CRC) to reduce the incidence of CRC or its mortality rate. In relation to the overall effectiveness of the program, a lower percentage than expected of patients with early-stage CRC (0-I) were attended in the HRC-CC. Longer follow-ups should be performed to analyze the capability of the program to detect early-stage CRC patients. As said before, at present, users are mainly referred to the HRC-CRC from other hospital services, which may explain the increased severity of the patients’ disease. Indicators that were not implemented in the present project, such as colorectal cancer that is diagnosed between two colonoscopies (interval CRC) or the overall effectiveness of the program to detect CRC, as well other patient-perceived indicators such as the previously mention impact of the program on the psychological and physical well-being of users, need to be measured in future studies.
Because the indicators have been applied in a single center and at baseline, this could lead to limitations in the interpretation of compliance with the quality standards. The fact that there are no published standards for each of the indicators, or the absence of consensus on some of them, hampers the interpretation of results. Further assessment of the HRC-CRC or the implementation of indicators in other prevention programs to assess the quality of care and to confirm the robustness and validity of the indicators is therefore required (Reference Wollersheim, Hermens and Hulscher28;29). In this study, modifications were made to improve the accuracy and validity of indicators, in terms of specification of the formula, description/justification, or even the standard if identified in the literature.
Limitations of the Study
Although the assessment of the quality of care based on the use of indicators appears to be relatively easy to perform, it has required a complex review of data quality. The retrospective nature of study and the use of computerized clinical records as the main source of information have hampered the processing of data for analysis and the calculation of some indicators. This database was not developed for research purposes, but for the clinical management of users. However, it was possible for the first time to assess data quality and completeness from information-based clinical records. Finally, it should be mentioned that the health of users changes over time, complicating the definition of their risk profile for statistical analysis. We used the most definitive diagnosis at the time of analysis, but users could have gone from a “healthy profile” to a “polyposis syndrome” in a short time, a fact that may have affected the accuracy of the association between adherence and CRC risk profiles.
CONCLUSION
Several indicators have been implemented to assess the quality of care in the HRC-CRC. They have demonstrated their feasibility and construct validity to identify areas of improvement of this new model, such as information systems, continuity of care, and communication among professionals. This is one of the first initiatives to have externally evaluated a CRC prevention program in a high-risk population in Catalonia. The fact that there are no published standards for each of the indicators makes the study an innovative assessment tool. However, further assessment or implementation of the indicators in other centers and in the HRC-CRC is still required to improve the interpretability of the results.
SUPPLEMENTARY MATERIAL
Supplementary Material: www.journals.cambridge.org/thc2013089
CONTACT INFORMATION
Victoria Serra Sutton, Sociologist, PhD, (vserra@aatrm.catsalut.cat), Catalan Agency for Health Research and Quality and CIBER Epidemiología y Salud Pública, Barcelona, Spain
Carmela Barrantes Serrano, MD, University Hospital Vall Hebrón, Preventive Medicine Service, Barcelona, Spain
Mireia Espallargues Carreras, MD, PhD, Catalan Agency for Health Research and Quality and CIBER Epidemiología y Salud Pública, Barcelona, Spain
CONFLICTS OF INTEREST
All authors report a grant to their institution from Quality Plan, Spanish Ministry of Health 2009.
Open access
