Decision analytic models play an increasingly important role in the comparison of alternative health technologies. The authors previously constructed a decision analytic model to determine the cost-effectiveness of a noninvasive program designed to prevent cavities arising from dental caries (Reference Warren, Pollicino and Curtis1). The structured preventive program, the Caries Management System (CMS), involves a noninvasive strategy designed to arrest and remineralize noncavitated lesions. The treatment goal of the CMS is to stop the progression of existing lesions, prevent new lesions, and reduce future needs for restorative care. Full details regarding the management protocol for the CMS has been reported elsewhere (Reference Evans, Pakdaman, Dennison and Howe2). All patients are encouraged to improve their tooth brushing skills through coaching. In addition, nonsurgical clinical care, involving fluoride varnish application to noncavitated lesions, the frequency of which is risk-determined (3-monthly applications for high risk patients and 6-monthly for medium risk patients) is instituted.
Originally the efficacy of the CMS was assessed within the context of a cluster randomized, controlled clinical trial (RCT) in the Australian private practice setting (The Monitor Practice Program) (Reference Curtis, Evans, Sbaraini and Schwarz3). The Monitor Practice Program was planned and implemented initially as a 3-year trial and during this period, contacts with the practices occurred frequently to monitor their adherence to the CMS protocols and to monitor outcomes. Curtis et al. (Reference Curtis, Warren and Pollicino4) demonstrated that, within the construct of the 3-year RCT, regular monitoring and the noninvasive management associated with the CMS protocol is effective in reducing the incremental DMFT (decayed, missing, and filled teeth) compared with standard care. An individual patient-simulation Markov model was developed previously, based on the incremental effectiveness observed in the 3-year RCT, to simulate the incidence and progression of dental caries, and resultant interventions, and to evaluate the lifetime (60 years) cost-effectiveness of the CMS versus standard dental care from the Australian private dental practitioner perspective. In that analysis, Warren et al. (Reference Warren, Pollicino and Curtis1) found that the CMS is most cost-effective in patients with a high risk of dental caries.
BACKGROUND
Posttrial Follow-up Study
The success of the original 3-year trial prompted the continuation of the program for a further 4 years (Reference Evans, Clark and Jia5). The purpose of the extended posttrial follow-up study was (i) to evaluate the CMS protocols in a real-life setting to determine whether the incremental benefits associated with the new mode of practice could be sustained in the longer-term in the absence of frequent monitoring visits from study coordinators, and (ii) to provide efficacy data (assessed at 5 years and 7 years postbaseline) to populate an ongoing assessment of its long-term cost-effectiveness.
During the extended posttrial follow-up (Reference Evans, Clark and Jia5), there was no involvement by the study coordinators other than requesting outcomes at year 5 and year 7. As a result, several dental practices were lost to follow-up (Reference Evans, Clark and Jia5). Of the nineteen dental practices included in the 3-year efficacy analysis (Reference Curtis, Warren and Pollicino4), eleven practices remained in the analysis at year 7 (four CMS and seven Control). Thus, the effectiveness analysis for the extended posttrial follow-up study (Reference Evans, Clark and Jia5) was limited to (a) those dental practices that provided consent to remain in the study, and (b) those dental practices that adhered to the CMS protocols after year three. Given that the earlier analyses (1;3;4) focused on the intent-to-treat (ITT) perspective (using the comparative results from the original RCT), the authors believed that the updated analysis would be most informative if the perspective taken was that of the real-life setting to determine whether the reduction in caries risk, as demonstrated at the end of the clinical trial, could be sustained.
To assess the comparative efficacy during the extended posttrial follow-up, mean DMFT increments during the periods: baseline to years 3, 5, and 7; years 4 and 5; years 4 to 7; and years 6 and 7 were determined through multilevel modeling analysis of co-variance, as the data were clustered by practice. Because the data for each patient is longitudinal and naturally clustered in practices, a multilevel model analysis is adopted. Each patient is modeled as a random effect to address the variation between practices and the large number of drop-outs. To maintain consistency, the authors used the same multilevel model to analyze each component of the DMFT score as was used to analyze mean DMFT increments.
It is evident from Evans et al. (Reference Evans, Clark and Jia5) that, if the CMS protocol is adhered to, the incremental clinical effect is sustainable over the long-term. Once the long-term effectiveness had been proven (Reference Evans, Clark and Jia5), the aim of this analysis was to re-evaluate the long-term cost-effectiveness of the preventive CMS approach in a real world per-protocol setting. Given that the extended posttrial follow-up study was limited to those dental practices that, in the absence of frequent and structured monitoring, adhered to the CMS protocol, it is important to note that the incremental cost-effectiveness ratio (ICER) in this updated economic analysis is conducted from a per-protocol perspective rather than an ITT perspective.
METHODS
Posttrial Follow-up Effectiveness Data
Traditionally, the composite measure of the DMFT is divided into three individual components for inclusion into an economic model (decayed, missing, and filled). The authors’ earlier economic analysis was constructed in this manner because the D (decay) component of the increment DMFT score was monitored and recorded separately during the initial 3-year trial (1;4). However, because monitoring visits ceased during the extended posttrial follow-up period, the D component could not be recorded during years 4 to 7. As a result, DMFT increments in the updated analysis are restricted to M+F increments. In other words, the efficacy results represent the number of restorative events avoided due to the CMS, and the ICER in this economic analysis is the additional cost per restorative event avoided.
Table 1 shows the comparative effectiveness results at year 5 and year 7. It should be noted that the number of restorative events observed for the time period from baseline to year 3 in Table 1 differ from those reported in Warren et al. (Reference Warren, Pollicino and Curtis1). This is because the results presented in Warren et al are based on the ITT population enrolled in the original 3-year trial, whereas the updated analysis is based on the dental practices included in the per-protocol analysis set (based on 302 patients).
DMFT IncrementsFootnote a of Patients by Intervention and Treatment Period - LOCF
Note. The p values are for comparisons between the treatment groups. * Significant at the 0.05 level.
a DMFT increment: The DMFT is a measure of caries experience at a specified point in time, calculated as the sum of the frequencies of untreated decayed teeth (those with cavities) [D] + missing/extracted teeth due to decay [M] + filled teeth due to decay [F]. The DMFT increment is the cumulative sum of incident frequencies of the separate DMF events during a specified period. However, at each data collection time during this study, only new M and F events were recorded.
b As per the original analysis (4), the F component comprises all restorative events: first time fillings; crowns; repeat fillings including repeat crowns; root fillings; bridge units; denture units, and implants. However, if the restorative event was a result of trauma (e.g. football injury, fall etc.), it was not included in the F increment.
c The small discrepancy between the DMFT increment and the sum of the separate M and F events is due to the mechanism of the multilevel model used in the analysis, which estimates the variation among practices separately when modelling different components.
CMS, Caries Management System; DMFT, decayed, missing and filled teeth; LOCF, last observation carried forward; N, sample size; SE, standard error.
Model Structure
The structure of the model has been described in detail elsewhere (Reference Warren, Pollicino and Curtis1). Briefly, a patient-level simulation model was constructed for 10,000 hypothetical patients with an age distribution similar to that of the Australian population. In each treatment arm, caries incidence, progression, and resulting dental treatment is modeled using eight independent Markov sub-models (with cycle length of 6 months) representing the impact of the CMS on the eight molar teeth separately. As per the previous model (Reference Warren, Pollicino and Curtis1), the effectiveness data from the posttrial follow-up is taken from all teeth (molars and anteriors) but has been applied to the eight molars in the model. This is appropriate given that the majority of decay occurs on the molar teeth (6).
Each of the eight independent Markov sub-models has the following eleven health states; No disease, Enamel caries, Dentine caries, Filling, Repeat filling, Root canal, Crown, Extraction, Bridge, Implant, and Death (an absorbing health state). Consistent with the previous economic analysis (Reference Warren, Pollicino and Curtis1), data from the Australian Institute of Health and Welfare (AIHW) (7) has been used to determine the baseline distribution of patients among the eleven health states in each Markov sub-model. As patients simulate through the Markov process, the number of restorative events they have incurred increases by one each time they enter any of the following three health states: filling, repeat filling and tooth extraction.
Therefore, if a patient receives a filling on one tooth and a filling on another tooth, the number of restorative events for that patient will increase by two. The computation of the patient's increment differs slightly from the authors’ earlier model in which a patient's DMFT increment also increased when they entered a fourth health state, the enamel caries health state. This modification is due to the fact that the per-protocol analysis presented here estimates the number of restorative events avoided due to the CMS. Thus, in the base-case analysis, an occurrence of new decay does not impact the number of restorative events incurred.
Transition Probabilities
As described above, the D (decay) component was not reported during the extended posttrial follow-up period. Thus, the underlying 6-monthly probability of new occurrences of enamel caries is assumed to be the same as the previous model (0.154), which was taken directly from the control arm of the original RCT. Furthermore, the relationship between enamel and dentine decay is identical to the previous economic analysis (i.e., based on the relationship reported in Arrow) (Reference Arrow8). Consistent with Warren et al. (Reference Warren, Pollicino and Curtis1), when compared with the control arm, the CMS resulted in a 40.3 percent reduction in the incidence of decay.
Even though decay was not monitored explicitly, it is reasonable to assume that new restorative events are due to new occurrences of decay (either enamel or dentine). Given that the number of restorative events experienced by a patient increases if they enter three health states (filling, repeat filling, and tooth extraction), the occurrence of any new filling-related event such as a crown or bridge is incorporated into the 6-month probability of a filling. This ensures that the increment in restorative events observed in the model corresponds to the F increment reported in Table 1.
As the mean age of patients in the CMS trial was 45 years, the 6-monthly probabilities calculated from the posttrial follow-up data are applied to 45-year-old patients in the model. For a 45-year-old patient in the standard care arm, the 6-month probabilities of progressing from enamel decay to a filling and from dentine decay to a filling are based directly on the incidence of new restorative events observed in the control group. The increment in filling-related events in the control arm over 7 years was 8.32 (Table 1). Given that it is not known whether these new fillings observed at year 7 in the study were due to enamel decay or dentine decay, the ratio of enamel decay to filling versus dentine decay to filling was assumed to be the same as the earlier model (Reference Warren, Pollicino and Curtis1).
Thus, the 6-month probability (for a 45-year-old patient in the standard care arm) of progressing from enamel decay to a filling was 0.288 and the 6-monthly probability of progressing from dentine decay to a filling was 0.225. When compared with the control arm, at year 7, the CMS resulted in a 55.0 percent reduction in the number of first-time fillings (Table 1). The corresponding 6-monthly probabilities of progressing from enamel decay to a filling and from dentine decay for the CMS arm are estimated by applying the percentage reduction in the number of first-time fillings (55.0 percent) to the age-specific 6-monthly probabilities for the standard care arm. Therefore, the 6-month probability (for a 45-year-old patient in the CMS arm) of progressing from enamel decay to a filling was 0.129 (= 0.288 – [0.288 × 55.0 percent]) and the 6-monthly probability of progressing from dentine decay to a filling was 0.101 (= 0.225 – [0.225 × 55.0 percent]). Consistent with the authors’ previous economic analysis (Reference Warren, Pollicino and Curtis1) other economic analyses of preventive programs (Reference Quinonez, Stearns, Talekar, Rozier and Downs9) and the use of multilevel linear models in the estimation of the DMFT (Reference Evans, Clark and Jia5), the model assumes a linear relationship between the incidence of fillings and age.
As with the previous economic analysis (Reference Warren, Pollicino and Curtis1), transition probabilities between future dental interventions (e.g., filling to repeat filling, filling to root canal) were based on 4 years of claims data from the second largest private health insurer in Australia, MBF (now BUPA).
When applying the comparative efficacy of the CMS versus standard care, repeat fillings are modeled separately from first-time fillings. When compared with the control arm, at year 7, the CMS resulted in a 32.1 percent reduction in the number of repeat fillings (Table 1). The 6-month probability of a repeat filling for patients in the CMS arm is estimated by applying a 32.1 percent reduction in the number of repeat fillings to the probability of a repeat filling modeled in the standard care arm (which was based on MBF data).
At year 7, the number of extractions was significantly higher for the CMS arm (Table 1). This anomaly is unlikely to be due to the CMS itself and is unlikely to be observed in clinical practice. It is most likely due to small sample sizes and/or occurred due to pre-existing conditions given that half of the CMS extractions occurred in the first three years. However, to ensure consistency with the clinical data, the rate of extractions used in the model was higher in the CMS arm than the rate in the standard care arm.
In the earlier ITT model it was assumed that the incremental benefit of the CMS ceased at 20 years due to the level of less than perfect adherence over the duration of the RCT. However, given the per-protocol analysis is restricted to those dentists that adhere to protocols, it is reasonable that the incremental benefit of the CMS is maintained indefinitely (providing the CMS protocols are followed).
Updated Costs Associated with Monitoring Dental Caries
The schedule for monitoring caries activity has been reported elsewhere (1;2). In the original economic analysis, unit costs were based on claims paid by MBF for services incurred in 2007. When conducting the updated analysis, it was not possible to obtain claims data from MBF. Unit costs included in this analysis are taken from the 2014 Australian Dental Association (ADA) fee survey (10). As shown in Table 2, the monitoring cost per visit is estimated to be AUD447.31 and AUD182.83 for the CMS and standard care arms, respectively. Given that the mean number of visits per 6-month period is 0.82 in both arms of the model (Reference Warren, Pollicino and Curtis1), the 6-monthly cost of monitoring caries is AUD369.01 and AUD150.82, respectively.
Per Visit Cost of Monitoring Caries Activity and Unit Cost of Dental Interventions (in 2014 Australian Dollars, AUD)
Note. The ADA codes used in the calculation of each item are reported in Warren 2010 (Reference Warren, Pollicino and Curtis1).
a Assumes one double bitewing radiographic survey per year.
b Applied as patient enters the health state.
c Weighted ADA charge based on respective utilization of individual ADA codes.
d Assumes one crown.
ADA, Australian Dental Association.
Updated Costs Associated with Operative Dental Interventions
The unit costs associated with operative dental interventions (fillings, root canals, etc.) have also been updated to 2014 Australian dollars using results from the ADA fee survey (Table 2). Similar to the previous economic analysis, the discount rate is 5 percent, and the perspective of the analysis is from the private practitioner view as no indirect or societal costs have been included.
Sensitivity Analysis
Univariate sensitivity analyses were performed on parameter values to determine the impact on ICERs. Where possible, variables selected for inclusion in the univariate sensitivity analysis were done so to enable a comparison with the results from the authors’ previous ITT analysis (Supplementary Table 1). Probabilistic sensitivity analysis is often performed to address the uncertainty in cohort models. However, in patient level simulation models, at least 10,000 patients are modeled to obtain a reasonable estimate of the mean value of the ICER. Therefore, given the extensive number of transitions allowed in this model within the eight independent Markov subgroups, the computational requirements have prevented the viability of performing probabilistic sensitivity analysis.
RESULTS
Cost-Effectiveness Results
Table 3 shows the ICER, incremental cost per restorative event avoided. Over the 7-year period, the discounted cost per patient was estimated to be AUD5,689 in the CMS group, and AUD3,613 in the control group. Over the same timeframe, the number of restorative events accrued was 5.46 in the CMS group, and 7.62 in the control group. Thus, the per-protocol ICER was estimated to be AUD964 over a 7-year period (USD686; 1 AUD = 0.71 USD), and the lifetime ICER is AUD1,980 (USD1,409). Given this exchange rate has fluctuated by 30 percent in the preceding 12 months, caution should be used when interpreting the USD values.
Discounted Cost (AUDFootnote a ) per Restorative Event Avoided
a 1 AUD = 0.71 USD. Given this exchange rate has fluctuated by 30% in the preceding 12 months, caution should be employed when interpreting the reported USD values.
b Incremental cost per restorative event avoided = incremental cost / incremental restorative events.
CMS = Caries Management System; SC = standard care.
Supplementary Figure 1 illustrates the cost-effectiveness scatter plot over the lifetime horizon, showing the individual incremental cost and effectiveness pairs for each of the 10,000 simulated patients. As expected, the vast majority of points occur in the upper right quadrant. However, a substantial number of points occur in the bottom right quadrant (which represents a claim of dominance as the CMS is more effective and less costly).
Certifying the Model's Structure/Assumptions through Validation of Updated Outputs
Performance evaluation is a key aspect of the model development process with the aim of improving the accuracy and credibility of these models. The earlier model was externally validated by comparing the 3-year results generated in the model with the 3-year results observed in the original RCT (Reference Warren, Pollicino and Curtis1). Due to the availability of extended posttrial follow-up data, we have reassessed the validity of the model. Supplementary Table 2 compares the effectiveness data generated by the model with the 7-year posttrial results. As described above, during the extended posttrial follow-up period, the D (decay) component was not reported. Thus, it is not possible to compare the overall DMFT increments. However, the other components (M+F) which constitute restorative events can be validated. It is evident from Supplementary Table 2 that the model appears to accurately predict the number of extractions. However, the model underestimates the number of filling-related events at 7 years. Consistent with Warren et al., the baseline age distribution of the modeled population was similar to that of the Australian population (12), with a mean age of 37 years. Given that there is a direct relationship between age and the incidence of fillings, the model logically amasses fewer restorative events than observed in the posttrial follow-up (in which the mean age at baseline was 45 years).
Sensitivity Analyses
Sensitivity analyses show the impact of different variables on the ICER. The results of one-way sensitivity analyses are shown in Table 4. Results from the one-way sensitivity analysis suggest that the ICER is most sensitive to the frequency of CMS monitoring, the ability of the CMS to arrest all tooth decay, and the duration over which the incremental benefit is assumed to continue. The results from the sensitivity analyses support the cost-effectiveness of the CMS, with the lifetime ICER ranging from AUD51 to AUD2,936 per restorative event avoided (USD: 36–2,089).
Sensitivity Analyses around the Cost-Effectiveness at 7 Years and over the Lifetime of the Model (AUD)
a CMS is less costly and more effective.
CMS, Caries Management System; DMFT, decayed, missing and filled teeth.
Even though the D component was not captured in the extended posttrial follow-up, the sensitivity analysis estimating the incremental cost per DMFT avoided enabled a comparison with the results from the previous ITT analysis (Reference Warren, Pollicino and Curtis1). It is logical that the lifetime ICER in the per-protocol analysis (AUD1,606; Table 4) is more favorable than the previously published ITT analysis (AUD1,795) given that the patients in the posttrial follow-up study were higher risk patients than those who were lost to follow-up. This validates the conclusion from the earlier study that the CMS is most cost-effective in patients with a high risk of dental caries.
DISCUSSION
The results of the extended posttrial follow-up study (Reference Evans, Clark and Jia5) demonstrate that, so long as efforts are maintained to reduce exposures to the important caries risk factors through home control of dental plaque level and restriction of between-meal exposures to sugar-containing foods and beverages on the one hand, and optimizing exposure to fluoride from drinking water, fluoride toothpaste, fluoride rinses, and professionally applied fluoride varnish on the other, the risk of decay can be reduced to near zero for some people and substantially reduced in others. Once the long-term incremental effectiveness had been proven (Reference Evans, Clark and Jia5), the aim of this current study was to re-evaluate the long-term cost-effectiveness of the preventive CMS approach in a real world per-protocol setting.
Consistent with other published economic models, we have reported cost-effectiveness in terms of cost per restorative event avoided (Reference Ney, van der Goes and Chi13). Although numerous trial-based economic evaluations exist in the dental care setting, the authors’ research provides one of the few modeled economic analyses of preventive dental programs (13–15). Our analysis both adjusts the age and baseline incidence of dental caries to the population being considered, and extrapolates the costs and outcomes beyond the duration of the clinical trial on which the efficacy results are based. Random effects of practices and random effects of patients were taken into account through the use of multilevel statistical modeling. Furthermore, our analysis models the impact of the CMS on the eight molar teeth separately, thereby increasing the accuracy of the model.
The model yields a lifetime cost per restorative event avoided of AUD1,980. Compared with Anderson (Reference Anderson16), who estimated the lifetime economic impact of a probably avoidable filling is over AUD3,300 (converted from USD 1,800 in 1997), the CMS appears cost-effective. This is despite the omission of many of the indirect costs associated with the provision of restorative services (time off work, productivity, and the opportunity cost of the dentist/auxiliary time in providing noniatrogenic services) from the analysis. Furthermore, this analysis does not incorporate the quality-of-life associated with (i) avoiding the pain and discomfort of ongoing restorative care, and (ii) the knowledge that tooth weakening is a consequence of repeat restorations (Reference Elderton17). Thus, the incremental cost associated with CMS monitoring may easily be justifiable to those patients whose prefer the nonsurgical approach.
The premise of the CMS is that regular monitoring of caries activity reduces the incidence of lesions and results in fewer operative interventions. This is consistent with the results of a 30-year preventive program which demonstrated that improved dental care behavior at home reduces the need for operative interventions and frequent monitoring (Reference Axelsson, Nystrom and Lindhe18). Given that the base-case analysis assumes that patients in both arms of the model undergo the same number of visits per 6-month period (0.82 per 6-month period), the base-case potentially overestimates the long-term incremental cost of the CMS program. The sensitivity analysis (Table 4) confirms that lifetime costs are substantially reduced when CMS monitoring frequency is halved and, consequently, cost-effectiveness improves considerably.
LIMITATIONS
As described above, the analysis estimates cost-effectiveness in terms of cost per restorative event avoided because the D component was not recorded during the extended posttrial follow-up period, thereby hampering comparisons with other published economic evaluations that used the DMFT measure (19–22). Over the lifetime horizon, the model most likely underestimates the number of new restorative events for several reasons. First, transitions between future dental interventions (e.g., filling to repeat filling) were based on 4 years of MBF claims data. In order for the transition probability to be calculated, both events must have occurred within the 4-year period. For certain paired events (such as repeat filling to extraction), it is unlikely that, in reality, these two events frequently occur on the same tooth in a 4-year period.
Second, the starting distribution of the modeled population among the Markov health states is based on AIHW data. Given the 10,000 modeled patients are randomly assigned a baseline tooth status, a substantial proportion of patients commence one of the eight molar sub-models in the “missing” and “implant” health states (Table 3 from Warren et al.). Given these patients start the Markov process in advanced health states, the model structure prohibits them from undergoing new restorative events on that particular tooth (as per Figure 1 from Warren et al.). All these assumptions potentially underestimates the incidence of dental interventions such as crowns and implants in the control arm, and as a result underestimates the long-term incremental benefit of the CMS.
If the CMS protocols are adhered to, the risk of decay is substantially reduced. The model assumes that the CMS reduces the incidence of enamel decay (as represented by the letter D on Figure 1 from Warren et al.). However, in practice, it is likely that the CMS will also prevent enamel decay from progressing to dentine decay. This benefit has not been incorporated into the model, and given that an equal number of fillings are done for reasons of enamel and dentine decay (Reference Brennan, Balasubramanian and Spencer23), this exclusion underestimates the incremental benefit of the CMS.
CONCLUSIONS
Even though adherence to the CMS protocol is associated with increased patient costs, it is evident from 7-year effectiveness results that, if the CMS protocol is adhered to, the incremental clinical effect is sustainable over the long-term. Therefore, dentists who wish to manage caries by nonsurgical methods, can do so successfully so long as regular risk-based monitoring and fluoride application is performed, thus validating the assumption in the original study that efficacy is maintained for as long as the CMS protocol is followed. The updated model is reasonably accurate at predicting restorative events (i.e., the incremental MFT at 7 years generated in the model is similar to that observed in the trial). The ICER for the per-protocol population is more favorable than the ICER generated for the ITT population, validating the conclusion from the earlier study that the CMS is most cost-effective in patients with a high risk of dental caries.
SUPPLEMENTARY MATERIAL
Supplementary Table 1: www.journals.cambridge.org/10.1017/S0266462316000246
Supplementary Figure 1: www.journals.cambridge.org/10.1017/S0266462316000246
Supplementary Table 2: www.journals.cambridge.org/10.1017/S0266462316000246
SOURCE OF FINANCIAL SUPPORT
This research has received funding from the Oral Health Foundation, NHMRC (project grants 402466 and 632715), The Dental Board of NSW, and the Australian Dental Research Foundation.
CONFLICT OF INTEREST
Emma Warren has received consulting fees from The University of Sydney.
Bradley Curtis is an employee of Eli Lilly and Company, Indianapolis, Indiana, USA.
Nan Jia is an employee of Eli Lilly and Company, Indianapolis, Indiana, USA.
Wendell Evans has an honorary appointment at The University of Sydney, Australia.
ETHICAL STANDARDS
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Open access
