One of the key tasks of governments and their advisory bodies is to decide what the nature and quality of the evidence should be which they use to support their policies. Evidence, as pointed out by Valerie Miké, is inherently incomplete, provisional, and subject to refutation (Reference Miké4). So when are we justified to presume sufficient value of a healthcare technology, warranting its public provision and funding?
The question is crucially important because of the serious consequences of policy errors: are important health benefits being denied to the public, or are scarce resources being spent on services that could have produced more significant health benefits elsewhere? When a physician errs, this may harm an individual patient. To be sure, scorning such consequences is a grave offense. However, when policy makers err, an entire community is affected. Paradoxically, at the policy level, fewer opportunities and mechanisms seem to exist for learning and rectification. Still, important progress has been made in this area.
First, we have come to realize that Bayesian approaches may be most appropriate to address these issues (Reference Esarey3). After all, a Bayesian analysis allows for an estimation of the relevant probabilities, for instance that a particular healthcare service will, in fact, be cost-effective, given the available evidence. Probability is here a measure of justified belief, of course.
Second, interesting experiments are being conducted with provisional funding arrangements, associated with evidence development. The most important feature of such arrangements may be their signaling function: the jury is out; the technology is still under probation. This is an important signal to healthcare providers and patients alike: there may be as yet unidentified benefits and risks, and you are engaged in a collective effort to learn more about them.
We also should accept that, at the end of the day, we may decide not to pursue this further. Whether the latter is a realistic option, or whether the arrangement will turn out to be a Pandora's Box, will still have to be borne out. In the meanwhile, it is of interest to know whether such arrangements are politically viable and can count on support from the public. This is exactly what Roger Chafe and his colleagues from the University of Toronto have investigated (Reference Chafe, Laupacis, Levinson, Merali and Martin1). The Ontario government has decided to restrict access to positron emission tomography, until further evidence becomes available. To build such evidence, it committed to funding five trials. The rather daring and precarious nature of this decision may be appreciated, when it is realized that no such restrictions were in place in neighboring Québec.
Chafe and his colleagues asked members of the Toronto Health Policy Citizens' Council to deliberate the Ontario policy. After 2 days' deliberation with input from various experts, the majority of the Citizens' Council considered the Ontario policy defensible. It expressed concerns, however, about the run time of the project. Producing the evidence apparently took more time than planned, partly because patient accrual lagged behind initially anticipated rates. This suggests that, in this context, researchers should preferably adopt a naturalistic design, being liberal with patient inclusion rather than highly restrictive. This, however, could be conflicting with the requirement of demonstrating efficacy. Clearly, several such methodological issues still need to be resolved.
The need for public participation in health technology assessment has been recognized from its early days (Reference Coates2). It touches upon basic questions as to how democratic societies arrive at decisions regarding science and technology. In their study, Chafe and his colleagues state that “Because policy makers and healthcare professionals cannot legitimately represent the views of lay citizens, direct public engagement is required.” This is an interesting observation, but is it true? After all, is this not exactly what representative democratic decision making is about? Of interest, there is wider dissatisfaction with contemporary democratic practice, giving rise to novel, deliberative concepts of democracy (Reference Smith and Wales5). These developments offer interesting opportunities for health technology assessment. Chafe and his colleagues should be commended for sharing with us their experience with public engagement in the area of evidence development and accountable policy making.
