To make the best use of limited resources for supporting health-related research, it is necessary to apply a suitable method to rank competing research options. The Child Health and Nutrition Research Initiative (CHNRI), an affiliated organization of the Global Forum of Health Research, has developed a new methodology for setting global health research priorities on topics that are relevant to the United Nation’s fourth Millennium Development Goal: ‘to reduce the mortality rate by two-thirds among children under five by 2015’(Reference Rudan, Gibson and Kapiriri1). The CHNRI methodology builds on existing approaches to establish research priorities in child health and nutrition, using a rationale, conceptual framework, application guidelines and strategies to address the needs of various stakeholders, as described in greater detail elsewhere(Reference Rudan, El Arifeen, Black and Campbell2). The major conceptual advance of this methodology is the recognition that health research options should be defined broadly, not just as investigation designed to produce new knowledge, but also as research activities carried out to provide information leading to more efficient implementation of existing knowledge, with the ultimate objective being the reduction of the current global disease burden(Reference Rudan, El Arifeen, Black and Campbell2).
Briefly, the CHNRI methodology uses a systematic and transparent approach to establish research priorities by considering the opinions of multiple experts and taking into account issues of the feasibility of successfully completing the research, its likely effects on subsequent programme implementation, the maximum potential impact of the resulting programmes on reducing disease burden, and the contribution of the intervention programmes to equity within the target population. CHNRI has now launched a series of projects to apply the priority-setting methodology to a broad range of research topics at both global and national levels. On the global level, the approach has been used to prioritize research options on several diseases that represent the major causes of child mortality and morbidity worldwide, including pneumonia, diarrhoea and birth asphyxia(Reference Rudan, El Arifeen and Black3).
To broaden experience with this priority-setting technique, we have also applied the method to rank possible research priorities concerning the control of Zn deficiency. Although Zn deficiency is not generally recognized as a direct cause of child mortality, except perhaps in its most severe forms(Reference Van Wouwe4), recent research indicates that it predisposes children to an increased incidence and severity of several of the major direct causes of morbidity and mortality, as summarized below. Thus, we applied the CHNRI methodology to develop and assess an expanded list of Zn-related research options.
Recent community-based intervention trials have found that Zn supplementation decreases the incidence of diarrhoea and acute lower respiratory-tract infection(Reference Bhutta, Black and Brown5, Reference Bhutta, Bird and Black6), two of the most important causes of child mortality in lower-income countries, and other studies have found that including Zn supplementation in the therapeutic regimen for treatment of diarrhoea reduces the severity and duration of illness(7). Notably, several studies have detected dramatically reduced death rates among children who received supplemental Zn(Reference Sazawal, Black, Menon, Dinghra, Caulfield, Dhingra and Bagati8–Reference Muller, Becher, van Zweeden, Ye, Diallo, Konate, Gbangou, Kouyate and Garenne11) and a recent large-scale trial found an 18 % reduction in mortality among children over 12 months of age, although no impact on mortality among infants under 12 months old(Reference Sazawal, Black, Ramsan, Chwaya, Dutta, Dhingra, Stoltzfus, Othman and Kabole12). Based on these results, a recent series on child mortality estimated that universal (>90 %) coverage with intervention programmes to prevent Zn deficiency would reduce child mortality by ♠5 % globally, which places such programmes among the top five available approaches for ensuring improved child survival in these settings(Reference Jones, Steketee, Black, Bhutta and Morris13). In addition to these effects of Zn on the risk of morbidity and mortality from common childhood infections, a number of studies indicate that preventive Zn supplements increase the linear growth and weight gain of stunted or underweight children(Reference Brown, Peerson, Rivera and Allen14), thereby reducing rates of malnutrition and malnutrition-related morbidity and mortality, as illustrated in Fig. 1.
Conceptual model of interactions between zinc deficiency and morbidity and mortality
Methods
A group of seven leading experts in the field of Zn research in child health were identified and invited to participate in this priority-setting exercise. The individuals were chosen to represent a wide range of expertise in Zn nutrition: community nutrition, dietetics, clinical nutrition and paediatrics, maternal nutrition, infectious diseases, public health nutrition programmes, health economics, global nutrition policy and the food industry. These individuals have longstanding experience in their respective field of research and all have worked extensively in lower-income countries in Africa, Asia and/or Latin America.
All seven invited experts agreed to participate and formed a technical working group (TWG). The TWG members were requested to list research options they believed to be important for reducing global Zn deficiency, with a particular focus on lower-income countries and restricting their priorities to those that would be expected to yield some impact on reducing disease burdens within 10 years. The target population was defined as children less than 5 years of age, and the target disease burden as all cases of mortality and morbidity related to Zn deficiency. A research option was broadly defined as a research project that proposes to generate new knowledge that would influence one or more health interventions to control Zn deficiency.
The TWG members were asked to categorize each research option into a particular research domain, depending on which aspect of the public health intervention was addressed by the research option. For example, the proposed research options related to the generation of new knowledge for improving the efficiency of already existing interventions were categorized into health policy and system research (domain 1). Research options that could improve existing interventions by increasing their affordability, deliverability or sustainability were listed in research domain 2. Finally, research options aiming to develop new (as yet non-existing) interventions that would be likely to be effective and have the potential to reduce disease burden were assigned to research domain 3(Reference Rudan, El Arifeen and Black15).
Once the TWG members submitted the full list of research options, we prepared a summary list by combining similar or redundant options, and we then organized the consolidated list by type of intervention strategy: Zn supplementation; Zn fortification; dietary modification; other new intervention strategies; or general issues concerning the control of Zn deficiency, such as novel methods for assessing Zn status or evaluating Zn-related interventions. The TWG members were then asked independently to score each of the research options on the consolidated list, applying the scoring system described by Rudan et al.(Reference Rudan, El Arifeen and Black15). Specifically, each research option was scored by answering three questions related to each of the following five criteria: (i) answerability and ethics of the research project; (ii) likely efficacy and effectiveness of the resulting intervention; (iii) estimated deliverability, affordability and sustainability of this intervention; (iv) its maximum potential for disease burden reduction; and (v) likely equity in achieved disease burden reduction. The questions were formulated so they could be answered only as yes or no, with one point credited when the answer reflected a positive evaluation with regard to the particular criterion. Intermediate scores were calculated for each criterion as the percentage of positive responses for that criterion, and the final research priority score (RPS) was calculated as mean of the five intermediate scores. In cases where a TWG member could not answer a question, the answer was counted as missing, so the results did not enter the numerator or denominator when calculating the percentage of positive responses. The global RPS estimates the value of each research option when all five criteria are given equal weight, i.e. it assesses the likelihood of the research option to generate new knowledge that would improve or develop an effective and deliverable intervention which would achieve the maximum disease burden reduction per investment in an equitable manner(Reference Rudan, El Arifeen and Black15). By using a global RPS, it is also possible to manipulate the weight assigned to each criterion to reflect the values of different stakeholders.
Although the CHNRI methodology recommends including the values and opinions of a broad range of stakeholders(Reference Kapiriri, Tomlinson, Gibson, Chopra, El Arifeen, Black and Rudan16), we have not yet subjected the current priority scores to review by a larger reference group of stakeholders globally. Nevertheless, the results are of interest with regard to the experience obtained when applying the priority-setting methodology to a risk factor for major childhood diseases, as well as to investigate the impact of assigning more or less importance to the different evaluation criteria. We accomplished the latter objective by modifying the weighting system for developing the final RPS, assigning each specific criterion as much weight as all the others combined and then re-ranking the research option according to this revised weighting system.
Results
The TWG members submitted a total of ninety research options, which we then consolidated into a final list of thirty-one research options categorized by the type of resulting intervention, as shown in shown in Table 1. In the final list of research options, more than a third of the projects (n 13) focused on Zn supplementation strategies, either therapeutic Zn supplementation for diarrhoea treatment or supplementation as a general preventive measure to control Zn deficiency. Of the remaining research options, six focused on Zn fortification, one on dietary modification and four on potential new intervention strategies. Seven research options dealt with general aspects of Zn nutrition, which eventually could benefit all of the intervention strategies.
Results from a priority-setting exercise in which technical experts scored thirty-one research options according to their potential to create new knowledge that could be applied to reduce zinc deficiency among children under 5 years of age in lower-income countries. Research options are ranked according to the final research priority score (RPS), as derived from the technical working group
EPI, epinephrine (adrenaline); AGA, appropriate for gestational age; SGA, small for gestational age; IM, intramuscular.
The research options in Table 1 are ranked by the final combined RPS provided by the TWG. This score takes into account the five criteria relevant for priority setting. Among the top five research options, four would create knowledge to improve the efficiency of Zn supplementation and one would provide information relevant for all intervention strategies to control Zn deficiency. Table 2 shows the average RPS and the range of the research options by Zn intervention strategy.
Mean, minimum and maximum research priority score (RPS) by type of zinc intervention strategy
NA, not available.
The mean RPS are shown by research domain in Table 3. Most research options in the final list aimed to improve the effectiveness of an existing intervention (health policy and system research, domain 1; n 18). In general, research options that fall into research domain 1, i.e. those that address questions on how to make an already existing intervention strategy more efficient, received higher priority scores than those that belong to research domains 2 and 3, which aim to make existing interventions more affordable, deliverable or sustainable or which offer to develop entirely new interventions.
Mean, minimum and maximum research priority score (RPS) by research domain
Table 4 shows the five highest-ranking research options and their RPS when each of the ranking criteria was assigned an equal weight to the weight of all the other criteria combined. These weighted scores provide a method to simulate the impact of different stakeholders who might value one evaluation criterion more than others. For example, if the criterion ‘answerability of the research option’ were weighted as much as all the other criteria combined, it would indicate that the stakeholders would find it most important that the research option be well-defined and answerable using an ethical research design. In all five scenarios, the research options ranked among the five choices did not vary very much. In most cases, the highest-ranking research options focus on the improvement of the effectiveness of Zn supplementation, regardless of the differential weighting of the evaluation criteria.
Mean research priority score (RPS) of the five highest-ranking research options after weighting one of the five criteria as much as all the remaining four criteria combined (weighed rank) and relationship to original unweighted ranking
EPI, epinephrine (adrenaline).
Discussion
The CHNRI methodology applied here has been applied by other TWG on other diseases on the global level(Reference Rudan, El Arifeen and Black3) and on the national level to address the seven main causes of death in South Africa(Reference Tomlinson, Chopra, Sanders, Bradshaw, Hendricks, Greenfield, Black, El Arifeen and Rudan17). The comparability of this method proposed by CHNRI with other priority-setting methods has been discussed elsewhere(Reference Rudan, El Arifeen, Black and Campbell2). We therefore limit our discussion to our application of the method for prioritizing Zn-related research.
The present project demonstrates the feasibility of: (i) convening a multidisciplinary group of expert scientists (the TWG) who are knowledgeable about different aspects of human Zn nutrition and public health to participate in a jointly enacted priority-setting exercise; and (ii) eliciting from these individuals a list of Zn-related research priorities, scored according to the newly proposed CHNRI methodology. The TWG was able to complete this task successfully following review of relevant background papers on the methodology and minimal additional instruction from the TWG coordinators, which was provided by electronic correspondence.
Interestingly, the research options that received the highest scores pertain to health and policy system research (domain 1), i.e. research on how to improve the efficiency of delivering already existing Zn intervention strategies. This result is probably attributable to the perception that research on scaling up proven efficacious interventions is likely to have the greatest effect on reducing children’s disease burden. The research options that were proposed most frequently focused on therapeutic and preventive Zn supplementation, possibly because these approaches have been tested more extensively to date, so there is more information available on their efficacy.
The TWG encountered several challenges when applying the CHNRI methodology to the problem of Zn deficiency. First, because Zn deficiency is considered a risk factor for other diseases that are primary causes of child mortality, the impact of Zn-related interventions on mortality reduction depends on the link between Zn deficiency and the risk of these other diseases. There is still some inconsistency in research results concerning the relationship between Zn deficiency and the risk of certain diseases, like malaria. Thus, it was not possible to estimate the ultimate impact of Zn interventions on mortality related to these diseases. Second, it was difficult to estimate the true impact of Zn-related interventions for reducing disease burden because of the limited available information on the global prevalence of Zn deficiency. The latter problem is related, in turn, to the difficulty in assessing Zn status. Nevertheless, a working group convened by WHO, UNICEF, the International Atomic Energy Authority and the International Zinc Nutrition Consultative Group has recently published recommendations for assessing population Zn status(Reference de Benoist, Darnton-Hill, Davidsson, Fontaine and Hotz18), so these recommendations can now be applied more broadly to determine the true extent of Zn deficiency and the likely impact of Zn interventions.
We encountered several other difficulties regarding the preparation of a consolidated list of research options using the CHNRI method. In retrospect, our attempt to reduce the list to a more easily manageable, relatively small number of research topics probably resulted in excessive aggregation of these options. For example, when attempting to estimate the potential impact of Zn supplementation, it became clear that the results would differ by age group and birth weight category. Thus, by aggregating all these groups of children under one research option, it was more challenging to provide a single estimate of the likely impact on disease burden reduction. Similar difficulties were encountered because of the need to consolidate these assessments at a global level. For example, the likely impact of a food fortification intervention would depend on the level of development of the food industry in a particular country.
Several other issues, such as the timing of inputs from the larger reference group, the proper interpretation of the rank order of research priorities and more explicit assessment of the cost of research, merit further consideration when applying or adapting the CHNRI priority-setting methodology. Although the current priority-setting methodology suggests consulting a broader reference group of individuals who might be end-users of research results once the list of research options has been developed, it might be preferable to include the perspectives of these stakeholders earlier in this process. This would ensure that the priorities of the broader reference group are represented within the list of research options, not just in the criteria for ranking these already established options.
The final ranking of research options displayed in the present summary should be considered qualitatively rather than quantitatively. The priority-setting method does not currently include statistical considerations, such as confidence limits, of the current rankings; and the relatively small size of the TWG does not lend itself to a more rigorous statistical analysis of the results. Thus, it is not possible to state with any degree of certainty that the first-ranked option is indeed valued more highly than those ranked slightly lower on the list, so it would probably be more appropriate to prioritize groups of research options ranked highly rather than looking at individual scores. The ranking does provide some sense of the general domains of research and research themes that are more appreciated than others according to the group of experts that was convened. We have not tested the reproducibility of this method, and it is not known to what extent the composition of the TWG determines the research priorities and eventually the final scores. To reduce any potential bias, a higher number of TWG members representing a variety of expertise and geographical regions may be required.
The actual cost of implementing the different research options is not explicitly considered in the current priority-setting approach, except perhaps under the criterion of ‘feasibility of research’. Thus, it is possible that the cost of implementing some of the more highly valued research options might exceed available donor resources. Nevertheless, these issues can be considered ‘down stream’ by donor agencies according to the amount of funds available to implement the proposed research priorities.
In summary, we found that it was possible to apply the CHNRI methodology to develop a list of research options concerning the control of Zn deficiency, a known risk factor for excessive child morbidity and mortality, and to assess the relative priority of these research options. The TWG concluded that research options aimed at increasing the efficiency of implementation of interventions known to be efficacious, particularly therapeutic and preventive Zn supplementation, were viewed as most likely to contribute to reducing child morbidity and mortality within the next 10 years in lower-income countries. This exercise should be repeated periodically, possibly with a larger TWG and reference group of stakeholders, as new information becomes available.
Acknowledgements
There are no conflicts of interest for any of the authors. We acknowledge the financial support by CHNRI. All authors contributed to the listing and scoring of the research options. K.H.B. and S.Y.H. coordinated the process, including consolidating research options, computing scores and drafting the manuscript. I.R. advised on the implementation of the CHNRI methodology and reviewed the manuscript.
