Methodologies to measure the coverage of vitamin A supplementation: a systematic review

Countries are increasingly transitioning from event-based vitamin A supplementation (VAS) distribution to delivery through routine health system contacts, shifting also to administrative, electronic-based monitoring tools, a process that brings certain limitations affecting the quality of administrative VAS coverage. At present, there is no standardised methodology for measuring the coverage of VAS delivered through routine health services. To address this gap, we conducted a systematic review of the literature to identify and recommend methods to measure VAS coverage, with the aim of providing guidance to countries on the collection of consistent data for planning, monitoring and evaluating VAS programmes integrated into routine health systems. We searched the PubMed®, Embase®, Scopus, Google Scholar and World Health Organization (WHO) Global Index Medicus databases for studies published from 1 January 2000 to 1 January 2021, reporting original data on VAS coverage and methodologies used for measurement. We screened 2371 original titles and abstracts, assessed twenty-seven full-text articles and ultimately included eighteen studies. All but two studies used a coverage cluster survey (CCS) design to measure VAS coverage, adapting the WHO Vaccination Coverage Cluster Surveys methodology, by modifying sample size and sampling parameters. Annual two-dose VAS coverage was reported from only four studies. Until electronic-based systems to collect and analyse VAS data are equipped to measure routine two-dose VAS coverage using administrative data, CCSs that comply with the 2018 WHO Vaccination Coverage Cluster Surveys Reference Manual represent the gold-standard method for effective VAS programme monitoring.


Introduction
Vitamin A supplementation (VAS) is a highly cost-effective public health intervention that reaches approximately 250 million children every year, protecting them from blindness and decreasing their risk of mortality from preventable causes (1) . In settings where vitamin A deficiency (VAD) is a public health problem, the World Health Organization (WHO) recommends two high-dose vitamin A supplements annually, spaced 4-6 months apart, for children aged 6-59 months (2,3) . UNICEF estimates that VAD affected about one-third of children aged 6-59 months in 2018, with the highest rates in sub-Saharan Africa (48 %) and South Asia (44 %) (4) .
Since 1998, VAS has been delivered mainly through campaigns, such as polio supplementary immunization activities (SIAs) (3) . Recently, the high cost of campaign delivery and the reduced frequency and geographical distribution of polio-SIAs due to effective eradication efforts, have encouraged countries to increasingly use routine health services for VAS delivery (4) . This change in delivery mechanism has coincided with a shift in programme monitoring tools, with many countries moving from paper-based to electronic-based administrative monitoring systems (4) .
The main indicator for VAS programme monitoring is VAS coverage, defined as the percentage of children aged 6-59 months of age receiving an age-appropriate vitamin A supplement in each of two annual semesters (5,6) .
In settings where VAS is distributed through campaigns, there is existing guidance on how to measure and validate VAS coverage after an event using survey methods, including the Post Event Coverage Survey (PECS) (7) , which employs the Expanded Programme on Immunization (EPI) cluster survey methodology (8) . However, this technique was revised by WHO in response to methodological concerns (9) . The main changes brought by the 2018 WHO Vaccination Coverage Cluster Surveys Reference Manual include the use of probability-based sampling methods at each stage; households (HHs) selected by a central group of planners rather than interviewers in the field; interview of every eligible child in the HH; and weighted analysis.
Other tools used to measure VAS coverage are represented by large-scale multi-topic HH surveys, such as the Demographic and Health Survey (10) or the UNICEF Multiple Indicator Cluster Survey (11) . However, such HH surveys have limitations in supporting VAS programme management needs, as they are not designed to measure annual two-dose VAS coverage; moreover, they are expensive and carried out too infrequently (i.e. every 10 years) to allow a real-time monitoring of VAS coverage aimed to identify and implement corrective actions (i.e. supplementary VAS activities in specific areas).
At present, there is no standardised methodology for measuring and validating the coverage of VAS delivered through routine health services. Strengthened methods are therefore required for accurate and timely measurement of VAS coverage. This is particularly important as countries integrate VAS into the routine health systems and shift to administrative, electronic-based monitoring tools, a process that brings certain limitations affecting the quality of administrative VAS coverage, which can impair effective VAS programme monitoring (5,6) .
To address this gap, we conducted a systematic review of the literature to identify and recommend methods to measure VAS coverage, with the aim of providing guidance to countries on the collection of consistent data for planning, monitoring and evaluating VAS programmes integrated into routine health systems.

Methods
The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (12) . The protocol for the review was not registered on the PROSPERO register of systematic reviews but is available on request.
We searched the PubMed®, Embase®, Scopus, Google Scholar and WHO Global Index Medicus databases for peer-reviewed studies reporting original data on VAS coverage among children under 5 years of age and methodologies used for measurement.
To focus on methods currently in use, we only included articles published from 1 January 2000 to 1 January 2021. Studies written in languages other than English, French, Portuguese or Spanish were excluded.
We used a combination of medical subject headings (MeSH) and text words, Boolean operators and synonyms in the thesaurus to create database-appropriate syntax (Table 1).
Bibliographic information was imported into a citation bibliographic management software for the storage and removal of duplicates. After duplicate citations were removed, titles and abstracts were independently screened for eligibility by authors. The reference lists of relevant articles were also checked to identify further eligible studies. In cases of disagreement, consensus was sought after reading the full-text article.
As all the studies included in the systematic review adopted a cross-sectional design, quality was assessed by two authors using the Newcastle-Ottawa Scale adapted for cross-sectional studies (13) . Disagreements in quality assessment were resolved through discussion.
Authors extracted the data using an electronic form. The summary of findings tables accompanied by a narrative synthesis was used to synthetise and present results.
The data collected included bibliographic information (authors, year and country of publication); study design; sample size and sampling procedures; data collection methods; data quality assurance methods; data analysis methods; ethical considerations; planning considerations (e.g. number of personnel, study length and month of study implementation) and outcome measured (routine/after event one-dose and/ or two-dose VAS coverage). The terminology used in the systematic review is provided in the glossary of terms (Supplementary material 1).
There was no relation between the sample dimension, the geographical level where studies were conducted and their design: CCs reported a sample size ranging between 210 units at the municipality level (17) to 5880 units at the national level (25) , the CLQAS (30) and LCS (27) sampled 855 and 10 454 units at the district level, respectively.
Within the selected clusters, all but one study (31) sampled HHs to find eligible children, with the majority (n 12) (15)(16)(17)(18)(19)(20)(21)(22)(23)(24)26,29) using   (2) Statistical test: (a) The statistical test used to analyse the data is clearly described and appropriate, and the measurement of the association is presented, including confidence intervals and the probability level (P value). * (b) The statistical test is not appropriate, not described or incomplete.
the WHO random walk method. Five articles (22,23,(27)(28)(29) reported that all eligible children within a selected HH were enrolled, while the other thirteen studies reported selecting only one eligible child by random selection. In all the reviewed studies, parents or caregivers were interviewed to collect information about VAS received. Overall, the number of selected clusters per strata ranged from four to eighty, with eight studies (15,16,18,20,21,(24)(25)(26) opting for thirty clusters. The number of sampled HHs per cluster ranged from seven to forty-five (mean 22⋅5; median 20).

Data collection and quality assurance
The data collection and quality assurance procedures used in the reviewed studies are presented in Table 5. Only two studies (28,30) used an electronic tool to collect information on the number of vitamin A supplements received by enrolled children, while the other studies used a standardised questionnaire, which was pre-tested and translated in the local language in thirteen studies (14,16,(19)(20)(21)(22)(23)(24)(25)27,(29)(30)(31) .

calculated weights
Among studies measuring two-dose VAS coverage (23,(27)(28)(29) , two (23,28) were conducted during the December-February period to estimate the proportion of children who had received two vitamin A supplements in the previous year. In the LCS (27) , children who received a first dose of vitamin A in the first semester of the year were followed up in the second semester to assess whether they had received a second dose.

Discussion
The systematic review revealed that across regions and time, CCS represented the principal method for measuring and validating VAS coverage, both after a vitamin A event distribution and via routine health contacts.
The majority of studies reviewed adapted the methodology of the WHO EPI cluster survey (8) , modifying sample size, sampling and data analysis procedures. In making such modifications, these studies partly implemented the new WHO recommendations contained in the 2018 Vaccination Coverage Cluster Surveys Reference Manual (9) .
In fact, over time, EPI surveys have increased in complexity, matching the evolution of the EPI since its inception in 1974 with the so-called '30 × 7 design' (7) .
Although the basic 30 × 7 EPI survey design has been a valuable programme management tool, the use of non-probability sampling and lack of standardised, well-documented quality control procedures may reduce confidence in the results (32) . To address these limitations, the WHO Vaccination CCS Reference Manual was updated in 2005 (8) and again in 2018 (9) and is still considered the standard guidance for conducting a CCS.
To calculate sample size, most studies used anticipated VAS coverage, a desired precision of ±5-10 %, a confidence level of 5 %, an average DEFF of 2⋅5 and a predefined number of clusters and respondents per cluster. In addition to these WHO-recommended parameters (8,9) , sample size should also be increased for an estimated non-response rate. However, this last parameter was only considered by four studies included in the systematic review (26,28,29,31) .
Rather than pre-establishing a certain number of clusters and HHs, the 2018 WHO Manual (9) recommends that at least thirty clusters be selected per stratum of a minimum of ten respondents each. This was done in a majority of reviewed studies.
When selected HHs are found empty or selected clusters become inaccessible (e.g. due to conflict, wildfires and flooding), a plan for cluster replacement and at least two HH revisits should be put in place. Yet overall, only one study (16) reported having a replace/revisit plan.
Within the selected HHs, most studies enrolled only one eligible child aged 6-59 months. To optimise resources and guarantee that the probability of selection for an individual is equal to the probability of selection for his or her HH, WHO (9) recommends including every eligible child in every selected HH, as was done in five reviewed articles (22,23,(27)(28)(29) . Moreover, because the target population of VAS programmes is children aged 6-59 months, it is recommended that the whole age group be included to both optimise resources and measure the percentage of children 6-59 months of age who received an age-appropriate vitamin A supplement in each semester (5) .
Digital data collection is beneficial because it eliminates the problem of illegible handwriting and can be directly linked, via data transmission, to a central location for storage and analysis. It also makes it easier to check the entries for mistakes and correct them before the data are transmitted (9) . For these reasons, electronic tools, as used in only two reviewed studies (28,30) , are preferred over paper-based questionnaires, where feasible. In certain contexts, in fact, digital data collection may not be feasible due to lack of electricity, internet connection and capacity in addition to data security issues.
In all the articles included in the systematic review, the CHC and parent recall were the main sources of information on the number of vitamin A supplements received by the child. If no home-based record of VAS was available, the next level of evidence was a verbal history of VAS by parents. In immunisation surveys, the validity of parental recall can be unreliable because of the complexity of immunisation schedules (34) . However, remembering the number of vitamin A capsules received by the child is more straightforward and can be facilitated by showing a sample capsule to the parent, as was done in eleven studies (15)(16)(17)(19)(20)(21)(22)24,27,30,31) . For these reasons, it is acceptable to collect VAS information by parental recall when CHC is not available.
According to WHO (8,9) , to guarantee the quality of collected data, it is necessary to provide interviewers with training and supervision. They should be organised in teams of two completing one cluster of a maximum of 30 HH/day. One supervisor should also be assigned to every two teams to monitor the quality of their work. WHO also recommends that interviewers be familiar with the clusters they are assigned and fluent in the local language. In line with these recommendations, most studies (n 14) (14)(15)(16)(17)(18)(19)(20)(21)(23)(24)(25)27,30,31) provided training and supervision to locally recruited interviewers, who surveyed an average of 29 HH/d.
Once data are collected, recommended quality actions (8,9) include double data checking, entry and cleaning. These actions were performed in the majority of reviewed studies (n 12) (14)(15)(16)(19)(20)(21)(22)24,25,27,29,31) .  (14) 72 7 NR No Yes Masanja (2006), Tanzania (15) NR NR NR Yes Yes , Guinea (16) 30 10 NR Yes Yes Ayoya (2007), Mali (17) NR NR NR NR NR Sachdeva (2009), India (18) NR NR NR NR NR Gebremedhin (2009), Ethiopia (19) 18 6 4 weeks Yes Yes Hodges (2013), Sierra Leone (20) 10 NR 1 week Yes Yes Nyhus (2013), Tanzania (21) 36 3 6 weeks Yes Yes Olusegun (2013), Zambia (22) NR Subsequently, under the multistage cluster sampling approach with PPS, data analysis must be weighted because sampling probabilities differ for different respondents. To derive a correct coverage estimate, sample weights need to be applied to each cluster to account for differences in population size and for non-response (8,9) . Overall, only four studies (16,20,24,30) calculated weights for analysis to account for differences in population size within the sampled clusters, and another three studies (15,21,22) adjusted VAS coverage estimates for non-response rate. On the other hand, the majority of studies (n 11) (15,17,19,(21)(22)(23)25,(28)(29)(30)(31) reported a 95 % CI of estimated VAS coverage, as recommended by WHO (8,9) , including the CLQAS (30) which design is not meant to measure the point of coverage estimates, but to identify whether an area (lot) has achieved a minimum level of coverage (32) . Although the main outcome of CLQAS is a binary classification of areas (lots) in accepted/rejected, without providing a point of coverage estimate, lot data can be aggregated according to a stratified weighted design to estimate coverage in the area. The main advantage of CLQAS is the small sample size required to classify lots with regard to coverage levels, but despite such advantage, the only reviewed CLQAS (30) selected 855 units, no more no less of the CCSs. Moreover, WHO discourages the use of this design to measure the point of coverage estimates, as it is not specifically conceived for this goal and uses a priori defined decision rules to classify coverage which contrast with the objective of coverage estimation (9) .
Because implementing a coverage survey is resource-intensive, efforts should be made to improve efficiency by measuring annual two-dose VAS coverage, presenting data by the age group (i.e. 6-11 and 12-59 months) and by the collection method. To do this, the survey should be performed during the December-February period, as done by two of the reviewed studies (23,28) .
Following-up with children to assess if they receive their second dose, as done in the LCS (29) , may introduce selection biases (e.g. by not considering population movement, including newly arrived children and children who age in or out of the eligible age range between the first and second dose). WHO underlines that an important sampling challenge is ensuring that no populations are missed, especially those that are difficult to reach (32) .
It is a limitation of this systematic review that six studies (14,16,18,20,24,27) did not provide information on sample size calculation procedures. Moreover, only five articles (19,21,25,29,30) were classified as being of high quality.
While a greater number of studies would have provided a wider evidence base upon which to draw conclusions, the Table 7. Data analysis procedures and outcome measured by studies included in the systematic review of methodologies to measure the coverage of Vitamin A Supplementation (n 18) lack of peer-reviewed publications itself demonstrates the need to strengthen methods for measuring the administrative coverage of VAS delivered through routine health services.

Conclusion and recommendations
In the current transition process towards routine health system contacts as the main VAS delivery platform and administrative electronic-based data collection systems, improving routine data quality is the best way to ensure stronger service delivery and monitoring of VAS programmes, as these data provide the most sustainable method for coverage estimation.
However, most VAS priority countries are in the early stages of this process and do not yet have the ability and full capacity to measure routine two-dose VAS coverage (4) .
Based on the results of this systematic review, these countries can adopt multistage CCS to measure VAS coverage, using the recommendations included in Table 8.
Consistent with WHO guidance (9) , the methodological recommendations provided will enable and support countries to collect reliable data for VAS coverage measurement (either after a vitamin A event distribution or via routine health contacts) in order to plan, monitor and evaluate VAS programmes in the current transition period and beyond.

Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/jns.2021.65. • Pre-test and translate data collection tools in the local language • Provide training to interviewers and ensure field supervision • Recruit interviewers that are familiar with the clusters they are assigned to and fluent in the local language • Ensure double data checking, entry and cleaning • Include a plan for households and clusters revisit and replacement

Planning considerations
• Divide interviewers into teams of two completing one cluster of a maximum of thirty households per day. Consider gender balance to deal with local customs, if needed • Ensure a maximum of one supervisor for every two teams • Consider recruiting an expert statistician • Consider an average of 22⋅5 d for fieldwork • Obtain ethical clearance • Request verbal informed consent from caregivers

Data analysis
• Calculate weighted VAS coverage accounting for differences in population size within the sampled clusters and for non-response rate • Report 95 % CI of estimated VAS coverage • Report one-dose and two-dose VAS coverage by the target age group (6-11 and 12-59 months) • Report VAS coverage by the collection method (card, recall and card plus recall)