1.1. Healthcare reforms in Rwanda

Rwanda has initiated comprehensive health reforms since 2000 to reduce maternal and child mortality and rebuild the healthcare system that was destroyed during the 1994 genocide (Bucagu et al., Reference Bucagu, Kagubare, Basinga, Ngabo, Timmons and Lee2012). After the success of pilot programs, in 2006, the government launched and expanded a couple of health policies including Facility-Based Childbirth Policy (FBCP), which provides a full package of ANC and FBD services free of charge regardless of insurance possession, CBHI, a universal health insurance scheme that works at the community level, and Performance-Based Financing Contracts (PBFC), which provide funds to health facilities based on their performance. The first two policies are intended to improve access to health services and the last is to enhance the quality of care.Footnote ² These attempts seem to be very successful; the FBD rate had increased from 37 percent in 2005 to 95 percent in 2010 thanks to the reforms, which is a notable increase compared to other sub-Saharan African countries with similar health policies (Doctor et al., Reference Doctor, Nkhana-Salimu and Abdulsalam-Anibilowo2018). At the same time, the Ministry of Health increased the number of trained staff (nurses, midwives, and doctors) and health centers and posts, the primary level facilities (Bucagu et al., Reference Bucagu, Kagubare, Basinga, Ngabo, Timmons and Lee2012).

Table 1. Rwanda health policy events, 1999–2010

Source: Bucagu et al. (Reference Bucagu, Kagubare, Basinga, Ngabo, Timmons and Lee2012).

While FBCP is the most relevant policy to ANC and FBD use, the other two policies also played a crucial role. Rwandan CBHI (also known as Mutuelles de Santé) increased both insurance coverage and healthcare use including maternal care (Smith Sulzbach, Reference Smith and Sulzbach2008; Wang et al., Reference Wang, Temsah and Mallick2016; Collins et al., Reference Collins, Saya and Kunda2016; Lu et al., Reference Lu, Chin, Lewandowski, Basinga, Hirschhorn, Hill, Murray and Binagwaho2012; Saksena et al., Reference Saksena, Antunes, Xu, Musango and Carrin2011), like other health insurance or user fee removal programs in low-income countries (Renard, Reference Renard2022; McKinnon et al., Reference McKinnon, Harper, Kaufman and Bergevin2015; Meessen et al., Reference Meessen, Hercot, Noirhomme, Ridde, Tibouti, Tashobya and Gilson2011). Although CBHI is operated at the community level, the main parameters such as benefits package, enrollment fees, subsidization mechanisms, organizational structure, management systems, etc. are standardized by law in 2008 (Lu et al., Reference Lu, Chin, Lewandowski, Basinga, Hirschhorn, Hill, Murray and Binagwaho2012). Total health expenditure per capita in Rwanda has increased since 2000 because of a variety of health initiatives, however, out-of-pocket and catastrophic health expenditures have decreased especially for those with insurance (Bucagu et al., Reference Bucagu, Kagubare, Basinga, Ngabo, Timmons and Lee2012). Wang et al. (Reference Wang, Temsah and Mallick2016) show that the increase in maternal health service was concentrated on FBD, and had a limited impact on ANC use. More importantly, although CBHI was supposed to favor disadvantaged households, the coverage has been highly skewed toward the rich (Wang et al., Reference Wang, Temsah and Mallick2016). The research on estimating the causal effect of Rwandan CBHI on mortality rates and other health outcomes such as illness or stunting is scant and the results are mixed; Pierce et al. (Reference Pierce, Heaton and Hoffmann2014) find positive effects but Binagwaho et al. (Reference Binagwaho, Hartwig, Ingeri and Makaka2012) show limited effects on health outcomes.Footnote ³

Likewise, the goal of PBF is to improve both access to and quality of healthcare. Low-quality health care has been recognized as an obstacle to improving people's health in developing countries (Das et al., Reference Das, Hammer and Leonard2008), and maternal health service is no exception (Godlonton Okeke, Reference Godlonton and Okeke2016). Under the PBF scheme, payments to health facilities are linked to the services delivery including ANC, FBD, postnatal care (PNC), family planning, and prevention of mother-to-child transmission of HIV (Bucagu et al., Reference Bucagu, Kagubare, Basinga, Ngabo, Timmons and Lee2012). Based on the success of the pilot program launched in 2001, the government decided to scale up it as a national program in 2006, with a staggered roll-out until 2008 (Rusa Fritsche, Reference Rusa and Fritsche2007). Chari Okeke (Reference Chari and Okeke2014); Basinga et al. (Reference Basinga, Gertler, Binagwaho, Soucat, Sturdy and Vermeersch2011) find that PBF increased FBD and the number of ANC visits, but it did not lead to a reduction in neonatal deaths.

In fact, FBCP, PBF, and CBHI interact with one another. FBCP and CBHI directly increase maternal service use, and so does PBF because maternal health service is included as a targeted intervention of PBF. Health insurance may increase the use of general health services, improving both maternal and child health (Ahmed et al., Reference Ahmed, Sarker, Sultana, Chakrovorty, Ahmed, Dorin, Mirelman, Islam, Rahman, Niessen, Rehnberg and Khan2018). In addition, if PBF had successfully improved the quality of health facilities and healthcare, then it may have reinforced the impact of maternal health services. Because of the similar timing of initiation and interactive characteristics, this paper estimates the impact of the holistic health reforms on maternal health service use and childhood mortality rates, rather than focusing on a specific policy.

1.2. Healthcare system and FBD in Rwanda

The healthcare system in Rwanda ranges from community-level care to national hospitals. At the most basic level, community workers at the village level visit households and identify each household member's healthcare needs. Health posts and centers are the primary care unit, performing promotional, preventive, and curative activities. Health posts are a relatively new type of health facility, which was expanded rapidly from 9 in 2008 to 1184 in 2021 (Rwanda Ministry of Health, 2022, 2017). Health posts provide similar, albeit more limited services than health centers, reaching out to the most remote portions of the country. To improve coverage of primary care, it is planned to have at least one health center per sector and health post per cell.Footnote ⁴Footnote ⁵ More complicated illness that cannot be treated in primary care units is referred to higher-level facilities, such as district hospitals (secondary) and provincial and national referral hospitals (tertiary) (Bucagu et al., Reference Bucagu, Kagubare, Basinga, Ngabo, Timmons and Lee2012; World Health Organization, 2017). In 2019, there were 12 referral and provincial hospitals, 36 district hospitals, 509 health centers, and 885 health posts (1,184 in 2021) (Rwanda Ministry of Health, 2019a, 2022).

Under this system, health centers handle most primary care including ANC without complications, normal child deliveries (vaginal delivery), and simple surgical procedures. District hospitals treat patients referred from health centers and posts, performing more complicated curative activities, such as medical and surgical activities, and importantly, difficult and cesarean deliveries (Lu et al., Reference Lu, Chin, Lewandowski, Basinga, Hirschhorn, Hill, Murray and Binagwaho2012; World Health Organization, 2017).Footnote ⁶ Most of the ANC and normal FBD service is provided by trained midwives and nurses in health centers (Lundeen et al., Reference Lundeen, Musange, Azman, Nzeyimana, Murindahabi, Butrick and Walker2019; Rwanda Ministry of Health, 2022).Footnote ⁷ In case of complication, the health centers transfer the woman to the nearest district hospital via an ambulance or transport arranged by the woman in case of no ambulance (Niyitegeka et al., Reference Niyitegeka, Nshimirimana, Silverstein, Odhiambo, Lin, Nkurunziza, Riviello, Rulisa, Banguti, Magge, Macharia, Habimana and Hedt-Gauthier2017). Cesarean section is typically performed at a district hospital by general practitioners (Niyigena et al., Reference Niyigena, Alayande, Bikorimana, Miranda, Rudolfson, Ndagijimana, Kateera, Riviello and Hedt-Gauthier2022; Rwanda Ministry of Health, 2019b), and thus the distance and transportation to district hospitals are important factors in seeking quality care in case of emergency.

2.1. Data sources

The main dataset used is the Rwandan Demographic and Health Survey (RDHS). I use the birth history data of 2005, 2008, 2010, 2014/15, and 2019/20 waves of RDHS and stack them according to birth year to create a continuous set of births from 2000 to 2018. The birth history data includes all births that the respondent (woman of reproductive age, 15–49 years old) gave retrospectively regardless of survival and age. For the children who have died, the respondents provide information on the age of death (in months when they died before 60 months). The RDHS also collects information on several birth and health statuses such as place of birth (facility or home), ANC, and PNC(postnatal care) visits,Footnote ⁸ vaccination history, and having fever and diarrhea for children under five years old. The data also provides useful information on households and mothers’ socioeconomic status (SES), such as household wealth and mothers’ education.Footnote ⁹ I use three measures of mortality rates: Newborn Mortality (death in 7 days, NMR7), Neonatal Mortality (death in 30 days, NMR), and Infant Mortality (death in one year, IMR). Following the traditional definition, the mortality rates are scaled per 1000 live births. When defining each mortality rate, I only include births that passed the one-month threshold for NMR7 and NMR and one year for IMR. Since the mortality rates had surged during the Rwandan genocide and thus led to a very different trend in the late 1990s, I only include births from 2000 although the RDHS asks for the full history of the birth career even happened more than five years ago.Footnote ¹⁰

One complication is the inconsistency in administrative districts over time. The Rwandan Government reformed its administrative areas from 12 provinces and 106 districts to 5 provinces and 30 districts in 2006. Because of this change, it is difficult to compare the spatial change of FBD and other health outcomes across time. Using the geocodes of the primary sampling units (PSU or clusters), I match 2008-2019 PSUs to the 106 old districts, for more variation, and construct a dataset with consistent geographic areas. Because the RDHS randomly displaces the GPS coordinates for privacy issues, some measurement error still exists.Footnote ¹¹ Random measurement error of explanatory variables typically biases the point estimate toward zero, which is a less of problem. Nevertheless, for robustness, I drop the 2005 RDHS, which has a different administrative boundary and use new district boundaries, which does not change my result much.Footnote ¹²

For additional information on the district characteristics, I use the Integrated Household Living Condition Survey (Enquête Intégrale sur les Conditions de Vie des ménages, EICV) of Rwanda, 2005 and 2014 waves. This survey provides information on changes in people's well-being, such as economic conditions, education, health and housing conditions, household consumption, etc. I use it as a supplementary data set to calculate the average insurance coverage rate, travel time to health facilities, and total population by district and year that RDHS does not provide. When calculating the average insurance coverage and travel time to hospitals by district, I restrict the EICV sample to women 15–49 years old as the RDHS sample consists of reproductive-aged women.

2.2. Descriptive statistics

Table 2 presents the summary statistics of the resulting data. I present the mean and standard deviation for all, treatment, and control groups, both for pre- and post-period relative to the health reform. Panels A and B specifically address the birth characteristics and the maternal/household characteristics, respectively.

Table 2. Summary statistics

Note: This table shows the mean and standard deviation (in brackets) of the entire data used. The unit of observation is each birth. In Panel A, variables related to children or births are presented. Basic demographic variables and survival status are available for all births but FBD and ANC information is for births within five years. Panel B presents the summary statistics of the variables of the mothers and households. Because the unit of observation is birth, some mothers are duplicated if she has more than one child. All of the statistics are weighted with the sample weights.

Three things are notable in Panel A. First, the FBD rate experienced a substantial increase, rising from 32 percent to 82 percent. This increase was particularly pronounced in the treated districts, jumping from 20 to 78 percent, in contrast to the control group's shift from 43 to 95 percent. On the other hand, the probability of utilizing ANC services did not witness a significant change, as nearly every pregnant woman had been engaging in ANC visits even in treatment districts before the reform. Post-reform, however, women exhibited earlier and more frequent ANC visits. While control districts maintained better statistics for ANC visits, the disparities between control and treated districts diminished in the post-period. Mortality rates exhibited a marked decline across all three measures. Specifically, NMR7 and NMR decreased by 9 and 12 deaths per 1000 births, respectively, and IMR nearly halved, plummeting from 78 to 40 deaths. The mortality rates were higher in treated districts during the pre-period, they converged in the post-period, indicating a relatively more substantial decline in the treated districts.

Panel B reveals a marginal improvement in household SES throughout the study period, with the exception of mothers’ education. The proportion of mothers with no formal education decreased from 26 to 17 percent, while years of education increased from 3.86 to 4.35 years. Control districts demonstrated superior education and urbanity levels in both pre-and post-periods. However, the differences between control and treated districts remained relatively consistent, suggesting parallel trends in SES. Other characteristics, such as age at birth, marital status, access to piped water, and possession of automobiles, did not undergo significant changes.

3.1. Main specification

To identify the causal effect of the health reforms on the use of maternal health services and mortality rates, I exploit a variation in baseline FBD rates across districts in a DiD framework. To be specific, I use the district-level home delivery rate (1-FBD) in 2000–2005 as the proxy for the intensity of exposure to health reforms. In other words, my identification relies on the hypothesis that districts with low baseline FBD rates had a greater scope to increase the use of maternal health services due to the reforms, and thus are more affected by the policy. A similar identification strategy is found in previous works like Bleakley (Reference Bleakley2007); Godlonton Okeke (Reference Godlonton and Okeke2016); Osili Long (Reference Osili and Long2008).

I take the districts with a low FBD rate (below median) as the treatment districts and call them low-FBD districts.Footnote ¹³ Figure 1 displays how the treatment and control districts are distributed in Rwanda. Kigali area (capital, center of the country) and other large cities are mostly the control districts; however, the treatment districts are well-distributed within the country.

Figure 1. Treatment/Control Assignment. Note: This figure shows the map of Rwanda with its administrative districts in 2005 (old districts). The treatment districts or the low FBD districts are depicted in the darker shade. The treatment districts are districts whose baseline average FBD rates are below the 50th percentile.

Specifically, the main regression equation is as follows:

(1) $$ \eqalign{ y_{idpmt} = \beta Low\ FBD_{d} \times {\opf 1} ( t\geq 2006) + \gamma X_{idpmt} + \tau_{d} + \eta_{m} + \theta_{t} + \alpha_{\,p}\times t + \varepsilon_{ipdmt},\; }$$

in which y _idpmt is an outcome for child i born in district d in province p in month m of year t. The outcome variables include an indicator for FBD use, ANC visits, and mortality rates. Low FBD _d is one if the district has a lower baseline FBD rate than the median (or higher home delivery rate) and ${\opf 1} ( t \geq 2006)$ is an indicator function whether t is greater or equal to 2006. X _idpmt is a vector of birth and household characteristics, including survey year fixed effect to correct wave-specific shocks. τ _d and θ _t are the district fixed effect and the birth year fixed effect, respectively. η _m is the birth month fixed effect, controlling any possible seasonality. I include province-level time trends to absorb any long-term linear trend in the outcome variables that may vary across provinces. 2005 RDHS is representative at the old district level while the rest of the waves are at the new district level. Thus, I use the corresponding districts to cluster the standard errors.Footnote ¹⁴ The coefficient of the interest is β, the reduced-form impact of the policy change on the outcome variables, capturing the difference in change in the outcome variables before and after the reforms between the districts of high and low FBD. To test the pre-trends and observe the dynamic effect of the policy, I modify Equation (1) to include lags and leads from 2006 and interact with the low FBD indicator in an event study framework. In some specifications, I consider the mother-fixed effect to control unobservable characteristics of mothers.Footnote ¹⁵ All regressions are weighted with the sample weights in RDHS.

3.2. Validity of identification strategy

I conduct a placebo test to reject the hypothesis that the effect on mortality rates is confounded by a concurrent change in other factors that affect childhood deaths. For example, mortality rates may have been reduced more in treatment areas, not because of the policy, but because mothers in the treatment areas got relatively healthier, more educated, or wealthier. To address this concern, I examine whether changes in observable characteristics are associated with the treatment status. While this test is not complete for exclusion restriction, the absence of significant association with observable characteristics suggests similar results for unobservable characteristics (Altonji et al., Reference Altonji, Elder and Taber2005).

Another important assumption is parallel trends of outcome variables between treatment and control districts in the absence of the policy intervention. A similar pre-trend between treatment and control districts suggests that the post-trend would have progressed similarly. In other words, the trend of treatment and control districts should evolve in parallel in the absence of the treatment. This assumption is fundamentally untestable because it requires post-period outcomes without policy intervention. In Appendix Figure A.1, I show the raw means of the use of FBD and ANC, and mortality rates by birth or conception year and by treatment status. In these figures, the pre-trends of outcome variables are similar in treatment and control districts. The control districts were better off in terms of maternal health service use and mortality rates in the pre-period, however, the differences converge after the policy implementation except for at least four ANCs. The event study specification also suggests strong graphical evidence for no visible violation of parallel trends in Figures 3 and 4. Nevertheless, I conduct an additional analysis utilizing an alternative estimator robust to violation of parallel trends suggested by Rambachan Roth (Reference Rambachan and Roth2023) and show the main specification is robust in Appendix Figure A.3.

Figure 2 graphically presents the result of the balance test. In this figure, I plot the coefficients of the event study estimator with 2005 as the baseline year for each outcome variable. If there is a pre- or post-trend, it indicates a differential change between the two groups. Most of the coefficients are not distinguishable from zero. Birth characteristics such as sex, birth order, twin indicator, and whether the mother is under 20 (young mother) are all uncorrelated. The treatment status is not associated with the socioeconomic status of the household such as the mother's marital status, education, and wealth index. However, the point estimate for living in an urban area shows an upward trend after 2006, although statistically insignificant. I suspect that urbanization was faster in treatment districts because they consist more of rural areas. Overall, results in Figure 2 indicate that treatment status balances almost all observable characteristics, and the research design is unlikely to be biased by changes in unobservable variables.

Figure 2. Balance Test. Note: This figure graphically shows the result of the balance test. I separately run regressions using Equation (1) with the dependent variables described in the title of each panel. Event study coefficients are presented with a 95% confidence interval. Controls are not included. Standard errors are clustered at the proper district level. (Old districts for the 2005 wave and new districts for the rest of the years.).

5.1. Travel time to health center

Most ANC and FBD occur in primary-level facilities, primarily health centers. In the event of pregnancy complications, patients are expected to be referred to a district hospital for advanced care.Footnote ¹⁷ However, this referral process poses challenges when the hospital is located too far away or transportation is inadequate (Niyitegeka et al., Reference Niyitegeka, Nshimirimana, Silverstein, Odhiambo, Lin, Nkurunziza, Riviello, Rulisa, Banguti, Magge, Macharia, Habimana and Hedt-Gauthier2017; Aggarwal, Reference Aggarwal2021). Proximity to a hospital plays a crucial role in the ease of accessing advanced care in the event of pregnancy complications, potentially reducing the likelihood of early mortality. This possibility is explored in Figure 5.

Figure 5. Heterogeneity by Travel Time to District Hospital. Note: This figure compares the event study coefficients in (new) districts where travel time to the district hospital is short (<180 minutes) and long (>180 minutes). The specification is the same as column 4 of Table 3. See the notes of Table 3 for the list of control variables. Standard errors are clustered at the proper district level. See Appendix Figure A.5 for heterogeneity by the linear distance to hospitals.

I use the district-level average travel time to the closest district hospital provided by the 2005 EICV to divide the sample into two groups.Footnote ¹⁸ While using average travel time may seem less precise than actual distance measurements, it incorporates factors such as transportation means used by the average household and road availability, making it a more relevant metric than linear distance.Footnote ^19, Footnote ²⁰

Because mothers mostly visit health centers for FBD and ANC, the impact on FBD and ANC use is similar for both groups of districts in Panels (a)–(d). However, for mortality rates, some differences are observed. NMR7 and NMR decreased right after 2006 in districts where the district hospitals are closer. The decline is clearer for IMR; there was an immediate drop in IMR in closer districts, and the decline lasted for years. Although the differences in Panels (e)–(g) are not statistically different, the figures clearly show that there was a clear decline in mortality rates in districts where hospitals are close, suggesting the importance of proximity to health facilities.

5.2. Quality of health facilities

The quality of the health facility where mothers deliver a child plays an important role in reducing neonatal mortality. For example, Godlonton Okeke (Reference Godlonton and Okeke2016) and Renard (Reference Renard2022) find that FBD did not have a significant effect on overall neonatal mortality, but only in areas where there is a qualified health facility nearby, in Malawi and Zambia, respectively. Unfortunately, the data on health facility quality in Rwanda is limited, making it difficult to conduct a similar analysis.

Alternatively, I exploit a policy relevant to facility quality to estimate the heterogeneity in effect. One important change in health reforms is the expansion of the PBF scheme. This scheme started as a pilot program in three health districts in 2000 and then expanded nationwide in 2006-2008 with a staggered roll-out.Footnote ²¹ The policy intends to improve the quality of healthcare by rewarding health facilities based on their performance. Thus, if PBF was successful, health districts that implemented PBF earlier may have better quality health facilities. The impact of PBF on FBD and ANC in Rwanda is shown in Basinga et al. (Reference Basinga, Gertler, Binagwaho, Soucat, Sturdy and Vermeersch2011); Chari Okeke (Reference Chari and Okeke2014), but the impact on mortality rates is unclear in the literature. In fact, the quality of health facilities is still low in Rwanda compared to high-income countries, with improper infrastructure and ill-trained staff members (Rwanda Ministry of Health, 2022; Rurangirwa et al., Reference Rurangirwa, Mogren, Ntaganira, Govender and Krantz2018). Thus, the improvement in healthcare quality due to PBF may not be sufficient enough to reduce mortality rates.

I divide the districts by PBF implementation year following (Chari Okeke, Reference Chari and Okeke2014). Earlier districts are districts where PBF was implemented before 2008. Figure 6 shows that later districts relatively perform slightly better than earlier districts in terms of maternal health services (a)-(d)), although the difference is not statistically significant. On the other hand, a marginal reduction in mortality rates is observed in PBF early districts, especially after 2011. However, the estimates are very imprecisely estimated and the differences are not large enough to conclude that PBF has a positive effect on mortality rates. This result is consistent with Chari Okeke (Reference Chari and Okeke2014). The DiD estimation in Appendix Table A.5 also presents a similar result. The reduction in mortality rates is larger in earlier districts, but the difference is not statistically different. Nevertheless, because the impact of PBF on care quality in Rwanda is not well-known, we cannot conclude that healthcare quality does not play an important role in reducing deaths in Rwanda.

Figure 6. Heterogeneity by PBF, Note: This figure compares the event study coefficients in (new) districts where PBF was implemented earlier and later than 2008. The specification is the same as column 4 of Table 3. See the notes of Table 3 for the list of control variables. Standard errors are clustered at the proper district level.