Measuring food security is challenging but important, as hundreds of millions of people around the world still lack access to sufficient food( Reference Barrett 1 ). The World Food Summit of 1996 defined food security as ‘a situation that exists when all people, at all times, have physical, social and economic access to sufficient, safe and nutritious food that meets their dietary needs and food preferences for an active and healthy life’( 2 ). Defining and measuring indicators for these concepts is difficult, since food security is multidimensional and has both objective and subjective dimensions. Measuring food security has been the object of ongoing debate in both the academic world and within the development arena.
Objective indicators of food security at household level include the well-known energy deprivation index, monetary indicators that can be used to construct a food poverty line, anthropometric measures and, more recently, dietary diversity scores that consider both micro- and macronutrients( Reference Headey and Ecker 3 – Reference Maxwell, Coates and Vaitla 8 ). Supporters of subjective approaches argue that objective indicators do not take into account important intangible aspects of food insecurity such as constant worries about the possibility of food deprivation or limited dietary variations. Several indicators have been developed that include these aspects( Reference Pérez-Escamilla 9 , Reference Marques, Reichenheim and de Moraes 10 ). One such indicator is the Household Food Insecurity Access Scale (HFIAS). This nine-item scale captures people’s perceptions about food insecurity using a range of indicators such as anxiety about food supply, limited dietary variety and quality, and insufficient food availability( Reference Coates, Swindale and Bilinsky 11 ). Such an index is easy to use and can be implemented at low cost, thus making it ideally suited for use by governments or non-governmental organizations to monitor and evaluate programme impacts.Footnote †
However, the simplicity of the HFIAS raises questions about its reliability. One can ask whether, in a cross-sectional survey, it is able to effectively discriminate between food-secure and food-insecure households. Previous studies( Reference Headey and Ecker 3 , Reference Knueppel, Demment and Kaiser 12 – Reference Naja, Hwalla and Fossian 18 ) showed that the HFIAS is correlated with objective food intake-based measures of food security and is thus cross-sectionally valid. In a longitudinal survey that follows the same households over time, the main concern is the inter-temporal validity of the index. This means that households experiencing an objective decline in food access over time should report feeling more food insecure than before. To date, few studies have investigated the inter-temporal validity of subjective food security indicators. A notable exception is a study in urban Burkina Faso( Reference Martin-Prevel, Becquey and Tapsoba 19 ) which concluded that the HFIAS is able to capture the impact of high food prices on households’ food security. However, a similar study in urban Ethiopia showed that female volunteer AIDS caregivers reported feeling more food secure during three subsequent survey rounds in 2008, despite higher food prices and the loss of food aid( Reference Maes, Hadley and Tesfaye 20 ). This latter finding calls into question the inter-temporal validity of the HFIAS.
The present study contributes to this literature by evaluating cross-sectional and inter-temporal validity of the HFIAS over a time span of 5 years for a representative sample of rural farmers in the north of Burundi.
Methods
Sampling and study design
Household surveys were conducted in Ngozi, a rural province in the north of Burundi, from mid-June until the end of July in 2007 and then in the same period in 2012. The surveys were carried out by an experienced team from the University of Burundi in collaboration with researchers from the Universities of Antwerp and Ghent (Belgium). Four of the ten enumerators and the team leader participated in both survey rounds. The interview period coincides with the dry season, when agricultural production is low (15 % of the annual total)( 21 – 23 ). There were also practical reasons for choosing this period: most villages in this region are accessible only during the dry season and farmers have lighter workloads in this period, allowing them time to spend on interviews. The questionnaire, which was drafted in French, was administered by a trained interviewer in approximately one hour in the local language, Kirundi. The enumerators were bilingual and a test phase sought to ensure that all enumerators translated and interpreted the questions similarly.
Ngozi is administratively divided into nine ‘communes’, which are further divided into villages, known as ‘collines’( 24 ). Within each of the nine administrative units, the surveys randomly selected ten villages, and four households from within each village, to participate in the study. Hence, a total of 360 households were interviewed.
In 2012, 340 out of the 360 households that had participated in the first round in 2007 were re-interviewed. However, twenty-six observations had to be disregarded due to missing variables or large outliers. Thus, the final data set contains 314 valid observations.
Participants were informed about the study and provided their verbal consent prior to being interviewed. No sensitive personal data were sought. Because of the approach used and the questionnaire content, no formal ethical approval was sought prior to performing the study.
The Household Food Insecurity Access Scale
The HFIAS was developed by a team of researchers at Tufts University as part of a Food and Nutrition Technical Assistance project funded by the US Agency for International Development( Reference Coates, Swindale and Bilinsky 11 , Reference Deitchler, Ballard and Swindale 16 ). The method assumes that food insecurity causes predictable reactions that are the same across countries and can be captured and quantified through a survey. Based on the eighteen questions of the US Household Food Security Survey Module (HFSSM), but adapted to the specific context of developing countries, the scale contains nine questions (see overview in Table 2 below). Together these questions cover a broad spectrum of experiences related to food security. The first asks about anxiety over food availability, the next three are related to food quality and the last five to the quantity of food intake. Each time a question elicits a ‘yes’ response, it is followed by a frequency-of-occurrence question with three options: ‘rarely’, ‘sometimes’ and ‘often’. Responses of ‘no’ to the initial question are coded as 0, whereas the answers ‘rarely’, ‘sometimes’ and ‘often’ are coded as 1, 2 and 3, respectively. Subsequently, the scores on the nine questions are summed to calculate the index. This results in a continuous food insecurity indicator that ranges from 0 (food secure) to 27 (severely food insecure)( Reference Coates, Swindale and Bilinsky 11 ).
Food production and consumption
The survey collected data on the total annual harvest of the main crops in Burundi (bananas, beans, cassava, coffee, maize, peanuts, peas, potatoes, rice, sorghum, sweet potatoes, soya and taroFootnote *) based on a one-year recall by the household head. The twelve selected crops account for more than 90 % of the energy intake of a household in Rwanda, a neighbouring country with a similar dietary pattern( 25 ).
Total annual food production was used as a proxy for food consumption and two different indicators for food consumption were constructed.Footnote † The first indicator aggregated total annual production in terms of its energy content. The second indicator also expressed the total annual harvest in these terms, but excluded bananas. There were two reasons for the construction of this second indicator. First, banana is a semi-cash crop that is both consumed in the household and sold on the market as the main ingredient for beer. Therefore, an increase in banana production does not necessarily directly entail an improvement in food security of the household because the additional revenues might be used to cover expenses not related to food consumption. Furthermore, there was a large drop in banana crop production between the survey years due to a bacterial disease. For both reasons, it was necessary to investigate whether bananas had a different impact on food security compared with other crops. Finally, both proxies for food consumption were expressed per capita and per day to make them more tangible.
Statistical analyses
To test the cross-sectional and inter-temporal validity of the HFIAS, we estimated the correlation between the HFIAS and the household and farm characteristics that are expected to contribute to food security. The models assumed that a household, i, rationally evaluates its own food insecurity status based on its underlying household-specific characteristics in each period, t. However, not all household characteristics are directly observable and this requires making a distinction between observable household characteristics (X i ), such as food consumption, and unobservable household characteristics (u i ), such as household-specific strategies to cope with stress in times of food shortages. Hence, the following model was estimated:
$$HFIAS_{{it}} =\alpha {\plus}\beta X_{{it}} {\plus}u_{i} {\plus}\gamma _{t} {\plus}{\varepsilon}_{{it}} .$$
Observable household characteristics are food consumption, coffee production, livestock ownership, off-farm work and household size. The production of banana bunches is included in a second set of analyses.
To test the cross-sectional validity of the HFIAS, we estimated equation (1) without taking into account the longitudinal nature of the data. Hence, equation (1) was estimated separately for both the 2007 and the 2012 samples excluding the year-fixed effects, γ t , and household-fixed effects, u i , as independent variables. This has the advantage that we do not assume the same correlation between household characteristics and food insecurity in both periods, but the drawback is that we cannot control for unobservable household characteristics. Equation (1) was estimated with a Tobit model, which yields unbiased estimates even when the dependent variable is truncated in nature, which is the case for the HFIAS( Reference Hsiao 26 ). Moreover, error terms were clustered at village level to avoid bias due to unobservable village characteristics.Footnote *
To test the inter-temporal validity of the HFIAS, the longitudinal nature of our data was exploited. Equation (1) was estimated with a random-effect Tobit model, which allowed us to control for unobservable household characteristics and to take into account the truncated nature of the data( Reference Hsiao 26 ). Inter-temporal validity was accepted if the year-fixed effect γ t was not significantly different from zero, because this condition is sufficient to ensure that all the variation over time of the HFIAS is explained through observable and unobservable household characteristics. Equation (1) was also estimated with a difference-in-difference approach. This means performing the regression of the change in the HFIAS between 2012 and 2007 v. changes in household characteristics.Footnote † This has the advantage that the estimations will not be biased by factors that might influence food security (such as education, soil quality or the household’s assets), which did not change between 2007 and 2012.
Several sensitivity analyses were performed.Footnote ‡ First, we checked for the possibility of enumerators interpreting the HFIAS questions differently (despite training prior to the survey), with some possibly consistently over- or underestimating households’ food security status. To control for this enumerator-specific effects were included in the models. The models were also re-estimated for a sub-sample restricted to the enumerators who participated in both rounds and for a sub-sample restricted to households that were interviewed by the same enumerator in both rounds.
All analyses were performed with the statistical software package STATA 11·0 SE.
Results
Descriptive statistics
Household size (5·8 on average, P=0·88), farm size (about 1 ha, P=0·49) and the number of households keeping livestock (about 20 %, P=0·18) hardly changed between 2007 and 2012 (Table 1). The proportion of households with at least one member engaged in off-farm activities decreased significantly, from 38 % to 18 % (P<0·01). The proportion of households growing coffee also decreased somewhat, from 63 % to 55 % (P<0·01). Households that did cultivate coffee harvested 441 kg in 2007 and 258 kg in 2012 on average. These figures may appear to indicate a sharp decline in coffee production, but coffee production in Burundi has a biannual harvest cycle in which an excellent harvest in one year is followed by a bad harvest in the next year( 27 ). Hence, it was no surprise that production in 2012 (a bad year) was lower than in 2007 (a good year).
Table 1 Sample descriptive statistics of small-scale rural farmers in Ngozi, north of Burundi, by round of data collectionFootnote †,Footnote ‡
Significant differences between rounds: (*)P ≤ 0·10, *P ≤ 0·05, **P ≤ 0·01.
† Values are presented as mean and standard deviation or percentage of households (n 314).
‡ P values were obtained with t test and χ 2 tests for means and percentages, respectively.
§ Banana production is expressed in harvested bunches: estimated average weight 15 kg/bunch.
However, there was marked decrease in total aggregated food production. Average production per day and capita equalled 10 142 kJ (2424 kcal) in 2007, but decreased by 30 % to 7372 kJ (1762 kcal) in 2012 (P<0·01). This decrease was mainly driven by an even sharper decrease in banana production, which fell from 139 bunches per household in 2007 to only fifty bunches per household in 2012 (P<0·01). This drop had a large effect on the total aggregate production as bananas are one of the main components of the Burundian diet( 28 ), have a high energetic value and were cultivated by more than 95 % of households in the sample. When we excluded bananas from total aggregate production figures, overall mean production did not change significantly between 2007 and 2012. The significant decrease in banana production was caused by the disease Xanthomonas wilt, which has infected many banana trees in the region and is threatening the livelihoods of many households in eastern and central Africa. Agricultural research has not yet found an effective prevention or treatment of the disease( Reference Tripathi, Mwangi and Abele 29 ).
Despite these downward changes, the responses to all nine questions of the HFIAS suggested an improvement in the food security situation between 2007 and 2012 (Table 2). For instance, in 2007, 80 % of the households claimed to have eaten a smaller meal than they needed at least once in the previous two weeks, compared with 70 % in 2012. When the frequency of occurrence questions were taken into account (Table 2), the HFIAS decreased significantly from a mean score of 13·9 in 2007 to 10·8 in 2012 (P< 0·01).Footnote * Thus, households reported feeling more food secure in 2012 than in 2007, despite the decrease in food, banana and coffee production. The internal consistency of the responses to the questions was assessed using Cronbach’s α. All questions related positively, and Cronbach’s α was 0·93 in 2007 and 0·95 in 2012. Principal component analysis and psychometric models confirmed the internal validity of the HFIAS (see detailed results in the online supplementary material).
Table 2 Responses to the nine questions of the HFIAS, by round of data collection, among small-scale rural farmers in Ngozi, north of Burundi
HFIAS, Household Food Insecurity Access Scale.
Significant differences between rounds: (*)P ≤ 0·10, *P ≤ 0·05, **P ≤ 0·01.
† Percentage of households (n 314); P values obtained with χ 2 tests.
‡ P values obtained with t tests.
Cross-sectional validity
Before turning to the regression analysis, the association between energy intake and the HFIAS was examined graphically (Fig. 1). Energy intake and the HFIAS were clearly related in the two periods. A classification of households by tertile based on energy intake per capita showed that food insecurity decreased significantly as energy intake increased (P<0·01). The difference in average HFIAS between the first (lowest energy intake) and the third tertile (highest energy intake) was more than 9 points in 2007 and just below 6 points in 2012 (P<0·01). This suggests that the HFIAS is a cross-sectionally valid indicator of food security.
Fig. 1 Household Food Insecurity Access Scale (HFIAS) score in relation to daily energy intake per capita (
, lowest energy intake; 
, medium energy intake; 
, highest energy intake) among small-scale rural farmers in Ngozi, north of Burundi, in 2007 and 2012. Tertiles of daily energy intake per capita correspond with cut-offs at 4895 kJ (1170 kcal) and 11 255 kJ (2690 kcal) in 2007 and 4268 kJ (1020 kcal) and 8201 kJ (1960 kcal) in 2012. Each group contains 104 or 105 observations
More formal regression analyses confirmed this finding (Table 3, models 1 and 4). Food production, cattle ownership and coffee production were positively and significantly correlated with food security in both periods. An increase of food production of 2615 kJ (625 kcal)/d per capita in 2007 was associated with a decrease of 1 point in the HFIAS, which is not a negligible effect, given that the mean HFIAS was 13·9 in 2007.
Table 3 Analyses with Tobit models of correlation between the HFIAS and farm characteristics in 2007 and 2012Footnote † among small-scale rural farmers in Ngozi, north of Burundi
HFIAS, Household Food Insecurity Access Scale.
Significant differences (n 314): (*)P ≤ 0·10, *P ≤ 0·05, **P ≤ 0·01.
† Regression models were estimated with Tobit models with errors clustered at village level; twenty-five and sixty-seven households had a minimum HFIAS score of 0 (left-censored observations) in 2007 and 2012, respectively; seven and eleven households had a maximum HFIAS score of 27 (right-censored observations) in 2007 and 2012, respectively.
‡ Banana production is expressed in harvested bunches: estimated average weight 15 kg/bunch.
§ Variable is mean-centred.
Keeping livestock was strongly and positively associated with food security. This correlation is consistent with cattle being a source of wealth and an important vehicle for saving in an environment characterized by imperfect credit markets( Reference Bundervoet 31 ). Hence, only richer households owned cattle. In addition, their manure is an important fertilizer in an environment where only a few households have access to chemical fertilizers and soil erosion poses a serious threat to agricultural productivity( Reference Cochet 32 ). The positive association between cattle ownership and food security was more pronounced in 2012 than in 2007, although there is no obvious explanation for this finding.
In both surveys, households that were not involved in the cultivation of coffee scored on average 1 point higher on the HFIAS than households that produced the average amount of coffee.
Engagement in off-farm activities correlated positively with food insecurity in both years, but the coefficient was statistically significant only in 2007. This correlation is consistent with the assumption that it was mainly very poor, nearly landless, families who worked as paid farm workers; a justifiable assumption as self-reported off-farm wages were very low (about €0·57/d), even by local standards. Off-farm activities should thus be interpreted as a coping strategy for food-insecure households, a strategy which has also been documented in Rwanda( Reference Ansoms and McKay 33 , Reference Rizzo 34 ).
Household size correlated negatively with food insecurity in both periods, but the association was stronger in 2012 than in 2007. Given that food production per capita was included in the regression analysis, this negative association may indicate that larger households have a higher income from off- and non-farm jobs and are therefore less food insecure.
In order to single out the effect of banana production on food security, banana production was included in the regression analyses as a separate independent variable (Table 3, models 2 and 5). The correlation between food security and banana production was positive and had a similar magnitude as the correlation between food security and food production.Footnote *
The essential finding of these cross-sectional models is that the correlation between the HFIAS and household and farm characteristics was of similar magnitude in both periods. The only important difference between the models was the constant term. The constant in the 2012 model is about 4 points less than the constant in the 2007 model. Hence, a household with exactly the same production characteristics in 2007 and in 2012 reported feeling more food secure in 2012 than in 2007, despite the decrease in overall food production in the region.
Inter-temporal validity
The preceding analysis has already provided some evidence that households reported feeling more food secure while their total food production decreased. A random-effect Tobit model confirmed this finding (Table 4, model 1).Footnote * The correlation between the HFIAS and the dependent variables is quantitatively similar to the base models (Table 3, models 1 and 4). The model also confirmed that, after controlling for the covariates, the average HFIAS was more than 4 points lower in 2012 than in 2007 (P<0·01). Hence this model rejects the inter-temporal validity of the HFIAS.
Table 4 Longitudinal models analysing correlation between the HFIAS and farm characteristics for different sub-samplesFootnote †,Footnote ‡ among small-scale rural farmers in Ngozi, north of Burundi
HFIAS, Household Food Insecurity Access Scale.
Significant differences: (*)P≤0·10, *P ≤ 0·05, **P≤0·01.
† Regression models with random-effect Tobit models.
‡ n 314 in models 1 and 2; 159 and 148 households were interviewed by one of the enumerators that participated in both survey rounds in 2007 and 2012, respectively; n 31 in model 4.
§ Variable is mean-centred.
A difference-in-difference model also rejected inter-temporal validity (Table 5, models 1 and 2). A household with the same farm characteristics reported a HFIAS score that was on average 4·5 points lower in 2012 than in 2007 (P<0·01). An increase in food or banana production, engaging in coffee farming or acquiring cattle between 2007 and 2012 were all significantly positively associated with an increase in households’ food security status. Households with fewer members in 2012 than 2007 reported feeling significantly more food insecure, while households with more members did not report any significant change. A decrease in household size from 2007 to 2012 is likely to occur if an adult child leaves the household, limiting the potential of the household to earn an off- and non-farm income and perhaps reducing landholdings if adult sons inherit already part of the land( Reference Van Leeuwen 35 ), leading to increased food insecurity of the household. Changes in off-farm work, no longer owning cattle or no longer cultivating coffee were not associated with changes in food security status. However, the positive correlations between changes in food security status and changes in production characteristics were too weak to explain many of the changes in the HFIAS between 2007 and 2012. For instance, a household that increased daily production by 4184 kJ (1000 kcal)/capita between 2007 and 2012 reported a decrease in the HFIAS of just 1 point less than a household that did not increase its production. Hence, the upward shift in the perception of food security of the households cannot be attributed to changes in food production, cattle ownership, off-farm work, coffee production or household composition.
Table 5 Analyses with difference-in-difference model of correlation between changes in the HFIAS and changes in farm characteristics between 2007 and 2012Footnote †,Footnote ‡ among small-scale rural farmers in Ngozi, north of Burundi
HFIAS, Household Food Insecurity Access Scale.
Significant differences: (*)P ≤ 0·10, *P ≤ 0·05, **P ≤ 0·01.
† n 314.
‡ Errors clustered at household level.
§ Banana production is expressed in harvested bunches: estimated average weight 15 kg/bunch.
Sensitivity analyses
The inclusion of enumerator-specific dummies revealed that some enumerators consistently over- or underestimated households’ food security in both periods (Table 3, models 3 and 6). For instance, the HFIAS of households interviewed by enumerator 3 in 2012 was on average 7·4 points higher than the average score of households interviewed by enumerator 1. Estimates for this categorical variable cannot be compared between models 3 and 6 because not all enumerators participated in both rounds. It should also be noted that these additional dummies did not capture location-specific effects, because enumerators interviewed different households within the same village. The inclusion of enumerator-specific dummies did not considerably affect the estimates of the main independent variables, but only improved the explanatory power of the models. In addition, constants in both models were quite similar to the base models. It seems that differences between enumerators mattered, but that all enumerators were able to discriminate between food-secure and food-insecure households. However, these regression models showed that scores on a subjective measure could be severely biased as a result of enumerators’ divergent interpretations of the questions. This aspect of subjectivity also has to be considered when choosing between using the HFIAS and other measures of food insecurity.
The differences between enumerators in assessing households’ food insecurity status are unlikely to explain the important finding of the lack of inter-temporal validity of the HFIAS (Table 4, models 2 to 4). The fixed-year effect remained negative and highly significant in the random-effect Tobit models that included enumerator-fixed effects (model 2), restricted the sample to enumerators who participated in both rounds (model 3) or to households that were interviewed by the same enumerator in both rounds (model 4). Only thirty-one households were interviewed by the same enumerator in both survey rounds. This sample size was too small to estimate a meaningful difference-in-difference model as an additional robustness check.
Discussion
The present study shows that the HFIAS is a cross-sectionally valid indicator of food security. This is in line with the literature. However, its inter-temporal validity can be questioned, because the self-reported food security status of households increased despite food production decreasing between the two surveys. This finding has not been often reported before in the literature on food insecurity indicators. Hence, we closely examine the factors that might invalidate this conclusion.
An important assumption in the present study is that food production is strongly correlated with food consumption; we did not collect detailed food expenditure data or food intake data. Several studies indicate that Burundian farmers mainly produce for subsistence purposes( 23 , Reference Detry 36 , 37 ). This is confirmed by our data. For instance, 35 % of the households sold sweet potatoes in 2012 and these households sold on average less than 30 % of their harvest. Only coffee and bananas were extensively marketed. Moreover, food production is the main source of wealth in rural Burundi (e.g. it is strongly correlated with assets such as land) and is thus expected to be significantly correlated with food security.
A second, closely related assumption concerns the timing of food consumption. The HFIAS only probes into households’ food security status over the last four weeks. As total food consumption in the four weeks before the interview is approximated by total annual food production, the lack of inter-temporal validity would also be observed if most households consumed considerably more in the four weeks before the interviews were conducted in 2012 than in 2007. This would explain the lack of inter-temporal validity, although it would not contradict a general decrease in food production over time. The fact that interviews were conducted in the same month in both rounds of data collection can only partially mitigate this concern. However, the finding that annual food production is strongly associated with the HFIAS in both periods suggests that this possibility is probably not driving the results.Footnote * Future studies would ideally make use of detailed food intake data to avoid this caveat.
The main finding of lack of inter-temporal consistency of the HFIAS hinges on the observation that food production decreased, or at least did not increase, between 2007 and 2012. This decline is confirmed by secondary data sets. The Food Balance Sheets published by the FAO show that daily food supply per capita in Burundi decreased from 6929 kJ (1656 kcal) in 2007 to 6711 kJ (1604 kcal) in 2009. Similarly, an aggregation of the total food production based on the main staple crops (published by FAO) shows that the total production in Burundi did not increase between 2007 and 2011, while the population grew considerably. Simple calculations based on these figures showed that food production per capita per day decreased from 9602 kJ (2295 kcal) in 2007 to 8899 kJ (2127 kcal) in 2012. A website recently launched by the Government of Burundi (in close collaboration with the FAO) provides agricultural statistics at the provincial level( 38 ). The reported trends of food production in Ngozi corroborate our findings. They found a 60 % decrease in banana production and an 80 % decrease in the production of sweet potatoes between 2007 and 2012, while production of the other main crops remained more or less stable. It should, however, be mentioned that the reliability of these figures is difficult to check. Nevertheless, we are fairly confident that the decrease in food production, and hence total income and food consumption, is a region-wide phenomenon.
Another competing explanation for the improvement in the perceived food security status of the households between 2007 and 2012 is an increase in food aid. However, food aid in Ngozi provided by the World Food Programme and its partners decreased, from 2750 tonnes in 2007 to 2570 tonnes in 2012, which corresponds to 4·16 and 3·89 kg per capita, respectively.Footnote † Moreover, there is no indication that food aid programmes were better targeted at food-insecure households in 2012 than in 2007. In 2012 more than 70 % of resources were devoted to school feeding programmes, which provide all primary-school children with a daily free meal, independent of their food security status. A final possibility is an increase in remittances between 2007 and 2012. However, the importance of this livelihood source is likely to be small as only five households in the sample claimed to receive remittances.
The present study did not focus on the internal validity of the HFIAS, as this aspect of the indicator has already been validated in previous studies( Reference Marques, Reichenheim and de Moraes 10 ). However, if the questions of the HFIAS do not reliably measure food security, this could explain the lack of inter-temporal validity. For instance, our findings would be biased if a question is interpreted differently by the respondents in 2007 compared with 2012. This point was examined with polytomous Rasch models (see online supplementary material). This revealed that question 9 may have been interpreted differently in 2012 v. 2007, with more households reporting ‘going a whole day and night without eating’ in 2007 than expected. This question contributed significantly to the overall decrease in the HFIAS from 2007 to 2012 (see Table 2). However, even if question 9 was excluded from the indicator, we would still observe an increase in perceived food security from 2007 to 2012. This suggests that lack of stability of the questions may partially explain the lack of inter-temporal validity, but it is unlikely to be the only reason. Additional psychometric studies are nevertheless required to examine this point in more detail.
A study in Ethiopia( Reference Maes, Hadley and Tesfaye 20 ), which found a similar inconsistency over time, stresses the possibility of ‘observation bias’ and ‘response shifts’. The former might occur if respondents pretend to be more food insecure than they really are in the first round of a survey because they expect that less food-secure households will receive food aid. In the second round, households would respond more honestly, reporting their ‘true’ food security situation. We believe that this bias is likely to be limited in our study because respondents were well informed on the research aim at the start of the interview. Moreover, a very limited number of international non-governmental organizations are active in the area and therefore respondents do not expect any food aid.
Finally, response shifts might arise if respondents shift their internal standards as their living conditions change over time. This theory predicts that individuals assess their well-being not only by comparing their current situation with the past but also by gauging their relative position within their community( Reference Sprangers and Schwartz 39 , Reference Günther and Maier 40 ). This lack of a common reference frame, both over time and between poor and rich households, is a general limitation of subjective indicators( Reference Headey and Ecker 7 ). The study in Ethiopia( Reference Maes, Hadley and Tesfaye 20 ) pointed to this phenomenon to explain why volunteer HIV/AIDS caregivers, frequently faced with individuals even worse off than themselves, reported feeling more food secure even though their food security situation (measured objectively) deteriorated. Objectively, these caregivers had indeed become more food insecure over time, but they were less affected than the households that they visited regularly and therefore felt more food secure. A similar effect may be at play in our study area. Agricultural production decreased in the entire region (primarily caused by the loss of banana trees) but, given the limited migration rates and lack of communication infrastructure, only a few households had access to information about living standards in other provinces in Burundi or in the capital. It is therefore likely that respondents compared their food security situation with that of their neighbours and evaluated their position within the local community instead of comparing their current situation with the past. This would simultaneously explain the cross-sectional validity and lack of inter-temporal validity of the HFIAS. Similar patterns have been found in research on happiness( Reference Luttmer 41 , Reference D’Ambrosio and Frick 42 ). Recently, new measures of poverty have been proposed that explicitly take into account this reference-dependent utility( Reference Günther and Maier 40 ).
Conclusion
The development of the HFIAS is an attempt to construct an indicator of food insecurity that is internally, cross-culturally, cross-sectionally and inter-temporally valid and that captures all aspects of food insecurity. Moreover, this indicator needs to be user-friendly so that food insecurity in rural areas can be easily monitored by non-governmental organizations and governments. Harmonizing these ambitious, and sometimes contrasting, objectives is a major challenge.
Results from the present study in the north of Burundi confirm the cross-sectional validity of the HFIAS, as it was significantly correlated with annual food production, livestock keeping, off-farm work, coffee production and household size. However, we are less convinced about the inter-temporal validity of the index, as perceived food security increased while total production declined over the same time period. As this is one of the first studies investigating the inter-temporal validity of this indicator of food insecurity over a long time period, additional studies are needed to confirm (or refute) our results in different settings.
The findings reported in the present study suggest that detailed production and consumption data will remain indispensable in the examination of the dynamics of food security. Consequently, studies which assume the inter-temporal validity of subjective indicators should be interpreted carefully, as this assumption is questionable( Reference Headey 43 , Reference Verpoorten, Arora and Stoop 44 ). Finally, the results raise the question of what the HFIAS actually measures and how households assess their own food security situation. Part of the answer might lie in ‘response shifts’ in which respondents reassess their internal standards over time due to a general decrease of the living standards within their community. This is an interesting avenue for further research.
Acknowledgements
Financial support: Data collection for this study was supported by the Flemish Inter-University Council–University Development Cooperation (VLIR-UOS by its Flemish acronym) through the Own Initiative Project (project number ZEIN 2007PR 336). S.D was supported by Ghent University (doctoral grant from the Special Research Fund (BOF) – UGent) and S.N. was supported by VLIR-UOS. Conflict of interest: None. Authorship: S.D., M.D. and S.N. designed the research; S.N. conducted the fieldwork; S.D. analysed the data; S.D and M.D wrote the paper, with S.D. having primary responsibility for final content; all authors read and approved the final manuscript. Ethics of human subject participation: Because of the approach used and the questionnaire content, no formal ethical approval was sought.
Supplementary material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1368980015000403
