Hostname: page-component-76d6cb85b7-ntvhh Total loading time: 0 Render date: 2026-07-13T07:09:02.066Z Has data issue: false hasContentIssue false

Survey ordering and the measurement of welfare

Published online by Cambridge University Press:  30 July 2025

Khanker Wahedur Rahman
Affiliation:
University of Oxford, Martin Program on the Future of Development, Oxford, United Kingdom and BRAC Institute for Governance and Development, BRAC University, Dhaka, Bangladesh
Jeffrey Bloem*
Affiliation:
International Food Policy Research Institute, Markets, Trade, and Institutions Unit, Washington, DC, USA
Marc F. Bellemare
Affiliation:
Department of Applied Economics, University of Minnesota, Saint Paul, MN, USA
*
Corresponding author: Jeffrey Bloem; Email: bloem.jeff@gmail.com
Rights & Permissions [Opens in a new window]

Abstract

Economic policy and research rely on the accurate measurement of welfare. In nearly all instances, measuring welfare requires collecting data via long household surveys. If survey response patterns change over the course of a survey to introduce measurement error, this measurement error can be either classical (i.e., changing distributions, leading to noise) or non-classical (i.e., changing expectations, leading to bias). We embed an experiment in a survey by randomly assigning a questionnaire with either the assets module near the beginning of the survey or the assets module at the end of the survey, delaying enumeration of assets by about 60 minutes. We find no evidence in the full sample that survey ordering introduces differential response patterns, either in the number of reported assets or the reported value of those assets. In exploratory analysis of heterogeneity, we find evidence of non-classical measurement error due to survey ordering within sub-samples of respondents who (i) are from larger households or (ii) have low levels of education. Our experimental design can be generalized to serve as an ex-post test of data quality with respect to questionnaire length.

Information

Type
Original Paper
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2025. Published by Cambridge University Press on behalf of the Economic Science Association.
Figure 0

Table 1 Survey module order

Figure 1

Fig. 1 Kernel Density Estimate of the Duration of Active Survey TimeNotes: Epanechnikov kernel. Full sample mean = 1.21 hours, median = 1.14 hours, 1st percentile = 0.64 hours, and 99th percentile = 2.38 hours. Sample size = 3,931. The average difference in duration by treatment status is not statistically significant. Regression results are shown in Table A.4

Figure 2

Table 2 Number of assets reported, full sample

Figure 3

Fig. 2 Effect on the Value Reported of Each AssetNotes: This figure shows coefficient estimates with associated 95 percent confidence intervals. When we adjust for multiple hypothesis testing using the method developed by Benjamini et al. (2006), as implemented by Anderson (2008), none of these effects are statistically different from zero

Figure 4

Table 3 Number of assets reported and total asset value, within sub-samples

Supplementary material: File

Rahman et al. supplementary material

Rahman et al. supplementary material
Download Rahman et al. supplementary material(File)
File 621.2 KB