Skip to main content Accessibility help
×
Home

How to use replicate weights in health survey analysis using the National Nutrition and Physical Activity Survey as an example

  • Carole L Birrell (a1), David G Steel (a1), Marijka J Batterham (a1) and Ankur Arya (a1)

Abstract

Objective:

To conduct nutrition-related analyses on large-scale health surveys, two aspects of the survey must be incorporated into the analysis: the sampling weights and the sample design; a practice which is not always observed. The present paper compares three analyses: (1) unweighted; (2) weighted but not accounting for the complex sample design; and (3) weighted and accounting for the complex design using replicate weights.

Design:

Descriptive statistics are computed and a logistic regression investigation of being overweight/obese is conducted using Stata.

Setting:

Cross-sectional health survey with complex sample design where replicate weights are supplied rather than the variables containing sample design information.

Participants:

Responding adults from the National Nutrition and Physical Activity Survey (NNPAS) part of the Australian Health Survey (2011–2013).

Results:

Unweighted analysis produces biased estimates and incorrect estimates of se. Adjusting for the sampling weights gives unbiased estimates but incorrect se estimates. Incorporating both the sampling weights and the sample design results in unbiased estimates and the correct se estimates. This can affect interpretation; for example, the incorrect estimate of the OR for being a current smoker in the unweighted analysis was 1·20 (95 % CI 1·06, 1·37), t= 2·89, P = 0·004, suggesting a statistically significant relationship with being overweight/obese. When the sampling weights and complex sample design are correctly incorporated, the results are no longer statistically significant: OR = 1·06 (95 % CI 0·89, 1·27), t = 0·71, P = 0·480.

Conclusions:

Correct incorporation of the sampling weights and sample design is crucial for valid inference from survey data.

Copyright

Corresponding author

*Corresponding author: Email cbirrell@uow.edu.au

References

Hide All
1. Valliant, R, Dever, JA & Kreuter, F (2013) Practical Tools for Designing and Weighting Survey Samples. New York: Springer.
2. Levy, PS & Lemeshow, S (2008) Sampling of Populations: Methods and Applications. Hoboken, NJ: John Wiley & Sons.
3. Valliant, R & Dever, JA (2018) Survey Weights – A Step-By-Step Guide to Calculation. College Station, TX: Stata Press.
4. Bell, BA, Onwuegbuzie, AJ, Ferron, JM et al. (2012) Use of design effects and sample weights in complex health survey data: a review of published articles using data from 3 commonly used adolescent health surveys. Am J Public Health 102, 13991405.
5. Heeringa, SG, West, BT & Berglund, PA (2010) Applied Survey Data Analysis. Boca Raton, FL: Chapman and Hall/CRC, Taylor & Francis Group.
6. Campbell, RT & Berbaum, ML (2010) Analysis of data from complex surveys. In Handbook of Survey Research, 2nd ed., pp. 221259. Bingley: Emerald Publishing Group Limited.
7. StataCorp (2017) Stata Survey Data Reference Manual. College Station, TX: Stata Press.
8. Saylor, J, Friedmann, E & Lee, HJ (2012) Navigating complex sample analysis using national survey data. Nurs Res 61, 231237.
9. Kim, Y, Park, S, Kim, N-S et al. (2013) Inappropriate survey design analysis of the Korean National Health and Nutrition Examination Survey may produce biased results. J Prev Med Public Health 46, 96104.10.3961/jpmph.2013.46.2.96
10. West, BT, Sakshaug, JW & Aurelien, GAS (2018) Accounting for complex survey sampling in survey estimation: a review of current software tools. J Off Stat 34, 721752.
11. Wolter, KM (2007) Introduction to Variance Estimation, 2nd ed. New York: Springer Verlag.
12. Australian Bureau of Statistics (2012) 4363.0.55.001 – Australian Health Survey: Users’ Guide, 2011–13. http://www.abs.gov.au/AUSSTATS/abs@.nsf/Lookup/4363.0.55.001Main+Features12011-13?OpenDocument (accessed December 2017).
13. Kott, PS (2001) The delete-a-group jackknife. J Off Stat 17, 521526.
14. Valliant, R (2004) The effect of multiple weighting steps on variance estimation. J Off Stat 20, 118.
15. Abdi, H & Williams, LJ (2010) Encyclopedia of Research Design. Thousand Oaks, CA: SAGE Publications, Inc.
16. Australian Bureau of Statistics (2012) 4364.0.55.001 – Australian Health Survey: First Results, 2011–12. http://www.abs.gov.au/ausstats/abs@.nsf/Lookup/4364.0.55.001main+features12011-12 (accessed December 2017).
17. Australian Bureau of Statistics (2018) Types of microdata. http://www.abs.gov.au/websitedbs/D3310114.nsf/home/Microdata+Entry+Page (accessed October 2018).
18. Binder, DA (1983) On the variance of asymptotically normal estimators from complex surveys. Int Stat Rev 51, 279292.
19. West, BT, Berglund, P & Heeringa, SG (2008) A closer examination of subpopulation analysis of complex-sample survey data. Stata J 8, 520531.
20. Allman-Farinelli, MA, Chey, T, Merom, D et al. (2010) Occupational risk of overweight and obesity: an analysis of the Australian Health Survey. J Occup Med Toxicol 5, 14.
21. Peng, Y, Wang, Z, Dong, B et al. (2017) Life’s Simple 7 and ischemic heart disease in the general Australian population. PLoS One 12, e0187020.
22. Australian Bureau of Statistics (2005) 4715.0.55.002 – Technical Manual: National Aboriginal and Torres Strait Islander Health Survey, Expanded CURF, 2004–05. http://www.abs.gov.au/AUSSTATS/abs@.nsf/Latestproducts/4715.0.55.002Main+Features3002004-05 (accessed February 2018).
23. Burden, S, Probst, Y, Steel, D et al. (2012) The impact of complex survey design on prevalence estimates of intakes of food groups in the Australian National Children’s Nutrition and Physical Activity Survey. Public Health Nutr 15, 13621372.

Keywords

How to use replicate weights in health survey analysis using the National Nutrition and Physical Activity Survey as an example

  • Carole L Birrell (a1), David G Steel (a1), Marijka J Batterham (a1) and Ankur Arya (a1)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed