Skip to main content
×
×
Home

Using pre- and post-survey instruments in interventions: determining the random response benchmark and its implications for measuring effectiveness

  • George C Davis (a1) (a2), Ranju Baral (a3), Thomas Strayer (a4) and Elena L Serrano (a1)
Abstract
Abstract Objective

The present communication demonstrates that even if individuals are answering a pre/post survey at random, the percentage of individuals showing improvement from the pre- to the post-survey can be surprisingly high. Some simple formulas and tables are presented that will allow analysts to quickly determine the expected percentage of individuals showing improvement if participants just answered the survey at random. This benchmark percentage, in turn, defines the appropriate null hypothesis for testing if the actual percentage observed is greater than the expected random answering percentage.

Design

The analysis is demonstrated by testing if actual improvement in a component of the US Department of Agriculture’s (USDA) Expanded Food and Nutrition Education Program is significantly different from random answering improvement.

Setting

USA.

Subjects

From 2011 to 2014, 364320 adults completed a standardized pre- and post-survey administered by the USDA.

Results

For each year, the statement that the actual number of improvements is less than the expected number if the questions were just answered at random cannot be rejected. This does not mean that the pre-/post-test survey instrument is flawed, only that the data are being inappropriately evaluated.

Conclusions

Knowing the percentage of individuals showing improvement on a pre/post survey instrument when questions are randomly answered is an important benchmark number to determine in order to draw valid inferences about nutrition interventions. The results presented here should help analysts in determining this benchmark number for some common survey structures and avoid drawing faulty inferences about the effectiveness of an intervention.

Copyright
Corresponding author
* Corresponding author: Email georgedavis@vt.edu
References
Hide All
1. Andreyeva T, Middleton AE, Long MW et al. (2011) Food retailer practices, attitudes, and beliefs about the supply of healthy foods. Public Health Nutr 14, 10241031.
2. Song HJ, Gittelsohn J, Kim M et al. (2009) A corner store intervention in a low-income urban community is associated with increased availability and sales of some healthy foods. Public Health Nutr 12, 20602067.
3. Martínez-Donate AP, Riggall AJ, Meinen AM et al. (2015) Evaluation of a pilot healthy eating intervention in restaurants and food stores of a rural community: a randomized community trial. BMC Public Health 15, 136.
4. Escaron AL, Martinez-Donate AP, Riggall AJ et al. (2016) Developing and implementing ‘Waupaca Eating Smart’: a restaurant and supermarket intervention to promote healthy eating through changes in the food environment. Health Promot Pract 17, 265277.
5. Lee RM, Rothstein JD, Gergen J et al. (2015) Process evaluation of a comprehensive supermarket intervention in a low-income Baltimore community. Health Promot Pract 16, 849858.
6. Baral R, Davis GC, Blake S et al. (2013) Using national data to estimate average cost effectiveness of EFNEP outcomes by state/territory. J Nutr Educ Behav 45, 183187.
7. Ott RL & Longnecker M (2001) An Introduction to Statistical Methods and Analysis, 5th ed. Pacific Grove, CA: Duxbury.
8. US Department of Agriculture (2016) Expanded Food and Nutrition Education Program (EFNEP). http://www.nifa.usda.gov/program/expanded-food-and-nutrition-education-program-efnep (accessed September 2016).
9. US Department of Agriculture (2016) Expanded Food and Nutrition Education Program. National Data. Years 2010–2014. http://www.nifa.usda.gov/efnep-national-data-reports (accessed September 2016).
10. Murphy SP, Kaiser LL, Townsend MS et al. (2001) Evaluation of validity of items for a food behavior checklist. J Am Diet Assoc 101, 751761.
11. George GC, Milani TJ, Hanss-Nuss H et al. (2004) Development and validation of a semi-quantitative food frequency questionnaire for young adult women in the southwestern United States. Nutr Res 24, 2943.
12. Lim SS, Gold A, Gaillard PR et al. (2015) Validation of 2 brief fruit and vegetable assessment instruments among third-grade students. J Nutr Educ Behav 47, 446451.
13. Bond TG & Fox CM (2007) Applying the Rasch Model: Fundamental Measurement in the Human Sciences, 2nd ed. Mahwah, NJ: LEA Publishers.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Public Health Nutrition
  • ISSN: 1368-9800
  • EISSN: 1475-2727
  • URL: /core/journals/public-health-nutrition
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords:

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 1
Total number of PDF views: 37 *
Loading metrics...

Abstract views

Total abstract views: 266 *
Loading metrics...

* Views captured on Cambridge Core between 21st December 2017 - 19th January 2018. This data will be updated every 24 hours.