Formative research is widely practised when designing interventions to improve public health(Reference Gittelsohn, Steckler, Johnson, Pratt, Grieser, Pickrel, Stone, Conway, Coombs and Staten1–Reference Horner, Romer and Vanable3). In-depth interviews, focus groups, card sorts and other methods are used to identify best components for an outreach, or to choose among media (print, television, personal encounter, and so on), or to draft evaluation instruments, or to improve recruitment and retention of clients.
Efforts to fight obesity and improve human nutrition have embraced such formative methods(Reference Ayala, Elder, Campbell, Engelberg, Olson, Moreno and Serrano4–Reference Strolla, Gans and Risica6). As with other issues in public health, however, nutrition projects’ use of formative strategies has tended to remain on a high level of construal(Reference Trope and Liberman7, Reference Trope, Liberman and Wakslak8). That is, preparatory studies have addressed broad and relatively global questions related to target behaviours or project goals, such as: ‘What are current food-use habits among our designated population?’ or ‘Which deficits in knowledge and attitudes need to be surmounted in order to convince people to eat a more nutritious diet?’
The present article differs by reporting the use of formative methods at a much lower level of construal: pre-testing messages intended to induce near-term, specific and concrete actions to improve nutrition. In the present case, the behavioural goal was clear. Households would be supplied with an ample amount of a fresh vegetable. That established, how could we best design recipes and food-use tips that would lead household cooks to increase their preparations of that vegetable within the next few days?
The present study, therefore, reports the use of formative methods in order to settle issues about the information design of intervention tools, and not to choose, say, among optional versions of a dietary intervention. Construal theory asserts that the manner in which an intervention frames and presents immediate tasks to be performed greatly influences whether or not people will procrastinate, or ignore the tasks altogether. As Schriver(Reference Schriver9) demonstrated, legibility (through adept use of typography, open space, organisation and illustrations) and pertinence (emphasis on vital information and de-emphasis of peripheral content) matter for achieving an intervention’s mission. Formative research on the particulars of information design is an underdeveloped aspect of nutrition education.
The present article offers a case study in how four methods of formative evaluation improved the content and graphic design of printed nutrition outreach aimed at low-income people. These methods adapt easily to preparing a nutrition intervention’s fundamental tools, whether intended for wide use with the general public or in more limited research projects.
Even in an era of electronic communication, print remains a valuable medium. Nutrition educators often wish to place information about healthy eating in the hands of household cooks, precisely at the moment when they are preparing meals and snacks in their kitchens. Cooks play a pivotal and obvious role in household diets. They can easily retrieve print, combining images and text, from nearby drawers or cabinets and carry printed information as they move from refrigerator to counter to sink to stove to serving table.
Below, we: (i) explain the background of our nutrition intervention to improve the use of fresh vegetables among clients of community food pantries; (ii) recount how the logistics of charitable supply dictated boundaries in the intervention’s overall approach; (iii) describe four methods of formative evaluation that sharpened decisions about the content and graphic design of recipes and food-use tips, the intervention’s fundamental documents; and (iv) share discoveries from formative studies about meeting information needs of low-income clients of pantries.
The four techniques do not have to be executed in a fixed order. Two of the evaluative methods – focus groups and paper-and-pencil evaluations – are familiar to most nutrition educators. Two others, however, are relatively novel. We relied on in-home cooking experiments to improve the intervention and also developed a technique that used Velcro™ boards to simulate the design of print flyers.
Improving the supply of fresh vegetables
The health benefits of a diet rich in fresh produce have been documented(Reference Khaw, Wareham, Bingham, Welch, Luben and Day10). However, low-income people often find these foods too expensive to purchase(Reference Monsivais and Drewnowski11), unavailable in nearby markets(Reference Sloane, Diamant, Lewis, Yancey, Flynn, Nascimento, McCarrthy, Guinyard and Cousineau12), or unappealing or difficult compared to convenience items(Reference Block, Scribner and DeSalvo13). Fortunately, America’s vast network of more than 250 food banks and 26 000 charitable community pantries has begun to solicit and distribute surplus fresh produce and other nutritious, perishable items(Reference Papa, Singhal and Papa14, Reference Parsons15). This infrastructure satisfies at least some of the food needs of the 25 million low-income people each year who are clients of the food banks’ community-based agencies. Most recipients are the working poor, children, or seniors. (For details about Feeding America, the network comprising the majority of food banks, see http://www.feedingamerica.org.)
Since 1991, two of the authors (S.H.E. and P.C.) have promoted the collection and distribution of healthy foods by the nation’s food banks and their network of community-based pantries. We have provided food banks with technical assistance and grants enabling them to launch programmes to solicit donations of fresh produce and to distribute these foods. Online reports about this work can be found at http://www.grass-roots.org/usa/whol.shtml and http://www.epa.gov/osw/conserve/materials/organics/pubs/wast_not.pdf. However, over the years we have observed that many clients do not know how to prepare this food in healthy ways, if at all. When a household’s cook is handed a bag of nutritious and perishable vegetables, he or she faces a challenge: What can I make out of this, right now, to feed myself and my family?
Pantries are underexploited venues for teaching recipients how to prepare nutritious meals. But, we discovered, healthy recipes that would be embraced by recipients did not exist. Our experiences in forty-seven states led to hundreds of consultations with pantry staffs, occasions to observe clients preparing meals at home, and exposure to the logistics of charitable food distribution. Those experiences shaped intensive interviews of pantry clients and early focus groups (not reported here), from which we concluded that recipes and tips would need to honour the eight principles summarised in Table 1.
Principles about information design
Improving the use of fresh vegetables
Whilst supplies of fresh vegetables have been increasing at charitable pantries (R Bella, personal communication), many household cooks continue to under-accept or under-utilise them because they find these items unfamiliar or challenging to prepare in meals(Reference Clarke, Evans and Hovy20). To help bridge this knowledge gap, we developed a computer-based system with vegetable recipes and tips for clients of community pantries; we call this Quick! Help for Meals(Reference Clarke and Evans21, Reference Hirst, DeMarco, Hovy and Parsons22). Pantry staff ask each client twenty-three questions about kitchen utensils and meal preparation preferences. Then, Quick! Help for Meals produces an individualised, colour-illustrated flyer with recipes and tips that meet the needs of each client. This type of customised computer system, often called ‘message tailoring’, uses software written for other health applications(Reference Wanner and Hovy23).
This intervention to improve nutrition among low-income people has been guided by social cognitive theory(Reference Bandura24). Along with many other approaches to health change, social cognitive theory distinguishes among knowledge, attitude and behaviour. Examples of these components include whether household cooks understand the recommended number of daily servings of vegetables, expectations that the family will enjoy consuming vegetables, and actually preparing meals using vegetables.
As theory emphasises, however, behaviour is the key to unlocking other psychosocial components of better eating. Learning to fix simple but tasty servings of vegetables creates a growth in favourable attitudes toward these challenging foods. Learning to make meals adds value to discovering abstract facts, such as the health benefits of vegetables or recommended daily consumption. Fixing servings, perhaps starting with easy preparations, paves the way toward greater self-efficacy about more advanced kitchen tasks and toward outcome expectations, the anticipation that cooked vegetables will contribute to well-received meals.
These considerations have driven the present intervention. We assumed it is vital that cooks discover how to peel or clean vegetables, cut them in pieces of the right size, combine them with other ingredients, cook and perform sequenced tasks(Reference Goody25). Cooks must be invited to manipulate foods they have been given, using implements that they already possess. They must be asked to prepare servings for recognisable needs: satisfying the appetites of young children, or producing a dish that contains Hispanic flavours, or meeting dietary requirements of someone with diabetes, for example.
Results from four methods of formative evaluation
Although the present case study uses computer software to individualise or ‘tailor’ flyers with recipes and tips customised to each household’s needs(Reference Noar, Benac and Harris26, Reference Suggs27), more familiar and standardised nutrition intervention tools require intense formative evaluation, as well. Each formative application is presented here as a separate Phase, though other nutrition projects may pursue the methods in a different sequence.
Phase 1: Focus groups and decisions they shaped about information design
The first step in our process was a series of six focus groups, each involving six to ten participants. These discussions helped determine the content of recipes, gather intelligence about what food-use tips might be helpful, and explore details about the presentation of this information. (Participants’ screening questionnaire and moderator’s guide are available from the authors.) Low-income household cooks were recruited from charitable pantry food lines, and given an information sheet about the focus group. All cooks were female and were compensated $25 each for their time. Discussions were held in facilities near the pantries (e.g. a trailer park’s recreation room) and either audiotaped or videotaped in order to replay passages, distilling both verbal and non-verbal contributions.
Clients expressed a wide-ranging appetite for recipes, from microwaving to crockpot to steaming to stir frying to soups. Subsets of clients desired kid-friendly preparations. Some leaned toward different ethnic flavourings (African American, Hispanic, Asian). Other distinguishing features of recipes also emerged – such as using garlic or not, mixing vegetables with meat or not, and length of preparation and cooking times.
Focus group participants expressed interests in more food-use tips than we had imagined. Nutrition information of practical use was high on the list of many. Some sought ideas for making baby food, while others were attentive to the chewing and digestive needs of seniors. Many other ideas for tips surfaced, such as how to store vegetables, how to freeze items, portion control for people with diabetes, and more.
Gradually, results from focus groups led toward a preferred design for recipes. Household cooks wanted ingredients listed first, followed by answers to six questions. These were: (i) preparation time; (ii) cooking time; (iii) number of servings yielded; (iv) whether the recipe could be made well ahead of consumption; (v) whether it could be frozen; and (vi) whether it was good for leftovers. Below this information, participants wanted preparation steps. Finally, they wanted a colour photograph of the finished product and, where needed, a picture of an unfamiliar intermediate step in preparation. They recoiled from line drawings, cartoons and abstract or decorative art work(Reference Schriver9).
Focus groups clarified additional details of information design. Photos of recipes should include a table fork or other familiar implement, or a human hand, to convey the size of food components. Recipes should separate panels of text using lines (rules, in graphic design), and also separate text from photographs with lines. Text should appear in a sans serif font, occasionally using italics to distinguish information. Recipes should be titled emphasising their essential ingredients or styles of preparation, instead of using exotic or clever names. Titles should be printed in green, connoting nature. All material should be printed on paper stock that is stiff enough to be propped up, for easy reading while making preparations.
Figures 1 and 2 show examples of recipes with these design elements. The second recipe contains two photos: one that shows an intermediate step of immersing beans in ice water to halt cooking and the second that shows the finished serving.
Sample recipes
Sample recipes
Altogether, 195 recipes were developed covering ten vegetables (broccoli, carrots, cabbage, cauliflower, green beans, onions, potatoes, sweet potatoes, zucchini, and ‘root vegetables’ combining turnips, parsnips and rutabagas). These vegetables probably embrace 95 % of the produce that charitable pantries ever make available (R Bella, personal communication).
All recipes were prepared and pre-tested in collaboration with nutritionists, adjusting ingredients to improve taste, texture, appearance and ease of preparation. For each vegetable, we also created at least twelve tips about its use, relying on guidance from focus group participants. Tips covered general serving suggestions (sometimes including how to cut the vegetable or use overlooked parts, like broccoli stalks); topics like how to store, freeze and prevent spoilage of a vegetable; how to make baby food; making snacks; how to prepare juices; modifying servings for people with diabetes; preparing the vegetable for just one or two people; and nutritional facts in general, and those relevant for children and for seniors. Figure 3 shows an example of a tip.
Sample tip
Focus groups, in brief, launched an odyssey into multiple and innovative methods of testing the design of both recipes and food-use tips. (A sample set of recipes and tips, illustrating both content and design solutions, is available from the authors.)
Phase 2: Lessons from on-site evaluations of sample recipes at WIC centres
The next evaluation step sought feedback from mothers who were primary cooks in their households. We created evaluation groups of thirty to thirty-five participants in the USDA’s Women, Infants and Children (WIC) Programme. All women fell below 185 % of the US poverty income guidelines, and received food vouchers as a benefit of attendance at WIC sessions, an incentive that made them resemble clients of community pantries.
Bilingual research assistants explained to the participants that the project wanted ‘to find out what you like and don’t like about some recipes that other people have developed’. Research assistants added that ‘it’s very important to find out what is good and not so good… because this will help us develop the best recipes to share with individuals and families across the country’.
Participants in a group received a packet of four recipes, each on a separate page and presented in the format already described, including colour photographs. A short paper-and-pencil questionnaire followed each recipe. It began by asking, ‘How appealing does this recipe look to you right now?’, accompanied by a three-point smiley-face rating scale. Subsequent questions asked how interested participants were in trying the recipe at home, how easy the ingredients would be for them to get, how easy the recipe’s steps were, how helpful the photograph was, whether they would pass the recipe along to a family member or friend, and whether or not they felt they could prepare the recipe at home. The bilingual staff spoke instructions and also read questions aloud to reduce a reliance on participants’ literacy.
As WIC cooks completed evaluations of each recipe, they slipped their questionnaires into an unmarked envelope. This, staff had pointed out, guaranteed confidentiality and anonymity. Assistants also added that ‘recipes are available at the front of the room’ (in both English and Spanish, though black-and-white only), and that participants could take copies if they liked. Assistants kept track of how many copies were removed at each group session, a behavioural indicator of the cooks’ interest in each recipe.
The present paper reports on just four preparations, illustrating the kinds of discoveries that helped shape the inventory of recipes and tips. Figure 4 shows the reactions of thirty-five WIC mothers to:
Percentages of WIC cooks making selected responses to four recipes (
, percentage of WIC women who took recipe when leaving the session; 
, percentage of WIC women who rated recipe with a smiling face; 
, percentage of WIC women ‘very interested’ in trying recipe at home)
• Potato Corn Chowder (called Soup below)
• Cold Green Bean and Tomato Salad
• Broccoli and Garlic Noodles
• One-Dish Potatoes and Baked Chicken.
Solid bars show the percentages of cooks who took each recipe away (behavioural measure). Grey bars show the percentages that rated each recipe with a positive face (graphic measure). Open bars show women who were ‘very interested’ in trying each recipe (verbal expression).
Results demonstrate general agreement about the recipes across three types of responses: behavioural, graphic and verbal. With occasional exceptions, this convergence persisted across measures of cooks’ feelings about other recipes, in addition to the four reported here. For example, with one set of diverse recipes the rank-order correlation was 0·71 between take a copy and graphic rating, was 0·74 between take a copy and verbal rating, and was 0·87 between graphic rating and verbal rating. For all practical purposes, these were equivalent ways of asking cooks what they thought they would or would not use.
The on-site evaluations helped identify recipes with near-universal appeal, middle-range support, and those that were less popular. The overall appeal of recipes varied widely. For example, Potato Corn Chowder, like other soups tested at WIC sessions, drew favour from well under half of cooks, typically one-quarter to one-third. On the other hand, the one-dish, chicken-and-vegetable preparation was very popular. Usually, recipes fell in the middle-range, where the chart shows the green bean and tomato salad and the preparation of broccoli and noodles. Each attracted its own segment of cooks, somewhere between 40 and 60 %.
If one were designing a print brochure for mass distribution, Potato Soup probably would not ‘make the cut’. The soup exemplifies several kinds of preparations that failed to draw a majority of fans, at least based on looking at just the recipes themselves.
In our project, however, because we could customise each pantry client’s flyer, it was easy to accommodate a choice such as soup. Based on WIC findings, and with the advantages of customisation at our disposal, we resolved to include a tailoring question in our protocol that asked about interest in soup-making. Quick! Help for Meals enabled us to provide a helpful recipe for just those cooks who wanted to make soups.
The popularity of the Chicken and Potatoes dish was instructive in a different way: the chart’s data and results with other recipes taught that meat–vegetable combinations are popular. Hence, Quick! Help’s protocol was expanded so that every vegetable includes a chicken or meat preparation.
Overall, across many evaluations and recipes, more than 80 % of the participants responded with the most optimistic and confident answers: (i) that they could obtain the ingredients easily; (ii) that the preparation seemed easy to do; (iii) that the photograph was helpful; and (iv) that they felt confident they could prepare this at home (regardless of whether they wanted to).
Where a recipe appeared problematic, it was either dropped or modified and tried with a new group of WIC cooks. Formative trials prevented disastrous mistakes.
Phase 3: Testing based on home-cooked experiences among WIC participants
Soliciting paper-and-pencil evaluations of recipes uncovers one level of judgement. Asking people to prepare the recipes and present servings to family members digs more deeply.
Staff invited WIC participants, who had not been involved in rating recipes, to accept two recipes and all the food that one would need to prepare them at home, including seasonings and oils. Quantities of food in each participant’s bag were consistent with her household’s size. WIC mothers were told to expect a telephone call in 3–4 days to learn how the cooking had gone and what household members thought about the dishes. They were promised a $20 gift certificate to a local store in appreciation, once the interview was completed.
We tested recipes in this way – a few at a time – scouring interviews for clues that would help us improve the servings or clarify how to prepare them. Phone questioning began by soliciting general comments or reactions to each recipe. Then, cooks were asked which household members had tasted the serving and whether or not each had liked it. Follow-up questions dealt with such matters as ease of preparation, whether or not the instructions were clear, whether the preparation time was accurate, whether the cook had changed anything during preparation, and whether she would make the dish again.
Responses from just twelve WIC mothers, among whom we assigned some of our more challenging recipes, illustrate the benefits to be gained from actual in-home cooking evaluations. With seventeen out of twenty-four of the preparations, cooks liked the dishes ‘a lot’ (and an equal share of other family members agreed). Nonetheless, these primary cooks were not shy about differentiating among household members, saying, for example, that her husband liked a dish just ‘a little’ and another adult or child liked it ‘a lot’ or ‘not at all’. Where negative reactions arose, the cooks were not at a loss for explanations: ‘it didn’t have enough flavour’, or ‘it was too crunchy’, or ‘it didn’t look pretty’. These responses helped in modifying or substituting recipes.
Almost uniformly, the participants considered the recipes ‘easy’ and the instructions ‘clear’. The investment in information design, based on these preparations at least, had paid off. We were especially concerned about our forecasts of preparation time. Would these be judged reliable or not? In nineteen out of twenty-four instances, cooks said the recipes had taken ‘about the same amount of time’ as promised. Where discrepancies arose, most preparations had actually required less time than recipes predicted.
Phase 4: Developing flyer layout using Velcro experiments
At this point, a wealth of recipes and tips had been assembled. But, what would be the preferred layout in a printed take-away flyer? Again we turned to testing.
We invented a research procedure using Velcro. We assembled all the recipes and tips about an individual vegetable (such as broccoli), mounting each recipe and tip on its own laminated card backed with a square of Velcro. Pantry clients were asked, one-by-one (and paid $20 for their time), to stick these cards onto felt boards that resembled pages of a flyer, in whatever pattern they wished. Participants discarded unwanted cards and expressed opinions about other things they wished had been provided. They moved cards around until satisfied with their ‘flyer’. We recruited seventeen pantry clients into these Velcro sessions.
In one respect, astonishing variability emerged from this stream of research, even when comparing participants alike in age, ethnicity, household composition and cooking experience. For example, a matched pair of participants often agreed on fewer than half the recipes and tips they wanted in their flyer, say about sweet potatoes. Participants differing in age or in other features agreed even less. (This individuality is what makes message tailoring such an appealing strategy for nutrition outreach.)
One common feature did emerge, however. Nearly all our subjects mounted recipes they desired on the opening pages of their hypothetical flyer and clustered tips they wanted on the closing pages. The flow of information on their ‘pages’ was strikingly similar.
Not surprisingly, we programmed Quick! Help for Meals to mimic this consensus about order of presentation. We already had prepared and tasted scores of recipes, and now we rendered the best of them in printed flyers that were informed by our design testing.
Discussion and implications for research and practice
This case study shares sample results from a series of formative research methods applied to the design of print materials in a nutrition outreach. Methods helped specify dozens of ways that the content and graphic design of the materials should appear.
We could not subject all design options to testing. As with nearly all formative research, the statistical power of tests we did conduct was weak. And our discoveries often rested on data from groups of people with unknown representativeness. Regardless of such limitations, though, the application of four differing formative methods was revealing.
Two of our methods were mainly verbal (focus groups and rating questionnaires) and two were largely behavioural (composing an ‘ideal’ document and home-testing of recipes). Each method contributed its own insights. And results from each strengthened our core premise: low-construal tools of an intervention should be pre-tested, as well as an intervention’s more abstract and over-arching purposes.
In short, the power of media hinges on informed decisions about even mundane issues. Strategic choices include layout, type font, use of white space (or silence and pacing in audio–visual presentations), inclusion and placement of pictures (or moving images), graphic devices such as line separators that chunk text into digestible units, order of presentation, and levels of literacy and numeracy assumed by the text or narrative. As others also have claimed, information architecture matters(Reference Wurman28). Graphic arts can play a central role in the social marketing of health behaviours(Reference Tyus, Freeman and Gibbons29).
It is inevitable that many results from formative studies of intervention tools remain largely sui generis, difficult to generalise to different health issues or circumstances. However, findings and the formative methods that led to them should provoke health educators to think in new directions. Hard-and-fast rules are scarce because user-centred design is in its infancy.
A few guiding principles have emerged. We know, for example, that using a difficult type font unconsciously signals to readers that tasks that the text promotes are harder to perform than if an easy-to-read font is used(Reference Song and Schwarz30). We understand some things about how combining pictures with language can increase comprehension of, or adherence to, recommended actions(Reference Houts, Doak, Doak and Loscalzo31). We have learned that use of red in a layout induces deep and detailed recall of content, whereas use of blue invites more expansive and association-rich processing of information(Reference Mehta and Zhu32). Some of these principles have identified or suspected roots deep in human biology or in culture(Reference Dunbar33, Reference Aslam34).
Even though the general applicability of lessons from formative tests of tools remains uneven, user-centred design is emerging as a field of practice. Contributions are mounting from the work of website creators and the graphic user interface community more broadly(Reference Vredenburg, Isensee and Righi35).
Questions raised by our formative evaluation methods may spur tool-builders of other nutrition interventions to examine their documents, or other media, more closely.
Acknowledgements
The present study was supported by the California Vitamin Cases Consumer Settlement Fund, the United States Department of Agriculture (CSREES grant 2006-55215-16730), the Ralph M. Parsons Foundation, and Kraft Foods. Research was performed by faculty and staff at the University of Southern California, Los Angeles, CA, USA. The authorship responsibilities are as follows: S.H.E. and P.C. were involved in all phases of this research; C.K. was involved in Phases 2 and 3 of the research. The authors have no conflict of interests to declare. We also thank Robyn Letter of Opinion Studies, Annette Besnilian, MHP, RD, and the Marilyn Magaram Center, California State University, Northridge.
