1 Introduction
Patients are often confronted with difficult medical decisions. Many of these decisions have to be made based on numerical information (e.g., information about chances and risks of treatments, see Reference Lipkus, Peters, Kimmick, Liotcheva and MarcomLipkus, Peters, Kimmick et al., 2010). Therefore, it is quite important that this information is understood correctly. Past research has shown that many people have difficulties understanding numerical risk information (Reference Gigerenzer and EdwardsGigerenzer & Edwards, 2003; Reference Visschers, Meertens, Passchier and de VriesVisschers, Meertens, Passchier et al., 2009), and that persons with low numeracy skills (the ability to understand numbers) are especially challenged by numerical information (Reference Lipkus and PetersLipkus & Peters, 2009; Reference PetersPeters, 2008). Therefore, not surprisingly, more and more studies show that low numeracy is associated with less understanding of medical information and unfavorable decision outcomes (see e.g., Reference Donelle, Arocha and Hoffman-GoetzDonelle, Arocha, & Hoffman-Goetz, 2008; Reference Schwartz, Woloshin, Black and WelchSchwartz, Woloshin, Black et al., 1997; Reference Tanius, Wood, Hanoch and RiceTanius, Wood, Hanoch et al., 2009; Reference Zikmund-Fisher, Fagerlin and UbelZikmund-Fisher, Ubel, Smith et al., 2008).
Different solutions have been proposed for improving the communication of medical information. Some authors suggest, for example, that numbers should be expressed as frequencies (Reference Gigerenzer and EdwardsGigerenzer & Edwards, 2003; Reference Hoffrage, Lindsey, Hertwig and GigerenzerHoffrage, Lindsey, Hertwig et al., 2000) or, especially for persons with low numeracy, conveyed in graphs (Reference Apter, Paasche-Orlow, Remillard, Bennett, Ben-Joseph, Batista, Hyde and RuddApter, Paasche-Orlow, Remillard et al., 2008; Reference Nelson, Reyna, Fagerlin, Lipkus and PetersNelson, Reyna, Fagerlin et al., 2008). One special type of graph combines these two recommendations for risk communication because the graph a) shows frequency information, and b) conveys numbers in a purely graphical way. These so-called pictographs show the number of people affected by a certain medical condition in a larger group of people (i.e., the denominator of mostly 100 or 1000, see Figure 1; for other examples of this type of graph, see also Reference Edwards, Elwyn and MulleyEdwards, Elwyn, & Mulley, 2002; Reference PalingPaling, 2003; Reference Schapira, Nattinger and McHorneySchapira, Nattinger, & McHorney, 2001). Therefore, this type of graph seems to be a promising tool for communicating medical information to persons with low numeracy.
Example for one of the conditions (lower probability, reference information present) used in Study 1.
Several studies show that pictographs help people with low numeracy understand medical information (Reference Galesic, Garcia-Retamero and GigerenzerGalesic, Garcia-Retamero, & Gigerenzer, 2009; Reference Hawley, Zikmund-Fisher, Ubel, Jancovic, Lucas and FagerlinHawley, Zikmund-Fisher, Ubel et al., 2008; Reference Zikmund-Fisher, Fagerlin and UbelZikmund-Fisher, Fagerlin, & Ubel, 2008). However, although pictographs seem to improve low-numerates’ direct understanding of the presented numbers (e.g., knowledge of how many persons are affected by a certain disease), it is not yet clear how this graph influences low-numerates’ risk perception. The influence of pictographs on risk perception, however, may be crucial because perceiving a risk as either high or low might have a greater impact on behavioral intentions than understanding the numerical information alone (Reference Zikmund-Fisher, Fagerlin, Keeton and UbelZikmund-Fisher, Fagerlin, Keeton, et al., 2007).
Generally, pictographs seem to evoke lower risk perceptions than other presentation formats such as the Paling perspective scale (Reference PalingPaling, 2003) or numerical frequencies (Reference Galesic, Garcia-Retamero and GigerenzerGalesic et al., 2009; Reference Keller and SiegristKeller & Siegrist, 2009; Reference Siegrist, Orlow and KellerSiegrist, Orlow, & Keller, 2008). Unfortunately, it is not possible to decide whether a reported risk perception is the “correct” one, because it is subjective in nature. To handle this difficulty, one can conduct an experiment to investigate whether different levels of probabilities evoke different levels of perceived risk (Reference Keller and SiegristKeller & Siegrist, 2009; Reference Siegrist, Orlow and KellerSiegrist et al., 2008). In this approach, participants are faced with either a higher or lower probability, and then estimate their perceived risk. We then analyze the extent to which participants confronted with the higher risk perceive the risk as higher than participants confronted with the lower risk. Results of two previous studies following this procedure using pictographs showed that a higher probability did not evoke a higher level of perceived risk than a lower probability (Reference Keller and SiegristKeller & Siegrist, 2009; Reference Siegrist, Orlow and KellerSiegrist et al., 2008). This result might suggest that, although some studies showed that pictographs seem to help persons with low numeracy to understand the numbers depicted in a graph, pictographs may not help them to evolve clearly distinguishable risk perceptions to the same degree. Thus, the type of task used in a study may influence the evaluation of pictographs. The role of numeracy in this perception process is, to our knowledge, not yet fully understood. We therefore conducted three studies to examine the influence of numeracy on people’s perceptions and, as a new approach to this question, on people’s processing of numerical medical information depicted in pictographs. To examine this issue, we chose the context of cancer screening test results, as some studies have shown that numeracy is important in this area (Reference Donelle, Arocha and Hoffman-GoetzDonelle et al, 2008 Reference Hanoch, Miron-Shatz and HimmelsteinHanoch, Miron-Shatz, & Himmelstein, 2010; Reference Schwartz, Woloshin, Black and WelchSchwartz et al., 1997).
Numeracy is defined as a person’s ability to understand and process numerical concepts (see e.g., Reference PetersPeters, 2008). It can be measured in two different ways. Objective measures assess people’s numeracy by letting them solve mathematical tasks (Reference Lipkus, Samsa and RimerLipkus, Samsa, & Rimer, 2001; Reference Schwartz, Woloshin, Black and WelchSchwartz et al., 1997). One problem with using such objective measures in mail-in surveys is that the respondents might use helping devices such as calculators. This would then bias the resulting numeracy score. Furthermore, respondents might find it annoying to fill in such questionnaires and, thus, might simply avoid them when they have the opportunity to do so (Reference Fagerlin, Zikmund-Fisher, Ubel, Jankovic, Derry and SmithFagerlin, Zikmund-Fisher, Ubel et al., 2007). To cope with this problem, Fagerlin and colleagues (2007) developed the subjective numeracy scale, which assesses self-reported numeracy skills. This measure offers the advantage of shorter administration and less reluctance from participants than objective measures (Reference Fagerlin, Zikmund-Fisher, Ubel, Jankovic, Derry and SmithFagerlin et al., 2007). On the other hand, this measure relies entirely on self-reported numerical ability and preference. Moreover, although it is positively correlated with objective numeracy (Reference Fagerlin, Zikmund-Fisher, Ubel, Jankovic, Derry and SmithFagerlin et al., 2007), it does not measure exactly the same construct as the direct measurement of mathematical skills in objective numeracy measures.
In short, the aim of our first study was to examine the influence of subjective numeracy on the perception of cancer screening test results presented in pictographs. Following Siegrist and Keller’s approach (Reference Keller and SiegristKeller & Siegrist, 2009; Reference Siegrist, Orlow and KellerSiegrist et al., 2008), we conducted an experiment to examine whether different levels of probabilities evoke different levels of perceived risk. To reach a deeper understanding into how pictographs might influence risk perception in relation to numeracy, we conducted a second study. We thereby directly examined the processing of cancer screening results depicted in pictographs and its association with numeracy. Finally, in Study 3, we explored the role of the sequence of the task (numerical understanding first vs. risk perception first) in the context of risk communication with pictographs and numeracy. With this manifold procedure, we aim to broaden the existing knowledge about numeracy in medical decisions by investigating the role of numeracy in risk perception. We aim to accomplish this by revealing the underlying process that might lead to differences between persons with higher and lower numeracy.
2 Study 1
In a previous study, pictographs showing either a higher or a lower probability test result did not evoke corresponding higher or lower risk perceptions among participants with higher or among participants with lower numeracy (Reference Keller and SiegristKeller & Siegrist, 2009). This finding could suggest that this type of graph does not evoke differentiated risk perceptions for different probability levels, irrespective of an individual’s numeracy. However, several other possible explanations exist for the pictograph’s lack of effect. Therefore, Study 1 aimed to rule out some of the possible factors that could have impeded the successful communication of different probability levels in this previous study (Reference Keller and SiegristKeller & Siegrist, 2009). We investigated whether modified pictographs could evoke differentiating risk perceptions in persons with higher and lower numeracy.
First, the size of the denominator of a pictograph may influence risk perception and understanding of the information in the graph (Reference Galesic, Garcia-Retamero and GigerenzerGalesic et al., 2009; Reference Hawley, Zikmund-Fisher, Ubel, Jancovic, Lucas and FagerlinZikmund-Fisher, Ubel, et al., 2008). Reference Keller and SiegristKeller and Siegrist (2009) used rather low risks depicted in pictographs with large denominators (1 in 1000, 9 in 1000, 21 in 1000, 167 in 1000). In a focus group study about the perception of different formats of risk communication, participants preferred pictographs with small denominators to pictographs with large denominators because the participants found the pictographs with small denominators easier to interpret (Reference Schapira, Nattinger and McHorneySchapira et al., 2001). The denominator in Reference Keller and SiegristKeller and Siegrist’s (2009) study may thus have been too large to efficiently depict such low risks because the large denominator of 1000 complicated the processing of the graphs. This might have made all of the risks seem equally low, even for persons with high numeracy. This mechanism could then have overshadowed a potentially beneficial effect of the pictographs. We therefore chose a smaller denominator in our study. More specifically, we aimed to investigate whether two levels of probability depicted in a pictograph of 100 icons led to different risk perceptions. We hypothesized that persons confronted with a higher probability would report a higher risk perception than when confronted with a lower probability (Hypothesis 1). We expected a similar effect for persons with higher and lower numeracy.
Another factor that may influence the decision of whether a given probability is high or low may be the absence of additional information that puts a risk in a broader context (see Reference LipkusLipkus, 2007). In everyday life, comparing one’s own risks to those of others seems to be done automatically: when faced with test results in a medical context, people compare their personal test results to what is communicated to them as the normal value (Reference Adelsward and SachsAdelsward & Sachs, 1996). In a study by Reference Dillard, McCaul, Kelso and KleinDillard and colleagues (2006), providing women with reference information in the form of higher risks of other women helped them to avoid overrating their own breast cancer risk. Furthermore, graphical or numerical reference risk information seemed to enable people to differentiate between a higher and a lower risk (Reference Siegrist, Orlow and KellerSiegrist et al., 2008). Thus, in Reference Keller and SiegristKeller and Siegrist’s (2009) study, pictographs may have failed to convey the difference between higher and lower risk because of the lack of reference information. Adding reference information to the pictograph could therefore help to evoke a differentiating risk perception. Hence, we hypothesized that participants confronted with a higher probability would report higher risk perceptions compared with participants confronted with a lower probability when there is a second pictograph with reference information (Hypothesis 2). We expected to observe the same effect for persons with higher and lower numeracy.
2.1 Method
2.1.1 Participants and procedure
Study 1 was an experiment that was part of a survey about health and nutritional information. The topic of the experiment reported here was different from the survey’s other content. Therefore, no carry-over effects were expected. The questionnaire was sent to a sample of households in the German-speaking part of Switzerland. The households were randomly chosen from the Swiss telephone book. In total, 589 questionnaires were returned, which resulted in a response rate of 38%. Of these 589 questionnaires, 56 were not completely filled in with regard to numeracy or the dependent variable of the experiment. Therefore, our analyses were based on the responses of 533 participants. Of the 533 participants, 296 (56%) were women; three persons did not specify their sex. Respondents were between 17 and 94 years old (M = 53.32 years, SD = 15.69). In our sample, 42 persons (8%) had finished primary or lower secondary school; 256 (48%), upper secondary vocational school; 79 (15%), upper secondary school; and 151 (28%), university/technical university. Five persons did not provide information about their educational level.
All respondents read the same hypothetical scenario about a woman (“Daniela”) who had a screening test for colorectal cancer. The doctor used a personalized pictograph to inform her about the test results. This pictograph was shown in the questionnaire and consisted of an array of 100 icons (10×10) with grey and white icons, which represented the probability of Daniela having colon cancer and the probability that she did not have cancer, respectively (see Figure 1). At the end of the scenario, all participants estimated the risk of Daniela having cancer on a 6-point scale (1 = very low probability to 6 = very high probability). This part of the procedure was the same for all participants.
Three factors were used for a 2 (probability level: higher vs. lower) × 2 (presence of reference information: yes vs. no) × 2 (subjective numeracy: higher vs. lower) between-subjects design. The level of the depicted probability for Daniela having colon cancer was either 17 in 100 (higher) or 2 in 100 (lower). Each participant thus saw either the lower- or higher-probability graph. This manipulation allowed us to analyze whether the participants’ risk perceptions differed across the two levels. Furthermore, half of the participants saw only the graph for Daniela’s probability (i.e., only the top half of Figure 1 consisting of the first text and the first graph), whereas the other half received reference information in the form of a second pictograph (see Figure 1). In this reference graph, we additionally depicted the average probability of a woman of the same age as Daniela having colon cancer (4 in 100). These manipulations resulted in four versions of the questionnaire, to which the participants were randomly assigned.
Because Study 1 was a mail-in survey, we could not directly control whether the respondents used calculators and filled in all questions. Therefore, we used the subjective numeracy scale (SNS, Reference Fagerlin, Zikmund-Fisher, Ubel, Jankovic, Derry and SmithFagerlin et al., 2007) to measure the participants’ numeracy. Another reason for using the SNS was that we assumed that more respondents would return the questionnaire with this scale than with an objective numeracy scale. The SNS is a self-reported measure of one’s ability to handle numbers, as well as one’s preference for numbers. The scale consists of 8 items (e.g., “How good are you at working with percentages?” “How often do you find numerical information to be useful?” assessed on 6-point scales) and results in an average numeracy score from 1 (low numeracy) to 6 (high numeracy).
2.1.2 Data analysis
To test whether probability level, reference information, and subjective numeracy influence risk perception, we utilized an analysis of variance (ANOVA). For ease of interpretation, we performed a median split on the subjective numeracy measure (higher vs. lower numeracy). However, as subjective numeracy is a continuous variable, we conducted an additional analysis of covariance (ANCOVA) with probability estimate as the dependent variable, the probability level and reference information as factors and subjective numeracy as a continuous covariate. All statistical analyses were performed with SPSS version 17.0 (SPSS, Inc.).
2.2 Results
Mean subjective numeracy was 4.17 (SD = .87, scale 1–6); the internal consistency of the SNS was good (8 items; Cronbach’s alpha = .82). We performed a median split on subjective numeracy (Mdn = 4.25), which resulted in a higher-numeracy group (n = 279) and a lower-numeracy group (n = 254).
The ANOVA showed that reference information did not play a significant role for risk perception, either as a main effect, F(1, 525) = .25, p = .62, or as an interaction effect with one or both of the other factors, Fs ≤ .89, ps ≥ .35. We found significant main effects for probability level, F(1, 525) = 20.82, p < .001, as well as for subjective numeracy, F(1, 525) = 4.66, p = .03, and a significant interaction effect for probability level × subjective numeracy, F(1, 525) = 4.82, p = .03.
Table 1 shows the average risk perception for each of the eight cells of the experiment. Planned independent t-tests following Hypotheses 1 and 2 showed that persons with higher numeracy differentiated between the higher and the lower probability levels, irrespective of whether there was reference information or not (see Table 1). Although the interaction effects with reference information were non-significant, there was an interesting and rather counter-intuitive t-test result in the lower numeracy groups. When there was no reference information present, persons with lower numeracy seemed to differentiate between higher and lower probabilities. However, when there was a reference information graph, persons with lower numeracy who had seen the lower risk did not have different risk perceptions than persons with lower numeracy who had seen the higher risk (see Table 1).
Note: 6-point scale: 1 (very low) — 6 (very high).
The ANCOVA showed a significant main effect for subjective numeracy, F(1, 525) = 4.06, p = .05, and a significant interaction effect of probability level × subjective numeracy on risk perception, F(1, 525) = 5.25, p = .02. All other effects, including the main effect for probability level, were not significant in the ANCOVA, Fs ≤ 1.72, ps ≥ .19. The interaction numeracy × probability level was thus significant in both analyses.
In sum, the analyses showed that persons with higher subjective numeracy differentiated between the two probability levels, whereas persons with lower subjective numeracy did not, or at least not to the same degree. Adding reference information did not significantly influence the participants’ risk perceptions in the multivariate analyses. However, planned comparisons revealed a tendency for reference information to impede the ability of persons with lower numeracy to have different risk perceptions.
2.3 Discussion
The results of Study 1 suggest that persons with higher subjective numeracy perceived more risk when confronted with a higher probability than when confronted with a lower probability. These results are partly in line with our first hypothesis. This contradicts the results of a previous study that suggest that pictographs neither influence risk perception for persons with high numeracy nor for persons with low numeracy (Reference Keller and SiegristKeller & Siegrist, 2009). This also seems to imply that pictographs can be useful for evoking a meaningful risk perception when some aspects of the pictograph are changed (probability level, size of denominator).
Adding reference information, however, changed this picture in an interesting and rather surprising way. Persons with higher subjective numeracy differentiated between the higher and the lower probability irrespective of the presence of reference information. For persons with lower subjective numeracy, on the other hand, reference information seemed to actually limit the perception of the difference between the higher and the lower probabilities. Our second hypothesis was thus not confirmed for the lower numeracy groups. On the contrary, our results seemed to suggest that pictographs that include reference information are not suitable for communication with persons with lower numeracy.
One possible explanation for this rather surprising impact of reference information may be explained by the fact that pictographs depict numerical information, albeit graphically illustrated, and that people may also tend to treat the pictographs like numerical information. According to Reference PetersPeters’ (2008) model of numeracy and the comprehension and use of numeric risk information, persons with high numeracy focus more on numerical information and draw more meaning from numbers than persons with low numeracy. Therefore, persons with higher numeracy may focus on the depicted numbers when they are looking at the pictographs so that the graphical reference information may actually lead to the mere comparison of two numbers for this group. Persons with lower numeracy, on the other hand, may not focus on these numbers. This, in turn, may have impeded the explanatory power of the pictographs with reference information, especially in the lower probability condition where target probability (2 in 100) and reference information (4 in 100) were rather close together. Therefore, it is possible that this different manner of processing the graph has led to different risk perceptions between these two groups. To test the idea that persons with higher numeracy pay more attention to the numerical information in pictographs than persons with lower numeracy, we conducted Study 2.
3 Study 2
We suggest that pictographs can be processed in different ways. Either one counts the icons and calculates how many persons are affected (focus on the numbers “behind” the graph), and/or one compares the marked icon area with the unmarked icon area (holistic processing of the graph). According to Reference PetersPeters’ (2008) model of numeracy and the comprehension and use of numeric risk information, persons with higher numeracy focus more on, and pay more attention to, numerical information than persons with lower numeracy. This implies that persons with higher numeracy may pay more attention to the actual numbers “hidden behind the pictograph”, whereas persons with lower numeracy process the pictograph on a more holistic level. To test this idea, we analyzed interviews with laypeople about the processing of a pictograph (10x10 icons) in regard to counting the icons. We expected that higher numeracy would be related to a higher tendency to count the icons and to look for the actual numbers depicted in the graph.
3.1 Method
Study 2 consisted of face-to-face interviews with 52 persons from the general population. These interviews were conducted in the context of a larger study about the processing of various graphical risk communication formats. The participants were recruited from an earlier study in which they had been asked whether they would participate in this study. Fifty-two persons agreed to participate (participation rate = 66%). Participation took about one hour (approximately 12 minutes of this hour were dedicated to the pictograph) and was financially compensated. Of the 52 participants, 16 (31%) were women. Respondents were between 22 and 73 years old (M = 52.25 years, SD = 13.95). Four (8%) had finished lower secondary school, 17 (33%) upper secondary vocational school, 6 (11%) upper secondary school, and 25 (48%) university/technical university.
The study took place in our test laboratory. All participants read a hypothetical scenario on a 15.4-inch computer screen about a man (“Hans”) who had a screening test for lung cancer. As in Study 1, the test result was communicated with a personalized pictograph (10x10 icons). The depicted probability for Hans having lung cancer was 14%, visualized as 14 marked icons in 100 (see Figure 2). All participants read the same scenario and looked at the same graph with the task to estimate the depicted probability level. After this, the experimenter conducted an interview about the processing of the graph, particularly dealing with the question about whether the participant had counted the icons or not. The interviews were then transcribed and coded as either 1, meaning the icons were counted, or 0, meaning the icons were not counted (variable “counting”). Furthermore, we analyzed the transcripts in regard to whether the participants had spontaneously mentioned (i.e., without us asking for this information) the numbers depicted in the graph in the form of percentages or frequencies (coded as 1 “yes” or 0 “no”; variable “mentioning numbers”).
Pictograph used in Study 2
To measure numeracy, we applied the same subjective numeracy scale as in Study 1 (Reference Fagerlin, Zikmund-Fisher, Ubel, Jankovic, Derry and SmithFagerlin et al., 2007) and a short and modified version of the objective numeracy scale used by Reference Hawley, Zikmund-Fisher, Ubel, Jancovic, Lucas and FagerlinLipkus and colleagues (2001). Because there had been ceiling effects when the original tasks were used in a Swiss sample (Reference Keller and SiegristKeller & Siegrist, 2009), we made the tasks more difficult to achieve a more balanced distribution of scores.Footnote 1 The scale we used consisted of seven mathematical tasks, and resulted in a minimum score of 0 and a maximum score of 7. Despite these changes, the distribution was negatively skewed. The mean subjective numeracy was 4.40 (SD = .74), and the mean objective numeracy was 5.44 (SD = 1.50). The internal consistencies of both scales were acceptable, with Cronbach’s alpha = .81 (8 items) and .63 (7 items), respectively, and the two measures showed a significant positive correlation, r = .44, p = .001. All statistical analyses were performed with SPSS version 17.0 (SPSS, Inc.).
3.2 Results
Thirty-four participants (65%) reported having counted the icons, and 14 (27%) reported that they had not counted the icons to estimate the probability. Four participants (8%) did not provide any or only unequivocal information about having counted the icons or not. Thirty-one participants (60%) spontaneously mentioned the depicted numbers either as percentages or as frequencies, whereas 21 (40%) did not mention the exact numbers.
The correlations between these two variables and subjective/objective numeracy are shown in Table 2. As expected, numeracy was associated with counting the icons and mentioning the numbers depicted in the graph. However, counting the icons was only significantly associated with objective numeracy, and mentioning the numbers was only significantly correlated with subjective numeracy. Subjective numeracy can be further broken down into ability and preference subscales (Reference Fagerlin, Zikmund-Fisher, Ubel, Jankovic, Derry and SmithFagerlin et al., 2007). Doing this showed that mentioning the numbers was correlated with the ability scale (r = .44, p = .001), but not with the preference subscale (r = .15, p = .30). Neither the ability nor the preference subscale was significantly associated with counting the icons (rs ≤ .16, ps ≥ .27).
Note: * p < .05
3.3 Discussion
The results of Study 2 supported our hypothesis. We found that persons with higher objective numeracy counted the icons slightly more often than persons with lower objective numeracy, and persons with higher subjective numeracy were more likely to mention the numbers depicted in the graph than persons with lower numeracy. This finding is in line with previous research highlighting that, overall, persons with higher numeracy focus more on numerical information and draw more meaning from these numbers than persons with lower numeracy (Reference PetersPeters, 2008). We assume that persons with lower numeracy may perceive the graph rather holistically (e.g., comparing the areas of the graph or judging the graph by a gut-feeling) because they pay much less attention to the numerical information than persons with higher numeracy.
Further analyses of the subscales of subjective numeracy showed that it was the self-reported ability and not the self-reported preference that was associated with the processing of the graph. Thus, it does not seem to be the liking of numbers that is related to the perception of the graph, but numeracy in the narrower sense, namely people’s mathematical skills.
In sum, Study 1 suggested that pictographs are useful for both persons with higher and lower subjective numeracy. However, this effect seems to be more stable for persons with higher numeracy because they differentiated between the higher and the lower probability, irrespective of the presence of a reference information graph. Study 2 implied that persons with higher numeracy seem to concentrate more on the numbers “behind the pictograph” than person with lower numeracy. Taken together, all of these results suggest that it may be useful to prompt persons with lower numeracy to count the icons of the pictograph or to focus on the number depicted in the graph to make the positive effect of pictographs also more stable for this group. To gain further insight into this relationship between processing pictographs and numeracy, we conducted Study 3.
4 Study 3
Prompting persons to count the icons of a pictograph may be effectively accomplished by carefully choosing the tasks that participants have to solve. On the one hand, a risk perception task, as we used in Study 1 (e.g., “how high is this probability?”), probably does not trigger a special type of processing. We therefore expect participants to choose their default way of processing the pictograph. Based on the results of Study 2, we assume that the default way of processing is focusing on numbers and counting the icons for persons with higher numeracy and perceiving the graph rather holistically for persons with lower numeracy. On the other hand, a numerical understanding task, such as those used in previous studies (e.g., “how many people are affected?”), may trigger all participants to count the icons because the answer to this question is an explicit number (see Reference Galesic, Garcia-Retamero and GigerenzerGalesic et al., 2009; Reference Hawley, Zikmund-Fisher, Ubel, Jancovic, Lucas and FagerlinHawley et al., 2008; Zikmund-Fisher, Fagerlin et al., 2008).
To test whether inducing a focus on the numerical information in the graph influences participants’ risk perceptions, we conducted Study 3. We had the following two hypotheses. First, we expected that persons with lower and higher numeracy who are not triggered to count the icons would differentiate between a lower and a higher probability depicted in a pictograph (i.e., replication of the effect in Study 1; Hypothesis 1). Second, we expected that persons with lower numeracy who were triggered to count the icons would differentiate more strongly between a higher and a lower probability than persons with lower numeracy who had not been triggered to count the icons (Hypothesis 2). We did not expect this effect for persons with higher numeracy, because their default way of processing the graph may be counting the icons. Therefore, we assumed that prompting to count the icons would have no further effect on this group of persons.
4.1 Method
4.1.1 Participants and procedure
An online questionnaire was sent to a panel of Swiss households and was answered by 601 persons. We excluded eleven of the respondents because their data were incomplete in regard to risk perception or numeracy. Our analyses were thus based on the answers of 590 participants. Of these, 304 were women (52%). The participants were between 18 and 69 years old (M = 38.69 years, SD = 12.42). Thirty-two respondents (5%) had finished primary or lower secondary school; 248 (42%) upper secondary vocational school; 142 (24%) upper secondary school; and 168 (29%), university/technical university.
We used the same scenario and the same graphs as in the no-reference conditions of Study 1 (upper part of Figure 1). On the first screen, all respondents read the hypothetical text about “Daniela” who had been tested for colon cancer and who had the test results communicated to her by means of a 10x10-pictograph with grey and white icons representing Daniela’s probability of colon cancer. Three factors were used for a between-subjects design in this study. First, as in Study 1, the level of probability participants saw on the screen was either lower (2 in 100) or higher (17 in 100). Following this procedure, we again examined whether the participants differentiated between the lower and higher probabilities in their risk perception. The second factor we manipulated was the order of the two tasks to test whether a task that triggers the counting of the icons influences how a risk is perceived. The two tasks were: a) a risk perception task with the question, “How high do you estimate the probability of Daniela having colon cancer?” and b) a numerical understanding task with the question, “How many people similar to Daniela have cancer?”. The latter question was intended to trigger the counting of the icons. Half of the participants saw the risk perception task first and then, on a second screen, the numerical understanding task (the “non-counting first” condition). The other half saw the numerical understanding task first and then the risk perception task (the “counting first” condition). This procedure resulted in four versions of the online questionnaire to which the participants were randomly assigned.
As a third factor, we took participants’ subjective numeracy into account, measured with the SNS (Reference Fagerlin, Zikmund-Fisher, Ubel, Jankovic, Derry and SmithFagerlin et al., 2007, see Study 1 for details about the scale). Using a median split, the respondents were again divided in two groups: a higher numeracy group and a lower numeracy group. Because this study was a self-administered online questionnaire, we did not measure objective numeracy for the same reasons as in Study 1, namely lack of control regarding whether the participants used a calculator, and an increased percentage of drop-outs with objective numeracy.
4.1.2 Data analysis
Again, we used the same analyses as in Study 1. For ease of interpretation, we performed a median split on subjective numeracy and included these two numeracy groups (higher vs. lower), probability level (higher vs. lower) and task sequence (counting first condition vs. non-counting first condition) in an analysis of variance (ANOVA) with the probability measure from the risk perception task as the dependent variable. To test which of the cells were significantly different, we used independent t-tests. However, as subjective numeracy is a continuous variable, we conducted an additional analysis of covariance (ANCOVA) with probability estimate as the dependent variable, the probability level and task sequence as factors and subjective numeracy as a continuous covariate. All statistical procedures were performed with SPSS version 18 (SPSS, IBM corp.).
4.2 Results
Mean subjective numeracy was 4.11 (SD = .90, scale 1–6); the internal consistency of the SNS was good (8 items; Cronbach’s alpha = .83). We performed a median split on subjective numeracy (Mdn = 4.25), which resulted in a higher-numeracy group (n = 296) and a lower-numeracy group (n = 294).
The ANOVA showed significant main effects for task sequence, F(1, 582) = 9.08, p = .003, and for probability level, F(1, 582) = 40.16, p < .001. Furthermore, we found a significant 3-way interaction effect between numeracy, probability level, and task sequence, F(1, 582) = 5.53, p = .02. All other effects were non-significant, Fs ≤ 2.19, ps ≥ .14. Planned t-tests showed that persons with higher subjective numeracy who had seen the higher probability judged this probability to be higher than persons with higher subjective numeracy who had seen the lower probability, irrespective of which task was first (see Table 3). For persons with lower subjective numeracy, on the other hand, whether they had to solve the numerical understanding task first or the risk perception task affected their perceptions. Only persons with lower subjective numeracy who were in the non-counting first condition (risk perception task first) showed a significant difference between the risk perceptions of the two probability levels. The risk perceptions of persons with lower numeracy in the counting first condition did not differ between the lower and the higher probability level.
Note: 6-point scale: 1 (very low) — 6 (very high).
The ANCOVA confirmed the 3-way interaction effect between task sequence, risk level and subjective probability, F(1, 582) = 4.77, p = .03. Furthermore, there was a significant 2-way interaction effect between task sequence and risk level, F(1, 582) = 4.52, p = .03. All other effects in the ANCOVA were non-significant, Fs ≤1.86, ps ≤ .17.
Furthermore, 257 of the 296 persons with higher numeracy (87%), and 217 of the 294 participants with lower numeracy (74%) gave the correct answer to the numerical understanding task (“2” or “17”). Significantly more persons with higher numeracy solved this task correctly than persons with lower numeracy, χ 2 (1, N = 590) = 15.82, p < .001. Hence, this confirmed the assumption that people with higher numeracy are better able to solve numerical problems than those with lower numeracy.
To check whether the results of the ANOVA described above were influenced by whether the participants had correctly answered the numerical understanding task, we recalculated the analyses, this time only including participants who had given the correct answer. This procedure did not change the results: The main effects for task sequence, F(1, 466) = 15.80, p < .001, and probability level, F(1, 466) = 67.50, p < .001, remained significant, as well as the 3-way interaction effect, F(1, 466) = 8.78, p = .003.
Overall, the analyses showed that task sequence was important for persons with lower numeracy to differentiate between the higher and the lower probability. In the lower numeracy group, solving the risk perception task first seemed to result in different risk perceptions in line with the different probability levels, whereas when persons with lower numeracy had to solve the numerical understanding first, they seemed to perceive the risks of the higher and lower probabilities as similar. Persons with higher subjective numeracy, on the other hand, differentiated between a higher and a lower probability irrespective of task sequence.
4.3 Discussion
We replicated the effect from Study 1 by showing that pictographs are a useful tool to evoke differentiating risk perceptions in persons with higher and lower numeracy. We were thus able to confirm our first hypothesis. However, our expectation concerning the second hypothesis was not met by our data. We expected that triggering persons with lower numeracy to count the icons would lead to a larger difference in risk perceptions. However, in contrast, the results of Study 3 suggest that guiding people with lower numeracy towards counting the icons of a pictograph may impede their ability to draw meaningful information from this type of graph. Persons with lower numeracy may draw the meaning of the information directly from the pictograph without focusing too much on the numbers behind the graph, when they are not prompted to count the icons first. However, when they are stimulated to count the icons first, they may have the exact number in mind. In this case, they may not be able to draw meaning from this number because of their lower numeracy skills (see Reference PetersPeters, 2008), so that their risk perceptions are not affected by the probability levels. This mechanism would explain why pictographs are useful for people with lower numeracy to understand medical information numerically (e.g., knowledge about how many people are affected by a certain disease, Reference Galesic, Garcia-Retamero and GigerenzerGalesic et al., 2009; Reference Hawley, Zikmund-Fisher, Ubel, Jancovic, Lucas and FagerlinHawley et al., 2008; Zikmund-Fisher, Fagerlin et al., 2008) but that the mechanism becomes more complex when it comes to evoking differentiating risk perceptions (Reference Keller and SiegristKeller & Siegrist, 2009). For persons with higher numeracy, focusing on the numbers depicted in a graph seems to be intuitive and advantageous, whereas this procedure may be rather counter-intuitive and impeding for persons with lower numeracy.
5 General discussion
Researchers have recommended using graphical displays such as pictographs to improve communicating risk to persons with low numeracy (Reference Apter, Paasche-Orlow, Remillard, Bennett, Ben-Joseph, Batista, Hyde and RuddApter et al., 2008; Reference Nelson, Reyna, Fagerlin, Lipkus and PetersNelson et al., 2008). Our results suggest that pictographs might be useful for persons with higher and lower numeracy—but for different reasons and under different conditions. To use pictographs for effective communication, it is helpful to understand these reasons and conditions. Our results imply that persons with higher numeracy may profit from this type of graph because they more often draw the exact numbers from it and turn these numbers into a subjective risk perception that enables them to differentiate between higher and lower levels risk. Thus, one could also provide this group with the numbers alone and the effect would probably be comparable. Persons with lower numeracy, on the other hand, seem to process this kind of graph differently. They seem to rely on a different type of information, and not on the numbers “hidden in the graph”. This is in line with Reference PetersPeters’ (2008) model of numeracy and the comprehension and use of numeric risk information. Even more, our results imply that guiding individuals with lower numeracy towards attending to the numbers in the graph may even be counterproductive and confusing for this group. All in all, our results suggest that pictographs for persons with lower numeracy should be as simple as possible to facilitate a processing of the graph that is relatively unaffected by numerical information or calculations. Some additional verbal information about the meaning of the information depicted in the pictograph, e.g., in the form of verbal labels, could also be useful for persons with lower numeracy to ease the understanding of this information (see Peters, Dieckmann, Mertz et al., 2009). However, this should be done carefully because labeling numbers might influence a person’s behavioral intentions (Reference Zikmund-Fisher, Fagerlin, Keeton and UbelZikmund-Fisher et al., 2007).
Overall, our studies provided rather clear indications of which information persons with lower numeracy do not rely on when they look at pictographs: namely, the numbers. However, we could only assume which information they do rely on to build up their risk perceptions. Based on the assumption that there are two basic ways of processing pictographs (focus on numbers and holistic processing), in Study 2, we suggested that persons with lower numeracy might perceive the graph rather holistically. However, further studies are needed that explore the crucial parts of information that are used by persons with lower numeracy to build their risk perceptions.
Both Studies 1 and 3 showed differences between persons with higher and lower numeracy in the lower probability condition, whereas the two groups gave rather similar answers in the higher probability conditions. Our procedure does not provide information about correct or incorrect answers because risk perception is subjective and, therefore, cannot be right or wrong. Thus, we cannot conclude from our results that persons with lower numeracy understand small probabilities less than persons with higher numeracy. However, we can conclude that lower probabilities rather than higher probabilities seem to be processed and judged differently by persons with higher and lower numeracy. Further studies are needed to shed more light on this crucial aspect of communicating risk to persons with lower numeracy.
Finally, some methodological issues and limitations of our studies should be considered. First, we had three rather different samples with regard to socio-demographic variables. Furthermore, the level of risk perception was higher in the first than in the third study although we used the exact same scenario. As the samples were quite similar in regard to numeracy levels, the discrepancy between the risk perceptions in Studies 1 and 3 can, therefore, probably be explained by the lower mean age of the sample in Study 3. The lower age in Study 3 can, in turn, be the result of the type of data gathering (online vs. paper-pencil questionnaire). Second, we used only one special type of pictograph in all studies. However, the pictographs’ characteristics, for example, the denominator or the order of the marked icons, can influence the perception and understanding of the depicted information (Reference Feldman-Stewart, Kocovski, McConnell, Brundage and MackillopFeldman-Stewart, Kocovski, McConnell et al., 2000; Reference Galesic, Garcia-Retamero and GigerenzerGalesic et al., 2009; Reference Zikmund-Fisher, Fagerlin and UbelZikmund-Fisher, Ubel et al., 2008). We, therefore, do not know whether our results can be generalized to all types of pictographs. Third, Study 2 was a qualitative and explorative study using unstructured interviews and a small sample with more men than women. Therefore, these results should be interpreted with caution and be confirmed in a larger and more representative sample. However, we think Study 2 provides an important and, above all, new input for the interpretation of pictographs by directly examining the processing of the graph rather than solely the understanding of the depicted information. Finally, we measured numeracy in Studies 1 and 3 with only the subjective numeracy scale, and not with an objective measure. As Study 2 showed, the two measures are significantly correlated, but not very highly. It is unclear whether we would have found the same results with an objective numeracy scale.
Open access
