In this comment, I attempt to understand the differences between the results of Shafir (Reference Shafir1993) and its ‘very close replication’ by Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) (see Brandt et al., Reference Brandt, Jzerman, Dijksterhuis, Farach, Geller, Giner-Sorolla and Van't Veer2014 for a discussion of very short replication). Both studies examined the effect of the framing of a selection task in terms of choosing the best alternative versus framing it in terms of rejecting the worst alternative. In both studies, participants were presented with the same seven two-alternative and one three-alternative selection tasks and were asked either to choose or to reject one of the alternatives, where one of the alternatives was an enriched alternative, an alternative with high attribute variability (i.e., both very positive and very negative attributes), and the other an impoverished alternative, an alternative with low attribute variability. In the original study, Shafir (Reference Shafir1993) found that in all the eight selection tasks, the enriched alternative was preferred significantly more than the impoverished alternative when the selection was presented in terms of choosing than when it was presented in terms of rejecting. Shafir understood these results as providing support to a compatibility hypothesis, which suggests that there is a stronger tendency to assign heavier weight to attributes (e.g., positive versus negative attributes) that are compatible with the response mode (choose versus reject, respectively) than to attributes which are not compatible to the response mode (reject versus choose, respectively).
Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) obtained the same results as Shafir in only two of the eight tasks. In two other tasks, they found the opposite pattern, and in four of the tasks, they found no significant differences between the accept and reject task. An even more contradictory finding was that of Klein et al. (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and Sowden2018), who attempted to replicate only one of Shafir’s eight tasks and found with regard to this task the opposite of what Shafir found (Klein et al., Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and Sowden2018, even note that among 28 studies they attempted to replicate, only three were in the opposite direction, and of these three, the replication of Shafir, Reference Shafir1993, had by far the strongest effect size). Finally, as Wedell (Reference Wedell1997) noted, Ganzach (Reference Ganzach1995) also reported results that are diametrically opposite to Shafir’s.
In an attempt to make sense of the contradictory results obtained by Shafir (Reference Shafir1993) and Ganzach (Reference Ganzach1995), Wedell (Reference Wedell1997) noted that in Shafir’s selection tasks, the enriched alternative was overall more attractiveFootnote
1
than the impoverished alternative, whereas in Ganzach’s (Reference Ganzach1995) tasks, the opposite was the case. Furthermore, he ingeniously suggested that, both in Shafir’s data and in Ganzach’s data (as well as in additional data he collected), the choice–rejection framing effect can be explained by the fact that the difference between alternatives is accentuated in the choice task in comparison to a rejection task (the accentuation hypothesis), leading preference toward the more attractive alternative to be higher in choice than in rejection.
There are two explanations for accentuation. Wedell’s (Reference Wedell1997) original explanation is motivational. It suggested that accentuation is the result of the decision maker’s need to justify his/her decision when choosing but not when rejecting. Thus, if alternative A is inherently viewed as better [worse] than B, the need to justify the decision leads the advantages [disadvantages] of alternative A to loom larger in a choice frame than in a rejection frame.
Chen and Proctor (Reference Chen and Proctor2017) offered a cognitive-effort explanation for accentuation. According to their explanation ‘Compared to the selection task, the rejection task is more difficult and leaves fewer cognitive resources available for discriminating the alternatives’ (p. 566), which results in people paying less attention to the differences between the alternatives. This results in worse discrimination between alternatives in the rejection task than in the choice task. Stated differently, rejection tasks are more noisy and therefore more regressive, and it is this regressiveness that leads to the choice–rejection framing effect (see the appendix for a formal development of this argument).
In trying to understand the difference between Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) and Shafir (Reference Shafir1993), we first note that compatibility and accentuation are not necessarily two competing explanations for the choice/rejection framing effect. They are better viewed as two independent processes that influence this effect. The effect of compatibility depends on the variability of the attribute values of the alternatives (i.e., their degree of enrichment/impoverished), and the effect of accentuation depends on mean values of these attributes (i.e., their attractiveness) (see Ganzach, Reference Ganzach1995 for this framework for analyzing alternatives’ characteristics). Whereas compatibility is reflected in the effect of attribute variability on the difference between choosing and rejecting, accentuation is reflected in the effect of attribute mean on the difference between choosing and rejecting. This view suggests that the effect of the choice/rejection framing on the preference for the enriched alternative depends on the relative influence of these two processes. If compatibility is dominant, a stronger preference for the enriched alternative in choice than in rejection is observed. On the other hand, if accentuation is stronger, a stronger preference for the attractive alternative in choice than in rejection is observed.
Based on this analysis, we suggest that Chandrashekar et al.’s (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) failure to replicate Shafir’s compatibility results in the choice/rejection tasks, coupled with the clear accentuation effect they found in their ‘extensions’ (i.e., analyses based on Wedell’s (Reference Wedell1997) analytical approach), is consistent with Chen et al.’s (Reference Chen and Proctor2017) cognitive-effort view of accentuation. Chandrashekar et al.’s (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) Mturk participants are very different from Shafir’s Princeton students’ participants, in that the former were likely to have less cognitive resources for discriminating between the alternatives in the more difficult rejection task. First, the motivation of Mturk workers to invest cognitive resources in performing experimental tasks is relatively low (Peer et al., Reference Peer, Rothschild, Gordon, Evernden and Damer2022), and this motivation event deteriorated during the last few years (Kennedy et al., Reference Kennedy, Clifford, Burleigh, Waggoner, Jewell and Winter2020); and second, their verbal ability, a talent which is necessary to extract information from written texts, is likely to be lower than that of the Ivy League students who participated in Shafir’s experiment (see Hauser et al., Reference Hauser, Paolacci and Chandler2019 for information about the characteristics of Mturk participants). This difference in cognitive resources between Chandrashekar et al.’s (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) and Shafir’s (Reference Shafir1993) participants could have resulted in accentuation [compatibility] exerting stronger [weaker] influence on the preferences in choice and rejection.
From this perspective, it is interesting to compare the tasks in which Chandrashekar et al. succeeded in replicating Shafir’s (Reference Shafir1993) results to those tasks in which they did not. Out of the seven two-alternative tasks that both studies examined, in five, the meaning of the attributes may have been depended on the cultural context in which the experiments were conducted, such as the social era or the geographical location. For example, as Shafir (Reference Shafir2018) noted, in the task involving choosing/rejecting a custodian parent, the attribute ‘lots of work-related travel’ in the description of the enriched alternative may had been considered more negative 30 years ago when he conducted his experiments than in Klein et al.’s failed replication in Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and Sowden2018
Footnote
2
. On the other hand, the two tasks that Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) succeeded in replicating involved choosing/rejecting an enriched lottery (a 60% chance to win $100, and a 40% chance to lose $5) over an impoverished one (a 20% chance to win $50, otherwise nothing). These are tasks which appear to be less sensitive to the cultural context in which they were conducted. Furthermore, unlike the two-alternative lottery tasks, an additional three-alternative lottery task involving selecting one of three alternatives was not replicated in Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021). Because of its increased complexity, such a task is likely to require more cognitive resources generating more noise and increased regressiveness in the rejection task, and as a result a strong accentuation effect.
Concluding remarks
Shafir (Reference Shafir2018) argues that, because of context effects, his original stimuli are inappropriate for a reliable test of the choice–rejection framing effect and the compatibility hypothesis. As my discussion of a possible context effect associated with these stimuli suggests, there may be some truth in this argument. Nevertheless, I view Chandrashekar et al.’s (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) replication as a worthwhile contribution to our understanding of this effect. Foremost is their clear finding that even though Shafir’s results are not robust with regard to compatibility, they are robust with regard to accentuation. Furthermore, our review of the existing evidence suggests that accentuation explains the differences between choice and rejection better than compatibility. By and large, all the studies I am aware of that were based on Shafir’s original design of pitting enriched versus impoverished alternatives in choice versus rejection tasks support the accentuation hypothesis over the compatibility hypothesis. In fact, none of these studies support the compatibility hypothesis. Shafir’s (Reference Shafir1993) original results are equally explained by compatibility and accentuation. Ganzach’s (Reference Ganzach1995) results are explained by accentuation but not by compatibility and so are those of Chen and Proctor (Reference Chen and Proctor2017), Klein et al. (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and Sowden2018), Reijnen et al. (Reference Reijnen, Kühne, Stöcklin and Wolfe2019), and Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021). Note, however, that this is not to say that compatibility, which was found to have a clear effect in quite a few areas of judgment and decision making studies (e.g., Rubaltelli et al., Reference Rubaltelli, Dickert and Slovic2012; Tversky et al., Reference Tversky, Sattath and Slovic1988), does not exert an influence on the choice–rejection framing. In fact, at least one study of choice–rejection framing, it was shown to have an effect which is consistent with the compatibility hypothesis (Ganzach & Schul, Reference Ganzach and Schul1995). However, this study did not rely on a choice task, but rather on a judgment task). Thus, it remains to be seen whether compatibility can be demonstrated with this later task.
Another conclusion that emerges from our analysis is that the stimuli Shafir used in his original 1993 paper, and Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) in their replication, are not really appropriate to test the compatibility hypothesis. In these stimuli, the alternatives in each task are characterized by the enriched alternative being more attractive than the impoverished one. Here, a higher preference for the enriched alternative in choice than in rejection does not necessarily support the compatibility hypothesis because, as discussed above, both compatibility and accentuation make the same prediction. The stimuli that are appropriate for testing the compatibility hypothesis are such that the enriched alternative is less attractive than the impoverished alternative. Here, compatibility and accentuation offer diametrically different predictions. Compatibility, but not accentuation is supported by a pattern in which the enriched alternative is preferred more in choice than in rejection, and accentuation, but not compatibility is supported by a pattern in which the impoverished alternative is preferred more in choice than in rejection. Interesting enough, as we mentioned above, such an experimental test is reported in Ganzach (Reference Ganzach1995). However, Ganzach’s (Reference Ganzach1995) study was not designed to test the compatibility hypothesis nor was it designed as a critical test between compatibility and accentuation. So, a pre-registered experiment that will examine this question on the basis of stimuli in which the impoverished alternatives are more attractive than the enriched alternatives is called for here.
Finally, if one accepts the ‘noisy rejections’ view of accentuation, then it is primarily, if not only, the difference in attractiveness between the focal alternatives that leads to the choice–rejection framing effect. Thus, testing for this framing effect among alternatives that do not vary in enrichment and vary only in attractiveness seems a valuable next step.
Open access
In this comment, I attempt to understand the differences between the results of Shafir (Reference Shafir1993) and its ‘very close replication’ by Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) (see Brandt et al., Reference Brandt, Jzerman, Dijksterhuis, Farach, Geller, Giner-Sorolla and Van't Veer2014 for a discussion of very short replication). Both studies examined the effect of the framing of a selection task in terms of choosing the best alternative versus framing it in terms of rejecting the worst alternative. In both studies, participants were presented with the same seven two-alternative and one three-alternative selection tasks and were asked either to choose or to reject one of the alternatives, where one of the alternatives was an enriched alternative, an alternative with high attribute variability (i.e., both very positive and very negative attributes), and the other an impoverished alternative, an alternative with low attribute variability. In the original study, Shafir (Reference Shafir1993) found that in all the eight selection tasks, the enriched alternative was preferred significantly more than the impoverished alternative when the selection was presented in terms of choosing than when it was presented in terms of rejecting. Shafir understood these results as providing support to a compatibility hypothesis, which suggests that there is a stronger tendency to assign heavier weight to attributes (e.g., positive versus negative attributes) that are compatible with the response mode (choose versus reject, respectively) than to attributes which are not compatible to the response mode (reject versus choose, respectively).
Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) obtained the same results as Shafir in only two of the eight tasks. In two other tasks, they found the opposite pattern, and in four of the tasks, they found no significant differences between the accept and reject task. An even more contradictory finding was that of Klein et al. (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and Sowden2018), who attempted to replicate only one of Shafir’s eight tasks and found with regard to this task the opposite of what Shafir found (Klein et al., Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and Sowden2018, even note that among 28 studies they attempted to replicate, only three were in the opposite direction, and of these three, the replication of Shafir, Reference Shafir1993, had by far the strongest effect size). Finally, as Wedell (Reference Wedell1997) noted, Ganzach (Reference Ganzach1995) also reported results that are diametrically opposite to Shafir’s.
In an attempt to make sense of the contradictory results obtained by Shafir (Reference Shafir1993) and Ganzach (Reference Ganzach1995), Wedell (Reference Wedell1997) noted that in Shafir’s selection tasks, the enriched alternative was overall more attractiveFootnote 1 than the impoverished alternative, whereas in Ganzach’s (Reference Ganzach1995) tasks, the opposite was the case. Furthermore, he ingeniously suggested that, both in Shafir’s data and in Ganzach’s data (as well as in additional data he collected), the choice–rejection framing effect can be explained by the fact that the difference between alternatives is accentuated in the choice task in comparison to a rejection task (the accentuation hypothesis), leading preference toward the more attractive alternative to be higher in choice than in rejection.
There are two explanations for accentuation. Wedell’s (Reference Wedell1997) original explanation is motivational. It suggested that accentuation is the result of the decision maker’s need to justify his/her decision when choosing but not when rejecting. Thus, if alternative A is inherently viewed as better [worse] than B, the need to justify the decision leads the advantages [disadvantages] of alternative A to loom larger in a choice frame than in a rejection frame.
Chen and Proctor (Reference Chen and Proctor2017) offered a cognitive-effort explanation for accentuation. According to their explanation ‘Compared to the selection task, the rejection task is more difficult and leaves fewer cognitive resources available for discriminating the alternatives’ (p. 566), which results in people paying less attention to the differences between the alternatives. This results in worse discrimination between alternatives in the rejection task than in the choice task. Stated differently, rejection tasks are more noisy and therefore more regressive, and it is this regressiveness that leads to the choice–rejection framing effect (see the appendix for a formal development of this argument).
In trying to understand the difference between Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) and Shafir (Reference Shafir1993), we first note that compatibility and accentuation are not necessarily two competing explanations for the choice/rejection framing effect. They are better viewed as two independent processes that influence this effect. The effect of compatibility depends on the variability of the attribute values of the alternatives (i.e., their degree of enrichment/impoverished), and the effect of accentuation depends on mean values of these attributes (i.e., their attractiveness) (see Ganzach, Reference Ganzach1995 for this framework for analyzing alternatives’ characteristics). Whereas compatibility is reflected in the effect of attribute variability on the difference between choosing and rejecting, accentuation is reflected in the effect of attribute mean on the difference between choosing and rejecting. This view suggests that the effect of the choice/rejection framing on the preference for the enriched alternative depends on the relative influence of these two processes. If compatibility is dominant, a stronger preference for the enriched alternative in choice than in rejection is observed. On the other hand, if accentuation is stronger, a stronger preference for the attractive alternative in choice than in rejection is observed.
Based on this analysis, we suggest that Chandrashekar et al.’s (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) failure to replicate Shafir’s compatibility results in the choice/rejection tasks, coupled with the clear accentuation effect they found in their ‘extensions’ (i.e., analyses based on Wedell’s (Reference Wedell1997) analytical approach), is consistent with Chen et al.’s (Reference Chen and Proctor2017) cognitive-effort view of accentuation. Chandrashekar et al.’s (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) Mturk participants are very different from Shafir’s Princeton students’ participants, in that the former were likely to have less cognitive resources for discriminating between the alternatives in the more difficult rejection task. First, the motivation of Mturk workers to invest cognitive resources in performing experimental tasks is relatively low (Peer et al., Reference Peer, Rothschild, Gordon, Evernden and Damer2022), and this motivation event deteriorated during the last few years (Kennedy et al., Reference Kennedy, Clifford, Burleigh, Waggoner, Jewell and Winter2020); and second, their verbal ability, a talent which is necessary to extract information from written texts, is likely to be lower than that of the Ivy League students who participated in Shafir’s experiment (see Hauser et al., Reference Hauser, Paolacci and Chandler2019 for information about the characteristics of Mturk participants). This difference in cognitive resources between Chandrashekar et al.’s (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) and Shafir’s (Reference Shafir1993) participants could have resulted in accentuation [compatibility] exerting stronger [weaker] influence on the preferences in choice and rejection.
From this perspective, it is interesting to compare the tasks in which Chandrashekar et al. succeeded in replicating Shafir’s (Reference Shafir1993) results to those tasks in which they did not. Out of the seven two-alternative tasks that both studies examined, in five, the meaning of the attributes may have been depended on the cultural context in which the experiments were conducted, such as the social era or the geographical location. For example, as Shafir (Reference Shafir2018) noted, in the task involving choosing/rejecting a custodian parent, the attribute ‘lots of work-related travel’ in the description of the enriched alternative may had been considered more negative 30 years ago when he conducted his experiments than in Klein et al.’s failed replication in Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and Sowden2018 Footnote 2 . On the other hand, the two tasks that Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) succeeded in replicating involved choosing/rejecting an enriched lottery (a 60% chance to win $100, and a 40% chance to lose $5) over an impoverished one (a 20% chance to win $50, otherwise nothing). These are tasks which appear to be less sensitive to the cultural context in which they were conducted. Furthermore, unlike the two-alternative lottery tasks, an additional three-alternative lottery task involving selecting one of three alternatives was not replicated in Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021). Because of its increased complexity, such a task is likely to require more cognitive resources generating more noise and increased regressiveness in the rejection task, and as a result a strong accentuation effect.
Concluding remarks
Shafir (Reference Shafir2018) argues that, because of context effects, his original stimuli are inappropriate for a reliable test of the choice–rejection framing effect and the compatibility hypothesis. As my discussion of a possible context effect associated with these stimuli suggests, there may be some truth in this argument. Nevertheless, I view Chandrashekar et al.’s (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) replication as a worthwhile contribution to our understanding of this effect. Foremost is their clear finding that even though Shafir’s results are not robust with regard to compatibility, they are robust with regard to accentuation. Furthermore, our review of the existing evidence suggests that accentuation explains the differences between choice and rejection better than compatibility. By and large, all the studies I am aware of that were based on Shafir’s original design of pitting enriched versus impoverished alternatives in choice versus rejection tasks support the accentuation hypothesis over the compatibility hypothesis. In fact, none of these studies support the compatibility hypothesis. Shafir’s (Reference Shafir1993) original results are equally explained by compatibility and accentuation. Ganzach’s (Reference Ganzach1995) results are explained by accentuation but not by compatibility and so are those of Chen and Proctor (Reference Chen and Proctor2017), Klein et al. (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and Sowden2018), Reijnen et al. (Reference Reijnen, Kühne, Stöcklin and Wolfe2019), and Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021). Note, however, that this is not to say that compatibility, which was found to have a clear effect in quite a few areas of judgment and decision making studies (e.g., Rubaltelli et al., Reference Rubaltelli, Dickert and Slovic2012; Tversky et al., Reference Tversky, Sattath and Slovic1988), does not exert an influence on the choice–rejection framing. In fact, at least one study of choice–rejection framing, it was shown to have an effect which is consistent with the compatibility hypothesis (Ganzach & Schul, Reference Ganzach and Schul1995). However, this study did not rely on a choice task, but rather on a judgment task). Thus, it remains to be seen whether compatibility can be demonstrated with this later task.
Another conclusion that emerges from our analysis is that the stimuli Shafir used in his original 1993 paper, and Chandrashekar et al. (Reference Chandrashekar, Weber, Chan, Cho, Chu, Cheng and Feldman2021) in their replication, are not really appropriate to test the compatibility hypothesis. In these stimuli, the alternatives in each task are characterized by the enriched alternative being more attractive than the impoverished one. Here, a higher preference for the enriched alternative in choice than in rejection does not necessarily support the compatibility hypothesis because, as discussed above, both compatibility and accentuation make the same prediction. The stimuli that are appropriate for testing the compatibility hypothesis are such that the enriched alternative is less attractive than the impoverished alternative. Here, compatibility and accentuation offer diametrically different predictions. Compatibility, but not accentuation is supported by a pattern in which the enriched alternative is preferred more in choice than in rejection, and accentuation, but not compatibility is supported by a pattern in which the impoverished alternative is preferred more in choice than in rejection. Interesting enough, as we mentioned above, such an experimental test is reported in Ganzach (Reference Ganzach1995). However, Ganzach’s (Reference Ganzach1995) study was not designed to test the compatibility hypothesis nor was it designed as a critical test between compatibility and accentuation. So, a pre-registered experiment that will examine this question on the basis of stimuli in which the impoverished alternatives are more attractive than the enriched alternatives is called for here.
Finally, if one accepts the ‘noisy rejections’ view of accentuation, then it is primarily, if not only, the difference in attractiveness between the focal alternatives that leads to the choice–rejection framing effect. Thus, testing for this framing effect among alternatives that do not vary in enrichment and vary only in attractiveness seems a valuable next step.
Appendix
In this appendix, I show that noisy rejections lead to a choice–rejection framing effect which is consistent with the accentuation hypothesis. In doing that, I rely on the standard approach to testing such a framing effect and calculate the sum of percentages of acceptance and rejection of the more (and less) attractive alternative, suggesting that significant deviations from 100% are indicative of a framing effect. Specifically, I show that under the assumption of noisy, or regressive, rejections, this sum is larger (smaller) than 100 for the more (less) attractive alternative.
Suppose that in the selection task participants have to choose or reject one of two alternatives, A and B, where the ‘true’ preference in the population is for alternative A, such that alternative A is preferred by 50+x percent of the population and alternative B by 50-x percent. For simplicity, assume that there is no noise when people reveal their preferences in choosing the preferred alternative, and that the sample is large enough and therefore the preferences for A and B in the choice task equal the true preferences in the population, that is, they are also 50+x and 50-x, respectively. The preferences in the rejection task are noisy and therefore regress toward the mean of 50 by r. Thus, in the rejection condition, the preference toward A is 50+x-r (the preference toward B is 50-x+r). In terms of rejection, the rejection rate of A is 100-[50+x-r] = 50-x+r (the rejection rate of B is 100-[50-x+r]=50+x-r). This state of affairs leads the sum of the percentages of choosing and rejection of alternative A to be 100+r, as opposed to a sum of 100 if choosing and rejecting are complimentary (similarly, in this case, the sum of the percentages of alternative B is 100-r).