1 Introduction
Many decisions occur in an interactive, social context (Reference Larrick, Mannes and SollLarrick, Mannes & Soll, 2012; Reference Sniezek and BuckleySniezek & Buckley, 1995). When we are uncertain about something, we often seek advice from others to enhance decision quality. Advice taking is considered an important, adaptive aspect of human decision making (Reference Yaniv and MilyavskyYaniv & Milyavsky, 2007). The increase in accuracy through combining independent judgments is a well-established finding (e.g., Reference Harvey and FischerHarvey & Fischer, 1997; Yaniv, 2004a, 2004b). Aggregated judgments based on multiple opinions are usually more accurate than individual judgments because aggregation balances out random errors (Reference GaltonGalton, 1907; Reference Herzog and HertwigHerzog & Hertwig, 2009).
Research has largely focused on how advice is used in judgment revision once it is received (e.g., Reference Harvey and FischerHarvey & Fischer, 1997; Yaniv, 2004a; but see Reference Gino, Brooks and SchweitzerGino, Brooks & Schweitzer, 2012, for an exception). The most consistent finding in this area is the underweighting of other people’s estimates, referred to as egocentric discounting (Reference Yaniv and KleinbergerYaniv & Kleinberger, 2000). Whereas averaging of estimates is usually the most successful strategy (e.g., Armstrong, 2001; Reference ClemenClemen, 1989; Reference Larrick and SollLarrick & Soll, 2006) people combine their own estimate with advice in only a subset of the cases (e.g., Reference Harvey and FischerHarvey & Fischer, 1997; Reference Soll and LarrickSoll & Larrick, 2009; Reference Soll and MannesSoll & Mannes, 2011; Reference Yaniv and Choshen-HillelYaniv & Choshen-Hillel, 2012) thereby failing to realize the full benefits of advice taking (Reference YanivYaniv, 2004a). The informational asymmetry account (Reference YanivYaniv, 2004b) asserts that egocentric discounting is due to differences in the accessibility of one’s own reasoning relative to that of others. More precisely, whereas one might have a number of reasons for one’s own judgment, one likely has less knowledge about another person’s judgment (Reference Tversky and KoehlerTversky & Koehler, 1994). Support for this assumption stems from studies reporting stronger reliance on advice with a decrease in self-reported knowledge (Reference Yaniv and Choshen-HillelYaniv & Choshen-Hillel, 2012) and an increase in task difficulty (Reference Gino and MooreGino & Moore, 2007). Both factors likely lead to reduced support for one’s initial judgment, thereby reducing informational asymmetry. In contrast, informational asymmetry should be more pronounced the more strongly advice differs from one’s own judgment.
1.1 Seeking advice
Some researchers have investigated the impact of multiple advisory judgments (Reference Yaniv, Choshen-Hillel and MilyavskyYaniv, Choshen-Hillel & Milyavsky, 2009) or advice generated by groups (Reference MannesMannes, 2009) on judgment revision, thereby (virtually) increasing the amount of information participants receive. However, this work does not take into account the fact that people can interact with their social ecology (Reference Bonaccio and DalalBonaccio & Dalal, 2006; Reference Gino, Brooks and SchweitzerGino et al., 2012). For instance, the frequency with which advice is consulted could vary across situations. The present research investigates advice seeking behavior. Initial research presented factors that affect people’s general willingness to consider a piece of advice (Reference Gino, Brooks and SchweitzerGino et al., 2012; Reference Gino and MooreGino & Moore, 2007). However, advice taking is often a sequential process that can be truncated at different points in time. Thus, depending on characteristics of the decision task, personal factors (e.g., knowledge), or the advice received, people could seek smaller or larger samples of advice.
Given that advice taking is an instance of interactive decision making, it appears highly artificial to assume that people’s only options to react to advice are integrating it with their own estimate or rejecting it. In contrast, we assume that receiving advice can also instigate additional advice seeking. Specifically, the informational asymmetry account (Reference YanivYaniv, 2004b) suggests that advice is evaluated with regard to how much information supports it. The advisee’s search for information might not be limited to her own mind. Thus, within a plausible range, advice might motivate people to consider more advice. Moreover, given the importance of independence of information to achieving substantial improvements by aggregation (Reference Einhorn, Hogarth and KlempnerEinhorn, Hogarth & Klempner, 1977; Reference Herzog and HertwigHerzog & Hertwig, 2009), the information samples people create by consulting different sources and quantities of advice constitute a necessary precondition for any improvement of decision quality through advice taking.
1.2 An ecological approach to advice taking
The present research expands the informational asymmetry account to investigate the process of advice taking — both seeking and integrating advice. We base this research on the premise that decision makers often have opportunities to consult multiple people and take an active role in the search for information. Such interactions between cognition and the information ecology have long been recognized (Reference BrunswikBrunswik, 1955; Reference SimonSimon, 1956) and have been studied recently as well (e.g., Reference Hertwig and ErevHertwig & Erev, 2009; Reference HertwigHertwig, 2015). Sampling as a theoretical approach assumes that biases in the information sample on which judgments are based might be a more parsimonious explanation for biases often assumed to reside within the individual’s mind (Reference FiedlerFiedler, 2000). The informational asymmetry account constitutes an instance of biased samples, on the basis of which the decision maker rationally chooses to integrate (or reject) advice (Reference Ravazzolo and RøislandRavazzolo & Røisland, 2011). However, the classical research paradigm never allowed decision makers to obtain additional information, leaving open the question of whether individuals spontaneously reduce the asymmetry by sampling. Related paradigms reveal that people often show profound biases in their tendency to search for different types of information (e.g., creating valence asymmetries in attitude acquisition; Reference DenrellDenrell, 2005; Reference Fazio, Eiser and ShookFazio, Eiser & Shook, 2004).
In the present research, we establish an advice taking paradigm that allows participants to sample any number of other people’s judgments. This paradigm allows for the investigation of whether the sample of advisory estimates is sensitive to features of the decision situation. We derive and test two straightforward assumptions of our ecological account:
Assumption 1: The frequency of sampling is greater when advice is distant from rather than close to the decision maker’s initial estimate. This distance has been shown to influence judgment revision (Reference YanivYaniv, 2004b; Reference Yaniv, Choshen-Hillel and MilyavskyYaniv et al., 2009), but to our knowledge, has not been applied to advice seeking. The distance of the advice from the decision maker’s initial estimate constitutes a cue to advice independence, a condition under which highest improvements in judgment accuracy can be expected (e.g., Reference Larrick and SollLarrick & Soll, 2006). Hence, distant advice should be preferred over close advice, if advice sampling is motivated by the prospect of acquiring new, independent information about the decision problem. Such a motive can be distinguished from a motive to be affirmed by advisors (Reference Kahneman and TverskyKahneman & Tversky, 1973). In this case, close advice should be preferred.
If sampling behavior is indeed a function of informational asymmetry and thus motivated by the informational value of advice, sampling should be more frequent when advice is distant rather than close. The present research manipulates advice distance in order to test this crucial assumption of the expanded informational asymmetry account. Specifically, we expect that distant as compared to close advice will lead to a higher frequency of additional sampling.
The next assumptions presume that the integration of advice depends on the samples drawn.
Assumption 2a: Advice is more likely to be integrated with a decision maker’s initial estimate if it is distant from the initial estimate rather than close. Distance of advice has repeatedly been shown to constitute an important moderator of egocentric discounting in advice weighting. Yaniv (2004a) postulated a monotone negative relationship between advice distance and weighting with decreasing integration of more distant advice. This pattern was reported in a number of studies (Reference Minson, Liberman and RossMinson, Liberman & Ross, 2011; Reference Yaniv and MilyavskyYaniv & Milyavsky, 2007; Reference YanivYaniv, 2004b). Recent research, however, reported a curvilinear relationship with the strongest integration of advice on intermediate distance levels, which is regarded most informative (Reference Moussaïd, Kämmer, Analytis and NethMoussaïd, Kämmer, Analytis & Neth, 2013; Reference Schultze, Rakotoarisoa and Schulz-HardtSchultze, Rakotoarisoa & Schulz-Hardt, 2015). Advice that is very distant from the decision maker’s perspective might not be integrated as it might become incredible to the advisee and is thus not regarded informative. Close advice also offers little new information to the decision maker. In contrast, it confirms the decision maker’s intial judgment. Close advice is thus not utilized for judgment updating, but rather for the updating of confidence (Reference Schultze, Rakotoarisoa and Schulz-HardtSchultze et al., 2015) serving a social validation function (Reference Kahneman and TverskyKahneman & Tversky, 1973; Reference Schulz-Hardt, Frey, Lüthgens and MoscoviciSchulz-Hardt, Frey, Lüthgens & Moscovici, 2000). In sum, these findings provide initial evidence for decision makers’ sensitivity to the informational value of advice. In the present research we compare relatively close and distant advice and predict that distant advice will be weighted more strongly than close advice.
Assumption 2b: Advice is more likely to be integrated with a decision maker’s initial estimate the more estimates sampled. Judgment updating should not be a function of only the distance of the advice from the decision maker’s initial estimate. The degree to which advice is used for judgment updating should depend on the degree of supportive information (i.e., its amount and consistency). That is, the more pieces of advice support an alternative position, the more strongly this position should be held. In fact, initial research indicates that advice weighting is sensitive to the number of advisors providing it (Reference MannesMannes, 2009). We thus expect that an alternative position will be weighted more strongly, the more pieces of advice support it. Therefore, advice integration should be a direct, positive function of sampling frequency.
In the present work, we focus on the number of pieces of advice sampled, rather than the number of reasons underlying a given judgment, for three reasons. First, in everyday life the judgments of other people can be observed more frequently than the reasons underlying a judgment. Second, when judgments are based on intuition, concrete reasons are often not available or constructed after the fact (Reference Nisbett and WilsonNisbett & Wilson, 1977). Third, we assume that a judgment is evaluated in accordance with the amount and consistency of information, as in an assessment of reliability, a principle that should be applicable to the number and consistency of advisory judgments as well as the number and consistency of supporting reasons (Reference KoriatKoriat, 2012).
1.3 Overview of experiments
We conducted three experiments that followed a general procedure. As in the classical approach to advice taking, participants were presented with an estimation task, in this case the caloric content of various foods per serving. After giving an initial estimate, participants received an initial piece of advice, supposedly sampled from a distribution of 100 participants in an earlier study. This piece of advice was either similar to or diverging from their initial estimate. All three experiments implemented this factor in a within-participants design. Experiment 1 resembled the classical research approach without a sampling phase and served as an initial test of the weighting functions of our operationalization of close and distant advice. In Experiments 2 and 3, participants could sample additional pieces of advice at will before giving their final estimate. While Experiment 2 implemented advice that was consistent among advisors, Experiment 3 utilized ecologically valid and thereby considerably less consistent advice.
1.4 Data analysis
As dependent measures, we analyzed the size of the self-determined samples as well as the integration of advice when forming a final judgment. As a measure of advice integration, we calculated the weight of advice (WOA; Reference Harvey and FischerHarvey & Fischer, 1997; Reference YanivYaniv, 2004a), defined as
where I, F, and A indicate the initial estimate, final estimate, and advice, respectively, on a given trial j in a given participant i. We calculated the mean of all pieces of advice received to arrive at an estimate for A. The WOA thus reflects the degree to which participants move towards the advice (i.e., a WOA of 1 indicates adoption of the advice; a WOA of 0 indicates adherence to the initial estimate; a WOA of 0.5 indicates an equal-weighting strategy). The WOA thus allows for an assessment of (increasing or decreasing) receptivity towards advice as a function of amount sampled or other variables.
The WOA is highly sensitive to outliers. Outliers primarily result from trials where the distance between initial estimate and advice is very small, so that even small alterations of the judgment may lead to values outside the range of 0 and 1. Many researchers thus truncate the WOA, setting values smaller than 0 to 0 and values greater than 1 to 1 (e.g., Gino, 2008; Reference Schultze, Rakotoarisoa and Schulz-HardtSchultze et al., 2015; Reference Soll and LarrickSoll & Larrick, 2009), arguing that this practice is unproblematic if it concerns less than 5% of all trials. As participants in our expanded paradigm (Experiment 2 and 3) received multiple pieces of advice that scattered, there were more trials with values of WOA outside the range of 0 and 1 than in Experiment 1. For the analyses reported in the main text, we opted to apply statistical criteria to identify and remove outliers on a trial-by-trial basis (Reference TukeyTukey, 1977) rather than altering values of WOA. Additional analyses applying the truncation practice yielded similar results, but led to a higher number of alterations in Experiment 2 and 3. Note that, for the influence of sampling on advice weighting, the qualitative pattern of results is unaltered even when no trials are excluded. However, the effect of advice distance on WOA reverses (Experiment 1 and 3) in the absence of any exclusion.
For all experiments, we calculated multilevel model analyses for all dependent measures to assess relationships on a trial-by-trial basis (Reference Judd, Westfall and KennyJudd, Westfall & Kenny, 2012). Analyses are based on 20 data points per participant. Following the recommendations by Judd and colleagues (2012), all models contained random intercepts for participants as well as items, which were fully crossed by design. The effects of our hypothesized predictors were always fixed. We determined the best-fitting model by sequentially including the hypothesized parameters and their interactions, one at a time and inspecting increase in model fit using the Kenward-Roger approximation (Reference Kenward and RogerKenward & Roger, 1997) for degrees of freedom. The resulting p-values as well as log-likelihood values and approximated Bayes factors (Reference MassonMasson, 2011) for each model are given in the Appendix. For the sake of brevity, we will report only the parameter estimates of the best-fitting model.
To summarize, we predict that sampling behavior will increase with the distance of advice from the decision maker’s initial estimate (Assumption 1). We test this assumption in Experiments 2 and 3, by looking for a main effect of advice distance on sample size for the multilevel model. We predict a larger sample size in distant as compared to close advice. We also expect advice integration to be sensitive to participants’ information samples. Specifically, we predict two main effects of advice distance (Assumption 2a; Experiments 1–3) and amount of information (Assumption 2b; Experiments 2 and 3) on the WOA in a multilevel model, such that the WOA will be larger with distant (rather than close) advice and larger (rather than smaller) information samples.
1.5 A-priori power analyses
We conducted a-priori power analyses to determine required sample sizes in all three experiments (Reference Faul, Erdfelder, Lang and BuchnerFaul, Erdfelder, Lang & Buchner, 2007). As there is no clear guideline for power analyses regarding multilevel modelling utilized in the present research, we based our calculation on repeated measures ANOVA designs. Detecting a medium-sized effect (f = .25) with sufficient power (β = .95) in Experiments 1 and 2 required collecting data of at least 20 participants. We assumed a smaller effect (f = .15) in Experiment 3 due to the increased variance in advice, requiring at least 54 participants. As the experiments were part of multi-experiment sessions, we increased sample size in accordance with other studies that required a larger sample size.
2 Experiment 1
The first experiment served as a conceptual replication of advice taking procedures that do not include a sampling phase using the same materials and basic manipulation as in the next two experiments. Specifically, we compared two advice distance conditions. The close condition was intended to confirm participants’ initial judgments leading to little judgment revision. In the distant condition, advice was intended to increase participant’s tendency to revise their judgment, as in the intermediate distances in Moussaïd et al.’s (2013) and Schultze et al.’s (2015) work.
2.1 Method
Participants. The sample consisted of 35 University of Tübingen students of different subjects (10 males; M age= 23.00 years, SD age= 6.18).Footnote 1 Participants took part in a one-hour session comprised of several social cognitive experiments and were compensated with either course credit or 7€ and a chocolate bar. Additionally, participants received a small bonus payment depending on task performance. Participants were recruited via e-mail and online social networks.
Design. The experiment implemented a 2 (distance of advice: close vs. distant) × 2 (judgment phase: initial vs. final judgment) within-participants design.
Materials and procedure. After participants signed a consent form, we asked them to estimate the caloric content of various dishes per serving. Names, pictures and caloric content of dishes were retrieved from a web page of a German nutrition magazine (http://www.essen-und-trinken.de).Footnote 2 We pretested 30 dishes using an online sample of 21 participants. Out of those 30 dishes, we selected the 20 dishes for which the average estimate most closely resembled the true caloric content of the dish (e.g., fish pasta). This procedure assured that the population had some knowledge about the caloric contents of those dishes (Reference GinoGino, 2008).
The order of dishes was randomized for each participant, as was the order of the 10 trials per distance condition across the 20 trials.
For each trial, participants first saw a picture of the dish along with a descriptive label and a response box to give their initial estimate. If participants submitted an estimate lower than 50 or higher than 1250 calories per serving, the software prompted them to give a plausible estimate. Next, they were shown another estimate for that same dish. These estimates were allegedly drawn from a pool of estimates of 100 participants in a preceding experiment. Depending on distance condition, the advice was simulated as a pseudo-normal distribution centering on a mean relatively close to the participant’s initial estimate (on close trials) or centering on a mean relatively distant from the participant’s initial estimate (on distant trials). Specifically, the mean of the distribution was simulated as i ± .5 t (on distant trials) and i ± .05 t (on close trials) where i is the participant’s initial estimate and t is the true value of the dish. If the participant underestimated in comparison to the true value, the proportion was added to the initial estimate; if the participant overestimated in comparison to the true value the proportion was subtracted. Consequently, the advice would generally point in the direction of the true value. However, if the initial estimate was very close to the true value, the advice could also lead participants away from the true value, by causing them to adjust too much. The standard deviation of the distribution was equal for close and distant advice. Random noise was added to obscure the artificial nature of advice, drawn from a uniform distribution in the range of [–8, +8] calories.
After receiving advice, participants gave a final estimate for the given dish. For each of these final estimates that fell in a range extending 10% on either side of the true caloric content of the dish, participants would receive a bonus payment of 0.10€. During the experiment, participants did not receive feedback on their accuracy or their cumulative bonus. The bonus was revealed only at the end of the experiment. In total, participants could collect a bonus of 2€. Finally, participants were probed for suspicion, debriefed, paid, and thanked.
2.2 Results
Distance of advice. As a manipulation check, we assessed the absolute difference between participants’ initial estimates and the initial piece of advice on the level of trials. Whereas participants’ initial estimate and the initial piece of advice differed by only M = 30.80 (SD = 21.19) calories on close trials, they differed by M = 241.85 (SD = 83.44) calories on distant trials.
We also calculated the normalized distance in line with Moussaïd et al. (2013):
where I and A denote the initial estimate of the participant and the advice, respectively, on a given trial j in participant i. Moussaïd et al. (2013) define values below 0.3 as “similar” advice, which they found to be related to an updating of confidence rather than an updating of one’s judgment. Values between 0.3 and 1.1 were defined as an “intermediate distance”, for which strong judgment updating was observed. Values above 1.1 were classified as “very distant” and were empirically related to diminishing influence on participants’ judgments. Employing the average advice on each trial as the standard of reference, the average Δ E amounted to M = .07 (SD = .06) in the close condition and to M = .53 (SD = .28) in the distant condition. Our operationalizations of the close and distant conditions thus resemble the “similar” and “intermediate distance” conditions of Moussaïd et al. (2013)
As a result of the distance manipulation, participants’ initial estimate and the advice enclosed the true value in 79% of the distant trials, but only 17% of the close trials.
Using advice. We excluded trials with a WOA < −0.42 and WOA > 0.67 (Reference TukeyTukey, 1977). In total, we excluded 49 of 700 trials (7.00%).
We tested the hypothesis that advice weighting is greater for distant as compared to close advice. We fitted the following model:
WOA ij = α0 + b 1Distance j + є,
where index i refers to participants and index j to questions. For this and all following models close advice was coded as 0 and distant advice as 1. The effect of distance thus indicates the consequences of increasing advice distance from close to distant advice. Replicating the results of other authors, advice distance significantly increased participants’ WOA, b 1 = .12, se = .01, t = 10.65 (Table 1), although fit indices indicated worse fit for this model over the null model in this specific data set. In all subsequent experiments, including distance as a predictor of WOA improved model fit (Appendix).
Note. Cells display mean values across participants and trials. Standard deviations are given in parentheses.
2.3 Discussion
Participants weighted the advice stronger when it was distant from rather than close to their initial estimates. These results validate our manipulation of advice distance replicating the finding of greater weighting for advice of intermediate rather than small distance (Reference Moussaïd, Kämmer, Analytis and NethMoussaïd et al., 2013; Reference Schultze, Rakotoarisoa and Schulz-HardtSchultze et al., 2015).
3 Experiment 2
The second experiment introduced a sampling phase that allowed participants to sample as many pieces of advice as desired. This experiment allowed us to assess whether distant advice indeed instigates a longer search for advice than close advice, and how several consistent pieces of advice are integrated during judgment revision. In order to investigate whether close advice in our paradigm actually led to higher levels of confidence, we introduced a confidence measure after each final estimate. We did that for only half of the participants to assess potential effects of reflecting on one’s confidence.
3.1 Method
Participants. The sample consisted of 44 University of Heidelberg students (14 males; M age = 26.48 years, SD age = 8.81). The study was run as part of an experimental session of approximately 45 minutes. Participants received either course credit or a monetary compensation of 6€. Additionally, participants received a performance-contingent bonus as in Experiment 1.
Design. This experiment implemented a 2 (distance of advice: close vs. distant) × 2 (judgment phase: initial vs. final judgment) within-participants design. Additionally, half of the participants [N = 22] were asked to provide confidence ratings for their final estimates.
Materials and procedure. Up to eight participants took part at the same time, each seated in a separate cubicle. Materials and procedure were identical to Experiment 1 with two exceptions. First, after receiving a close or a distant piece of advice on the caloric content of a given dish, participants were allowed to sample as many other estimates as desired. The sampling phase was however capped at 20 pieces of advice. Advice was consistently drawn from either the close or distant distribution depending on distance condition. Second, participants in the confidence ratings condition also rated their confidence on a 7-point scale anchored at –3 = “very unconfident” and 3 = “very confident” after each trial, while participants in the no confidence ratings condition proceeded without any further questions. As the presence or absence of confidence ratings did not exert effects on any dependent measure, we will not discuss this factor any further.
3.2 Results
Distance of advice. As a manipulation check, we assessed the absolute difference between participants’ initial estimates and the average advice for each trial. Whereas participants’ initial estimate and the advice differed by only M = 26.34 (SD = 15.75) calories on close trials, they differed by M = 246.18 (SD = 77.57) calories on distant trials.
The Δ E to the average advice amounted to M = .06 (SD = .06) in the close condition and to M = .54 (SD = .26) in the distant condition. Hence, we realized a “similar” and “intermediate” distance condition in terms of Moussaïd et al. (2013). As a result, participants’ initial estimate was enclosed by the range of advice for 51% of close trials but for none of the distant trials. Initial estimate and range of advice enclosed the true value on 22% of the close and 84% of the distant trials.
Sampling of advice. We first inspected the average number of pieces of advice drawn as a necessary precondition for analyzing moderating variables of advice seeking behavior. Participants sampled more advice than the one piece classically offered. The average of M = 8.05 (SD = 7.00) advisory estimates sampled by participants across distance conditions was significantly larger than one, t(43) = 7.27, p < .001, d = 1.10.
Further, we expected greater informational asymmetry cued by distant as compared to close advice to increase sampling frequency. For advice sampling, the model with the best fit was the following:
Sample size ij = α0 + b 1Distance j + є .
Advice distance significantly increased sample size, b 1 = 1.53, se = .20, t = 7.82. That is, receiving advice that was distant from their initial estimate led participants to consider about 1.5 pieces of advice more than when receiving advice that was close to their initial estimate (Table 1).
Using advice. We excluded trials with WOA < −0.90 and WOA > 1.40, amounting to 41 of 880 data points (4.66%; Reference TukeyTukey, 1977). Consistent with Experiment 1, we expected distant advice to lead to greater reliance on advice. Testing our assumption of sensitivity to the sampled information, we expected increased sampling to increase reliance on advice. The following model had the best fit:
WOA ij = α0 + b 1Distance j + b 2Sample size ij + є .
Replicating the results of Experiment 1, advice distance increased participants’ WOA, b 1 = .23, se = .02, t = 11.21 (Table 1). Repeatedly encountering a consistent piece of advice increased the WOA as indicated by the effect of sample size, b 2 = .015, se = .003, t = 5.66.Footnote 3
The analysis of advice weighting strongly supports our assumption of increased reliance on an advisory estimate when it is supported by additionally sampled, consistent pieces of advice.
To complement the traditional WOA analysis, we also assessed how the weighting strategy employed by participants fit with the normative rule of equal weights averaging (Reference MannesMannes, 2009). We therefore calculated the normative WOA participants should have assigned to the mean advice on a given trial as N advice / (N advice + 1), where N advice is the number of advisory estimates received on a given trial (Reference MannesMannes, 2009), and subtracted it from the actual WOA participants assigned to receive a Δ WOA. For instance, if on a given trial a judge received nine pieces of advice, from an equal weighting perspective, the normative WOA should be 0.9. If the actual WOA on that trial is 0.6, the Δ WOA would amount to –0.3, indicating that the judge underweighted the advice. A positive score in this case implies overweighting of advice in comparison to the normative rule of equal weights averaging. The best fitting model for this deviation score was the following:
Δ WOA ij = α0 + b 1Distance j + b 2Sample size ij + b 3Distance j × Sample size ij + є.
Replicating numerous studies on failure to follow the normative rule of equal weights averaging (e.g., Mannes, 2009), participants fall short of this rule on both, close and distant trials (Table 1). However, assigned weights more closely matched the normative rule when advice was distant rather than close, b 1 = .17, se = .03, t = 5.81.
Although sampling increased the weight participants placed on an advisory estimate, it did so suboptimally, as the deviation from the normative rule increased with increasing sample size, b 2 = –.012, se = .003, t = –3.81. This effect was more pronounced for sampling on close as compared to distant trials, as indicated by the interaction, b 3 = .008, se = .003, t = 2.78.
Confidence. In line with other authors (Reference Moussaïd, Kämmer, Analytis and NethMoussaïd et al., 2013; Reference Schultze, Rakotoarisoa and Schulz-HardtSchultze et al., 2015), we assumed that receiving close as compared to distant pieces of advice would lead to higher levels of confidence. Additionally, we wanted to assess whether increased sampling of close versus distant advice has an effect on confidence. The following model had the best fit:
Confidence ij = α0 + b 1Distance j + є.
Indeed, increasing distance of advice decreased confidence, b 1 = –.63, se = .07, t = – 9.28 (Table 1). Additional sampling of consistent advice did not affect confidence, as indicated by the missing increase in model fit when including sample size as an additional predictor (Appendix).
3.3 Discussion
Allowing participants to sample additional pieces of advice, the results of Experiment 2 testify to participants’ willingness to obtain additional information. Further, the effect of advice distance on sample size demonstrates that receiving diverging as compared to similar pieces of advice increased participants’ sampling frequency. In line with our expanded informational asymmetry account, participants were sensitive to their information samples, shifting more strongly towards the advisory opinion the higher the number of consistent pieces of advice supporting this opinion they sampled.
Further validating our distance manipulation as providing information supporting or diverging from participants’ initial judgments, confidence was significantly lower on distant as compared to close trials.
4 Experiment 3
The third experiment was intended to increase the ecological validity of our approach, as the conclusions drawn from Experiment 2 are subject to a number of limitations. First, there was a cumulative effect of distance of advice. That is, if a participant, who received a distant (close) piece of advice, decided to consider an additional piece of advice, she received another distant (close) piece of advice. This cumulative effect could per se have an influence on participants’ sampling behavior, as it successively confirms the advisors’ (and in the case of close advice one’s own) position. Second, the advice always pointed in the direction of the true value, so that advice was generally of better quality than the own judgment. To eliminate these limitations, we replicated Experiment 2 with a more ecologically valid advice simulation method. Specifically, we implemented a procedure in which only the very first piece of advice differed between distance conditions. All subsequently sampled pieces of advice were drawn from the same distribution, without regard to condition, at random.
4.1 Method
Participants. The sample consisted of 58 University of Tübingen students (13 males; M age= 21.10 years, SD age= 3.34)Footnote 4, who took part in exchange for course credit or a monetary compensation of 7€ and a chocolate bar. The experiment was part of a session that lasted about 60 minutes. Additionally, participants received a performance-contingent bonus as in the previous experiments.
Design. This experiment implemented a 2 (distance of advice: close vs. distant) × 2 (judgment phase: initial vs. final judgment) within-participants design.
Materials and procedure. Materials and procedure were identical to Experiment 2, with two exceptions. First, the generation of advice differed in the following way. For each dish, the distribution of advice consisted of 100 values drawn at random from a normal distribution with the mean and standard deviation of the initial estimates from participants in Experiment 2. Depending on distance condition (close vs. distant advice) participants received an initial piece of advice that was either drawn from the same decile as their initial estimate (on close trials) or from the fifth next decile, above or below their initial estimate, depending on which decile their initial estimate fell into (on distant trials). For instance, if a participant’s initial estimate fell into the 6th decile, she would receive an initial piece of advice from the 6th decile (close trials) or from the 1th decile (distant trials). All additional pieces of advice received during the sampling phase were randomly drawn from the distribution of advice for the respective dish, independent of distance condition. The mean standard deviation of the distributions of advice experienced by participants was M = 163.61 with a range of [125.08; 214.29]. Thus, the advice received was more inconsistent than in Experiment 2, where the average standard deviation of the sampled advice was M = 24.93 with a range of [6.36; 46.67]. As a second deviation from Experiment 2, all participants provided confidence ratings for their final estimates.
4.2 Results
Distance of advice. As a manipulation check, we assessed the absolute difference between participants’ initial estimates and the initial piece of advice for each trial. Whereas participants’ initial estimate and the initial piece of advice differed by only M = 24.88 (SD = 29.28) calories on close trials, they differed by M = 274.05 (SD = 76.82) calories on distant trials. In contrast, the distance between participants’ initial estimate and the average advice received, if participants chose to sample, converged as we manipulated only the distance of the first piece of advice (close: M = 98.95, SD = 85.11; distant: M = 148.53, SD = 104.71).
Likewise the average Δ E between the decision maker’s initial estimate and the first piece of advice amounted to M = .06 (SD = .08) in the close condition and M = .60 (SD = .32) in the distant condition. Hence, we realized a “similar” and “intermediate” distance condition in terms of Moussaïd et al. (2013). Averaging across all pieces of advice, Δ E becomes more similar between conditions with M = .24 (SD = .32) in the close condition and M = .33 (SD = .32) in the distant condition.
As a result of the greater variance of advice compared to the previous experiments, participants’ initial estimate was included in the range of advice for 80% of close trials and for 77% of distant trials. Participants’ initial estimate and the sampled advice enclosed the true value on 83% of close and 95% of distant trials.
Sampling of advice. Participants again sampled significantly more advisory estimates than the one piece received in previous research, M = 10.26 (SD = 5.98), t(57) = 10.52, p < .001, d = 1.77. The following model had the best fit for sample size:
Sample size ij = α0 + b 1Distance j + є .
Although only the first piece of advice differed as a function of the distance manipulation, advice distance increased sample size, b 1 = .87, se = .18, t = 4.83 (Table 1).
Using advice. In analyzing the WOA we excluded trials with WOA < –0.65 and WOA > 1.03, amounting to 119 of 1160 trials (10.26%; Reference TukeyTukey, 1977). The following model had the best fit for WOA:
WOA ij = α0 + b 1Distance j + b 2Sample size ij + є .
Increasing advice distance increased participants’ WOA, b 1 = .06, se = .02, t = 3.64, (Table 1). Albeit being less consistent among advisors than in Experiment 2, sampling of advice again increased the WOA, b 2 = .007, se = .002, t = 3.40.
We again complemented traditional WOA analysis by investigating participants’ behavior in comparison to the normative rule of equal weights averaging (see Experiment 2). The model with the best fit was:
Δ WOA ij = α0 + b 1Distance j + b 2Sample size ij + є.
As in Experiment 2, participants fell short of the normative rule on both close and distant trials (Table 1), but came closer when advice was distant, b 1 = .05, se = .02, t = 3.23. Again, while sampling increased the WOA, it did so suboptimally, b 2 = –.009, se = .002, t = –4.19.
Confidence. For confidence, the following model had the best fit:
Confidence ij = α0 + b 1Distance j + b 2Sample size ij + є.
Replicating the findings of Experiment 2, advice distance decreased confidence, b 1 = –.26, se = .05, t = –5.24 (Table 1). Interestingly, additional sampling of advice further reduced confidence, b 2 = –.035, se = .008, t = –4.46.
4.3 Discussion
Experiment 3 replicated the findings of Experiment 2 under ecologically more valid conditions. Even though the distance of advice was manipulated via a single instance of advice only, initially receiving one diverging as compared to similar piece of advice led participants to sample more advice and to weight advice more strongly. Despite being less consistent among advisors than in Experiment 2, sampling of additional advice increased reliance on advice. We also replicated the effect of advice distance on confidence. Moreover, in contrast to the previous experiment, sampling of advice generally reduced confidence, as advice was less consistent.
5 General discussion
We analyzed advice taking from a broader perspective, covering both advice seeking and weighting. Specifically, we expanded the informational asymmetry account (Reference YanivYaniv, 2004b), conceptualizing egocentric discounting as an instance of biased samples. Three experiments tested the main assumptions of this expanded informational asymmetry account using a research paradigm that allows for the sampling of advice. The results show that participants were generally disposed to sampling additional advice. Participants’ sampling behavior was sensitive to features of the information ecology, with increased sampling of more informative, diverging advice (Assumption 1). Moreover, advice integration was sensitive to the information samples created by sampling, with stronger reliance on distant advice (Assumption 2a) and a greater reliance on advice that is supported by additional pieces of advice (Assumption 2b). The current approach thus expands research on advice taking by revealing first relations between advice seeking and advice integration and advances current theoretical approaches.
5.1 A sampling approach to advice taking
By investigating the process of sequential advice seeking, the present research answers a repeated call from other scientists (Reference Bonaccio and DalalBonaccio & Dalal, 2006; Reference Gino, Brooks and SchweitzerGino et al., 2012). The current results reveal advice seeking should no longer be neglected in research on advice taking. Previous research already pointed towards a generally large willingness to consider advice (Reference Gino and MooreGino & Moore, 2007). However, by allowing participants to decide on the amount of advice, we created more variance and thereby the possibility to assess factors influencing advice seeking (which was prevented by ceiling effects in the above mentioned study by Gino and Moore [2007]) and to investigate how participants react to different types of advice. The current research is thus the first to assess the sequential process of seeking advice.
The present experiments not only demonstrate a general willingness to sample advice but also show that advice seeking is susceptible to features of the information ecology, such as the distance of advice from the decision maker’s initial estimate (Assumption 1). That is, distant as compared to close advice increased participants’ propensity to consider further advice. These findings challenge less optimistic conceptualizations of advice taking (Reference YanivYaniv, 2004a). Specifically, in the classical empirical approach, participants show egocentric discounting that is attributed to unequal information samples (Reference YanivYaniv, 2004b). The present results show that individuals do not stick to these information samples but expand them when given the opportunity. We found this to be the case particularly when (initial) advice deviated from their own views. That is, the more the advisory estimates differed from the own initial judgment, the more people searched for additional information. Interestingly, a post-hoc comparison of average sample sizes in Experiments 2 and 3 reveals that sampling frequency when advice was close was significantly higher in Experiment 3 (b exp = 2.54, se = 1.16, t = 2.18; b exp*distance = –0.67, se = 0.267, t = –2.49), hinting at overall variance or inconsistency as another moderator of sampling frequency.
Advice weighting was sensitive to the sampled information with distant advice (Assumption 2a) and frequency of sampling increasing the weight of advice (Assumption 2b). The latter finding is supported by a comparison of the WOA for Experiments 1 and 2. The opportunity to sample additional advice significantly reduced egocentric discounting (i.e., increased the WOA) on both close (b exp = .095, se = .036, t = 2.66) and distant trials, (b exp*distance = .132, se = .025, t = 5.22).
In sum, the present results demonstrate a high degree of adaptiveness of advice taking and add to the controversy about the roots of egocentric discounting (Reference Soll and MannesSoll & Mannes, 2011; Reference Yaniv and Choshen-HillelYaniv & Choshen-Hillel, 2012). Building on informational asymmetry as a core property of advice taking, the results show that egocentric tendencies can be reduced via the sampling of information.
The present research complements findings from a related research area that investigates sampling behavior prior to the choosing of options. In research on optional stopping, for instance, participants sample (costly) offers until they stop sampling and keep the latest offer (e.g., Reference Bearden and RapoportBearden & Rapoport, 2005; Reference Rapoport and TverskyRapoport & Tversky, 1966; Reference Seale and RapoportSeale & Rapoport, 1997). In research investigating decisions from experience, participants are usually presented with two risky lotteries and explore their outcomes before deciding which one of these lotteries to play for actual profit (Reference Hertwig, Barron, Weber and ErevHertwig, Barron, Weber & Erev, 2004; Reference Hertwig and ErevHertwig & Erev, 2009). Although there is some debate about the optimal sample size (e.g. Reference Hertwig and PleskacHertwig & Pleskac, 2010), this research generally demonstrates that participants sample substantially, investing money and/or time. Moreover, as in the present research the amount of sampling is adaptive, being sensitive to the variability of the sample (e.g., Reference Lejarraga, Hertwig and GonzalezLejarraga, Hertwig & Gonzalez, 2012; Reference Mitra, Reiss and CapellaMitra, Reiss & Capella, 1999; Reference Pachur and ScheibehennePachur & Scheibehenne, 2012), the importance of the decision (Reference Hau, Pleskac, Kiefer and HertwigHau, Pleskac, Kiefer & Hertwig, 2008), and competition for the choice options (Reference Phillips, Hertwig, Kareev and AvrahamiPhillips, Hertwig, Kareev & Avrahami, 2014). In these paradigms, participants try to maximize their profit. In the present research, the structure of motives is different in that the task invites both a motive for affirmation and an accuracy motive. The latter is only indirectly linked to financial outcomes. Moreover, participants are oriented towards the central tendency (or consistency) of the sampled information. Nevertheless, the two research areas converge on the adaptiveness of sampling.
At first glance, the present findings are at odds with an analysis presented by Reference Soll and MannesSoll and Mannes (2011). In an attempt to test whether differential access to reasons caused the differential weighting of own and others’ judgments, they created a paradigm in which participants predicted outcomes from experimentally controlled cues. Whether or not these cues were presented during judgment revision did not affect advice integration. From this null-effect, we could conclude that the differential availability of reasons does not explain egocentric discounting effects. Alternatively, we could conclude that the mere repetition of information underlying the initial judgment was not effective in increasing egocentric tendencies. In the present paradigm, we investigated the effects of seeking additional information on advice integration. In contrast to Reference Soll and MannesSoll and Mannes (2011), we show that an increase of the advisory information base by sampling reduces egocentric tendencies.
5.2 Limitations and future directions
The present results demonstrate the adaptiveness of advice taking in terms of its flexibility in light of certain affordances in the environment. Another connotation of the term “adaptiveness”, however, concerns the improvement that comes with this flexibility. The present research did not address the second type of adaptiveness for two reasons. First, the current research was focused on whether and under what conditions decision makers consult advisors. Second, the manipulated advice generally pointed towards the true value (except for the sampled advice in Experiment 3), so that we refrained from reporting effects on accuracy, as those are bound to this artificial setting. Future research should thus ask whether the sort of flexibility shown here improves accuracy under ecologically valid conditions.
The laboratory setting also limits the generalization of the present findings to real-life decision making. It is well established that people preferably turn to close and similar others for advice (Reference McPherson, Smith-Lovin and CookMcPherson, Smith-Lovin & Cook, 2001; Reference Suls, Martin and WheelerSuls, Martin & Wheeler, 2002) and that they even assume these others to be representative of the actual population distribution (Reference Galesic, Olsson and RieskampGalesic, Olsson & Rieskamp, 2012). One might thus argue that people would not even encounter diverging advice in their ecology. However, two findings render us optimistic. First, people exhibited substantial sampling even in light of similar advice. Thus, the chance of encountering even rare events during this sampling process is not negligible. Second, advice in Experiment 3 was simulated based on the actual population distribution parameters by a previous sample from the same student population, the same group of peers that participants would most likely turn to for advice in real life.
To assess preferences for close versus distant advice in an even stricter manner, both types of advice should be offered concurrently. As people preferably turn to close and similar others for advice (Reference McPherson, Smith-Lovin and CookMcPherson et al., 2001; Reference Suls, Martin and WheelerSuls et al., 2002), participants might appear to be less well adapted when being offered both types of advice simultaneously, preferring advice consistent with their opinion. Moreover, the present paradigm can be extended to advisor characteristics that were shown to affect conformity in advice taking and other, social psychological paradigms such as persuasion. For instance, from the present perspective it can be expected that characteristics that communicate competence and trustworthiness such as expertise and reputation would strongly influence advisor choice. However, research has also shown that individuals sometimes utilize communicator characteristics that can be misleading (e.g., Chaiken, 1979). Consequently, it remains to be investigated whether the adaptiveness of advice seeking shown here can be extended to advisor choice.
The present research is the first to look at the sampling of advice. Based on the informational asymmetry hypothesis, we focused on the seeking of advice and its effect on judgment revision. We demonstrated that the breadth and consistency of the advisory estimates influences the degree to which they are integrated with the decision maker’s initial estimate. At the same time, however, the present research neglects the role of the breadth and consistency of the decision maker’s initial knowledge base. We assume that an assessment of the breadth and consistency of the decision maker’s knowledge base will result in a feeling of confidence (Reference KoriatKoriat, 2012). Specifically, decision makers who have different pieces of information supporting a given judgment will feel more confident than decision makers with very little or inconsistent information. Presumably, the decision maker’s confidence will affect the amount of advice sampled, such that highly confident decision makers will sample less advice than less confident ones. Moreover, even if decision makers sampled the same amount of information, the degree of integration of the advice with their own estimate should depend on their confidence in their initial estimate. In sum, the present perspective should be complemented by an orientation towards the individual knowledge base in future research.
Although the present work focused on factors instigating sampling behavior, little is known about the factors determining the truncation of the sampling process in advice seeking. Recent work in the domain of decisions from experience assumes that judges set preference thresholds for the choice options (Reference Markant, Pleskac, Diederich, Pachur and HertwigMarkant, Pleskac, Diederich, Pachur & Hertwig, 2015). Sampling is terminated as soon as the preference state exceeds a given threshold. This model is in line with the influences of decision importance (i.e., a manipulation of reward magnitude) on sampling frequency in this paradigm (Reference Hau, Pleskac, Kiefer and HertwigHau et al., 2008). Such threshold assumptions could also be made for the sampling of advice. For instance, recent research has shown that people are more strongly inclined to receive advice when they are anxious (Reference Gino, Brooks and SchweitzerGino et al., 2012) or when the task is difficult (Reference Gino and MooreGino & Moore, 2007). Thus, it is conceivable that people compare their present level of knowledge to a threshold of desired knowledge, causing people to sample more information as long as this threshold is not reached and more information is still available. Hence, more frequent sampling could both be due to lower levels of knowledge as well as higher knowledge thresholds. Future research should thus investigate whether such thresholds are set prior to sampling and to what degree these thresholds are sensitive to the sampling process.
Despite the beneficial effects of sampling on advice weighting, averaging proper was not a dominant integration strategy in the present experiments. While the normatively desirable WOA is 0.5 (giving equal weight to one’s own position and the average advice), the largest average WOA achieved in the present experiments was 0.4. Presumably, final information samples were still biased towards the initial judgment. It is hardly conceivable that the sampled advice matched informational richness within individuals’ minds, so that information samples likely remain uneven, especially in terms of qualitative arguments underlying each judgment. Moreover, it is possible that motivational factors also influence advice sampling and weighting. One aspect to consider is that we asked participants to give an initial estimate and, thus, form an opinion. This procedural detail could also be responsible for lingering egocentric tendencies in the present research. Previous research (Reference Koehler and BeauregardKoehler & Beauregard, 2006; Reference Yaniv and Choshen-HillelYaniv & Choshen-Hillel, 2012) has shown that judgments were less influenced by advice when participants gave an initial judgment. Without an initial judgment, participants were influenced by advice much more strongly (without being aware of that influence). Possibly, the sampling and weighting of advice would then depend mainly on the consensus among the advisors (Reference Budescu and YuBudescu & Yu, 2007), and equal weighting strategies would be more frequent.
5.3 Conclusion
The present research introduces a sampling approach to advice taking. Our findings corroborate the importance of the information ecology as a central influencing factor for people’s receptivity for advice (Reference Gino, Brooks and SchweitzerGino et al., 2012; Reference Gino and MooreGino & Moore, 2007; Reference Sniezek and BuckleySniezek & Buckley, 1995). Three experiments showed substantial sampling of advice that further increased when advice was different from rather than confirming participants’ own positions. Participants integrated diverging advice more strongly into their own judgment. Moreover, participants were sensitive to the information sampled, relying more strongly on advice when it was supported by additionally sampled advisory estimates. In summary, advice taking is sensitive to the information ecology and the human mind appears adaptive when allowed to be.
Appendix: Sequential pair-wise model comparisons
Experiment 1
Experiment 2
Experiment 3
Note. Degrees of freedom are based on the Kenward-Roger approximation (Reference Kenward and RogerKenward & Roger, 1997). For each dependent variable (DV), fit of each model was compared to the fit of the model in the row above. Approximate Bayes factors were calculated using the BIC (Reference MassonMasson, 2011). The approx. Bayes factor indicates the conditional probability of the model being true given the data obtained in comparison to the model in the row above.
Open access
