In the present chapter we investigate how reward-rich environments can lead to the persistence of (initial) biases. More specifically, we argue that frequent rewards invite the exploitation of a supposedly best option which in turn will reinforce the biased preference. Because feedback is often contingent on the choices made, exploitation will result mostly in the aggregation of information about the exploited option. This, in turn, restricts the extent to which beliefs can be updated, with downstream consequences for further decisions. This dynamic might be responsible for why false beliefs about the outcomes of behavioral options can be maintained even when decision makers are motivated to choose the best choice alternative. We present data from simulations and empirical work to support this argument and conclude that the exploration–exploitation tradeoff serves as a particularly vivid example of the interplay between one’s cognition (goal-directed) behavior, and the sample that is aggregated.