1. Introduction
How do consumers come to know what they like? This question is increasingly relevant in economies characterized by high innovation and differentiation. Consumers usually face several varieties of one good like different brands or different musical styles. Within a period, they may consume sequentially more than one variety and experience heterogeneous products from each variety to learn what they like. The multiplicity of varieties of each good, of unknown products within each variety, and the need to experience these products sequentially generate choice uncertainty in a dynamic setting. However, the dynamic and stochastic nature of consumer choice is usually masked by statistics of consumption data through a process of product and time aggregation. Hence, while observational data show the quantities of all varieties consumed in a discrete period as if consumers had solved a static problem of choice among a set of homogeneous varieties, we conjecture that rational consumers have been facing a stochastic multi-armed bandit problem through repeated consumption of different varieties and products. This conjecture about real-world behavior will be tested in an experiment that forces subjects to make repeated consumption choices among heterogeneous varieties of an experience good thus revealing the underlying sequential decision-making process hidden by observational data.Footnote 1 Restoring the hidden dimensions of consumer choices in a controlled environment will provide a rational interpretation of random utility in discrete consumer choices which does not have to rest on unobservable perception and/or decision errors in a static setting.
We focus on the dynamic and stochastic nature of consumer choices by examining how people ‘learn by consuming’ in a unique lab experiment. Given our focus on consumption and preference for goods, the experiment will be conducted in the natural context of music consumption. Our experiment offers 50 repeated choices between four musical styles (i.e., varieties) and measurement of post-choice satisfaction. In addition, it contains two treatments providing unequal opportunities for learning. To focus on the dynamics of preference discovery, goods can be acquired for free. Subjects repeatedly consume the musical style they select, which provides non-pecuniary incentives to choose what they like.
We chose music because of its external validity: Consuming music in the lab is no different from streaming music in isolation. Moreover, music appears to be especially fit for uncovering the individual strategies masked by statistical aggregation of consumption data in the limited time of an experiment. Indeed, it is sufficiently universal to concern all subjects, sufficiently pleasant to avoid boredom, sufficiently unfamiliar to allow learning, and sufficiently heterogeneous to avoid rapid satiation. Finally, music allows us to contrast behavior when faced with familiar or unfamiliar styles. Thanks to the nice properties of music, this experiment is perhaps the first to investigate repeated consumption choices in the lab, with direct measure of experienced utility. Having many repetitions enables us to uncover exploration strategies, including random exploration and diversification that would otherwise be hidden in aggregate data.
This repeated consumption task can be viewed as a four-armed bandit problem in which consumers who seek to maximize their experienced utility over an extended period choose one musical style at a time, unaware of which music piece will be played and how much utility they will derive from it. Each consumption reveals an experienced utility that provides additional information on the distribution of rewards between musical styles. This problem is non-trivial, however, because even though consumers evaluate precisely the experienced utility from the products they consume after each round, their choice yields no information about non-selected varieties. Therefore, they need to explore different varieties to acquire new information, while the exploitation of the optimal decision so far would yield the highest expected utility conditional on prior information. In essence, they seek to learn which varieties are best for themselves while not spending too much time exploring. Moreover, while adoption of an optimal policy allows the consumer to converge toward stable latent preferences, preference learning is complicated in the short run by the phenomenon of satiation in consumption goods. This mechanism generates a preference for diversity.
We have two treatments, distinguishing between familiar music and unfamiliar music, to check whether consumers make more stable and concentrated choices, and less random exploration, when faced with more familiar styles. We estimate nested Bayesian bandit models of learning tastes on the data from the two treatments. They are adapted to consumer choices by allowing for satiation of utility with repeated consumption of goods in the form of diminishing experienced marginal utility. Our best performing model features decreasing random exploration of the choice options. Early exploration is more intensive in the unfamiliar treatment and persists throughout the experiment in both treatments. Overall, subjects make choices that deviate from their best prediction 61.5% of the time in the unfamiliar treatment versus 44.7% of the time in the familiar treatment. The nature of exploration however differs between treatments: subjects adopt a strategy of randomization in the unfamiliar treatment (50% always randomize), while they also adopt a strategy of diversification in the familiar treatment (they never choose randomly all the time).
We observed substantial heterogeneity in consumer behavior. Estimated exploration rates were five times higher (resp. 3.4) for subjects with high openness in the unfamiliar (resp. familiar) treatment. We also observe differences between high explorers and low explorers. In the familiar treatment, the diversification strategy adopted by high explorers protected them from excessive satiation and resulted in higher experienced utility overall. However, exploration was not beneficial overall in the unfamiliar treatment as high explorers, choosing randomly, took the risk of consuming too much of a variety they did not like, and low explorers converged sooner on what they liked better but exposed themselves to stronger satiation.
The rest of the paper is organized as follows. Section 2 reviews the related literature. Section 3 describes the experimental design and data. A Bayesian bandit model of consumer choice with satiation and random exploration is then presented in Section 4. A reduced form estimation is provided in Section 5, and the structural estimation appears in Section 6. Section 7 provides evidence of the exploration-exploitation trade-off and of individual differences in exploration intensity. Section 8 concludes.
2. Related literature
Conventional economic theory often views consumption choices as static and deterministic, reflecting known and stable individual preferences. However, both consumption and experimental data suggest that choices are inherently dynamic and stochastic. Stigler and Becker (Reference Stigler and Becker1977) reconciled the dynamic nature of consumption behavior with traditional economic theory by proposing that preferences are acquired through the accumulation of consumption-specific human capital. For example, repeated exposure to music develops ‘musical capital,’ thereby enhancing one’s taste for music. Plott (Reference Plott, Arrow, Colombatto, Perlman and Schmidt1996) explained experimental inconsistencies of choice by suggesting that individuals have stable latent preferences, but these preferences are initially unknown and must be discovered through experience (similar ideas can be found in Smith, 1994; Binmore, Reference Binmore1999; Camerer & Ho, Reference Camerer and Ho1999). Preference reversals are then caused by some type of myopia when subjects face unfamiliar tasks. With incentives and experience, however, choices become rational and reflect stable preferences. Using data on theater consumption, Lévy-Garboua and Montmarquette (Reference Lévy-Garboua and Montmarquette1996) proposed and tested an adaptive model of discovered preferences. Delaney et al. (Reference Delaney, Jacobson and Moenig2020) examine preference discovery focusing on the question of what is to be learned or not.
Another account of the stochastic nature of preferences is random utility theory, in which the utility of a discrete choice cannot be known with full certainty (e.g., Becker et al., Reference Becker, DeGroot and Marschak1963; Loomes & Sugden, Reference Loomes and Sugden1995). Intrinsic randomness would be due to measurement errors, preference changes, taste for diversity, or choice inconsistencies. Hence, the model cannot forecast what choice is made, but it can estimate the probability that a given option in the choice set be selected as a ratio of utilities (McFadden, Reference McFadden2001).
Our experiment describes the process of preference learning as a multi-armed bandit problem. Theoretical literature on multi-armed bandit problems has grown exponentially during the last decades (see Gittins and Jones, Reference Gittins, Jones and Gani1974 for a deterministic solution in an infinite horizon with exponential discounting, Bergemann and Valimaki, Reference Bergemann and Valimaki2006 for economic applications, Scott, Reference Scott2010 and Burtini et al., Reference Burtini, Loeppky and Lawrence2015 for surveys of stochastic models of exploration-exploitation). However, few experimental tests of single-agent bandit problems have appeared so far in the literature. Banks et al. (Reference Banks, Olson and Porter1997) first implemented a two-armed bandit where one arm is safe and the other is a simple win-lose (Bernoulli) distribution. Implementing an exponential bandit model, Hudja and Woods (Reference Hudja and Woods2023) observed that subjects respond to changes in the prior belief, safe action, and discount factor in the predicted direction, yet under‐respond to these changes. Steyvers et al. (Reference Steyvers, Lee and Wagenmakers2009) asked 451 participants to solve a number of bandit problems. Using Bayesian model selection measures that assess how well people adhere to an optimal decision process compared to simpler heuristic decision strategies, they related individual performances to observed differences in cognitive abilities and personality traits. Lévy-Garboua et al. (Reference Lévy-Garboua, Gazel, Berlin, Dul and Lubart2024) designed an experiment where 169 high school students solved a bandit problem with a known optimal solution to draw a parallel between their measured creativity and their ability to perform optimally in a bandit task.
3. Preference learning in the lab
3.1. Experimental design
Conducting a lab experiment on preference learning requires goods that satisfy at least four conditions. First, subjects must be able to choose and consume the goods repeatedly in the lab. Second, the goods should be relatively unfamiliar to the subjects so that there is room for preference learning. Third, consuming the goods should be sufficiently pleasant to avoid boredom. Fourth, although relatively uniform, two units of the same good should be sufficiently different to limit satiation. To meet these challenges, we conceived an experiment in which subjects chose and consumed music. Besides, music is a good experimental ‘model,’ as music consumption has been shown to produce strong rewards, similar to those of food and money (Salimpoor et al., Reference Salimpoor, Benovoy, Larcher, Dagher and Zatorre2011), and consuming music in the lab is no different from streaming music in isolation.
We conducted two treatments. In the ‘familiar’ treatment, 68 subjects had to choose in each round between four popular musical styles: Pop/Rock, Classical, Rap/RnB, and Jazz/Blues. In the ‘unfamiliar’ treatment, 72 subjects had to choose in each round between traditional music from four regions of the world: Africa, America, Europe, and Asia.Footnote 2 Unlike the musical styles in the familiar treatment, our subjects (mostly university students) were unlikely to have been exposed to traditional music prior to the experiment. As a result, the treatments create asymmetric opportunities for subjects to learn their own preferences through listening experience.
Special care was taken when constructing the set of 400 pieces of music. The objective was to select sufficiently different tracks within each style, while having styles properly partitioned and an equal amount of heterogeneity within each style. To confirm that the tracks selected met these criteria, we conducted a categorization task with external judges drawn from the same subject pool (see Appendix I for the description of the task and the results).
At the start of the experiment, subjects filled out a questionnaire where they indicated their age, gender, personality (Brief Big Five, Barbot, Reference Barbot2011; Costa & McCrae, Reference Costa and McCrae1992), and prior frequency of listening to the four musical styles selected for the experiment on a 4-level scale.
The experiment consisted of 50 rounds, each partitioned in 3 stages. In stage 1, the subject selected one of four musical styles. In stage 2, a one-minute sample from the musical style chosen by the subject in stage 1 was played. The subject wore closed-back headphones and had to listen to the entire sample while facing a blank screen. The duration of the sample (one minute) was chosen to be sufficiently long to allow a precise rating of the experience, but short enough to avoid ear fatigue. In addition, an acoustic adjustment was applied to homogenize sound levels within and across music samples so that listening experiences were both comfortable and homogeneous. In stage 3, the subject was asked to rate his listening experience on a 10-point scale. This rating is used as a proxy for the subject’s experienced utility (see, for instance, Kahneman et al., Reference Kahneman, Wakker and Sarin1997). This rating automatically captures any change in utility resulting from increased satiation or appreciation. Furthermore, it can capture the subject’s risk attitude, since the decision process takes place in a state of uncertainty.Footnote 3
Subjects received a show-up fee of 15 euros. In addition, they had strong non-pecuniary incentives to choose a musical style carefully in stage 1, given that they would immediately experience the consequence of their choice (i.e., listening to a one-minute sample) and that they would have to make a large number of choices. To encourage truthful reporting in stage 3, each subject received a personalized 10-track digital recording designed to mirror the experienced utility reported during the experiment. Specifically, a subject was told that 5 sets of 10 randomly-drawn rounds would be compiled at the end of the experiment to create their CD. For each set, we calculated the average experienced utility reported by the subject for the music samples he chose in those rounds. The 10 tracks from the set with the highest average experienced utility were recorded on a device which was then offered to the subject.Footnote 4 Thus, subjects had extrinsic incentives to rate their listening experience carefully.
3.2. From a static to dynamic look at the data
In the familiar treatment, average listening frequencies show a preference for Pop/Rock (40% of choices) and Rap/RnB (34%) versus Classical (15%) and Jazz/Blues (11%). In the unfamiliar treatment, we observe a low frequency of choices for Asian style (18% of choices) while the remaining choices are almost equally spread between Africa (28%), Europe (25%), and America (29%). From a static perspective, our data can be interpreted as if a representative consumer exhibited a preference for diversity and consumed all musical styles during the experiment.
Figure 1 describes the evolution of the listening shares for each style across the 50 rounds of the experiment. In both treatments, subjects explore a lot during the first 12 rounds or so. Convergence toward preferred styles occurs quickly in the familiar treatment while convergence is limited and delayed in the unfamiliar treatment, essentially away from the Asian style. Consumers maintain diversified preferences, probably in response to the decline in experienced marginal utility of more heavily consumed styles and to the persistent uncertainty regarding the specific sample they will have to listen to in each round.
Evolution of shares of the musical styles over 50 rounds. (a) Familiar treatment. (b) Unfamiliar treatment
Fig. 1 Long description
The image A shows a line graph titled 'Familiar treatment' with the x-axis labeled 'Period' and the y-axis labeled 'Share (percent) number of times the style was chosen'. It displays four lines representing Pop/Rock, Rap, Classic and Jazz over 50 periods. Pop/Rock starts around 35 percent, fluctuates and ends near 35 percent. Rap begins near 5 percent, rises and stabilizes around 15 percent. Classic and Jazz both start below 5 percent, with Classic rising slightly and Jazz remaining low. The image B shows a line graph titled 'Unfamiliar treatment' with the same axes. It features lines for Africa, Europe, America and Asia. All styles fluctuate between 5 and 30 percent, with no clear dominant style. Africa and Europe show more variability, while America and Asia maintain moderate shares throughout the periods.
3.3. Exploration and convergence
We provide a characterization of the intensive exploration phase and a synthetic proof of convergence by comparing the Herfindahl index (H) calculated over three time-intervals: 1–12, 13–25, and 26–50. The Herfindahl index is defined as 
$H = \mathop \sum \limits_{j = 1}^4 shar{e_j}^2$ where 
$shar{e_j}$ denotes the share of consumption style j for a subject over a given time interval. It captures how subjects’ selection of musical styles is concentrated during a given interval, notwithstanding which style they choose. For instance, subjects who listened three times to each of the four competing styles over 12 consecutive rounds will have a minimal H index value of 0.25. Meanwhile subjects who listened three times to two styles, but only twice to a third style and four times to the fourth style, yield the second smallest H value of 0.2639.
Table 1 reports the Herfindahl values for the two treatments. The time interval (1–12) seems to be a phase of intensive exploration in both treatments, such that the H index takes its minimal value on average. Convergence toward more concentrated choices occurs after this initial phase though, as expected, preferences are less concentrated for unfamiliar styles. After this exploration phase, the mean value of H increases significantly for familiar and unfamiliar musical styles, remaining almost constant thereafter.Footnote 5
Mean Herfindahl across time intervals and treatment
Table 1 Long description
The table measures mean Herfindahl scores across three time intervals for familiar and unfamiliar styles, indicating preference consistency. Familiar styles have higher scores than unfamiliar ones, with scores of 0.42, 0.54, and 0.52 compared to 0.29, 0.37, and 0.37 for unfamiliar styles. Significant differences are observed between the first two intervals for both styles, with p-values less than 0.001. However, scores between the second and third intervals show no significant change, with p-values of 0.522 for familiar and 0.843 for unfamiliar styles. The consistent scores in later intervals suggest stable preferences over time.
Note: 
$H = \mathop{\sum}\nolimits^{4}_{j = 1}share_{j}^{2}$ is computed for each individual, where 
$shar{e_j}$ is the proportion of rounds in which style j was chosen over a given time-interval. H may vary between 0.25 and 1.
a Two-sided paired t-tests (equality of means).
b Two-sided unpaired t-tests (equality of means).
3.4. Repeated consumption and satiation
Table 2 reports the mean experienced utility in both treatments across time intervals 1–12, 13–25, and 26–50. Experienced utility appears uniformly lower in the unfamiliar treatment, but the two treatments follow a parallel trend. Subjects experience significantly lower utility levels on average during the second half of the experiment, although they explore less in later rounds and are thus more prone to choose their preferred styles. This confirms the erosion of marginal experienced utility through repeated consumption, that is, satiation.
Mean experienced utility across time intervals and treatment
Table 2 Long description
The table measures mean experienced utility across three time intervals for familiar and unfamiliar treatments. Familiar experiences have mean utilities of 5.66, 5.82, and 5.47 across intervals 1-12, 13-25, and 26-50, respectively. Unfamiliar experiences show mean utilities of 5.07, 4.96, and 4.72 for the same intervals. A significant decrease in utility is observed for familiar experiences from intervals 13-25 to 26-50 (p=0.001), while unfamiliar experiences show a less pronounced decrease (p=0.012). The t-tests indicate significant differences between familiar and unfamiliar experiences across all intervals, with p-values of 0.002, <0.001, and <0.001, respectively. These results suggest that familiarity impacts the stability of experienced utility over time.
a Two-sided paired t-tests (equality of means).
b Two-sided unpaired t-tests (equality of means).
In the next section, we introduce a Bayesian bandit model of taste and consumer’s choice that will be estimated econometrically on the experimental data to provide further insights on the dynamic and stochastic aspects of learning.
4. Bayesian bandit model of learning taste
4.1. Learning model of taste with satiation
Consumers face, for each experience good like wine or music, a set of heterogeneous products classified into varieties sharing common features. For example, when selecting a wine, one could choose between Pinot and Cabernet, each forming a variety. Likewise, when listening to music, one could choose between Pop/Rock and Classical music, each forming a different style. The model assumes that consumers have a taste for each variety of good, by choosing for each round one variety of this good and consuming one product randomly drawn from this variety.
Set-up. Each round 
$t \geqslant 1$ consists of three phases. In phase 1, agent i selects a variety j among 
$J$ mutually exclusive alternatives. His discrete choice is described by a set of dichotomous variables 
${x_{jt}}$ s.t. 
${x_{jt}} = 1$ and 
${x_{j{^{\prime}}t}} = 0\,\forall \,j' \ne j$.Footnote 6 In phase 2, the agent consumes a randomly drawn product from variety j. When he selects a variety in phase 1, the agent does not know precisely the marginal utility he will experience from consumption of the product randomly drawn from this variety in phase 2. In phase 3, the agent updates their beliefs based on the marginal utility they experienced from that consumption.
Tastes. We assume that the agent has a stable but imperfectly known latent taste for each variety. The taste for variety j follows a normal distribution 
${v_{jt}}\sim \mathcal{N}\left( {{\mu _j},{h_j}} \right)$, where the mean 
${\mu _j}$ is the unknown latent taste and 
${h_j}$ is the variance assumed to be known by the consumer.Footnote 7 
${\mu _j}$ is to be learned by the agent through repeated consumption over time.
Taste for variety j in round t, 
${\upsilon _{jt}}$, is not directly experienced by the consumer but can be discovered by him through realizations of the experienced marginal utility 
${u_{jt}}$. The latter generally decreases with repeated consumption of the variety within the duration of the experiment, due to satiation, and deviates deterministically from 
${\upsilon _{jt}}$. We describe this relation by:
\begin{equation}{u_{jt}} = {\upsilon _{jt}} + \frac{{{\alpha _j}}}{{1 + {n_{jt}}}}\end{equation}where 
${n_{jt}},\,\left( {0 \leqslant {n_{jt}} \leqslant t} \right)$ designates the number of times variety j was consumed in the first 
$t$ rounds (i.e., 
${n_{jt}} = \mathop{\sum}\nolimits^{t}_{\theta = 1} {x_{j\theta }}$), and 
${\alpha _j}$ is a variety-specific parameter assumed to be known by the agent. We assume that 
${\alpha _j}$ is non-negative and describes the law of decreasing marginal utility.Footnote 8 While we expect satiation to occur, marginal experienced utility might increase for certain varieties if taste was acquired through repeated exposure.Footnote 9 Unlike the distribution of taste 
${v_{jt}}$, the distribution of experienced marginal utility is not stable over time when 
${\alpha _j} \ne 0$, that is 
${u_{jt}}\sim \mathcal{N}\left( {{\mu _j} + \frac{{{\alpha _j}}}{{1 + {n_{jt}}}},{h_j}} \right)$.
After consuming a product from variety j, the agent discovers his taste for this variety, 
${\upsilon _{jt}}$.Footnote 10 He needs to sample many products from the same variety to get a precise estimate of his mean taste 
${\mu _j}$ for this variety.
If the agent knew every latent taste 
${\mu _j}$, then his latent preferences would produce a stable ranking for the 
$J$ varieties. However, because he does not know 
${\mu _j}$ in any round 
$t$, consequently latent ranking is also unknown. Instead, the agent’s choices entail an unstable ranking of the 
$J$ varieties. Unlike latent preferences, choices can be path-dependent and endogenous.
Taste Learning. In our model of taste learning, the agent’s choices converge toward his latent preferences. After consuming a random product from variety j in round t (i.e., 
${x_{jt}} = 1$), the agent updates his prior beliefs (i.e., estimate of own taste) by applying Bayes’ rule, based on the information contained in the signal 
${u_{jt}}$. For varieties which are not consumed in round t, beliefs do not change. The prior distribution of beliefs is assumed to be normal, with known mean and variance 
$\left( {{\mu _{j0}},{k_{j0}}} \right)$. We will also assume in the following that 
${k_{j0}}$ is homogeneous across agents. The agent predicts he will have taste 
${\mu _{j0}}$ for variety j at round 1. After consuming a random product from this variety, he transforms the mean of his normally distributed prior taste in a Bayesian fashion (see DeGroot, Reference DeGroot1970, chapter 9). The law of motion for the predicted taste 
${\mu _{jt}}$ and variance 
${k_{jt}}$ in subsequent rounds is as follows (see Appendix III)Footnote 11:
\begin{equation}{\mu _{jt}} = \left( {1 - {\rho _{jt}}} \right)\!{\mu _{j,t - 1}} + {\rho _{jt}}{\upsilon _{jt}};\,{k_{jt}} = \left( {1 - {\rho _{jt}}} \right)\!{k_{j,t - 1}};\,{\text{with}}\,{\rho _{jt}} = {x_{jt}}\frac{{{k_{jt}}}}{{{h_j}}}\end{equation} 
${\rho _{jt}}$ can be interpreted as a ‘reinforcement rate’ 
$\left( {0 \leqslant {\rho _{jt}} \lt 1} \right)$, that is, the weight given to the informative signal 
${\upsilon _{jt}}$ relative to that of the prior 
${\mu _{jt - 1}}$. The three equations (2) determine the three unknowns 
${\mu _{jt,}}{k_{jt}},{\rho _{jt}}$ in each round t, given the fixed parameters 
${\mu _{j0}}$, 
${h_j}$, 
${\alpha _j}$, and the observables 
${x_{jt}}$ and 
${u_{jt}}$.
Alternatively, the first equation in (2) can be written as 
${\mu _{jt}} - {\mu _{jt - 1}} = {\rho _{jt}}\left( {{\upsilon _{jt}} - {\mu _{jt - 1}}} \right)$ and interpreted as an adjustment of predicted taste proportionally to the ‘reward prediction error’ 
$\left( {{\upsilon _{jt}} - {\mu _{jt - 1}}} \right)$.Footnote 12 The predicted taste 
${\mu _{jt}}$ of an experienced variety converges through the learning process and the reinforcement rate tends toward zero when a given variety is experienced repeatedly. Note that convergence of the reinforcement rate toward zero predicts the tendency for learning curves to be steep initially and then flatter (see Blackburn, Reference Blackburn1936). Equation (2) also demonstrates that first impressions matter a lot for less familiar varieties, characterized by a high variance of prior beliefs 
${k_{j0}}$ and, plausibly, by a low prior taste 
${\mu _{j0}}$.
Since taste is not directly experienced by the consumer in the presence of satiation, we estimate the law of motion of mean experienced marginal utility 
$\mu {'_{jt}} \equiv {\mu _{jt}} + \frac{{{\alpha _j}}}{{1 + {n_{jt}}}}$, for all 
$t \geqslant 0\,$, provided parameter 
${\alpha _j}$ is known by the econometrician. It can be derived from the law of motion of taste in equation (2), with the help of definition (1) to express taste as a function of experienced utility. An explicit recurrence equation for updating the mean experienced marginal utility 
$\mu {'_{jt}}$ is provided by:
\begin{equation}\mu {'_{jt}} = \left( {1 - {\rho _{jt}}} \right)\left( {{{\mu {^{\prime}}}_{j,t - 1}} - \frac{{{\alpha _j}{x_{jt}}}}{{\left( {1 + {n_{jt}}} \right)\left( {1 + {n_{jt - 1}}} \right)}}} \right) + {\rho _{jt}}{u_{jt}},\,\forall t \geqslant 1\end{equation}which is the law of motion of taste with an additional term 
$ - \left( {1 - {\rho _{jt}}} \right)\frac{{{\alpha _j}{x_{jt}}}}{{(1 + {n_{jt)}}\left( {1 + {n_{jt - 1}}} \right)}}$. The latter converges to 0 when 
${n_{jt}}$ rises indefinitely. After only four experiences, it is divided by 20. Thus, even if 
${\alpha _j} \ne 0$, the predicted mean taste and mean experienced marginal utility should eventually converge with experience to a common value 
$\left( {\mu {'_j} = {\mu _j}} \right)$.
4.2 Decision model with random exploration
We assume that the agent’s decision utility 
${U_{jt}}$ in phase 1 of round t incorporates a random component 
${\xi _{jt}}$ observed by the agent during this phase.Footnote 13 If the agent correctly anticipates a decrease in experienced marginal utility if variety j were to be consumed in the next round, then his stochastic decision utility is given by equation (4):
\begin{equation}{U_{jt}} = {\mu _{j,t - 1}} + \frac{{{\alpha _j}}}{{2 + {n_{jt - 1}}}} + {\xi _{jt}},\end{equation}where 
${\mu _{j,t - 1}} + \frac{{{\alpha _j}}}{{2 + {n_{jt - 1}}}}$ is the predicted marginal experienced utility and 
${\xi _{jt}}$ is a realization of the random decision component 
${\xi _{jt}}\sim Logistic\left( {0,\tau } \right)$ of zero mean and standard deviation 
$\tau $.
Several reasons for introducing a stochastic element in consumer choices can be found in the ‘random utility’ literature, which rests on a static decision model. In contrast, our consumer faces a stochastic bandit problem, and we interpret the random component of decision utility as originating from the nature of experience goods and from the random exploration of different varieties. We do not estimate the standard version of the bandit model in which the reward distribution is not assumed because it is difficult to apply and restricted to special cases. Instead, we adopt the ‘randomized probability matching’ heuristic put forward by the multi-armed bandit literature, which is compatible with Bayesian updates of the posterior distribution. This Bayesian modern approach can be more easily implemented and may even outperform the standard ‘optimal’ solution, as explained by Scott (Reference Scott2010, p.640).Footnote 14 In equation (5) below, the agent deliberately matches the computable probability of selecting variety j in round t with its probability of being optimal conditional on his updated beliefs. Thus, randomized probability matching endogenizes the random utility model. Ex ante (i.e., before 
${\xi _{jt}}$ is realized), the probability that variety j be selected in round t is:
\begin{equation}{P_{jt}} = P\left( {{x_{jt}} = 1} \right) = \mathop{\smallint}\limits_{\Theta_{jt}} \phi \left( {{\xi _t}} \right)d{\xi _t}\end{equation}where 
${\xi _t} = \left( {{\xi _{1t}}, \ldots ,{\xi _{Jt}}} \right)'$, 
${{{\Theta }}_{jt}} = \left\{ {{\xi _t}\left| {{U_{jt}}} \right. \gt \,{U_{j{^{\prime}}t}}\,\forall j{^{\prime}} \ne j} \right\}$ and 
$\phi \left( . \right)$ is the pdf of the logistic distribution with zero mean and covariance matrix 
${\tau ^2}{I_j}$ (Train, Reference Train2009).
Observe that this decision model is general since it captures the full range of behavior from pure exploitation to pure exploration. When 
$\tau = 0$, decision and predicted utilities are equal, according to equation (4), and decision behavior is deterministic. In every round t, the agent maximizes his predicted marginal utility conditional on the information acquired from prior experiences and selects 
${j^*} = Argma{x_{j \in J}}\left( {{\mu _{j,t - 1}} + \frac{{{\alpha _j}}}{{2 + {n_{jt - 1}}}}} \right)$, so that 
${P_{{j^*}t}} = 1$. This is a pure exploitation scenario. In contrast, an agent does not systematically select the variety he currently believes is the best for himself when 
$\tau \gt 0$. Instead, he learns his preferences through exploration, by randomly sampling varieties. When 
$\tau $ is large, every variety is selected with almost equal probability, leading to a scenario of pure exploration.
Random exploration is necessary for an agent to learn their latent preferences and draw the most from consumption experience in the long run. Indeed, as in the standard Bayesian learning model (see DeGroot, Reference DeGroot1970), an agent is only guaranteed to fully learn his latent preferences (i.e., 
$\left( {{\mu _{jt}},{k_{jt}}} \right) \to ({\mu _j},0)\,\forall j$) when every variety has a positive probability of being sampled almost all the time. This is not the case when 
$\tau = 0$ as some varieties may only be experienced a finite number of times.Footnote 15 In contrast, 
${P_{jt}} \gt 0$ when 
$\tau \gt 0$, in which case the agent fully learns his preferences.
4.3. Decreasing random exploration
An agent relies on random exploration to learn about varieties of goods. However, once the agent has acquired better knowledge of his preferences, random exploration can be expected to fade. This is especially true in finite horizon. If decline in random exploration is slow enough, the agent will select his preferred style 
${j^*} = Argma{x_{j \in J}}\left( {{\mu _j}} \right)$ with a probability that converges to 1. To model decreasing random exploration, we specify the exploration parameter as 
${\tau ^i} = {\tau _0}{\text{exp}}\left( {X\beta - \lambda t + {\delta ^i}} \right)$, where 
$t$ is the round, 
$\lambda \geqslant 0$ captures decreasing exploration, 
${\tau _0}$ is a positive constant, 
$X$ a vector of individual characteristics and 
${\delta ^i}\sim \mathcal{N}\left( {0,\sigma _\tau ^2} \right)$ is the individual-specific random effect in equation (4). 
$\lambda = 0$ yields the Bayesian model with constant random exploration.
4.4. Summary
In the Bayesian bandit model of consumer choice, an agent selects variety j with highest stochastic decision utility in phase 1 of round 
$t$; then consumes j in phase 2, which generates an experienced marginal utility 
${u_{jt}}$ drawn from 
$\mathcal{N}\left( {\mu {'_j},{h_j}} \right)$; and in phase 3, the agent applies Bayes’ rule to update his beliefs about 
$\mu {'_j}$ from 
$\mathcal{N}\left( {\mu {'_{j,t - 1}},{k_{j,t - 1}}} \right)$ to 
$\mathcal{N}\left( {\mu {'_{jt}},{k_{j,t}}} \right)$ based on the information contained in 
${u_{jt}}$. Over time, 
$\mu {'_{jt}}$ converges toward 
${\mu _j}$ for all j, and the agent fully learns his latent taste for this variety.
We estimate this Bayesian bandit model of consumer choice on our experimental data in two steps. We first implement a reduced form analysis in Section 5 and conduct a structural estimation of the model in Section 6.
5. Reduced form analysis
In this section, we report panel data estimation of experienced utilities. For each musical style, we first convert the reported listening frequency into a prior marginal utility 
$\mu {'_{ij0}}$ using linear interpolation at the individual level.Footnote 16 This makes priors comparable to the reported marginal experienced utilities. Then, we conduct a panel data regression analysis to estimate experienced utilities:
\begin{equation}{u_{ijt}} = \mu {'_{ij}} + {\varepsilon _{ijt}} = {\mu _{ij}} + \frac{{{\alpha _j}}}{{1 + {n_{ijt}}}} + {\varepsilon _{ijt}} = {\bar \mu _j} + \frac{{{\alpha _j}}}{{1 + {n_{ijt}}}} + {\eta _{ij}} + {\varepsilon _{ijt}}\end{equation}where 
${\bar \mu _j}$ is average latent taste for variety j across subjects, 
${\eta _{ij}}\sim \mathcal{N}\left( {0,\sigma _{{\eta _j}}^2} \right)$ is a variety and individual-specific random effect, and 
${\varepsilon _{ijt}}\sim \mathcal{N}\left( {0,{h_j}} \right)$ is the residual of the random experienced utility. This allows us to estimate the 
${\alpha _j}{\text{s}}$ and the 
${h_j}{\text{s}}$ which are assumed to be known by subjects but not observed by the econometrician. We implemented this analysis at two levels of aggregation, considering varieties, first as treatments of familiar and unfamiliar music (Table 3), then as musical styles within each treatment (Table 4). In both Tables, the average prior experienced utilities 
${\overline {\mu '} _{j0}}$ (the average value of 
$\mu {'_{ij0}}$ across subjects) and results of the panel regressions of equation (6) are reported.
Aggregate varieties: Familiar versus unfamiliar treatment
Table 3 Long description
The table compares experienced utility between familiar and unfamiliar treatments, focusing on mean utility, DMU term, and mean latent taste. The familiar treatment has a higher mean utility of 5.63 compared to 4.28 for the unfamiliar treatment, with a significant t-test result (t = 5.18, p < 0.001). The DMU term is significantly higher in the familiar treatment (1.84) than in the unfamiliar treatment (0.82), with a chi-square test result of 2.94 (p = 0.09). Mean latent taste is also higher in the familiar treatment (5.48) compared to the unfamiliar treatment (4.81), with a chi-square test result of 13.38 (p < 0.001). The parameter h is slightly higher in the familiar treatment (4.86) than in the unfamiliar treatment (4.50), with a chi-square test result of 4.94 (p = 0.03). These results suggest significant differences in utility and taste between the two treatments.
* significant at the 10%-level.
** significant at the 5%-level.
*** significant at the 1%-level.
a Mean 
$\mu {'_{ij0}}$, across individuals 
$i$ and across styles 
$j$; see footnote 16. Two-sided unpaired t-test (equality of means).
b Wald test of comparison of the two alpha parameters.
c Wald test of comparison of the two intercepts of the model (one for the Familiar treatment, one for the Unfamiliar treatment).
d Likelihood ratio test of comparison of a model in which all parameters are allowed to differ (including the 
$h$) and a model in which all parameters but the 
$h$ are allowed to differ.
Disaggregated varieties: Musical styles within treatments
Table 4 Long description
The table presents a two-step estimation of a Bayesian bandit model, focusing on experienced utility across musical styles and treatments. Key findings include high DMU terms for Rap/RnB (2.68) and Africa (3.98), both significant at the 1% level. Mean latent taste is significant across all styles, with Pop/Rock (5.43) and Classic/Jazz (5.24) showing high values. The mean latent taste per style parameter is consistently significant, indicating strong latent preferences. Log-likelihood values vary, with the lowest for America (-2014.64) and highest for Asia (-1327.97). Standard deviations are provided in parentheses, highlighting variability in parameter estimates.
Estimation of equation (6): Linear random-effect model, estimated by maximum likelihood.
** significant at the 5%-level.
*** significant at the 1%-level.
a Mean 
$\mu {'_{ij0}}$, see footnote 16.
In parentheses, we report the standard deviations of the variables 
$\mu {'_{j0}}$, not the standard deviation of the means, which are much lower.
Table 3 presents the results of a variant of equation (6), in which all styles within a treatment are aggregated, and all model parameters (
$\sigma _\eta ^2$, 
$h$, 
$\overline \mu $, 
$\alpha $) are allowed to vary across treatments. This allows us to test the statistical significance of treatment effects to validate our assumption that traditional world music would be less known a priori than the so-called ‘familiar’ genres. The results confirm that subjects, on average, had more prior experience of familiar genres than unfamiliar ones 
$\left( {t = 5.18,\,p \lt 0.001} \right)$ and a higher latent taste as well (Wald test 
${\chi ^2} = 13.38,\,p \lt 0.001$).
In the within-treatment analysis (Table 4), Classical music and Jazz/Blues are aggregated into one style to increase their share and facilitate the econometric analysis.Footnote 17 As expected, subjects have clear priors about their least and most liked music in the familiar treatment (
${\overline {\mu '} _{j0}}$ is highest for Pop/Rock and lowest for Classical/Jazz), while subjects assign similar, intermediate values to all four musical styles on average in the unfamiliar treatment.
The DMU term 
$({\alpha _j})$ is positive and significantly different from 0 only for the most popular styles, Pop/Rock, Rap/RnB, African, and European style (two-sided test), suggesting that music listening is not subject to strong satiation as it takes lots of repetition to identify a significant declining trend in marginal experienced utility. However, Table 3 shows that DMU terms are positive on average at an aggregate level in both treatments at the 5% level of significance at least.
Table 4 shows that the 
${\bar \mu _j}{\text{s}}$ and the 
${\overline {\mu '} _{j0}}{\text{s}}$ behave differently in both treatments both in terms of magnitude as well as ranking. In the familiar treatment, initially low-ranked styles (Classical and Jazz) are liked more than anticipated, whereas the highest-ranked style (Pop/Rock) is liked less. At the end of the learning process, Classical/Jazz comes close to Pop/Rock and even surpasses Rap/RnB.Footnote 18 By contrast, in the unfamiliar treatment, the 
${\bar \mu _j}{\text{s}}$ never differ significantly from the 
${\overline {\mu '} _{j0}}{\text{s}}$.
For a rational subject with stable known preferences, marginal utility would be equalized over the whole experimental period for all consumed musical styles. A comparison of the distribution of mean latent tastes 
$({\bar \mu _j} = \bar \mu {'_j})$ with prior experienced utility 
$\,({\overline {\mu '} _{j0}})$ in Table 4 reveals that this is approximately verified in the familiar treatment but not in the unfamiliar treatment. We observe a significant difference between prior experienced marginal utilities and a partial convergence in the familiar treatment, yet a diverging trend in the unfamiliar treatment.Footnote 19 While learning has taken place in both treatments, the process is converging in the familiar treatment but remains incomplete in the unfamiliar treatment. In this treatment, there is a striking difference between a quasi-indifference in the prior experienced utilities and the wide appreciation gap between Asian music (least appreciated in our sample) and other styles. Clearly, subjects consciously avoided the musical style that they liked least, as already apparent in Figure 1b.
Finally, the variances 
${\hat h_j}$ are precisely estimated in Tables 3 and 4, which legitimates our assumption of a known variance of taste 
${h_j}$.Footnote 20 Moreover, variances are relatively similar across musical styles and treatments, which provides some evidence that tracks used in the experiment portray a similar degree of heterogeneity within each style. There are two exceptions, however, as variances of taste are lower for Classical/Jazz and traditional Asian music. Subjects had strong prior beliefs about what they would like least. As a matter of fact, they were wrong for Classical/Jazz and right for Asian musical style.
6. Structural estimation
6.1. Specification and method
The remaining structural parameters to estimate are variances of initial beliefs 
${{\boldsymbol{k}}_0} = \left( {{k_{1,0}},{k_{2,0}},{k_{3,0}},{k_{4,0}}} \right)$ for the non-familiar treatment and 
${{\boldsymbol{k}}_0} = \left( {{k_{1,0}},{k_{2,0}},{k_{3,0}}} \right)$ for the familiar treatment; and 
${\tau ^i}$, the (individual) exploration parameter. The estimation procedure combines the choice probabilities 
${P_{ijt}}\left( {{{\boldsymbol{k}}_0},{\tau ^i}} \right) = P\left( {{x_{ijt}} = 1} \right)$ in equation (5) with the Bayesian law of motion in equation (3), where 
${h_j}$ is replaced by 
${\hat h_j}$ which has been estimated in the previous section.
To account for possible heterogeneity in learning, we specify 
${\tau ^i}$ as:
\begin{equation*}{\tau ^i} = {\tau _0}\exp \left( { - \lambda t + \omega Ope{n_i} + {\delta ^i}} \right)\end{equation*} In practice, the model is estimated as follows, to ensure that 
${\tau _0}$ stays positive.
\begin{equation}{\tau ^i} = {\text{exp}}\left( {{\beta _0} - \lambda t + \omega Ope{n_i} + {\delta ^i}} \right)\end{equation} As mentioned above, 
$\lambda \geqslant 0$ captures decreasing random exploration. Observed heterogeneity is captured by one of the Big Five personality traits, namely openness.Footnote 21 Unobserved heterogeneity is captured by the individual-specific random effect 
${\delta ^i}\sim \mathcal{N}\left( {0,\sigma _\tau ^2} \right)$.
When the model does not include random effects (i.e., 
$\sigma _\tau ^2 = 0$), 
${{\boldsymbol{k}}_0}$, 
${\beta _0}$, 
$\lambda $ and 
$\omega $ are estimated by Maximum Likelihood based on the likelihood function:
\begin{equation*}L\left( {{{\boldsymbol{k}}_0},{\beta _0},\lambda ,\omega } \right) = \mathop \prod \limits_{i,j,t} {\left[ {{P_{ijt}}\left( {{{\boldsymbol{k}}_0},{\beta _0},\lambda ,\omega } \right)} \right]^{{x_{ijt}}}}\end{equation*}where 
${x_{ijt}} = 1$ if subject 
$i$ has chosen the style 
$j$ in round 
$t$, and 
$0$ otherwise.
In this case, the model is a multinomial logit model with a non-linear index. It is the non-linearity of the index that allows identification of the 
${\beta _0}$, 
$\lambda $ and 
$\omega $ parameters.
When the model includes random effects (i.e., 
$\sigma _\tau ^2 \ne 0$), it is a mixed multinomial logit model, once more with a non-linear index. In this case, we use a Maximum Simulated Likelihood approach (see Train (Reference Train2009) for example). The unconditional choice probability may be written:
\begin{equation}{\bar P_{ijt}}\left( {{\mathbf{k}_0},{\beta _0},\lambda ,\omega ,{\sigma _\tau }} \right) = E\left[ {{P_{ijt}}\left( {{\mathbf{k}_0},{\tau ^i}\left( {{\delta ^i}} \right)} \right)} \right]\end{equation}where the expectation operator is taken as a function of individual-specific random effect 
${\delta ^i}$. We use Monte Carlo simulations to approximate numerically 
${\bar P_{ijt}}\left( {{{\boldsymbol{k}}_0},{\beta _0},\lambda ,\omega ,{\sigma _\tau }} \right)$. Specifically, we generate sequences of 
$R$ pseudo-random numbers 
${\left\{ {\delta _i^r} \right\}_{r = 1, \ldots ,R}}$ from 
$\mathcal{N}\left( {0,\sigma _\tau ^2} \right)$ (more precisely, we use Halton sequences with 
$R = 1000$), and we replace the expectation in (8) by its empirical counterpart. The likelihood is then approximated by:
\begin{equation*}\tilde L\left( {{{\boldsymbol{k}}_0},{\beta _0},\lambda ,\omega ,{\sigma _\tau }} \right) = \mathop \prod \limits_{i,j} \left\{ {\frac{1}{R}\mathop \sum \limits_{r = 1}^R \mathop \prod \limits_t {{\left[ {{P_{ijt}}\left( {{{\boldsymbol{k}}_0},{\tau ^i}\left( {\delta _i^r} \right)} \right)} \right]}^{{x_{ijt}}}}} \right\}\end{equation*} This second step allows us to estimate the variances of initial beliefs 
${{\boldsymbol{k}}_0}$ along with components of the exploration parameter. As mentioned above, we do not use true values of 
${h_j}$, but their first-step estimates 
${\hat h_j}$ instead. Because of this, the second-step matrix of variances-covariances is biased. We correct this bias by using the Murphy and Topel (Reference Murphy and Topel1985) correction.
6.2. Estimation results
Six versions of structural estimation of the Bayesian bandit model of consumer choice are reported below, for the familiar treatment in Table 5 and the unfamiliar treatment in Table 6. M1 (col. 1) is the simple model of pure exploitation (with 
${k_0}$ only), M2 (col. 2) includes exploration (with 
${\tau _0} \gt 0$) without unobserved heterogeneity, M3 (col. 3) is M2 + unobserved heterogeneity 
${\sigma _\tau }$, M4 (col. 4) is M3 + Openness which accounts for observed heterogeneity, M5 (col. 5) is M3 + decreasing exploration 
$\lambda $, and M6 (col. 6) is M5 + Openness.
Structural estimation of Bayesian bandit model: Familiar treatment
Table 5 Long description
The table presents a two-step estimation of a Bayesian bandit model focusing on familiar treatment across different models. Key coefficients k1,0, k2,0, and k3,0 are significant across all models, with M1 showing the highest k1,0 at 8.88. The tau parameter, representing the effect of openness, varies significantly, with M5 + Open. showing the highest tau at max Openness at 11.17. The variance of the random effect in tau, denoted as sigma_tau, is consistent across models M3 to M6, ranging from 0.72 to 0.79. The log-likelihood values indicate model fit, with M6 having the highest at -2989.50. Standard errors are corrected using the Murphy and Topel method, which may affect the interpretation of significance levels.
a 
$\tau \,\left( {{\text{at}}\;mean(Openness)} \right)$ for models M4 and M6.
** significant at the 5%-level.
*** significant at the 1%-level.
Standard errors corrected by using the Murphy and Topel (Reference Murphy and Topel1985) correction, to consider the use of 
${\hat h_j}{\text{s}}$ instead of the true 
${h_j}{\text{s}}$.
Structural estimation of Bayesian bandit model: Unfamiliar treatment
Table 6 Long description
The table presents structural estimations of a Bayesian bandit model under unfamiliar treatment across different models. Key findings include significant openness effects in models M4 and M6, with $ au$ values at mean openness being significant at various levels. Model M4 shows $ au$ values of 3.68 at minimum openness and 12.93 at maximum openness, indicating a strong impact of openness. Variance in random effects (${au _0}$) is consistently significant across models, with model M5 showing the highest $ au$ value at 26.84. The log-likelihood values suggest model M6 has the best fit, with the lowest value of −4672.51. Standard errors are corrected using Murphy and Topel's method, ensuring robustness in the estimates.
a 
$\tau $ (at 
$mean(Openness$)) for models M4 and M6.
* Significant at the 10%-level.
** significant at the 5%-level.
*** significant at the 1%-level.
Standard errors corrected by using the Murphy and Topel (Reference Murphy and Topel1985) correction, to consider the use of 
${\hat h_j}{\text{s}}$ instead of the true 
${h_j}{\text{s}}$.
In both treatments, a considerable increase of the log-likelihood can be observed when random exploration and decreasing exploration are added respectively to the model, which is consistent with predictions of the multi-armed bandit literature. Unobserved heterogeneity also improves the model’s fit with high significance. Openness is the only observable factor of heterogeneity that was found significant, but it has a strong impact on exploration intensity, which conforms with intuition.
The estimated variances of the initial priors, 
${k_{j0}}$, are quite sensitive to the specification and appear to be biased upward when exploration is not accounted for. These variances are smallest for Classical/Jazz and Pop/Rock, the musical styles that subjects were most familiar with before the experiment. Even with styles that were more familiar to them, subjects’ prior knowledge of preferences was imperfect, since the estimated variances of the initial priors are always highly significant. Thus, there was room for learning in both treatments. However, variances of the prior distribution of tastes are significantly larger in the unfamiliar treatment, which confirms our assumption that subjects knew little about their taste for traditional world music before the experiment.
The exploration parameter 
${\tau _0}$ is between 1.8 and 3.8 times higher in the unfamiliar treatment, which indicates that subjects relied more on random exploration to learn about the musical styles that they were less familiar with. Decreasing exploration 
$(\lambda )$ is higher in absolute value for the unfamiliar treatment, however. The combined result on 
${\tau _0}$ and 
$\lambda $ indicates that early exploration was more intensive for less familiar musical styles, yet exploration decreased more rapidly. The half-life duration for the exploration parameter is of 12.84 rounds (s.d. = 0.95) in the unfamiliar treatment and of 18.73 rounds (s.d. = 1.52) in the familiar treatment.
To better appreciate learning differences across subjects, we introduce unobserved heterogeneity in model M3 of Tables 5 and 6. The term 
${\sigma _\tau }$ is always highly significant and hardly diminishes when a personality trait like openness is introduced in model M4. Therefore, subjects appear to have learned their preferences in different ways during the experiment: some were more likely to choose the style they most preferred at the time, whereas others tended to sample styles to better identify their preferences.
The impact of openness on the exploration rate 
$\tau $ is visible in models M4 and M6. In our preferred model M6, exploration rates of subjects with the maximum score of openness are 3.4 times those of subjects with the minimum score in the familiar treatment and 5.0 times those in the unfamiliar treatment. Thus, while everybody tends to explore more when musical styles are unfamiliar, more open people explore far more than others.
In the next section, we further examine the convergence process and the heterogeneity of learning strategies used by subjects.
7. Discussion
7.1. Initial and final beliefs
Table 7 reports summary statistics for the average distance between predicted utility and latent utility at the beginning (columns 1 and 4) and end (columns 2 and 5) of the experiment. The initial distance 
$\left| {\mu {'_{ij0}} - \bar \mu {'_{ij}}} \right|$ reflects the average ‘projection bias’ of subjects in predicting their latent utility (Loewenstein & Adler, Reference Loewenstein and Adler1995; Loewenstein et al., Reference Loewenstein, O’Donoghue and Rabin2003) whereas the final distance 
$\left| {\mu {'_{ij50}} - \bar \mu {'_{ij}}} \right|$ is a measure of the average convergence of consumers toward their latent taste. Every mean 
$\left| {\mu {'_{ij0}} - \bar \mu {'_{ij}}} \right|$ significantly differs from the corresponding mean 
$\left| {\mu {'_{ij50}} - \bar \mu {'_{ij}}} \right|$ and all those mean values differ significantly from 0 (
$p \lt 0.001)$. The projection bias, which is large relative to the prior and relatively larger in the unfamiliar treatment, has been significantly reduced by learning during the experiment as shown in columns (3) and (6). However, convergence remains partial since the gap remains significant by the end of the experiment.
Average distance between predicted utility and latent utility of musical styles at the beginning and end of the experiment
Table 7 Long description
The table measures the average distance between predicted and latent utility of musical styles and regions at the start and end of an experiment. Pop/Rock and Africa have the highest initial utility differences, 1.88 and 2.28 respectively, which decrease significantly by the end of the experiment to 0.50 and 0.43. All musical styles and regions show statistically significant differences in means, with p-values less than 0.001. Classical/Jazz has the smallest initial difference at 0.70, reducing to 0.31. The t-tests confirm significant changes in utility perception across all categories, indicating a consistent trend of reduced utility differences over time. The standard deviations of individual differences are higher than those of the average distances, suggesting variability in individual experiences.
a The standard deviation of the individual experienced utility difference is displayed in parentheses below the average distance (i.e., the absolute value of this difference). The (non-reported) standard deviation of the average distance is much lower than the standard deviation of the variable.
7.2. Evidence about the exploration-exploitation tradeoff
7.2.1. Learning curves
Learning curves measure the impact of the exploitation strategy for each musical style. Figure 2 shows the theoretical and empirical learning curves, which depict the reinforcement rates 
${\rho _{jt}}$ after 
$t$ rounds of theoretical or actual experience respectively. These curves are plotted using equations (2) and (3), for the estimated values of the parameters (kj 0, etc.).Footnote 22 The reinforcement rates vary widely across musical genres in the first rounds of the experiment and always converge toward zero as subjects accumulate experience.
Theoretical and empirical learning curves 
$\left( {{{\boldsymbol{\rho }}_{{\boldsymbol{jt}}}}} \right)$. (a) Theoretical learning curve. (b) Empirical learning curve
Fig. 2 Long description
The image contains two graphs. The first graph, labeled 'a Theoretical learning curve', plots 'rho' on the y-axis against 't' on the x-axis from 0 to 50. It shows curves for Africa, Europe, America, Asia, Pop, Rap and JazzClassic, all converging towards zero. The second graph, labeled 'b Empirical learning curve', also plots 'rho' on the y-axis against 't' on the x-axis from 0 to 50. It displays similar musical styles with more variation initially, but also converging towards zero over time. Both graphs illustrate the learning curves for different musical styles, with theoretical and empirical data showing reinforcement rates decreasing as experience accumulates.
The estimated rates of the theoretical learning curves, computed midway in the first exploration time interval (1–12), were equal to 8% for Pop/Rock and Classic/Jazz, and 14% for Asia and Africa, with an intermediate value of 13% for Rap/RnB, Europe and America. The disparity between these reinforcement rates essentially reflects a disparity between the estimated prior knowledge kj ,0s in Tables 5 and 6 which capture the prior knowledge of musical styles. These results confirm that subjects learned in both treatments and that the learning curve was steeper for less familiar musical styles.Footnote 23
For empirical curves, posterior variances 
${k_{jt}}$ and observed reinforcement rates 
${\rho _{jt}}$ do not depend directly on t but on the frequency of listening to a style j during the first t rounds, since they can only be revised when musical style j is selected. For instance, on average, Rock was listened to 19.8 times during the 50 rounds of the experiment, which resulted in an empirical reinforcement rate of 7.2% per round, whereas Classical/Jazz was listened to 13.1 times only, leading to a higher empirical reinforcement rate of 8.1% per round.
7.2.2. How much do consumers randomize?
Random exploration contributes to long run optimization in consumers’ choices. To interpret estimated exploration parameters 
$\tau $ and 
$\lambda $, Figure 3 plots the average probability of not choosing the best predicted style during the experiment. This probability for individual 
$i$ at time 
$t$ is equal to 
$1 - \bar P\left( {{x_{ijt}} = 1} \right)$, where 
$j$ is the predicted utility-maximizing musical style at round 
$t$ for 
$i$ and 
$\bar P\left( {{x_{ijt}} = 1} \right) = {\bar P_{ijt}}\left( {{{\widehat {\boldsymbol{k}}}_0},{{\hat \beta }_0},\hat \lambda ,\hat \omega ,{{\hat \sigma }_\tau }} \right)$ is the estimated probability of choosing this style, given by equation (8) as a function of the estimated parameters 
${\hat k_{0j}}$ and 
${\tau ^i}\left( {{\delta ^i}} \right)$. Figure 3 illustrates to what extent the average consumer chose a style that did not maximize his predicted utility during our experiment.
Share of subjects not choosing the expected utility maximizing style
Fig. 3 Long description
A line graph showing the share on the y-axis and period on the x-axis, ranging from 0 to 50. It includes two main lines: 'Non Familiar' and 'Familiar', each with a corresponding linear fit. The 'Non Familiar' line fluctuates more prominently, while the 'Familiar' line remains relatively stable. Shaded areas around the lines indicate variability. The graph includes dashed lines for linear fits, labeled as 'Non Familiar - linear fit' and 'Familiar - linear fit'.
Random choices are most frequent throughout the early exploration time interval 1–12, while the frequency of non-maximizing choices stabilizes at a lower level during the rest of the experiment. The latter frequency reaches a high level of 61.5% for unfamiliar styles and 44.7% for familiar styles, and the difference is significant after the first time interval. The evolutions of the two curves are parallel, as shown by a linear fit on time intervals 1–12 and 13–50. Figure 3 is a vivid illustration of the randomness of experienced utility in discrete consumer choices.
7.2.3. Diversification or randomization?
The different degree of knowledge imperfection between familiar and unfamiliar musical styles changes the nature and impact of exploration behavior across treatments in our experimental context. Figure 4 displays the histograms of the Herfindahl indexes for both treatments. Recall that the Herfindahl is an inverse index of exploration intensity.
Histograms of the Herfindahl concentration index for both treatments
Fig. 4 Long description
The image contains two histograms. The first histogram is labeled 'Herfindahl Index distribution - Familiar treatment' and shows the distribution of the Herfindahl Index for familiar treatment. The x-axis is labeled 'Fraction' and the y-axis is labeled 'Density'. It includes lines for kernel density, mean Herfindahl Index and median Herfindahl Index. The second histogram is labeled 'Herfindahl Index distribution - Unfamiliar treatment' and shows the distribution for unfamiliar treatment. The axes are similarly labeled and it also includes lines for kernel density, mean Herfindahl Index and median Herfindahl Index.
The Herfindahl index distribution is skewed to the right in both treatments, with mean higher than median, yet major differences exist between treatments. The distribution is far more concentrated in the high exploration zone for the unfamiliar treatment. A Kolmogorov-Smirnov test of comparison of distributions rejects the hypothesis that the Unfamiliar group is more concentrated than the Familiar group 
$\left( {D = 0.527\,(p - value \lt 0.001)} \right)$ and does not reject the hypothesis that the Familiar group is more concentrated than the Unfamiliar group 
$\left( {D = - 0.003\,\left( {p - value = 0.999} \right)} \right)$.
Almost 50% of participants randomize all the time to choose a style of traditional world music (median Herfindahl is 0.279), and the cutoff lies between those who randomize throughout the experiment and those who do so only part time, with time left to exploit satisfying experiences. In comparison, amidst the familiar musical styles, participants never choose randomly all the time (first percentile Herfindahl is 0.334) and there is a clear divide between subjects who diversify their music choices more and those who do not.
To summarize, high explorers look like diversifiers who took advantage of exploration in the familiar treatment and randomizers who failed to do so in the unfamiliar treatment, as will be apparent in the next section.
7.3. Did more intensive early exploration benefit consumers?
If rational consumers have an imperfect knowledge of their preferences, they should learn what they like most and least through early exploration and make better decisions in the long run by exploiting received information. Early exploration is thus expected to benefit consumers more in the unfamiliar treatment than in the familiar treatment.
We test the prediction of a beneficial effect of exploration in Table 8, which reports the mean experienced utility depending on the early exploration intensity, that is, during the first 1–12 time-interval. A proxy is used to capture an individual’s early exploration intensity, which is their position relative to the mean Herfindahl index for the initial time interval (1–12). A higher concentration means here a lower early exploration intensity. We defined low (resp. high) explorers as subjects whose Herfindahl index is above (resp. below) the mean value in each treatment.
Mean experienced utility and early exploration intensity in both treatments
Table 8 Long description
The table measures mean experienced utility and early exploration intensity across familiar and unfamiliar contexts, comparing low and high early exploration treatments. For familiar experiences, the mean utility is higher in the high early exploration group (5.68) compared to the low group (5.43), with a statistically significant p-value of 0.004. In contrast, unfamiliar experiences have the same mean utility (4.87) in both exploration groups, with a non-significant p-value of 0.983. The t-tests for both familiar and unfamiliar experiences show significant results with p-values less than 0.001, indicating strong evidence against the null hypothesis of equal means. These findings suggest that early exploration intensity impacts familiar experiences but not unfamiliar ones.
a Two-sided unpaired t-tests (equality of means).
b Two-sided unpaired t-tests (equality of means).
No matter how much effort they spend on exploration, consumers experience lower utility on average in the unfamiliar treatment, as means differ at the 0.1% level. High explorers in the familiar treatment reported on average a higher mean experienced utility compared to low explorers and benefited from higher choice diversification by listening more frequently to a musical style which they ultimately appreciated (Classical/Jazz). In contrast, high explorers in the unfamiliar treatment were no better-off than low explorers overall. Indeed, while high explorers in this treatment devoted more time to musical styles that they liked less (Asian style), low explorers on the other hand suffered in the long run from listening too much to musical styles they liked.
Table 9 focuses on the evolution of mean experienced utility over the three time-intervals. Comparison of the two treatments shows that the Unfamiliar group experiences on average significantly lower utility than the Familiar group in all time-intervals and for all levels of early explorationFootnote 24 except for low explorers in the 1–12 time-interval (p = 0.916).
Mean experienced utility and early exploration intensity per time interval in both treatments
Table 9 Long description
The table measures mean experienced utility and early exploration intensity across three time intervals for familiar and unfamiliar treatments. In the familiar treatment, low early exploration decreases significantly from 5.81 in the 13-25 interval to 5.14 in the 26-50 interval, with a p-value of 0.001 indicating statistical significance. High early exploration remains relatively stable across intervals. In the unfamiliar treatment, low early exploration drops from 5.58 to 4.64 between the first two intervals, with a significant p-value of 0.009, while high early exploration shows a slight increase from 4.93 to 5.05. The t-tests compare high and low exploration within each treatment, revealing significant differences in some intervals. Interpretation should consider the statistical significance indicated by p-values and the context of exploration intensity changes.
a Two-sided paired t-tests (equality of means).
b Two-sided unpaired t-tests (equality of means).
In the familiar treatment, experienced utility is not significantly different between the first and second intervals no matter how intensive was early exploration. This observation corroborates the conventional economic assumption that people have little to learn with familiar music since they have good, albeit imperfect, prior knowledge of their preferences. However, high early exploration slows down the decline in experienced marginal utility for the second half of the experiment as consumers counterbalance the decreasing marginal utility of the more heavily consumed styles through a greater diversification of consumption. Setting the cutoff at the median or mean does not make a difference and does not alter the conclusion.
The situation is very different for the unfamiliar treatment. Low early exploration is now associated with a very significant decline in experienced marginal utility as soon as the second time interval (13–25). In contrast, high early exploration appears like a costly investment. While mean experienced utility is significantly reduced during the exploration phase (one-tailed test, p = 0.036), the benefits drawn from this investment are small and short-lived. Indeed, the experienced marginal utility of high explorers rises insignificantly in the second interval and suffers from a significant fall in the second half of the experiment to the level attained by low explorers. Our results mitigate the benefits from early exploration in the unfamiliar treatment. Indeed, early exploration does not guarantee higher average reward over a finite horizon, since it may also reveal bad surprises that cannot be fully compensated.
8. Conclusion
The traditional economic assumption that consumers have innate, stable preferences and make utility-maximizing choices has been increasingly challenged by findings from econometric models and psychological experiments. This view seems too rigid, especially for experience goods in a world with many varieties and constant product innovation.
We believe that the main virtue of our experimental data about the repeated choices of musical styles has been to uncover the sequential nature of the decision-making process that is usually masked by the product and time aggregation of consumption data. Consumers need time to explore and discover which variety they prefer. They do not take a series of independent static choices but typically face a stochastic multi-armed bandit problem over their lifetime. Rational consumers are uncertain about the mean utility different varieties will provide, and they must choose to explore lesser-known options. Their optimal policy thus contains two types of randomness, one due to their prior ignorance of products and varieties and the other due to their conscious exploration behavior. The two-stage mechanism of decision-making (evaluation and decision) described by neuroscientists (Glimcher, Reference Glimcher, Glimcher, Camerer, Fehr and Poldrack2009) is consistent with this dual randomness of utility.
Our experimental approach enabled us to observe satisfaction and choice separately, thus disentangling experienced utility from decision utility as advocated by Kahneman et al. (Reference Kahneman, Wakker and Sarin1997). The notions of experienced utility and decision utility coincide in conventional economics due to the neglect of preference uncertainty. We however never assume that experienced utility deviates from decision utility because consumers make errors of perception and/or decision. Instead, their difference is the consequence of rational, or quasi-rational, exploration behavior.
Using disaggregated individual data, we estimated a Bayesian bandit model of consumer choice that accounts for satiation and declining random exploration. Our model explains seemingly unstable choices as optimal behavior under incomplete preferences. Over time, choices stabilize, but short-term fluctuations in marginal utility lead consumers to switch between varieties – which static models interpret as a preference for diversity.
People may know precisely what they like after consuming a single product, but the multiplicity of varieties and products in real life makes exploration essential to discover which one they like better. This may be the main reason for random utility.
Supplementary material.
The supplementary material for this article can be found at https://doi.org/10.1017/eec.2026.10050.
Acknowledgements
We especially thank Peter Klibanoff, Olivier L’Haridon, and Chantal Marlats for helpful comments on previous drafts. We are grateful to Aurélien Baillon, Ido Erev, Charles Noussair, and Véronique Simonnet for early discussions on the paper, and to Todd Lubart for his help in the conceptualization of the categorization task. We acknowledge the contribution of Olivier Armantier in a previous version of this paper. We thank seminar and conference participants at the Paris School of Economics, Erasmus University in Rotterdam, Asfee in Lyon, ESA in Zurich, AEW in Rome, BEELab in Florence, Afse in Lyon, ACEI in Montreal, SABE in Lake Tahoe, University of Bilbao, JMA at Montpellier, UNU-MERIT in Maastricht, SABE-IAREP meeting in Dundee. Louis Lévy-Garboua acknowledges Agence Nationale pour la Recherche for providing experiment funding (ANR-10-CREA-008 MACCAN). The experiments were conducted at LEEP (Paris School of Economics and Centre d’Economie de la Sorbonne). The replication material for the study is available at https://doi.org/10.5281/zenodo.18866032.
Open access
