1 Introduction

Risk attitude is a crucial factor influencing economic behaviour. As a consequence, experimenters are interested in eliciting the risk-attitude of their subjects. This can be done in two ways: either directly, using the context of a particular experiment to estimate the risk-aversion that best explains behaviour; or indirectly, eliciting risk aversion in a separate part of the experiment, and using the elicited value to explain behaviour in the main experiment. It is to those experimenters that use the latter approach to whom this paper is addressed.

Economic theory posits that decisions under risk depend on how people evaluate, and hence decide between, risky lotteries. By these we mean lotteries where the outcomes are risky, and where the probabilities are known. Clearly how people evaluate lotteries depends not only on the lotteries, but also on the preference functionals of the decision-maker (DM). In the literature there are a number of proposed preference functionals, the best-known of which is the Expected Utility functional. All of these embody the idea of an underlying utility function u(.); it is the degree of concavity of this when it is defined over money that indicates the degree of risk-aversion. It is this that we are trying to elicit.

There are a number of methods that are used in the literature to elicit risk aversion. Possibly the most popular is that known as the Holt–Laury Price List, introduced by Holt and Laury (Reference Holt and Laury2002), and which we will refer to as HL. While the detail may vary from application to application, the basic idea is simple: subjects are presented with an ordered list of pairwise choices and have to choose one of each pair. The list is ordered in that one of the two choices is steadily getting better or steadily getting worse as one goes through the list. There are many variants on the basic theme: sometimes one of the two choices is a certainty, and that is getting better or worse through the list; sometimes either just one or both of the choices are risky choices and one of them is getting better or worse through the list. Because of the ordered nature of the list, subjects should choose the option on one side up to a certain point thereafter choosing the option on the other side. Some experimenters force subjects to have a unique switch point; others leave it up to subjects.Footnote ¹

A second method is to give a set of Pairwise Choices, but separately (not in a list) and not ordered. We will refer to this as PC. Typically the pairwise choices are presented in a random order. This has been used by Hey and Orme (Reference Hey and Orme1994), amongst many others. Some argue that this method, whilst being similar to that of Price Lists, avoids some potential biases associated with ordered lists. Usually the pairwise choices are chosen such that they are distributed over one or more Marschak-Machina triangles.Footnote ²

A method which is elegant from a theoretical point of the view is the Becker–DeGroot–Marschak mechanism proposed by Becker et al. (Reference Becker, DeGroot and Marschak1964). This we will later denote by Lottery Choice (LC) because of the way that we implement it. The method centres on eliciting the value to a subject of a lottery—if we know the value that a subject places on a lottery with monetary outcomes, we can deduce the individual’s attitude to risk over money. There are two variants of this mechanism that are used in the literature: one where the DM is told that they own the lottery, and hence have the right to play it out or to sell it; and one where the DM is offered the chance to buy the lottery, and, if so, to then play out the lottery. The subject’s valuation of the lottery as a potential seller is the minimum price for which they would be willing to sell it, while the subject’s valuation of the lottery as potential buyer is the maximum price for which they would be willing to buy it. Here we describe the mechanism as it relates to a potential buyer—the mechanism is the same, mutatis mutandis, if it relates to a potential seller. The subject is asked to state a number; then a random device is activated, which produces a random number between the lowest amount in the lottery and the highest amount. If the random number is less than the stated number, then the subject buys the lottery at a price equal to the random number (and then plays out the lottery); if the random number is greater, then nothing happens and the subject stays as he or she was. IfFootnote ³ the subject’s preference functional is the expected utility functional then it can be shown that this mechanism is incentive compatible and reveals the subject’s true evaluation of the lottery. Analytically it is equivalent to a Second Price Auction. The problem is that subjects do seem to have difficulty in understanding this mechanism, and a frequent criticism is that subjects understate their evaluation when acting as potential buyers and overstate it when acting as potential sellers. So in our experiment we implement this mechanism in a new way, which, we hope, is easier for subjects to understand, and which we will explain later.

The Allocation method, which we shall denote by AL, was originally pioneered by Loomes (Reference Loomes1991). It was then revived by Andreoni and Miller (Reference Andreoni and Miller2002) in a social choice context, and later by Choi et al. (Reference Choi, Fisman, Gale and Kariv2007) in a risky choice context. This method involves giving the subject some experimental money to allocate between various states of the world, with specified probabilities for the various states, and, in some implementations, with given exchange rates between experimental money and real money for each of the states. This method seems easier for subjects to understand than BDM.

This paper is a follow-up, and complement to, the previous literature and, in particular, the paper by Loomes and Pogrebna (Reference Loomes and Pogrebna2014), in which the authors compare three of the elicitation methods described above—specifically Holt–Laury price lists, Ranking and Allocations. Our paper complements theirs, not only in the elicitation methods we consider, but also in that our experimental design (and crucially the numbers of questions asked for each method), as well as the data analysis, are completely different. We also consider a slightly different set of elicitation methods.

The purpose of this paper is to report on the results of an experiment in which subjects were asked to perform each of the four methods described in Sect. 1 above. The paper is organised as follows. We start with a literature review of the various elicitation methods in Sect. 2, and a survey of previous experimental results in Sect. 3. In Sect. 4 we describe how our experiment was organised and how the various methods were implemented in it, giving more detail about each of the methods. As we adopt an econometric methodology of fitting preference functionals to the data, we specify, in Sect. 5, the preference functionals that we fit to the data and describe the functional forms that we assume, and the parameters in them that we estimate. In Sect. 6, we describe how we analysed the data, detailing the stochastic assumptions that we make. Section 7 contains the results. In Sect. 8 we discuss what else we might have done; and Sect. 9 concludes.

2 Review of the various elicitation methods

One clear difference between the methods is in the information that the answers give. Pairwise Choices (on which Price Lists are built) merely tell us which of two lotteries is preferred, but not by how much. In contrast both the LC and AL give us a continuous measure, which is (should be) the outcome of an optimising decision. This suggests that the latter two might be more informative, though of course more cognitive effort might have to be expended by the subjects. A discussion of the various methods can be found in Charness et al. (Reference Charness, Gneezy and Imas2013). A complete list, with links to descriptions, is given in Table 1. We briefly comment on them below.

Table 1

Elicitation methods

Allocation	Loomes (Reference Loomes1991)
Angling risk task	Pleskac (Reference Pleskac2008)
Balloon analog risk task	Lejuez et al. (Reference Lejuez, Read, Kahler, Richards, Ramsey and Stuart2002)
Becker–DeGroot–Marschak mechanism	Becker et al. (Reference Becker, DeGroot and Marschak1964)
Bomb risk elicitation task	Crosetto and Filippin (Reference Crosetto and Filippin2013)
Columbia card task	Figner et al. (Reference Figner, Mackinlay, Wilkening and Weber2009)
Cups task	Levin and Hart (Reference Levin and Hart2003)
Deal or no deal	Deck et al. (Reference Deck, Lee, Reyes and Rosen2008)
Devil’s task	Slovic (Reference Slovic1966)
Distribution builder	Goldstein et al. (Reference Goldstein, Johnson and Sharpe2008)
Dynamic experiments for estimating preferences: risk	Toubia et al. (Reference Toubia, Johnson, Evgeniou and Delquié2013)
Eckel and Grossman method	Eckel and Grossman (Reference Eckel and Grossman2002)
First price auction	Isaac and James (Reference Isaac and James2000)
Gneezy and Potters method	Gneezy and Potters (Reference Gneezy and Potters1997)
Lowa gambling task	Bechara et al. (Reference Bechara, Damasio, Damasio and Anderson1994)
Multi-outcome risky decision task	Lopes and Oden (Reference Lopes and Oden1999)
Pairwise choices	Hey and Orme (Reference Hey and Orme1994)
Price list	Holt and Laury (Reference Holt and Laury2002)
Ranking task	Carbone and Hey (Reference Carbone and Hey1994)
Reyna and Ellis risk task	Reyna and Ellis (Reference Reyna and Ellis1994)
Risk-taking propensity measures	MacCrimmon and Wehrung (Reference MacCrimmon and Wehrung1985)
Sequential investment task	Frey et al. (Reference Frey, Rieskamp and Hertwig2015)
Two-outcome risky decision task	Lauriola et al. (Reference Lauriola, Levin and Hart2007)

This table was built from this site provided by the Society for Judgment and Decision Making and augmented with other methods not listed there

We classify the various methods into four main categories (and variants on them), namely: pairwise choice, multi-item choice, ‘continuous’Footnote ⁴ choice and ranking. We do not give details but instead categorise the types of the methods and the underlying idea behind them.

In the first category we can obviously include the PC method and the Holt–Laury price list method (which comprises sets of ordered pairwise choices). The ‘Cups Task’, the ‘Reyna and Ellis Risk Task’ and the ‘Risk-taking Propensity Measures’ are special cases of pairwise choice in which one of the lotteries is a certainty. The ‘Two-Outcome Risky Decision Task’ is also a special case where both lotteries are two-outcome lotteries but one is risky while the other is ambiguous.Footnote ⁵ The ‘Deal or No Deal Method’ and the ‘Dynamic Experiments for Estimating Preferences’ are dynamic versions of pairwise choice.

In the multi-option category there is the Eckel and Grossman method (with choice over 5 lotteries) and the ‘Iowa Gambling Task’ (with dynamic choice over four options with initially unknown probabilities).

In the ‘continuous’ choice category, we start with five methods that have strong similarities: the ‘Angling Risk Task’, the ‘Balloon Analog Risk Task’, the ‘Columbia Card Task’, the ‘Devil’s Task’ and the ‘Sequential Investment Task’. They all share the same basic idea: subjects have to choose a probability; then either some random event does occur with that probability or it does not occur with the residual probability; if it does occur the subject gets nothing, and, if it does not occur, the subject gets paid some increasing function of the probability. The chosen probability is an indicator of risk aversion. In an obvious sense the subject is choosing a distribution of his or her payment. The same is true with the Allocation method (which we have described above), the Becker–Degroot–Marschak method (which we have described above and which is analytically equivalent to a Second Price Auction), the ‘Distribution Builder’ task (in which subjects are asked to state a preferred distribution), the First Price Auction method (in which subjects have to state their valuation of a lottery), the Gneezy and Potters method (in which subjects are asked to allocate some money between a safe asset and a risky asset), and the ‘Sequential Investment Task’.

The ranking method offers subjects a choice of lotteries and asks them to rank them in order of preference (with an appropriate incentive mechanism). This has been used by Bateman et al. (Reference Bateman, Day, Loomes and Sugden2006) and Loomes and Pogrebna (Reference Loomes and Pogrebna2014), in addition to Carbone and Hey (Reference Carbone and Hey1994).

3 Previous experimental findings

Table 2 summarises previous experimental work comparing different elicitation methods. One recurring message from these previous studies is that there is little correlation in the measured degree of risk aversion across different methods. However there is one outstanding feature of most of this literature apparent from Table 2: namely the very small number of observations per subject: in most of them (16 out of the 23 excluding the present paper) only one task is implemented for each subject. So for each method there is only one observation. Moreover, in most of these (depending upon the elicitation method), this observation can lead only to an interval estimate of risk-aversion. In stark contrast, our paper has many tasks for each elicitation method and we can get a point estimate of risk attitude. Loomes and Pogrebna come closest to what we do, but they make a different use of the data, carrying out comparisons across tasks within an elicitation method. In contrast we use all our data from each method to estimate risk-aversion. True, it may be the case, that with large numbers of tasks, subjects get fatigued and hence possibly noisier, but offsetting that is the increased precision from having more observations.

Table 2

Previous experimental investigations of different elicitation methods

	Number of subjects	Allocation	Balloon analog risk task	Becker–DeGroot–Marschak	Bomb risk elicitation	Deal or no deal	Eckel and Grossman	First price auction	Gneezy and Potters	Holt Laury	Pairwise choice	Ranking
Charness and Viceisza (Reference Charness and Viceisza2012)	91								46(1)	45(1)
Crosetto and Filippin (Reference Crosetto and Filippin2016)	350				88(1)		88(1)		86(1)	88(1)
Dave et al. (Reference Dave, Eckel, Johnson and Rojas2010)	881						881(1)			881(1)
Deck et al. (Reference Deck, Lee, Reyes and Rosen2008)	75					75(1)				75(1)
Deck et al. (Reference Deck, Lee, Reyes and Rosen2013)	203		203(1)			203(1)	203(1)			203(1)
Harbaugh et al. (Reference Harbaugh, Krause and Vesterlund2010)	128			128(6)							128(6)
Isaac and James (Reference Isaac and James2000)	34			34(4)				34(40)
Loomes and Pogrebna (Reference Loomes and Pogrebna2014)	423	423(13)								423(5)		423(2)
Reynaud and Couture (Reference Reynaud and Couture2012)	30						30(1)			30(1)
Zhou and Hey (this paper)	96	96(81)		96(54)						96(48)	96(80)

The numbers in the table indicate the number of subjects with that elicitation method and (in brackets) the number of problems within that method

There are slight differences in the way that different methods were implemented in these papers. For example the number of rows in the price lists of HL (usually 10 or 11), but these are relatively unimportant. There is a common theme to the way that the data is analysed in many of these papers: usually it is assumed that the preference functional of the subjects is CRRA; under this assumption, intervals for the subjects’ risk aversions can be derived from the subjects’ decisions. This is then compared across elicitation methods. Some papers go further—trying to fit a CRRA preference functional to the data. One example is Crosetto and Filippin (Reference Crosetto and Filippin2016). To do this they have to convert their one observation from each method into several observations. Take, for example HL. They have one price list with 10 rows and thus 10 ‘decisions’. They then interpret these in their econometrics as 10 independent observations. We (as will be seen) go to the opposite extreme—interpreting them as completely dependent—our view being that subjects will choose a ‘switch point’ and then fill in the table. It is not clear which is the correct assumption. Crosetto and Filippin (Reference Crosetto and Filippin2016) go even further with their Gneezy and Potters data where they say that “we transformed the GP data into 40 binary choices”—assumed to be independent choices.

Some papers estimate an average risk-aversion index (rather than estimating subject-by-subject) and some try to explain this with various demographics and control variables.

Crosetto and Filippin (Reference Crosetto and Filippin2016) also include something particularly interesting when they write: “We show by means of a simulation exercise that part of the often observed heterogeneity of estimates across tasks is due to task-specific measurement error induced by the mere mechanics of the tasks.” This, we suspect, is due to the way that the data from the different methods is analysed—using just one task to elicit an (interval) estimate of risk-aversion. Our method avoids this possible bias as we estimate across a number of tasks.

We should note in passing that the ‘framing’ of the different elicitation methods appears to have an influence on the elicited values. This is well-documented by Weber et al. (Reference Weber, Blais and Betz2002) and Lévy-Garboua et al. (Reference Lévy-Garboua, Maafi, Masclet and Terracol2012). It might seem that this is only marginally relevant to this paper, though it could be argued that different methods are simply one method differently framed.

Although much research about the relationship between elicitation methods and elicited risk preferences has already been carried out, we feel that our paper makes a positive contribution to the literature. Our paper is different, not only in the elicitation methods considered and contrasted, but also in our experimental design and in the number of tasks posed to subjects. We illustrated lotteries as pictures, rather than in words, believing that a pictorial representation aids understanding, with a consequent reduction in noise. We have a different way of implementing the Becker–DeGroot–Marschak method. Our data analysis also differs from virtually all of the previous literature: we fit preference functionals to the data. We use maximum likelihood techniques to estimate the parameters of our preference functionals. We consider not only EU (Expected Utility) preferences but also RD (Rank-Dependent Expected Utility), and also two different utility functions—CRRA and CARA. Thus we estimate different preference functionals, EU-CRRA, EU-CARA, RD-CRRA, and RD-CARA; and not only compare the four elicitation methods, but also the four preference functionals. We can do this as we have a lot more data than most studies.

4 The experimental implementation

We investigated a subset of four of the 20 methods listed in Table 1. We wanted them to be different; many of these 20 ‘overlap’. We chose Holt–Laury (because of its popularity), Pairwise Choice (as it is simple), Becker–Degroot–Marschak (again because of its popularity) and Allocation (because we think that it is more informative than pairwise choice and easier to understand than BDM). These four cover many of the 20 of our review. Our experiment was in four parts, corresponding to these four methods, with different ordering of the parts for different subjects.Footnote ⁶ We describe the four parts here. All parts of the experiment concerned lotteries. The complete set of tasks can be found in https://www.york.ac.uk/economics/exec/research/zhouandhey1/.

Throughout the experiment, lotteries are visually displayedFootnote ⁷ on the subjects’ computer screens in two dimensions, with the amount of money on the vertical axis and the chances on the horizontal axis. We used the random lottery incentive mechanism. If a particular lottery was chosen by this mechanism for paying out at the end of the experiment, the subject would draw a disk from a bag of 100 disks numbered from 1 to 100. The subject got paid the amount of money corresponding to the number on the disk. Let us give an example. Take the lottery in Fig. 1; this represents a lottery where there is a 1 in 2 (50 in 100) chance of gaining £5 and a 1 in 2 (50 in 100) chance of gaining £15. If this was played out at the end of the experiment, if the numbered disk was between 1 and 50 inclusive, the subject was paid £5; if it was between 51 and 100 inclusive the subject was paid £15. One of the possible advantages of this way of portraying lotteries is that the area indicates the expected value of the lottery.

Fig. 1

Representation of a lottery

Part 1 presented a set of 48Footnote ⁸

Holt–Laury price lists each containing 10 pairwise choices. An example is shown in Fig. 2a, b; Fig. 2a showing how it was first seen by the subject and Fig. 2b showing it after its possible completion by a subject. These are screen shots from the experimental software; they appeared full-screen in the experimental interface. In Fig. 2a, the thing that is staying constant is the lottery on the left (which is a 70% chance of £15 and a 30% chance of £0); the thing that is changing is that on the right—in this case a certainty—which increases from a certainty of £1.50 to a certainty of £15. Subjects were asked, for each pair in the list, to click on the preferred item; when doing so, the item on the other side turned grey. Figure 2b shows a possible set of responses—with the lottery being preferred to the certainty until the value of certainty reached £6. To avoid problems with subjects switching at several pointsFootnote ⁹ within the list, the software forced subjects to choose a unique switching point. The 48 Price Lists spanned a variety of cases; details in https://www.york.ac.uk/economics/exec/research/zhouandhey1/. We denote this method HL (as it comes from Holt–Laury mechanism).

Fig. 2

a A price list; b a completed price list

Part 2 asked subjects to respond to 80Footnote ¹⁰ pairwise choice questions. An example is shown in Fig. 3.

Fig. 3

A pairwise choice

In this pairwise choice, subjects had to choose between a lottery which give an 80% chance of £10 and a 20% chance of £5 and a lottery which give a 40% chance of £15 and a 60% chance of £5. The set of 80 pairwise choice questions spanned lotteries with outcomes of £0, £5, £10 and £15 with probabilities of 0, 0.2, 0.4, 0.6, 0.8 and 1.0; details in https://www.york.ac.uk/economics/exec/research/zhouandhey1/. We denote this method PC.

Part 3 asked subjects to respond to 54Footnote ¹¹ Becker–DeGroot–Marschak problems. Typically subjects are shown a lottery and asked to state their maximum willingness-to-pay or minimum willingness-to-accept for the lottery. Many experimenters have reported confusion among subjects with understanding this mechanism, so we adopted a new way of implementing it. Suppose that we want to find the subject’s certainty equivalent of a lottery which pays £x with probability p and £y with probability 1 − p, where x > y. The subject is asked to choose a number £z. We want z to be the certainty equivalent. To obtain this in an incentive compatible way,Footnote ¹² we could tell the subjects that a random number Z will be generated from a uniform distribution over the interval (y, x) and that they will get paid Z if Z > z and will get to play out the lottery if Z ≤ z. The optimal choice of z is the subject’s certainty equivalent of the lottery. Consider the implications in terms of what they are choosing: their choice of z implies the choice of a lottery, which is a compound of the original lottery and the uniform distribution. To illustrate this, consider the lottery in Fig. 4a, where the payoffs are £5 and £15. If they state z = 5 they get to play out the lottery; if they state z = 15, they are opting for the lottery in Fig. 4b—that is a uniformFootnote ¹³ distribution over (5,15); if they state some number in between, for example 11, they are opting for the lottery in Fig. 4c. As z is varied from 5 to 15, the lottery in Fig. 4c varies from that in Fig. 4a to that in Fig. 4b. We simply asked them to choose their preferred lottery; they did this by moving the slider below the graph and then clicking on ‘Confirm’. The implied value of z given by a choice of the lottery in Fig. 4c is 11; this is the observed certainty equivalent.

Fig. 4

a The lottery (in the Becker-Degroot-Marschak mechanism); b the uniform distribution (over the range of the lottery); c a lottery choice

We feel that this is a simpler and more understandable implementation of the Becker–DeGroot–Marschak mechanism, and is a major contribution of our paper. We denote it by LC—Lottery Choice—as they are choosing their preferred lottery. The 54 LC problems spanned lotteries with outcomes of £0, £5, £10 and £15, and with probabilities ranging from 0.0 to 1.0 in steps of 0.1; details in https://www.york.ac.uk/economics/exec/research/zhouandhey1/.

Part 4 asked subjects to respond to 81 allocation problems.Footnote ¹⁴ An example is shown in Fig. 5. In this example, the two states (red and yellow) have probabilities 0.7 and 0.3 respectively. Subjects have 100 tokens to allocate, and the exchange rates between tokens and money are 1 token = 17.5p for red, and 1 token = 10p for yellow. They made their allocation with the slider, with the figure showing the implied amounts of money (and their probabilities). The 81 allocation problems spanned probabilities ranging from 0 to 1 in steps of 0.1 with varying exchange rates; details in https://www.york.ac.uk/economics/exec/research/zhouandhey1/. We denote this method AL.

Fig. 5

An allocation

5 Functional forms assumed

While we are primarily interested in the differences in the estimated risk attitudes between the different elicitation methods, in order to understand these differences we need to model behaviour and hence estimate the risk attitude. To model the behaviour weFootnote ¹⁵ need to choose preference functionals. We do not know the preference functionals of our subjects, so we have to choose a set of such functionals and use our data to find the best-fitting one(s). We choose the most popular in the literature, namely Expected Utility (EU) and Rank Dependent expected utility (RD).

Let us denote by V the value to a subject of a 3-outcome lottery which pays x _i with probability p _i (for i = 1,2,…,3), and let us order the payoffs so that x ₁ ≥ x ₂ ≥ x ₃, then we have

$\text{for}\text{EU:}u\left(V\right)=p_{1}u\left(x_1\right)+p_{2}u\left(x_2\right)+p_{3}u\left(x_3\right)$

$\text{for}\text{RD:}u\left(V\right)=w\left(p_1\right)u\left(x_1\right)+\left(w\left(p_1+p_2\right)-w\left(p_1\right)\right)u\left(x_2\right)+\left(1-w\left(p_1+p_2\right)\right)p_{3}u\left(x_3\right)$

In these expressions, u(.) is the underlying utility function of the subject and w(.) is the rank-dependent weighting function.

We need to specify the utility function u(.) which appears in both these functionals. We adopt both the constant Relative Risk aversion (RR) form and the constant Absolute Risk aversion (AR) form. These are given by

$\text{for}\text{RR:}u\left(x\right)=x^{1-r}/\left(1-r\right),r\ne 1;ln\left(x\right),r=1$

$\text{for}\text{AR:}u\left(x\right)=-exp\left(-rx\right),r\ne 0;x,r=0.$

We note that in both cases r = 0 corresponds to risk-neutrality and increases in r imply greater risk aversion, but there is no mapping between the r for RR and that for AR. This is because the r in RR is a measure of relative risk-aversion, while the r in AR is a measure of absolute risk-aversion.

In fitting the RD specifications we also need to specify a weighting function for the probabilities. This we take to be of the following form (Tversky and Kahneman Reference Tversky and Kahneman1992):

$w\left(p\right)=p^g/{\left(p^g+{\left(1-p\right)}^g\right)}^{1/g}$

In the results that follow we fit, for all four elicitation methods, the four possible combinations of the two preference functionals and the two utility functions, using the obvious notation RREU, RRRD, AREU and ARRD. Essentially we want to see which of these best explains the data and we also want to see whether the estimated parameters differ across the elicitation methods; we do this on a subject-by-subject basis, as it is clear that subjects are different.

6 Our stochastic assumptions and econometric methodology

We should comment on our econometric methodology, as it is different from that used by others. We treat subjects as different, so we analyse subject-by-subject.Footnote ¹⁶ We also use simultaneously all the responses of the subjects on all problems of a particular elicitation method (and use them for estimation), rather than compare responses on particular problems. The latter is what Loomes and Pogrebna (Reference Loomes and Pogrebna2014) and many others have done. So, for example, in their Table 1 on their page 578, they look at the distribution of responsesFootnote ¹⁷ for particular decision tasks and compare these distributions across tasks. They note that the distributions are different across tasks, sometimes significantly so. This could be the case because of noise in subjects’ responses but they present no way of modelling this noise, though the use of a statistical test (in this case a Mann–Whitney test) does necessarily involve some implicit assumption about stochastics.

Another difference between our methodology and that of others is in the number of problems we present to our subjects: pre-experimental simulations show that one needs large numbers of observations to get precise estimates since there is a lot of noise in subjects’ behaviour. Unfortunately there is a downside to having a large number of problems if subject fatigue is increased and because the financial incentive per decision is reduced, we might get increased noise and decreased precision.

Our econometric methodology is to fit, for each of RREU, RRRD, AREU and ARRD, the models to the decisions of the subjects, for each of the four elicitation methods, and hence obtain estimates of the risk aversion index (and also the other parameters). We do this by Maximum Likelihood, using Matlab (the program is available in https://www.york.ac.uk/economics/exec/research/zhouandhey1/). To do this we need to make assumptions about the stochastic nature of the data. This arises from errors made by the subjects. We largely follow convention.

• • HL: we assume that the subject calculates the utility difference between the two lotteries for each pair in the list, but makes an error in the calculation. Further, embodying the fact that the list is presented as a list, we assume that the subject makes the same error for each pair; we further assume that this error has a normal distribution with mean 0 and precision (the inverse of the standard deviation) s. Then the switch-point decision is taken on the basis of where this utility difference plus error changes from positive to negative or vice versa.

• • PC: we assume that on each problem the subject calculates the utility difference between the two lotteries, but makes an error in this calculation; we further assume that this error is independent across problems and has a normal distribution with mean 0 and precision (the inverse of the standard deviation) s. So that the decision is taken on the basis of the sign of the utility difference plus error.

• • LC: we assume that the subject calculates the certainty equivalent of the lottery, but makes an error in this calculation; we further assume that this error has a normal distribution with mean 0 and precision (the inverse of the standard deviation) s. So that the observed certainty equivalent is the subject’s true equivalent plus error.

• • AL: we assume that the subject calculates the optimal allocation that he or should make, but makes an error in this calculation; we further assume that this error has a normal distribution with mean 0 and precision (the inverse of the standard deviation) s. So that the observed allocation is the optimal allocation plus error.

We note that the s in the HL and PC stories are on a different scale than the s in the PC and AL stories—the former being on utilities and the latter on money.

8 Qualifications

We have explored one dimension of the key issue of how ‘best’ to elicit the risk attitudes of subjects—though the issue is clouded by the fact that we do not, and cannot, know the ‘true’ risk attitude of our subjects. This is, of course, assuming that there is a risk-attitude out there to measure. There are some who think that risk attitude varies with the context, and it may be that which we are picking up. If that is true, then, if one is planning to use the elicited risk attitudes to explain behaviour in some economic problem, it would seem sensible to elicit the risk attitude in the same context as the economic problem. The simplest way of doing this is by eliciting the risk attitude from behaviour in the economics problem itself—that is, by finding the risk attitude which best explains their behaviour in that problem. So the risk attitude is elicited as a by-product of studying behaviour in some economic problem.

If, however, one believes that there is a risk attitude out there, one can criticise what we have done on other grounds. There are several dimensions to the problem. We have studied two: the elicitation method and the functional specification. But there is also the dimension of the stochastic assumptions used when analysing the data. As Wilcox (Reference Wilcox2008) points out “choices of stochastic models may be far more consequential than choices of structures such as expected utility or rank-dependent utility.” But notice that he is comparing different functionals (which is what we do) and not different elicitation mechanisms. In that paper he shows that, when using the pairwise choice method for eliciting risk attitudes, the stochastic assumptions can lead to markedly changed elicited values. Once again one does not know the ‘true’ risk attitude, so one cannot declare one stochastic specification to be the ‘best’. Chapter 6 of Bardsley et al. (Reference Bardsley, Cubitt, Loomes, Moffatt, Starmer and Sugden2010), Blavatskyy and Pogrebna (Reference Blavatskyy and Pogrebna2010) and Stott (Reference Stott2006) make the same point. Wilcox (Reference Wilcox2008) looks solely at pairwise choice problems. He considers a variety of stochastic specifications, most varying in their heteroscedasticity. Our stochastic specifications are all homoscedastic. Given the different nature of our methods, it is clearly not the case that we can apply the same set of heteroscedastic models to each method, so it is not clear what comparisons one can make. But to go some way to meeting this criticism we have refitted our allocation data with an heteroscedastic specification—a Beta distribution.Footnote ²¹ The figure of the results can be found in https://www.york.ac.uk/economics/exec/research/zhouandhey1/. In the figure, the left-hand graph is a scatter of the risk-aversion estimates for RREU obtained using the allocation method with a beta stochastic specification against that with the normal specification that we have used. The right-hand graph is the corresponding graph for RRRD. It will be seen that the estimates do differ a little with the stochastic specifications but the correlations are high (0.985 and 0.933 respectively).

So there are variations in the estimates produced with the different elicitation methods, and (smaller?) variations with different stochastic specifications. The latter may cancel out the former, and indeed it may be the case that with each pair of elicitation methods there is a pair of stochastic specifications for which the same estimates are produced. This is unlikely but suggests an interesting project for future research.