1 Introduction

Elicitation of time preferences in the discounted utility (DU) model requires simultaneous estimation of the felicity and discount functions. To demonstrate, consider a subject whose preferences over consumption streams are represented by a time-invariantFootnote ¹ DU model, and decides at time 0 on her consumption in periods $t$ and $t+k$ . Her utility is given by $U\left(c_t,,,c_{t+k}\right)=D\left(t\right)u\left(c_t\right)+D\left(t,+,k\right)u\left(c_{t+k}\right)$ , where $D\left(·\right)$ is the subject’s discount function, and $u\left(·\right)$ is her felicity function. Estimation of a discount function that is based on indifference between consumption of $c^1$ at time $t$ and $c^2\left(>,c^1\right)$ at time $t+k$ (and nothing in the other period), e.g. through multiple price list (MPL), implies that $D\left(t,+,k\right)/D\left(t\right)=u\left(c^1\right)/u\left(c^2\right)$ . It is well known that if the researcher assumes linear $u\left(·\right)$ while the true felicity function is concave, it will bias the estimated $D\left(t,+,k\right)/D\left(t\right)$ downwards.Footnote ²

To cope with this difficulty Andersen et al. (Reference Andersen, Harrison, Lau and Rutström2008) used the fact that under the standard Discounted Expected Utility model risk and time preferences are intimately linked: a concave utility function exhibits both atemporal risk aversion and a desire for intertemporal smoothing of consumption. Their double multiple price list (MPL) procedure uses one atemporal multiple price list to estimate risk preferences and a second intertemporal multiple price list to estimate time preferences. They use the curvature of the atemporal utility function in order to adjust the estimation of the discount function.

Andreoni and Sprenger (Reference Andreoni and Sprenger2012a, abbreviated exchangeably as AS in the following) proposed an interesting alternative according to which a single instrument can be used to jointly estimate the felicity and discount functions, without explicitly relying on the subject’s risk preferences. Andreoni and Sprenger’s convex time budgets (CTB) are a convexification of pairwise choices made on lines in the intertemporal MPL and allow the economist to directly measure intertemporal substitution. In their design the subject faces linear experimental budgets, which allow her to choose interior allocations between payments at two time periods ( $t$ and $t+k$ ). One can rationalize such interior allocations if the subject’s preferences between $c_t$ and $c_{t+k}$ are (weakly) convex. It thus provides a way to directly adjust the measurement of the subject’s discount function for intertemporal substitution without the need to explicitly invoke expected utility.Footnote ³ Andreoni and Sprenger (Reference Andreoni and Sprenger2012a), and their closely related study (Andreoni and Sprenger Reference Andreoni and Sprenger2012b), have been followed by a large number of applications and comments.Footnote ⁴

The current paper provides commentary and guidance for economists who wish to use CTB to measure time preferences. Specifically, we discuss a methodology for measuring the consistency of subject-level choices with a very general model of intertemporal choice (more general than the DU model). A key element of this methodology requires the inclusion of two convex budgets that differ only in their income level in the CTB design, which makes a direct test of wealth monotonicity possible.Footnote ⁵ We illustrate our approach using the data set of AS (on time allocation of money) and on the most influential application of CTB to date—the work of Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015), which investigates allocation of effort over time.

In the AS study, we find surprisingly high rates of violations of the general model of intertemporal choice that we consider. For the subjects who did not exhibit any curvature in their CTB choices, we directly estimate their discount factor based on the three choice lists and the corresponding CTBs assuming linearity of the felicity function, for the sake of comparison. We find WARP violations between choices made on CTB and choice lists for these subjects, and most of these violations are in the direction of exhibiting lower impatience in CTB than in choice lists. This could be an explanation for why Andreoni and Sprenger (Reference Andreoni and Sprenger2012a) obtain reasonably high CTB discount factors for these subjects even though their discount factors are not adjusted upward (as there is no evidence that their felicity function is concave). In the Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) paper we find substantial rates of demand monotonicity violations, especially in their replication study. The latter violations are accompanied by violations of classical monotonicity, which in turn are empirically associated with time inconsistent behavior. Choices that violate classical monotonicity cannot be rationalized by a monotone utility function, a fact that relates this finding to the literature on “decision-making quality” (Choi et al. Reference Choi, Kariv, Müller and Silverman2014)—if rationalizability of choices by a utility function is a marker of choice quality, then there is definitely some relation between the decision making quality and adherence to the normative standard of time consistency. We believe these surprising findings highlight the importance of implementing our suggested methodology before and after using CTB data for estimation.

In what follows, we suggest possible behavioral mechanisms (for example, magnitude effect, reference dependence, subject confusion, experimental design) that may generate the observed inconsistencies. We hope that our work encourages future research to further address these questions.

2 Literature review

Andreoni and Sprenger (Reference Andreoni and Sprenger2012a, Reference Andreoni and Sprengerb) have been among some of the most influential experimental papers in recent years, generating a significant amount of academic discussion about the experimental methodologies of estimating and understanding risk and time preferences. The most significant contribution of Andreoni and Sprenger (Reference Andreoni and Sprenger2012a) is the parsimony of the CTB framework for estimating time preferences without explicitly relying on expected utility in order to adjust the discount function for the curvature of the felicity function. The authors also do a very convincing and careful job of equalizing the subject convenience and confidence for present and future payments to measure present bias separately from the confounds of differential transaction costs. Andreoni and Sprenger (Reference Andreoni and Sprenger2012a) find very little evidence for present bias and curvature in the atemporal felicity function. The closely related paper Andreoni and Sprenger (Reference Andreoni and Sprenger2012b) compares CTB decisions in which payments on both dates are certain, to ones in which payments are risky, as realized by two independent lotteries. The authors hypothesise that subjects’ choices are governed by an (atemporal) utility function that is more concave than the one employed under conditions of certainty, as subjects choose more balanced portfolios of sooner and later payments under the risky condition.

Andreoni and Sprenger (Reference Andreoni and Sprenger2012a, Reference Andreoni and Sprengerb) led a host of follow-up studies which have used this methodology for estimation purposes. CTB has been otherwise used to study the evidence for present bias among particular sectors of the population (e.g. Giné et al. Reference Giné, Goldberg, Silverman and Yang2016; Ashton Reference Ashton2014; Kuhn et al. Reference Kuhn, Kuhn and Villeval2014; Carvalho et al. Reference Carvalho, Meier and Wang2016). Aside from being a huge influence on the experimental literature, Andreoni and Sprenger (Reference Andreoni and Sprenger2012b) has also generated a range of comments on the robustness and interpretation of its findings. Cheung (Reference Cheung2015) investigates the robustness of AS (Reference Andreoni and Sprenger2012b) to alternative experimental design. In a particularly interesting translation of the key AS (Reference Andreoni and Sprenger2012b) CTB treatment into a double MPL environment, Cheung uses an MPL where payments on both dates are received with 50% probability, and contrary to AS (Reference Andreoni and Sprenger2012b) he finds “very little evidence of difference of any systematic deviation in discounting behavior under risk as compared to certainty.” Cheung also finds evidence for non-linearity in intertemporal preferences when, in the absence of diversification opportunities (the risks across time being correlated), the proportion of interior allocations falls between those of no risk and independent risks. Miao and Zhong (Reference Miao and Zhong2015) utilize two additional CTB risk treatments (one of them similar to that of Cheung) to show that the behavior exhibited in temporal risk environments is more consistent with a model which separates risk attitudes and intertemporal substitution (like Epstein and Zin Reference Epstein and Zin1989; Halevy Reference Halevy2008) than the one suggested in AS (Reference Andreoni and Sprenger2012b). Epper and Fehr-Duda (Reference Epper and Fehr-Duda2015) demonstrate that probability weighting in rank-dependent utility models that take their entire temporal portfolios into account are able to explain subjects’ preference for intertemporal diversification as well as their proneness to intertemporal common-ratio violations and, therefore, all the major AS findings. Schmidt (Reference Schmidt2014) offers a different perspective: if the monetary payments in AS (Reference Andreoni and Sprenger2012a, Reference Andreoni and Sprengerb) are interpreted as income instead of as consumption, then arbitrage and portfolio risk minimization in a DEU framework could justify why subjects choose more interior solutions in the correlated temporal risk task than in the deterministic temporal task.

Harrison et al. (Reference Harrison, Lau and Rutström2013) is another critical comment that directly addresses Andreoni and Sprenger (Reference Andreoni and Sprenger2012a). One of the key arguments of Harrison et al. (Reference Harrison, Lau and Rutström2013) is that the large number of corner choices in the CTB data generates a bi-modal data set which is not well suited to analysis via non-linear least squares estimation techniques. The authors note that more appropriate econometric techniques, which attempt to match the full distribution of the data, imply that the data is best rationalized with a convex utility function. Furthermore, they argue that convex utility functions are a priori implausible in this environment and they therefore question the quality of the data.

Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) has been the most successful behavioral application of the CTB design. The authors use CTB to show present bias while using primary rewards (effort tasks). For sake of comparison, they pair this effort study with a companion monetary discounting study and find very limited time inconsistency in monetary choices. We analyze in greater detail demand monotonicity violations in the effort domain in Sect. 4.

3 Andreoni and Sprenger (2012)

The Andreoni and Sprenger (Reference Andreoni and Sprenger2012a) design includes nine choicesets per subject, where each choiceset is a collection of five CTB tasks between payments at $t$ and at $t+k$ (where $t=0,7,35$ and $k=35,70,98$ measured in days). Eight out of the nine choicesets contain a wealth shift which could be used to test for wealth monotonicity. Demand monotonicity is tested by the other four CTB tasks within a choiceset. Impatience is tested by comparing across choicesets belonging to the same subject. When evaluating wealth monotonicity we allow for the non-generic possibility of linear preferences with marginal rate of substitution between $c_t$ and $c_{t+k}$ equal to the gross interest rate over k days in which the wealth shift occurs, i.e. $1+r=1.25$ . In this case, the demand is a correspondence and wealth monotonicity as defined above need not hold.Footnote ⁷

AS included three choice lists (MPL) that correspond to three choicesets. Each one of these choice lists included four pairwise choices that corresponded to CTB. In other words, on these lines of the choice list a subject was asked to make a pairwise choice between the two points in which each CTB intersects the horizontal axis ( $c_{t+k}=0$ ) and the vertical axis ( $c_t=0$ ). In the CTB task the menu of allocations the subject was allowed to choose from included these two allocations and all interior allocations. We use this set-up to test for violations of the Weak Axiom of Revealed Preference (WARP), which requires that if an alternative is chosen from a menu and is available in a sub-menu then it should be chosen from the sub-menu as well. If in the pairwise choice a subject chooses one corner while in the CTB she chooses the opposite corner this contradicts WARP. The implication is that there exists no complete and transitive preference that can rationalize these choices.

3.2 WARP violations

Out of the 36 subjects who made all corner choices in CTB, we found 43 violations of WARP.Footnote ⁸ This is especially impressive if one considers that 17 of them always chose later consumption in the CTB and switched immediately in the choice lists (always chose later consumption). Therefore WARP violations could be detected only among the remaining 19 subjects. The direction of WARP violations is not random: 34 violations are in the direction of exhibiting less impatience in CTB than in choice list, while only 9 are in the opposite direction.

Fig. 1

Choice list versus CTB estimates of discount factor for the 36 all-corner subjects

Since these subjects did not exhibit any curvature in their CTB choices, we can directly estimate their discount factor based on the three choice lists and the corresponding CTBs assuming linearity of the felicity function. One should not adjust for curvature for these subjects, since their intertemporal decisions did not suggest any concavity of the felicity function.Footnote ⁹

The results are plotted in Fig. 1.Footnote ¹⁰ We find that for 11 subjects the discount factor estimated from CTB data would be higher than the one estimated from choice list data, while for two subjects the relation between the discount factors would be in the opposite direction. Note that the choices made by the 17 subjects who always chose later consumption can be rationalized with a discount factor of 1, and one cannot form a point estimate of the discount factors of 4 other subjects who always chose immediate consumption in at least one of the three CTBs.Footnote ¹¹

Among the other 61 subjects who made at least a single interior choice in the 45 CTB tasks we find a similar directional effect of WARP violations. If one of the three choicesets that has a comparable choice list has all corner choices, we find 23 WARP violations in the direction of exhibiting lower impatience in CTB than in choice list and none in the opposite direction. In choicesets with interior CTB choices (where the potential to observe direct WARP violation is smaller) we found ten violations in the direction of exhibiting lower impatience in CTB than in choice list and five in the opposite direction. It is important to note that owing to the WARP violations, there is no model of complete and transitive preferences that could potentially help us understand the correlation between impatience parameters estimated via CTB and MPL (DMPL) techniques within or across studies.

The WARP violations indicate an inconsistency between choices elicited via CTB and choices elicited via a choice list, and the strong correlation between WARP violations and impatience measures suggest that the inconsistency is, in some sense, structural rather than random. One possible interpretation of the WARP violations [following Ok et al. (Reference Ok, Ortoleva and Riella2015)] is that CTB induces a form of reference dependence by moving away from binary choices.Footnote ¹² However, it could be argued that a choice list may also be susceptible to reference dependence—consider for example a list starting from low interest rates compared to a list starting from high interest rates [see Andersen et al. (Reference Andersen, Harrison, Lau and Rutström2006) for further discussion].

While the presence of inconsistency between the choices in the CTB and MPL tasks does not necessarily privilege either elicitation procedure, the results are nevertheless both disconcerting and very interesting. We believe that understanding the nature of the discrepancies between the two elicitation mechanisms is an important open question.

3.3 Demand and wealth monotonicity

As the 36 subjects with all corner choices did not take advantage of the convexification offered by the CTB, we believe it would be misleading to include them in evaluating CTB for internal consistency (monotonicity). Hence, the analysis below concentrates on the remaining 61 subjects with at least one interior choice.

3.3.1 Frequency

Table 1 reports the frequency of choicesets that have wealth or demand monotonicity violations as a function of the number of interior choices made in a choiceset.

Table 1

Demand and wealth monotonicity violations as a function of number of interior choices

# of interior choices in a choiceset	# of choicesets	# of choicesets that exhibit demand monotonicity violations	# of choicesets that exhibit wealth monotonicity violations	# of choicesets that exhibit either wealth or demand monotonicity violations
0	435a	1	9	10
1	101	10	26	34
2	78	5	31	34
3	80	6	47	48
4	63	6	47	47
5	116	42	56	76
Total	873	70	216	249

^a324 out of the 435 choicesets with no interior choice (almost 75%) belong to the 36 subjects with only corner solutions

The frequency of demand monotonicity violations is below 10% for choicesets that contain 4 or fewer interior choices. However, more than 36% of choicesets with all interior choices have demand monotonicity violations. The frequency of wealth monotonicity violations is considerably higher: around half of the choicesets with at least one interior choice have a wealth monotonicity violation.

Table 2

Joint frequency of number of interior choicesets (by subjects) and number of interior choicesets that do not violate (demand and wealth) monotonicity (by subject), restricted to subjects who have at least one interior choiceset

# of interiora	# of monotone interiora choicesets										Total
choicesets	0	1	2	3	4	5	6	7	8	9
1	0	2									2
2	1	0	0								1
3	2	0	0	2							4
4	0	2	0	0	1						3
5	1	1	0	0	2	1					5
6	0	2	0	0	0	1	1				4
7	0	0	1	1	0	0	0	2			4
8	0	3	2	0	0	2	1	1	0		9
9	1	8	5	4	2	0	2	2	0	5	29
Total	5	18	8	7	5	4	4	5	0	5	61

The bolded entries highlights subjects that had violations in half or more of their interior choicesets

^aA choiceset is considered “interior” if at least a single choice (out of 5) is not at the corners of the budget line

Table 2 reports, for the 61 subjects with at least one interior choiceset, the distribution of subjects satisfying wealth and demand monotonicity as a function of the number of interior choicesets. A choiceset is considered interior if at least a single choice (out of five) is not at the corners of the budget line ( $c_t,c_{t+k}>0$ ).Footnote ¹³ Table 2 reveals that more than half of the 61 subjects violate monotonicity in at least half of their interior choicesets (the bolded entries in the table).

3.3.2 Magnitude

The two tables above demonstrate the high frequency of non-monotone choices in interior choicesets, especially as a response to wealth changes. We now turn to measure the magnitude of these behaviors. We calculate the magnitude of a wealth monotonicity violation by the number of tokens required to be reallocated in order to eliminate the violation at the higher wealth level. Our wealth monotonicity measure differs substantially from that reported in footnote 25 by Andreoni and Sprenger (Reference Andreoni and Sprenger2012a). We find that there are 216 violations of wealth monotonicity, with an average size of 24.46 tokens, which is 24.46% of the experimental budget or $4.89 of $c_t$ at the higher wealth level.Footnote ¹⁴ That is, conditional on violating wealth monotonicity, the magnitude of the measure is almost as high as the equivalent measure calculated for random choice: if choices are generated at random using a uniform distribution over the tokens allocated to $c_t$ , independently among the two budget lines, the expected value of our measure would be approximately 27 tokens.Footnote ¹⁵ Andreoni and Sprenger report an average adjustment of just 1.67 tokens to restore wealth monotonicity. There are three differences between our calculation procedure and the one used in AS (presented in decreasing order of importance).

First, we take the absolute value of violations. AS mistakenly defined non-monotonicity that is expressed as an over-allocation to $c_{t+k}$ (and under-allocation to $c_t$ ) as a negative number, while non-monotonicity that is expressed as an under-allocation to $c_{t+k}$ (and over-allocation to $c_t$ ) as a positive number. Because both over- and under-allocation to $c_{t+k}$ are prevalent across the population, violations cancel out at the aggregate level. Taking the absolute value of the violations accounts for 24% of the total discrepancy: starting from the 1.67 tokens reported in AS, correcting for this increases the measure to 7.03 tokens. Second, accounting for almost all of the residual discrepancy, we include only choicesets with a wealth monotonicity violation in the denominator. In contrast, AS use a denominator that includes all choicesets with a wealth shift, rather than just choicesets with a wealth monotonicity violation. We believe that the AS approach, by including the 36 subjects who made only corner choices (and had no wealth monotonicity violation), artificially dilutes the magnitude of monotonicity violations performed by subjects who responded to the convexification offered by the CTB design by making interior choices. Of course, this is mostly an accounting decision, and hence, we consider it less important than our first point of departure. Lastly, we measure violations using whole numbers of tokens, thereby reflecting the choice environment presented to subjects, accounting for less than .5% of the total discrepancy. AS use integer number of tokens when calculating the magnitude of demand monotonicity violations, but not when calculating the magnitude of wealth monotonicity violations.

Turning now to demand monotonicity, we calculate the magnitude of demand monotonicity violations by finding the minimal amount of $c_t$ that needs to be reallocated per choiceset to restore monotonicity. There are 70 choicesets with demand monotonicity violations, with an average size of 17.4 tokens and a value (at time $t$ ) of $3.02.Footnote ¹⁶

Another measure of the degree of non-monotonicity within a choiceset is to calculate the smallest number of choices that must be removed from a choiceset to restore monotonicity.Footnote ¹⁷ For the 249 choicesets that exhibit at least one non-monotonicity, the average number of data points that must be removed is 1.2. This figure includes the 179 choicesets that exhibit only wealth non-monotonicity and therefore require the removal of only a single data point; for the 70 choicesets that exhibit demand non-monotonicity the average number of data points that must be removed is 1.8.

There is a possibility that the income non-monotonicity we identify is an outcome of subjects exhibiting different temporal preferences for different stakes, known in the literature as the magnitude effect (see Thaler Reference Thaler1981; Frederick et al. Reference Frederick, Loewenstein and O’Donoghue2002). In studies that vary the outcome sizes, subjects appear to exhibit greater patience toward larger rewards. There are 152 instances of wealth monotonicity violations consistent with the subjects exhibiting greater patience, and 64 instances of the same in the opposite direction. The average size of reallocation required to restore monotonicity is 19.77 and 30.90 tokens, respectively. As a result, we suspect that the magnitude effect by itself is not sufficient to explain the frequency and magnitude of wealth monotonicity violations resulting from miniscule changes in budget wealth. In any case, whether “magnitude effect” is the correct interpretation of this phenomenon is an open question that future research would hopefully shed light on. One step in that direction is the recent work of Sun and Potters (Reference Sun and Potters2016), which reports a significant “magnitude effect” in CTB tasks.

4 Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015)

One critique that can be levelled against measuring time preferences using monetary payments, as in AS (Reference Andreoni and Sprenger2012a), is that subjects’ responses may be driven by their access to credit and savings instruments rather than their underlying time preferences over consumption bundles. Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) build on this argument, and compare the preferences elicited through CTB design using both monetary payments and effort tasks, where the effort tasks are possibly non-fungible and assumed to impose a dis-utility on the subject and therefore allow a direct measurement of time preferences with respect to the work-leisure trade off. In their first study (henceforth original experiment) subjects allocate both cash and units of effort over two dates, using a within-subject design. They also run a second study (henceforth replication experiment) in which they implement a between-subject design to replicate the findings of the first study. Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) identify two key differences between the monetary tasks and the effort tasks. First, present bias is found only in the effort domain and, second, the proportion of interior choices is much higher in the effort domain. This result supports the critique of monetary tasks as a tool for measuring impatience due to the fungibility of money.

In the following subsections we analyze the rate of violations of our behavioral desiderata across both the monetary and effort tasks. We believe it is a useful exercise for three reasons. First, Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) has been the most influential application of CTB on “primary versus monetary rewards”, a topic that is contested between some behavioral and experimental economists. Second, the interface used in the Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) is different from the one used in AS (Reference Andreoni and Sprenger2012a), so it allows us to evaluate if the demand monotonicity we documented in the latter is a consequence of interface subjects faced in the original AS study. Third, one key implication of our data analysis for the design of future CTB experiments revolves around the divisibility of effort tasks, and we will discuss this issue in detail in Sect. 4.1.1.

4.1 Data Analysis

Table 3

Original study, work data

# of interior	# of demand					Total
choices	monotonicity violations
	0	1	2	3	4
0	77	0	0	0	0	77
1	150	0	1	0	0	151
2	15	8	1	0	0	24
3	26	15	4	0	0	45
4	28	12	2	0	2	44
5	224	62	51	22	20	379
Total	520	97	59	22	22	720

Number of interior choices (rows) crossed with number of demand monotonicity violations (cols)

Table 4

Original study, money data

# of interior	# of demand
choices	monotonicity violations			Total
	0	1	2
0	289	0	0	289
1	35	2	0	37
2	9	0	0	9
3	5	7	1	13
4	3	2	2	7
5	23	5	2	30
Total	364	16	5	385

Number of interior choices (rows) crossed with number of demand monotonicity violations (cols)

Because the Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) design does not include a wealth shift, we are not able to test their data for wealth monotonicity. Instead, we calculate the rate of demand monotonicity violations for the effort tasks in the original study and find the frequency of violations to be higher than the rate of violations in AS (27.8% compared to 8.0%).Footnote ²¹ ${}^{,}$ Footnote ²² As in AS, the rate of violations is higher when all choices are interior (Table 3), 40.9% of choicesets with all interior choices have demand monotonicity violations (the corresponding proportion in AS is 36.2%) Table 4 displays the number of demand monotonicity violations in the monetary allocations; the rate of violations is rather low, as might be expected given that almost all choices are corner choices. For impatience monotonicity, there is very little evidence of violations for both effort tasks and monetary choices in the original study, and there was no scope for impatience monotonicity violations in the replication study.

4.1.1 Rounding of choices and classical monotonicity violations

One key aspect to consider when implementing CTB over effort tasks rather than over monetary rewards is the divisibility of the units. Because the effort tasks in Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) are discrete, there is a complication in offering finely distributed discrete choices on a budget line. Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) deal with this in two different ways: in the original experiment they vary the possible work allocations in the nearer period on a regular integer grid (presented to subjects as a slider), and then round-down the corresponding values of the other farther period choice using the budget equation.Footnote ²³ This rounding method can, for some interest rates, lead to the availability of dominated allocations. For example, when the task rate is 1.5, and $e_t$ may be chosen to lie between 0 and 50, then the allocations $(e_t,e_{t+k})=(2,32)$ and $(e_t,e_{t+k})=(1,32)$ are both available.

In the replication, the authors considered all possible pairs of earlier and later effort choices that would be on the budget line, and then rounded both of them (independently) to the nearest integer. This rounding method can create situations in which for a given “task rate” subjects are offered allocations which are below or above the budget line. For example $(e_t,e_{t+k})=(41,17),(40,17)\text{and}(40,18)$ appeared as possible choices (and each was chosen by at least some subjects) for the same budget line.Footnote ²⁴ This implies that certain subjects chose allocations that are strictly dominated by other available allocations. We identify such choices as violations of classical monotonicity. The rate of such violations is alarmingly high in the effort treatment of the replication data: 62 of the 95 subjects selected 8 or more (out of a maximum possible 18) dominated allocations. Because of the nature of the slider interface presented to subjects we think that subjects were probably simply unaware that dominating choices were available.

Table 5

Replication study, work data

# of interior choices	# of demand monotonicity violations									Total
	0	1	2	3	4	5	6	7	8
0	11	1	0	0	0	0	0	0	0	12
1	9	0	0	0	0	0	0	0	0	9
3	1	1	0	0	0	0	0	0	0	2
5	0	1	1	1	1	0	0	0	0	4
6	0	0	1	0	0	0	0	0	0	1
7	5	2	2	1	0	0	0	1	0	11
8	7	5	1	1	1	0	1	1	0	17
9	12	25	43	26	12	6	2	1	7	134
Total	45	35	48	29	14	6	3	3	7	190

Number of interior choices (rows) crossed with number of demand monotonicity violations (cols)

Table 6

Replication study, work data

# of interior choices	# of demand monotonicity violations								Total
	0	1	2	3	4	5	7	8
0	11	1	0	0	0	0	0	0	12
1	9	0	0	0	0	0	0	0	9
3	2	0	0	0	0	0	0	0	2
5	0	1	3	0	0	0	0	0	4
6	0	1	0	0	0	0	0	0	1
7	5	4	0	1	0	0	1	0	11
8	7	6	2	0	0	1	1	0	17
9	37	34	29	15	11	3	0	5	134
Total	71	47	34	16	11	4	2	5	190

Number of interior choices (rows) crossed with number of demand monotonicity violations (cols) after “correction” for classical monotonicity violations

The frequency of demand monotonicity violations is also high in the effort treatment of the replication: only 45 out of the 190 total choicesets have no demand monotonicity violations, which is a failure rate of 76.3%. Table 5 shows the number of demand monotonicity violations for the effort tasks in the replication study. We recognize that this high frequency of demand monotonicity violations might have been due to subjects choosing “above budget-line” and “below budget-line” allocations on different offered budget sets, and that these choices may have been driven by an unawareness of other nearby feasible allocations. For each observation which would constitute a demand monotonicity violation along with a choice at the adjacent lower discount rate, we first note if the higher, later-period effort choice at the higher discount rate was above the budget line due to experimental design. In this case, we “correct” the higher later period choice by moving it onto the budget line (thus decreasing it), in an attempt to “remove” the demand monotonicity violation. Similarly, for observations related to demand monotonicity violations, we note if the lower later period effort at a lower discount rate was below the budget line. As before, we “correct” this lower later period choice by moving it on the budget line (thus increasing it), in an attempt to “remove” the demand monotonicity violation. Using this modified data set, in Table 6, we report a more conservative frequency of demand monotonicity violations. The frequency of violations is still quite high (Table 6) as close to 63% of adjusted choicesets exhibit demand monotonicity violation.

Another reasonable hypothesis is that higher frequency of failing demand monotonicity in the replication experiment could be due to the fact that there are 9 discount rates rather than 5 (as in AS and the original experiment of Augenblick et al.). One could select 5 of the 9 discount rates, and evaluate demand monotonicity on that smaller set of choices for comparison.Footnote ²⁵ The demand monotonicity violation rates in the reduced exercise are still high: 74 (before aforementioned “correction”) or 67 (after “correction”) choicesets out of 190 choicesets (38.9 or 35.3%) have demand monotonicity violations. For comparison, in the original experiment the frequency was 200/720 (27.8%). Our conclusion is that the higher failure rate cannot be attributed solely to the higher number of interest rates.

We think that the high frequency of classical monotonicity violations and demand monotonicity violations, particularly the former, point out that certain participating subjects were not always well versed with the choice environment, thus failing to recognize and consider strictly better choices. Our suggestion to future studies planning to implement CTB effort tasks, is to either use divisible effort tasks or else follow the truncation of choices used by Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) in their original effort task experiment.

4.1.2 Classical monotonicity violation and time preferences

Finally, we touch on the question if there is a correlation between dynamically inconsistent choices and classical monotonicity violations. Given that estimation of present bias or lack thereof is one of the primary goals of Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015), we think the relation between the frequency of violations and observed temporal preferences is of primary importance. We use the following non-parametric method to identify time consistent choices. Each subject makes 2 decisions of $(e_t,e_{t+k})$ for every discount rate, once at $t=0$ and again at $t>0$ . For every discount rate, if the subject allocates the same amount of effort at date $t$ in both her choices ( $|e_t^0-e_t^t|\le 1$ , i.e, allowing a tolerance of 1), we identify that pair as time-consistent, otherwise we label the pair as dynamically inconsistent. Given that subjects make such pairs of choices for 9 different discount rates, the subjects can have 0–9 total pairs of time-consistent choices. In Table 7 we tabulate this number against the number of classical monotonicity violations.Footnote ²⁶ We find that time inconsistency is associated with classical monotonicity violations, so subjects with fewer dynamically inconsistent choice pairs make fewer classical monotonicity violations than the ones who have more.

The association between failure of classical monotonicity and time-inconsistency is an interesting empirical relation that provides some additional insight into the Augenblick et al. (Reference Augenblick, Niederle and Sprenger2015) choice environment. Abiding by classical monotonicity is a marker of “decision-making quality”—choices that can be rationalized by an increasing utility function (Choi et al. Reference Choi, Kariv, Müller and Silverman2014). It follows that there is a relation between the decision making quality and adherence to the normative standard of time consistency. Moreover, “low-quality” decisions that are associated with time inconsistent choices cannot be rationalized by any utility function, let alone by quasi-hyperbolic discounting one.Footnote ²⁷ As before, with violations of income monotonicity and WARP, we think this is a fascinating topic worthy of independent future study.

Table 7

Replication study, work data

# time- inconsistent choices	# of classical monotonicity violations		Total
	0–7	8–16
0–3	12	12	24
4–6	13	13	26
7–9	8	37	45
Total	33	62	95

Number of time inconsistent choices (rows) crossed with number of classical monotonicity violations (cols). A Fisher’s exact test rejects the null hypothesis of independence at standard significance levels (p $=$ 0.04), suggesting a positive association between dynamic inconsistency and classical monotonicity violations