2.1.1. Baseline – one job problem

There is one available job, and the DM wants to hire a worker to maximize their profits. There are two candidate workers: Ann and Bob. Only one of Ann or Bob applies for the job, each with an equal chance. The DM has to decide one wage for the job before observing who applies. Ann and Bob have different willingnesses to accept (θ_A and θ_B) and bring different revenues (v_A and v_B). We assume Ann always brings higher revenue than Bob ( $v_A \gt v_B$), Ann’s willingness to accept is lower than that of Bob ( $\theta_A \lt \theta_B$), and the revenue Ann brings is higher than Ann’s willingness to accept ( $v_A \gt \theta_A$). A worker will accept the job if and only if the wage the DM chooses is greater than or equal to their willingness to accept. Since Bob’s willingness to accept is greater than Ann’s, there is no wage that Ann rejects and Bob accepts. If the worker who applies rejects the wage, then the DM’s payoff will be 0. In this problem, the set of states of the world is $\Omega=\{\mathbf{A}, \mathbf{B} \}$, where A represents the state where Ann applies for the job and B represents the other.

Niederle and Vespa Reference Niederle and Vespa(2023) describe the contingent reasoning process as thinking of all hypothetical states and computing payoffs for each action-state pair.Footnote ⁶ We slightly modify this concept to define contingent reasoning. As a first step, we define the action-derived states as contingencies. Specifically, for any wage choice w, each state can be extended to be one where either a worker Accepts or Rejects (for simplicity, A and R, respectively). Thus, for the entire set of wage choices, $\mathcal{R}_+$, we can consider a Cartesian product of extended states $\{A, R\}\times \{A, R\} = \{(A, A), (A, R), (R, A), (R, R)\}$, where the first element of the pair represents Ann’s decision and the second element represents Bob’s decision.Footnote ⁷ Each wage w will trigger exactly one of these extended states. We define each element of this Cartesian product as a contingency.

Next, for each of these contingencies, we call the wage that maximizes their utility in that contingency the contingency-optimal wage. Thus, there will be four contingency-optimal wages in our problem. We then define two levels of contingent reasoning: First, does the DM choose one of the contingency-optimal wages? Second, do they choose the utility-maximizing wage from that set?

In our baseline setting, we can simplify the set of contingencies. Since $\theta_A \lt \theta_B$, we can rule out (R, A) and consider three relevant contingencies for any wage w: (1) $w \geq \theta_B$, which triggers (A, A), (2) $\theta_A \leq w \lt \theta_B$, which triggers (A, R), and (3) $w \lt \theta_A$, which triggers (R, R). Assuming expected utility, the DM evaluates each w according to

\begin{equation*} U(w)= \begin{cases} \frac{1}{2}u(v_A-w)+\frac{1}{2}u(v_B-w) \;\;\;\; & \mbox{if}\ w \geq \theta_B\\ \frac{1}{2}u(v_A-w) \;\;\;\; & \mbox{if}\ \theta_A \leq w \lt \theta_B \\ 0 \;\;\;\; & \mbox{if}\ w \lt \theta_A, \end{cases} \end{equation*}

where $u(\cdot)$ is an increasing function with $u(0)=0$. Since $u(\cdot)$ is an increasing function, U decreases in w. Thus, the contingency-optimal wage is the threshold value in each contingency: $w=\theta_B$, $w=\theta_A$, and $w \in [0,\theta_A)$ respectively. Our first question is whether the DM chooses one of these three contingency-optimal wages. Then, we ask if they choose optimally from that set. Since $v_A \gt \theta_A$, it is trivial that $w \lt \theta_A$ would give a lower utility than θ_A, so we can rule out $w \in [0,\theta_A)$. Thus, fully successful contingent reasoning requires choosing the wage that maximizes the DM’s utility from $\{\theta_B, \theta_A\}$.

2.1.2. Various notions of dominance

As one dimension of complexity, we consider the dominance property of choices. Specifically, we introduce a modified concept of dominant and obviously dominant strategies, called dominant and obviously dominant choices. In addition to the well-known dominant strategies concept, Li Reference Li(2017) suggests a concept called “obviously dominant strategies” and shows that obviously dominant strategies lead to rational choices more often than dominant strategies. We apply this notion, restricting choices first to those that are contingent-optimal.Footnote ⁸

Specifically, let $W^*$ be the set of contingency-optimal choices. By a slight abuse of notation, let u be a function of wage and state, (w, s), in this subsection. We say that $w^*$ is dominant (or a dominant choice) if $u(w^*,s) \geq u(w,s)$ for all $ s \in \Omega$ and for all $w \in W^*$. We say that $w^*$ is obviously dominant (or an obviously dominant choice) if $\min_{s \in \Omega}u(w^*,s) \geq \max_{s \in \Omega}u(w,s)$ for all $w \in W^*$. Note that any obviously dominant choice is also a dominant choice. For convenience, a “dominant choice” henceforth refers to a dominant but not obviously dominant choice. We regard a decision problem without a dominant choice to be more complex than one with a dominant choice, and a decision problem with a dominant choice is regarded as more complex than one with an obviously dominant choice.

In the baseline setting, if $v_B-\theta_B \leq 0 \leq v_A -\theta_B$, then θ_A is a dominant choice, as in Panel (b) of Figure 1. If $v_B-\theta_B \leq 0$, then θ_A is an obviously dominant choice (Panel (c) of Figure 1). When a decision problem does not have a dominant choice, the success in contingent reasoning is determined by the DM’s risk preference. For example, as in Panel (a) of Figure 1, $w=\theta_A$ has a higher mean but also higher variance compared to $w=\theta_B$. In this case, the optimal choice depends on the DM’s risk preference. How we elicit a risk preference in the experiment is explained in Subsection 2.2.

Fig. 1

Dominance property of choices and the payoffs given in each state when (a) neither choice is dominant, (b) $w=\theta_A$ dominates $w=\theta_B$, and (c) $w=\theta_A$ obviously dominates $w=\theta_B$.

2.1.3. Reducible problem

Oprea Reference Oprea(2020) proposes several notions of complexity in a decision problem based on the automata literature. “Reduciblity” is one such notion. If a decision problem can be reduced to an equivalent problem with fewer states, then the non-reduced (we call it reducible) problem is more complex than the reduced problem. In line with this measure of complexity, we construct a decision problem with a reducible state that can be reduced to the one job problem.Footnote ⁹

Consider the following decision that corresponds to this complexity. Similar to the one job problem, suppose that there is only one available job position. In the one job problem, there is always only one applicant, either Ann or Bob. In our reducible problem, there are now three states: Ann applies (A), Bob applies (B), and both apply (AB). If the true state is that both apply for the job (AB), then a randomization device decides who will get the job. The DM has to decide the wage w before knowing which is the true state.

Let a probability that the randomization device assigns to Ann in state AB be r and let the probability of a state ω being realized be p_ω. If we assume that the DM satisfies the reduction of compound lotteries axiom then this decision problem is equivalent to a decision problem with $\Omega=\{\mathbf{A}, \mathbf{B}\}$ with probabilities $(p_{\mathbf{A}}+p_\mathbf{AB}r,p_{\mathbf{B}}+p_\mathbf{AB}(1-r))$. If $p_{\mathbf{A}}+p_\mathbf{AB}r=p_{\mathbf{B}}+p_\mathbf{AB}(1-r)=\frac{1}{2}$, then the problem can be reduced to the one job problem. We regard this reducible problem as more complex than the corresponding (reduced) one job problem.

Within the reducible problems, we can also create additional complexity due to the dominance property of choices (without dominant choices, with dominant choices, and with obviously dominant choices) by using the same parametrizations as the one job problems.

2.1.4. Two jobs problem: Increasing the number of contingencies

Suppose now that there are two jobs, job x and job y. There is only one available position for each job. There are two workers, Ann and Bob, who differ in their willingness to accept for each job (θ_ij, i is Ann or Bob, and j is job x or job y), and the revenue they bring for each job is $(v_{ij})$. We impose the parameter conditions that Bob’s willingness to accept is higher than Ann’s for both jobs ( $\theta_{Bj} \gt \theta_{Aj},\;j=x,y$), Ann brings more revenue than Bob for both jobs ( $v_{Aj} \gt v_{Bj},\;j=x,y$), both workers have a higher willingness to accept for job x ( $\theta_{ix} \gt \theta_{iy}$, $i=$Ann, Bob), and Ann brings more revenue than her willingness to accept for both jobs ( $v_{Aj} \gt \theta_{Aj},\;j=x,y$).

With equal chances, (1) Ann applies for job x and Bob applies for job y, or (2) Ann applies for job y and Bob applies for job x. That means the states of the world are in $\Omega=\{(\mathbf{AB}), (\mathbf{BA})\}$ where first letter indicates which worker applies to job x. Under this uncertainty, the DM must decide one wage for each job, namely, w_x and w_y. Like in the one job case, a worker will accept a wage offer as long as it is greater than or equal to their willingness to accept.Footnote ¹⁰ The payoff of the DM is the sum of their earnings from the two jobs.

Even if the number of states is the same as the one job case, the number of contingencies is different. After ruling out non-feasible contingencies, there are nine contingencies: $\{$Both accept job x, Only Ann accepts job x, Both rejects job x $\} \times \{$Both accept job y, Only Ann accepts job y, Both rejects job y $\}$.Footnote ¹¹ Assuming an expected utility representation, we have the following form:

\begin{align*} &U(w)=\\ &\quad \begin{cases} \frac{1}{2}u(v_{Ax}-w_x+v_{By}-w_y)+\frac{1}{2}u(v_{Bx}-w_x+v_{Ay}-w_y) \;\;\;\; & \mbox{if}\ w_x \geq \theta_{Bx}, w_y \geq \theta_{By}\\ \frac{1}{2}u(v_{Ax}-w_x)+\frac{1}{2}u(v_{Bx}-w_x+v_{Ay}-w_y) \;\;\;\; & \mbox{if}\ w_x \geq \theta_{Bx}, \theta_{Ay} \leq w_y \lt \theta_{By}\\ \frac{1}{2}u(v_{Ax}-w_x)+\frac{1}{2}u(v_{Bx}-w_x) \;\;\;\; & \mbox{if}\ w_x \geq \theta_{Bx}, w_y \lt \theta_{Ay}\\ \frac{1}{2}u(v_{Ax}-w_x+v_{By}-w_y)+\frac{1}{2}u(v_{Ay}-w_y) \;\;\;\; & \mbox{if}\ \theta_{Ax} \leq w_x \lt \theta_{Bx}, w_y \geq \theta_{By}\\ \frac{1}{2}u(v_{Ax}-w_x)+\frac{1}{2}u(v_{Ay}-w_y) \;\;\;\; & \mbox{if}\ \theta_{Ax} \leq w_x \lt \theta_{Bx}, \theta_{Ay} \leq w_y \lt \theta_{By}\\ \frac{1}{2}u(v_{Ax}-w_x) \;\;\;\; & \mbox{if}\ \theta_{Ax} \leq w_x \lt \theta_{Bx}, w_y \lt \theta_{Ay}\\ \frac{1}{2}u(v_{By}-w_y)+\frac{1}{2}u(v_{Ay}-w_y) \;\;\;\; & \mbox{if}\ w_x \lt \theta_{Ax}, w_y \geq \theta_{By}\\ \frac{1}{2}u(v_{Ay}-w_y) \;\;\;\; & \mbox{if}\ w_x \lt \theta_{Ax}, \theta_{Ay} \leq w_y \lt \theta_{By}\\ 0 \;\;\;\; & \mbox{if}\ w_x \lt \theta_{Ax}, w_y \lt \theta_{Ay}\\ \end{cases} \end{align*}

By a similar logic as the one job case, we can rule out wages $w_j \lt \theta_{Aj}$ for any job j as they are dominated by $w=\theta_{Aj}$. Then the only possible optimal wages are either θ_Aj or θ_Bj, for $j \in\{x, y\}$. Thus, the contingency-optimal wage profiles $(w_x, w_y)$ are given by $\mathrm{W}^*=\{(\theta_{Ax},\theta_{Ay})$, $(\theta_{Ax},\theta_{By})$, $ (\theta_{Bx},\theta_{Ay})$, $(\theta_{Bx},\theta_{By})\}$. In other words, after eliminating trivially dominated cases, the number of relevant contingencies is four.

Like in the one job problem, the range of parameters can be divided into three cases: (1) $0 \leq v_{Bj}-\theta_{Bj} \leq v_{Aj}-\theta_{Bj} \leq v_{Aj}-\theta_{Aj}$ for all $j \in \{x,y\}$, (2) $v_{Aj}-\theta_{Aj} \geq v_{Aj}-\theta_{Bj}$ for all $j \in \{x,y\}$, and (3) $\min\{v_{Aj}-\theta_{Aj}, 0\} \geq \max\{v_{Aj}-\theta_{Bj}, v_{Bj}-\theta_{Bj}\}$ for all $j \in \{x,y\}$. The first case does not have a dominant choice, thus an optimal choice depends on the DM’s risk preference. Again, how we elicit a risk preference will be explained in Subsection 2.2. The second case is one with a dominant choice, and the third case is one with an obviously dominant choice. In these two cases, $(\theta_{Ax},\theta_{Ay})$ is the optimal wage profile.

2.1.5. Power of certainty

Martínez-Marquina et al. Reference Martínez-Marquina, Niederle and Vespa(2019) show that if a decision problem is represented in two different ways, one in a probabilistic version and the other one in a deterministic version, then failures in contingent reasoning increase in the probabilistic version. They call the difference in failure rates due to the existence of uncertainty the Power of Certainty. Even though the PoC might not be directly related to complexity, we also test whether the PoC exists in our setting with a simple modification.

The PoC environment offers the deterministic version of the wage decision problem we have discussed thus far. The PoC environment is equivalent to the main model in terms of the contingency and (expected) outcomes but has no randomness in the states of the world. We implement this by having two jobs, one for Ann and one for Bob. A worker will accept a wage as long as it is greater than or equal to their willingness to accept. The DM chooses only one wage that is used for both positions. In other words, the DM does not choose different wages for Ann and Bob. They choose only one wage, and Ann and Bob decide whether to accept the offer independently. Thus, they can hire two workers, one worker, or no worker depending on the chosen wage w. If w is greater than or equal to a worker’s willingness to accept, the worker accepts the offer. The DM is paid the sum of their earnings from both jobs. To have consistency in payoffs with the one job problem, we divide total profits by two. We refer to this as the one job PoC problem since it mimics the original one job problem but features deterministic problems.

Contingent reasoning requires choosing a wage from the set $\{\theta_A,\theta_B\}$. With an expected utility representation, we have

\begin{equation*} U(w)= \begin{cases} u(\frac{1}{2}(v_A-w)+\frac{1}{2}(v_B-w)) \;\;\;\; & \mbox{if}\ w \geq \theta_B\\ u(\frac{1}{2}(v_A-w)) \;\;\;\; & \mbox{if}\ \theta_A \leq w \lt \theta_B \\ 0 \;\;\;\; & \mbox{if}\ w \lt \theta_A \end{cases} \end{equation*}

This is equivalent to the original one job problem not only in a contingency sense but also in a utility sense if the DM is risk-neutral.

The two jobs PoC problem is also modified in a similar way. There are two kinds of jobs, job x and job y, and there are two available positions for each job. Both Ann and Bob apply for both jobs, and they can work for both jobs if they accept both jobs. Each worker’s willingness to accept and the revenue they bring are known to the DM. The DM submits one wage for each job, i.e., one wage for job x, and one wage for job y. Depending on the wage chosen, the DM can hire two workers, one worker, or no workers for each job. For consistency, we again divide total profits by two. The contingency-optimal wages are $W^*=\{(\theta_{Ax},\theta_{Ay}),(\theta_{Ax},\theta_{By}), (\theta_{Bx},\theta_{Ay}),(\theta_{Bx},\theta_{By})\}$. Thus, the relevant contingencies and preferences are the same as the original problem if the DM is risk-neutral. We can derive the expected utility from similar arguments. Note that since there is no uncertainty, risk preferences do not affect optimal choices in either PoC treatment.

The PoC treatment cannot include reducible problems because certainty is a key feature of that design. Thus, we exclude reducible problems.

2.2.1. Main treatment

Table 1 summarizes our design and hypotheses. The main treatment of the experiment employs a within-subjects design consisting of two parts: (1) wage choice problems and (2) lottery choice problems. Each subject experiences both. The first part of the experiment contains the labor market setting where a subject chooses a wage(s) for a job(s) as complexity in decision problems varies. In total, there are nine types of questions: {one job, two jobs, reducible} × {no dominant choice, dominant choice, obviously dominant choice}. Each question type has three questions with different parameters; thus, there is a total of 27 questions. That means each cell of Table 1 has three questions. After the 27 wage decision problems, subjects have 27 lottery choice problems that are equivalent to the wage decision problems. The purpose of the lottery choice problems is to measure subjects’ risk preferences. By doing so, we can identify subjects’ optimal wage choices for the problems where there is no dominant choice in the equivalent wage choice problem.

Table 1.

Complexity and hypotheses

From this structure, we say that a subject fails in contingent reasoning if they choose (1) any wage other than workers’ minimum willingness to accept (for example, in the one job case, any wage other than θ_A and θ_B) or (2) a wage in the original problem that does not correspond to their chosen lottery in the lottery choice problem. With this concept of failures in contingent reasoning, we have the following three hypotheses.

The first hypothesis can be tested by comparing the first (one job problems) and second row (two jobs problems) results in Table 1. That means if a subject is more successful in the nine questions from the first row compared to the nine questions from the second row, we can argue that failures in contingent reasoning increase in the number of contingencies.

This hypothesis is also related to the reducibility argument in Oprea Reference Oprea(2020). The questions in the third row (reducible problems) in Table 1 can be further reduced to the first row (one job (reduced) problems). Thus, if subjects perform better in questions corresponding to the first row compared to the third row, then this hypothesis is supported.

This hypothesis follows from Li Reference Li(2017) and can be tested by comparing frequencies of choosing optimal choices in the first (obviously dominant choice), second (dominant choice), and third (no dominant choice) columns of Table 1.

The second part of the experiment, lottery choices, is devised to identify optimal choices in problems without a dominant choice. As stated in the previous section, when there is no dominant choice, there is no objectively optimal wage from the set of contingency-optimal choices. Instead, the utility-maximizing choice depends on the DM’s risk preferences.

Rather than assuming subjects are risk averse, we introduce the lottery choice problems that are induced by the contingency-optimal wages from part 1 and check whether choices are consistent between the two. That means we create lotteries using only $\{\theta_A,\theta_B\}$ or ${\{(\theta_{Ax},\theta_{Ay}),(\theta_{Ax},\theta_{By}),(\theta_{Bx},\theta_{Ay}),(\theta_{Bx},\theta_{By}) \}}$. Let $\mathscr{L}(p_1,z_1;\dots;p_k,z_k)$ be a simple lottery that gives each z_j with probability p_j. Similarly, let $\bar{\mathscr{L}}(p_1,\mathscr{L}_1; \dots;p_k,\mathscr{L}_k)$ be a compound lottery that gives each simple lottery $\mathscr{L}_j$ with probability p_j. Then, each lottery choice problem has one of the following three types:

1. (1) A choice between $\mathscr{L}(\frac{1}{2},v_A-\theta_A ;\frac{1}{2},0)$ and $\mathscr{L}(\frac{1}{2},v_A-\theta_B ;\frac{1}{2},v_B-\theta_B)$, for the one job problems, which corresponds to $w=\theta_A$ and $w=\theta_B$ respectively,

2. (2) A choice between

   $\bar{\mathscr{L}}( \frac{1}{3},\mathscr{L}_1(\frac{1}{2},v_A-\theta_A; \frac{1}{2},0); \frac{1}{3},\mathscr{L}_2(1,v_A-\theta_A; 0,0); \frac{1}{3},\mathscr{L}_3(0,v_A-\theta_A;1,0)$ and

   $\bar{\mathscr{L}}( \frac{1}{3},\mathscr{L}_1(\frac{1}{2},v_A-\theta_B; \frac{1}{2},v_B-\theta_B); \frac{1}{3},\mathscr{L}_2(1,v_A-\theta_B; 0,v_B-\theta_B); \frac{1}{3},\mathscr{L}_3(0,v_A-\theta_B;1,v_B-\theta_B)$ for the reducible problems, corresponding to $w=\theta_A$ and $w=\theta_B$ respectively,

3. (3) A choice among

   $\mathscr{L}(\frac{1}{2},v_{Ax}-\theta_{Ax} ;\frac{1}{2},v_{Ay}-\theta_{Ay})$, $\mathscr{L}(\frac{1}{2},v_{Ax}-\theta_{Ax}+v_{By}-\theta_{By} ;\frac{1}{2},v_{Ay}-\theta_{By})$,

   $\mathscr{L}(\frac{1}{2},v_{Ax}-\theta_{Ay};\frac{1}{2},v_{Ay}-\theta_{Ay}+v_{Bx}-\theta_{Bx})$, and $\mathscr{L}(\frac{1}{2},v_{Ax}-\theta_{Bx}+v_{By}-\theta_{By};\frac{1}{2},v_{Ay}-\theta_{By}+v_{Bx}-\theta_{Bx})$ for the two jobs problems, corresponding to $(w_x,w_y)=(\theta_{Ax},\theta_{Ay})$, $(w_x,w_y)=(\theta_{Ax},\theta_{By})$, $(w_x,w_y)=(\theta_{Bx},\theta_{Ay})$, and $(w_x,w_y)=(\theta_{Bx},\theta_{By})$ respectively.

In problems without a dominant choice, the success of the contingent reasoning behavior is determined by the consistency between the wage decision problem and the lottery choice problem. If a subject chooses a wage other than one of the contingency-optimal wages, then regardless of their lottery choice, the subject is said to fail in contingent reasoning. If the subject chooses one of the contingent-optimal wages but chooses a lottery that is induced by a different wage in that set, then it is still a failure in contingent reasoning since the subject does not have a consistent preference. The subject succeeds in contingent reasoning only if they choose one of the contingency-optimal wages in the wage choice problem to accept and the equivalent lottery in the correspondent lottery choice problem.

Even though problems with a dominant choice and an obviously dominant choice have optimal solutions regardless of the DM’s risk preference, we still ask subjects to choose between two/four lotteries to see whether they behave consistently.

2.2.2. Power of certainty treatment

For the Power of Certainty treatment, each subject has 18 questions presented from the first two rows (one job and two jobs problems) of Table 1 with a deterministic representation (one job PoC and two jobs PoC problems). For consistency, we present payoffs as $\frac{1}{2}(v_i-w)$ if a worker i accepts wage offer w. Since optimal wages are determined regardless of the DM’s risk preference, subjects in this treatment do not have lottery choice problems. Following Martínez-Marquina et al. Reference Martínez-Marquina, Niederle and Vespa(2019), we hypothesize that the PoC treatment decreases failures in contingent reasoning.

2.2.3. Procedure

At the start of the experiment, subjects were given instructions and comprehension check problems. Next, they faced 27 wage decision problems in random order and then 27 lottery choice problems in random order. In the PoC treatment, subjects instead answered 18 wage decision problems in random order, but were not given lottery choice problems.

In total, 162 Ohio State undergraduate students participated, 113 subjects for the main treatment and 49 subjects for the Power of Certainty treatment.Footnote ¹² Subjects were recruited using ORSEE (Greiner, Reference Greiner2015). The participation fee was $6 for the main treatment and $3 for the PoC treatment. Subjects also received an additional payment from one randomly selected question. Each point in the selected question is converted to $0.30. The experiment was programmed by oTree (Chen et al., Reference Chen, Schonger and Wickens2016).