2.1 Public goods experiments

In a first phase, we collected studies searched for on Scopus in November 2012 using different combinations of keywords [experiment(s), voluntary contribution(s), public goods, linear, laboratory, subjects, participants], and the studies listed in the meta-study of Zelmer (Reference Zelmer2003). We restricted the search to published studies in English language, that could be accessed online via the library of Tilburg University. We also sent around private e-mails to authors of collected studies asking for information on certain variables, for cases where such information was difficult to extract from the study.

In a second phase, in February 2015, we sent around the references of the collected studies to the ESA Google Groups mailing list for experimental methods discussion. In the accompanying message, we asked to send us references of studies reporting on treatments fulfilling the above conditions and not included in the list.

In a third phase, we included three more studies that were missing, of which two were listed in Ledyard (Reference Ledyard1995). We ended up with 71 studies covering 122 treatments. Section 1 of the electronic supplementary material gives an overview of the collected studies, including an overview of the available data on the nature of feedback.

We recorded the initial endowment and the average contributions across all rounds and contributions in the first round as a percentage of the initial endowment, whenever possible, as well as the following range of binary variables that refer to the feedback participants get after each period of play: aggregate feedback about choices in one’s group (F.aggchoice), feedback about each group member’s choice (F.indichoice), feedback about each group member’s payoff (F.indipayoffs), and feedback about one’s own payoff (F.ownpayoff).Footnote ⁷ We also recorded a range of control variables, including group size and the marginal per capita return of the public good (MPCR). MPCR and group size have been shown to increase contributions (see Isaac et al. Reference Isaac, Walker and Williams1994). Our dependent variable in the analysis based on the public goods data is the average contribution as a percentage of the initial endowment.

2.2 Oligopoly experiments

As with the public goods experiments, we only used data from published studies in English language that could be accessed online via the library of Tilburg University. In a first phase, we collected the studies listed in the meta-study of Engel (Reference Engel2007), and studies covering oligopoly experiments that fulfilled the above-mentioned conditions listed in the references of Potters and Suetens (Reference Potters and Suetens2013). We also sent around private e-mails to authors of collected studies asking for information on certain variables, for cases where such information was difficult to extract from the study.

In a second phase we used the ESA Google Groups mailing list for experimental methods discussion to ask for references reporting on treatments that fulfill a range of conditions, but that were not included in our list collected in the first phase.

In a third phase, we included three more studies that were listed in Engel (Reference Engel2015). We ended up with 18 studies covering 50 treatments. Section 2 of the electronic supplementary material gives an overview of the collected studies, including an overview of the available data on the nature of feedback.

We recorded the average choices across all rounds and the choice in the first round, as well as the same range of feedback dummy variables as in the public goods experiments. In addition, we recorded a range of theoretical benchmarks, including the SPNE choice, the (symmetric) joint-payoff-maximizing choice, and the ‘deviation’ payoff (payoff one receives if one best-responds to the rest of the group fully cooperating).Footnote ⁸ If these theoretical benchmarks were not given, we calculated them ourselves using the parameters given. Out of the theoretical benchmarks, we composed the following index that serves as a measure for the scope for tacit collusion in repeated games along the lines of Friedman (Reference Friedman1971): $\frac{\text{joint-payoff-maximizing payoff}-\text{SPNE payoff}}{\text{deviation payoff}-\text{joint-payoff-maximizing payoff}}$ . We refer to this index as the ‘Friedman’ index, and expected it would have a positive effect on collusion.Footnote ⁹ We also recorded group size and the nature of the strategic environment (strategic complements versus substitutes). We expected more competition (higher output and lower prices) in large groups (see Huck et al. Reference Huck, Normann and Oechssler2004) and more collusion in games with strategic complements than in games with strategic substitutes (see Embrey et al. Reference Embrey, Mengel and Peeters2015; Engel Reference Engel2007; Potters and Suetens Reference Potters and Suetens2009; Suetens and Potters Reference Suetens and Potters2007).

Our dependent variable in the analysis based on the oligopoly data is the degree of collusion, defined as the average choice minus the SPNE choice divided by the joint-payoff-maximizing choice minus the SPNE choice.Footnote ¹⁰ The degree of collusion measures the extent to which participants deviate from the SPNE in the direction of joint-payoff maximization. To illustrate, if under Bertrand price competition the average price is higher (lower) than the SPNE price, then the degree of collusion is larger (smaller) than zero. In contrast, under Cournot quantity competition, an average quantity higher (lower) than the SPNE quantity, implies a degree of collusion below (above) zero. Our independent variables in both analyses are the feedback variables and the above-listed control variables.

2.3 Descriptives

Table 1 provides summary statistics of the dependent and independent variables, and of average numbers of independent observations (groups or markets) and participants by data point. To illustrate, public goods experiments have on average about 10 independent observations in their treatments, and oligopoly experiments have on average 7.7. The average number of participants by treatment is 46 in public goods experiments and 27 in oligopoly experiments. The table further shows that both samples have variation in all feedback variables. We should further point out that the negative mean degree of collusion reported in the table may be attributed to our selection criterion that excludes two-player groups. In oligopolies with more than two players, behavior is typically quite rivalistic, especially under Cournot competition (e.g., Huck et al. Reference Huck, Normann and Oechssler2004).

Table 1

Summary statistics

Variable	Public goods					Oligopoly
	N	Mean	SD	Min.	Max.	N	Mean	SD	Min.	Max.
Share contributed	116	0.40	0.14	0.11	0.74
Degree of collusion						50	−0.15	0.50	−2.40	0.90
F.indichoice	119	0.45	0.50	0	1	50	0.44	0.50	0	1
F.aggchoice	118	0.78	0.42	0	1	50	0.64	0.48	0	1
F.indipayoffs	119	0.11	0.31	0	1	50	0.28	0.45	0	1
F.ownpayoff	117	0.87	0.34	0	1	50	0.98	0.14	0	1
Group size	122	5.61	5.25	3	40	50	3.84	0.93	3	8
MPCR	122	0.46	0.14	0.03	0.75
Strategic complements						50	0.26	0.44	0	1
‘Friedman’ index						46	0.83	0.55	0.13	3
# Indep. obs.	121	9.96	7.08	1	34	50	7.74	3.63	1	17
# Participants	121	45.99	29.32	4	136	50	26.78	9.88	5	48
# Periods	122	12.29	6.32	4	50	50	42.94	28.12	12	120

The table reports summary statistics. Variables that have a minimum of 0 and a maximum of 1 are binary variables. F.aggchoice refers to aggregate feedback about choices in one’s group, F.indichoice to feedback about each group member’s choice, F.indipayoffs to feedback about each group member’s payoff, and F.ownpayoff to feedback about one’s own payoff

Table 2

Cooperation depending on feedback

	F.indichoice		F.aggchoice		F.indipayoffs		F.ownpayoff
	Yes	No	Yes	No	Yes	No	Yes	No
Average share contributed (all)	0.41	0.39	0.40	0.41	0.35	0.41	0.40	0.41
Average share contributed (MPCR < 0.7)	0.40	0.38	0.39	0.39	0.28	0.40	0.38	0.41
Average degree of collusion	−0.19	−0.12	−0.09	−0.26	−0.53	−0.00	−0.15	−0.06

The table reports the average share of endowment contributed (for public goods experiments) and the average degree of collusion (for oligopoly experiment) depending on the particular feedback present. F.aggchoice refers to aggregate feedback about choices in one’s group, F.indichoice to feedback about each group member’s choice, F.indipayoffs to feedback about each group member’s payoff, and F.ownpayoff to feedback about one’s own payoff

Table 3

Correlations between feedback variables

	F.indichoice	F.aggchoice	F.indipayoffs
Public goods experiments
F.aggchoice	−0.465
	(0.000)
F.indipayoffs	0.330	−0.401
	(0.000)	(0.000)
F.ownpayoff	0.092	0.110	0.136
	(0.323)	(0.238)	(0.145)
Oligopoly experiments
F.aggchoice	−0.678
	(0.000)
F.indipayoffs	0.704	−0.368
	(0.000)	(0.009)
F.ownpayoff	0.127	−0.107	0.089
	(0.381)	(0.459)	(0.538)

The table reports the correlation coefficients for each pair of feedback variables based on the data used in Tables 4 and 6. p values are in parentheses. F.aggchoice refers to aggregate feedback about choices in one’s group, F.indichoice to feedback about each group member’s choice, F.indipayoffs to feedback about each group member’s payoff, and F.ownpayoff to feedback about one’s own payoff

Table 2 gives a first impression of the effect of feedback on cooperation. It provides summary statistics on cooperation—the average share of the endowment contributed for public goods experiments and the average degree of collusion for oligopoly experiments—depending on the nature of feedback. For the public goods data, we also show averages across all experiments where the MPCR is lower than 0.7. The reason why we report the latter results is that few experiments implement an MPCR larger than 0.7 (see Sect. 3.1 for further motivation). The table shows that feedback about individual payoffs (F.indipayoffs) has the largest effect on cooperation in both sets of experiments, and the effect is negative. In the public goods data, the effect is particularly large for the games with an MPCR lower than 0.7. To illustrate, the share of the endowment individuals contribute when they have feedback about the payoffs is overall 6 percentage points lower than when they do not have such feedback. In the oligopoly games the difference in degree of collusion equals 0.53 points (notice that these points do not correspond to percentage points). The effects of the other feedback variables is less visible in the table, except that feedback about the aggregate choice seems to substantially increase the degree of collusion in the oligopoly experiments, whereas feedback about individual choices rather decreases it. We return to this finding later in the paper.

Finally, Table 3 reports pairwise correlations between the different feedback variables. As can be seen, some of the feedback variables are highly correlated. For example, F.indichoice and F.indipayoffs are positively and significantly correlated. It is important to be aware of these correlations for the interpretations of some of the regression results reported in Sect. 3.

3.1 Main results

Tables 4 and 5 report results from regressions based on the public goods data. Table 4 is based on all data and Table 5 on data from experiments where the MPCR is smaller than 0.7. As can be seen in the table, only 12 data points drop out, implying that in 12 of the data points the MPCR is above or equal to 0.7. All reported regression results are based on regressions that use numbers of independent observations by treatment as weights.Footnote ¹¹ Both tables have results from regressions that include the feedback variables one by one [specifications (1) to (4)], from a regression that includes all feedback variables at once without controlling for other effects [specification (5)], and from regressions that include all feedback variables controlling for other variables [specifications (6) to (8)].Footnote ¹²

Table 4

Regression results public goods

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Dep. var.: Share of endowment contributed
F.indichoice	0.030				0.042	0.056**	0.046	0.061**
	(0.257)				(0.149)	(0.044)	(0.123)	(0.032)
F.aggchoice		−0.007			−0.014	−0.015	−0.015	−0.016
		(0.835)			(0.723)	(0.697)	(0.707)	(0.676)
F.indipayoffs			−0.048		−0.073	−0.120**	−0.075	−0.123**
			(0.267)		(0.143)	(0.013)	(0.136)	(0.012)
F.ownpayoff				−0.002	0.004	0.032	0.010	0.039
				(0.974)	(0.927)	(0.482)	(0.846)	(0.404)
MPCR						0.446***		0.450***
						(0.000)		(0.000)
Group size							0.004	0.004
							(0.478)	(0.353)
Constant	0.404***	0.426***	0.425***	0.419***	0.411***	0.174**	0.389***	0.145*
	(0.000)	(0.000)	(0.000)	(0.000)	(0.000)	(0.025)	(0.000)	(0.082)
$R^2$	0.011	0.000	0.011	0.000	0.035	0.165	0.040	0.172
Adj. $R^2$	0.003	−0.009	0.002	−0.009	−0.001	0.126	−0.005	0.125
N	116	115	116	114	113	113	113	113

The table reports results from linear regressions based on data from public goods experiments. The numbers of independent observations by unit of observation (treatment) are used as weights (p values in parentheses). Stars ***, ** or * indicate that the effect of the variable is statistically significant at the 1, 5 or 10% level, respectively. F.aggchoice refers to aggregate feedback about choices in one’s group, F.indichoice to feedback about each group member’s choice, F.indipayoffs to feedback about each group member’s payoff, and F.ownpayoff to feedback about one’s own payoff

Table 5

Regression results public goods with MPCR < 0.7

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Dep. var.: Share of endowment contributed
F.indichoice	0.024				0.051*	0.053*	0.054*	0.057*
	(0.397)				(0.083)	(0.070)	(0.072)	(0.054)
F.aggchoice		0.028			0.019	0.006	0.018	0.004
		(0.457)			(0.638)	(0.892)	(0.649)	(0.922)
F.indipayoffs			−0.173***		−0.181***	−0.180***	−0.182***	−0.181***
			(0.001)		(0.001)	(0.001)	(0.001)	(0.001)
F.ownpayoff				−0.015	−0.009	0.012	−0.004	0.020
				(0.764)	(0.803)	(0.808)	(0.927)	(0.691)
MPCR						0.300*		0.317*
						(0.070)		(0.058)
Group size							0.003	0.004
							(0.561)	(0.412)
Constant	0.397***	0.387***	0.421***	0.421***	0.386***	0.244**	0.367***	0.210**
	(0.000)	(0.000)	(0.000)	(0.000)	(0.000)	(0.011)	(0.000)	(0.045)
$R^2$	0.007	0.006	0.098	0.001	0.125	0.155	0.128	0.161
Adj. $R^2$	−0.003	−0.004	0.089	−0.009	0.088	0.110	0.082	0.107
N	104	103	104	102	101	101	101	101

The table reports results from linear regressions based on data from public goods experiments. The numbers of independent observations by unit of observation (treatment) are used as weights (p values in parentheses). Stars ***, ** or * indicate that the effect of the variable is statistically significant at the 1, 5 or 10 level, respectively. F.aggchoice refers to aggregate feedback about choices in one’s group, F.indichoice to feedback about each group member’s choice, F.indipayoffs to feedback about each group member’s payoff, and F.ownpayoff to feedback about one’s own payoff

First, we consider Table 4. As shown in specifications (1) to (5), none of the feedback variables have a statistically significant effect on the share of endowment contributed to the public good if no other variables are controlled for. In terms of magnitude, the effects of F.aggchoice and of F.ownpayoff are negligible. The effect of F.indichoice, and, in particular, that of F.indipayoffs, is larger in magnitude. If controlling for group size, as is done in specification (7), results are very similar because group size has no significant effect. If controlling for the MPCR, however, as is done in specifications (6) and (8), the positive effect of F.indichoice and the negative effect of F.indipayoffs have a larger magnitude and turn significant at the 5% level. The share contributed tends to be about 6 percentage points higher than in cases where players receive feedback about the distribution of choices in their group. Provision of feedback about individual payoffs tends to reduce the share contributed to the public good by about 12 percentage points. Also, in line with the expectations, the MPCR has a large, positive effect on contributions.

Next, we turn to Table 5, which reports results from regressions based on public goods games with an MPCR lower than 0.7.Footnote ¹³ We include these results because there are few data points on public goods games with an MPCR higher than 0.7 (only 12). It happens to be the case that for these data points F.indichoice and F.indipayoffs are highly positively correlated.Footnote ¹⁴ Consequently, including these data points may lead to an underestimation of the ‘strength’ of the negative effect of F.indipayoffs in ‘typical’ public goods games. Results related to feedback variables F.indichoice, F.aggchoice and F.ownpayoff are qualitatively very similar to those based on all data; F.aggchoice and F.ownpayoff have basically no effect and F.indichoice increases the percentage contribution by about 5–6 percentage points if controlling for other variables (significant at the 10% level). With respect to F.indipayoffs we observe that its effect becomes larger in magnitude and highly statistically significant even without controlling for MPCR. The marginal effect is now 17–18 percentage points. In addition, as shown in specifications (6) and (8), the effect of MPCR decreases somewhat in magnitude.

Next, we consider the results for the oligopoly games, which are presented in Table 6. The table has eight specifications depending on whether the feedback variables are included one by one, and depending on whether controls are included. There are three things that come to mind when looking at the table. The first is that F.indipayoffs has a negative and significant effect across all specifications, thereby leading to a drop in degree of collusion of 0.48 to 0.78 points, depending on the specification. Just like in the public goods experiments, transparency about individual payoffs of group members thus decreases cooperation. Second, F.indichoice has a negative (but not significant) effect on the degree of collusion when included on its own and has a positive and (mostly) highly significant effect when included together with F.indipayoffs. The reason behind this shift in estimate is that F.indichoice and F.indipayoffs are highly positively correlated (see Table 3); in many oligopoly experiments both pieces of feedback are provided. Therefore, if included on its own, F.indichoice picks up the negative effect of F.indipayoffs thereby producing a biased estimate. For the public goods data, a similar tendency is observed although that the effect of F.indichoice does not become negative when included on its own [compare specifications (5) to (8) to specification (1) in Tables 4 and 5]. A third result is that F.aggchoice has a significantly positive and substantial effect on the degree of collusion across all specifications. When players receive aggregate information about the past choices of group members, it thus seems that competition, which can be quite fierce in oligopoly settings in groups larger than two, seems to be substantially reduced. The marginal effect depends somewhat on the specification, but, overall, having aggregate information about choices of others increases the degree of collusion by about 0.30 to 0.40 points.

Table 6

Regression results oligopoly

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Dep. var.: Degree of collusion
F.indichoice	−0.188				0.556***	0.499***	0.156	0.491***
	(0.131)				(0.003)	(0.003)	(0.428)	(0.004)
F.aggchoice		0.306**			0.335**	0.404***	0.358***	0.414***
		(0.015)			(0.016)	(0.002)	(0.005)	(0.002)
F.indipayoffs			−0.481***		−0.782***	−0.643***	−0.398**	−0.639***
			(0.000)		(0.000)	(0.000)	(0.027)	(0.000)
F.ownpayoff				−0.132	−0.026	−0.101	0.594	−0.063
				(0.790)	(0.947)	(0.780)	(0.192)	(0.862)
Complements						0.374***	0.459***	0.374***
						(0.002)	(0.000)	(0.002)
‘Friedman’ index							0.307**
							(0.021)
Group size								−0.053
								(0.358)
Constant	−0.117	−0.395***	−0.061	−0.060	−0.395	−0.464	−1.339**	−0.313
	(0.131)	(0.000)	(0.327)	(0.903)	(0.344)	(0.221)	(0.012)	(0.447)
$R^2$	0.047	0.116	0.255	0.001	0.403	0.522	0.567	0.532
Adj. $R^2$	0.027	0.098	0.239	−0.019	0.350	0.468	0.500	0.467
N	50	50	50	50	50	50	46	50

The table reports results from linear regressions based on data from oligopoly experiments. The numbers of independent observations by unit of observation (treatment) are used as weights (p values in parentheses). Stars ***, ** or * indicate that the effect of the variable is statistically significant at the 1, 5 or 10% level, respectively. F.aggchoice refers to aggregate feedback about choices in one’s group, F.indichoice to feedback about each group member’s choice, F.indipayoffs to feedback about each group member’s payoff, and F.ownpayoff to feedback about one’s own payoff

Regarding the control variables, in line with our expectations, the degree of collusion is higher in the case of strategic complementarity, and is positively related to the ‘Friedman’ index (both significant). Against the expectation, group size does not have a significant effect.

In summary, transparency about individual payoffs of group members leads to a substantial and significant decrease in cooperation in both dilemma settings. Transparency about individual choices tends to increase cooperation, but the effect is smaller in magnitude. Transparency about aggregate choices increases cooperation in oligopoly games, and not so in public goods games.

At first sight, it may seem remarkable that feedback about choices of group members in an aggregate format dramatically increases cooperation in oligopoly settings but not so in public good settings. We speculate that the differential effect may be due to the fact that, unlike in linear public goods games, in oligopoly settings there is scope for behaving more competitively than in the SPNE. Since ‘baseline’ behavior is typically more competitive than in the SPNE, particularly in settings with strategic substitutes (Engel Reference Engel2007; Suetens and Potters Reference Suetens and Potters2007), there is more potential for transparency about choices to increase cooperation. Regressions based exclusively on games with strategic substitutes suggest that the effect of transparency about aggregate choices is indeed particularly large in these environments (based on 37 data points, not reported). In games with strategic complements, no such effect is observed.Footnote ¹⁵

3.2 Publication bias

The regression results reported so far are based on data stemming from, on the one hand, experiments that either implement one particular feedback setting, and, on the other hand, from experiments that explicitly test for the effect of feedback on behavior. The data set thus consists of data from (1) experiments with treatments that all have the same feedback conditions, and (2) experiments where feedback conditions vary between treatments within the experiment. In the current section, we investigate whether the earlier reported results are robust to removing data from treatments of studies that were explicitly designed to study the effect of feedback on behavior. It may be that due to a publication bias studies that generate significant effects (for example, a significant effect of transparency about earnings on collusion) are more likely to be published. If the above-reported results no longer hold after removal of data from experiments with variation in feedback, this would be an indication of publication bias.

Table 7

Regression results reduced sample

	Pg	Oligopoly
	(1)	(2)	(3)
F.indichoice	0.062**	–	–
	(0.035)
F.aggchoice	−0.006	−0.070	−0.359**
	(0.875)	(0.793)	(0.042)
F.indipayoffs	−0.114**	−0.584**	−0.861***
	(0.042)	(0.041)	(0.000)
F.ownpayoff	0.036	0.227	−0.114
	(0.453)	(0.750)	(0.571)
MPCR	0.450***
	(0.000)
Complements		0.228	0.187
		(0.139)	(0.197)
‘Friedman’ index		0.141
		(0.636)
Group size	0.004		0.013
	(0.421)		(0.725)
Constant	0.142*	−0.413	0.260
	(0.096)	(0.715)	(0.345)
$R^2$	0.163	0.823	0.800
Adj. $R^2$	0.113	0.774	0.755
N	107	24	28

The table reports results from linear regressions based on data from public goods [specification (1)] and oligopoly experiments [specifications (2) and (3)] that have no treatment variation in one of the feedback variables. The dependent variables are share of endowment contributed, and degree of collusion, respectively. The numbers of independent observations by unit of observation (treatment) are used as weights (p values in parentheses). Stars ***, ** or * indicate that the effect of the variable is statistically significant at the 1, 5 or 10% level, respectively. F.aggchoice refers to aggregate feedback about choices in one’s group, F.indichoice to feedback about each group member’s choice, F.indipayoffs to feedback about each group member’s payoff, and F.ownpayoff to feedback about one’s own payoff

Our approach is thus to remove those data from experiments that have within-treatment variation in one of the feedback variables, and to run the earlier-reported regressions on the reduced data set. Table 7 reports the results from regressions based on the public goods and oligopoly data. We report the specifications where all feedback variables and controls are included, and refer to Tables S4 and S5 in the Sect. 3 of the electronic supplementary material for other specifications. Because the public goods experiments typically do not vary feedback conditions between treatments, only few data drop out. This is in contrast to the oligopoly data, where less than half of the data points are left.

As can be seen in Table 7, the results for the public goods experiments are similar to the above-reported results. In the small sample of remaining oligopoly data [specifications (2) and (3)], the above-reported main result related to transparency about individual earnings carries through. This is reassuring because it suggests that this is a robust finding. The result related to feedback about aggregate choices does not carry through, however.