Skip to main content Accessibility help
×
Home

Disaggregating Deliberation’s Effects: An Experiment within a Deliberative Poll

  • Cynthia Farrar, James S. Fishkin, Donald P. Green, Christian List, Robert C. Luskin and Elizabeth Levy Paluck...

Abstract

Using data from a randomized field experiment within a Deliberative Poll, this paper examines deliberation’s effects on both policy attitudes and the extent to which ordinal rankings of policy options approach single-peakedness (a help in avoiding cyclical majorities). The setting was New Haven, Connecticut, and its surrounding towns; the issues were airport expansion and revenue sharing – the former highly salient, the latter not at all. Half the participants deliberated revenue sharing, then the airport; the other half the reverse. This split-half design helps distinguish the effects of the formal on-site deliberations from those of other aspects of the treatment. As expected, the highly salient airport issue saw only a slight effect, while much less salient revenue-sharing issue saw a much larger one.

Copyright

References

Hide All

1 See, e.g., Luskin, Robert C., Fishkin, James S. and Jowell, Roger, ‘Considered Opinions: Deliberative Polling in Britain’, British Journal of Political Science, 32 (2002), 455487; Barabas, Jason, ‘How Deliberation Affects Policy Opinions’, American Political Science Review, 98 (2004), 687701; Druckman, James N., ‘Political Preference Formation: Competition, Deliberation, and the (Ir)relevance of Framing Effects’, American Political Science Review, 98 (2004), 671686.

2 See Fishkin, James S., The Voice of the People: Public Opinion and Democracy (New Haven, Conn.: Yale University Press, expanded paperback edition, 1997); Luskin, Robert C., Fishkin, James S. and Jowell, Roger, ‘Considered Opinions: Deliberative Polling in Britain’, British Journal of Political Science, 32 (2002), 455487; Fishkin, James S. and Luskin, Robert C., ‘Experimenting with a Democratic Ideal: Deliberative Polling and Public Opinion’, Acta Politica, 40 (2005), 284298.

3 Marie Jean Antoine Nicolas Caritat (the Marquis de) Condorcet, Essai sur l’Application de l’Analyse à la Probabilité des Décisions Rendues à la Pluralité des Voix (Paris, 1785); Arrow, Kenneth, Social Choice and Individual Values (New York: Wiley, 1953).

4 See Luskin, Robert C., Fishkin, James S. and Hahn, Kyu S., ‘Deliberation and Net Attitude Change’(paper presented at the biennial General Conference of the European Consortium for Political Research, Pisa, Italy, 2007); List, Christian, Luskin, Robert C., Fishkin, James S. and McLean, Iain, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy: Evidence from Deliberative Polls’ (unpublished, London School of Economics, 2007).

5 See Luskin, Fishkin and Hahn, ‘Deliberation and Net Attitude Change’; List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

6 This is a plausible worry for DPs conducted during and about referendum or election campaigns, as well as in scattered other cases, including DPs about educational policy in Northern Ireland, just as the current accord was being reached, and about foreign policy in the United States on the eve of the Iraq war.

7 Of course, both samples are corroded – unequally – by non-response/non-participation, depriving these comparisons of the full authority of true random assignment. Barabas’s analysis of much the same question involves broadly similar quasi-control groups (created in his case by matching), though with a highly non-random participant sample and hence lesser external validity.

8 See also Fishkin, James S. and Luskin, Robert C., ‘Bringing Deliberation to the Democratic Dialogue: The NIC and Beyond’, in Maxwell McCombs and Amy Reynolds, eds, A Poll with a Human Face: The National Issues Convention Experiment in Political Communication (Mahwah, NJ: Lawrence Erlbaum, 1999); Luskin, Robert C., Fishkin, James S., McAllister, Ian, Higley, John and Ryan, Pamela, ‘Information Effects in Referendum Voting: Evidence from the Australian Deliberative Poll’ (paper presented at the annual meeting of the American Political Science Association, Boston, Mass.; 2002); Barabas, ‘How Deliberation Affects Policy Outcomes’.

9 The changes could conceivably if implausibly stem simply from the stimulation of the initial interview, which even without the invitation to and anticipation of the DP should produce some heightened attention and thought. Luskin, Fishkin, McAllister, Higley, and Ryan, ‘Information Effects in Referendum Voting: Evidence from the Australian Deliberative Poll,’; Luskin, Robert C. and Fishkin, James S., ‘Deliberative Polling, Public Opinion and Democracy: the Case of the National Issues Convention’ (paper presented at the annual meeting of the American Association for Public Opinion Research, 1998); and Luskin, Robert C., Iyengar, Shanto and Fishkin, James S., ‘Considered Opinions on U.S. Foreign Policy: Face-to-Face versus Online Deliberative Polling’ (Center for Deliberative Democracy, Stanford University, 2006) address this concern by comparing to quasi control groups likewise interviewed before as well as after the deliberations.

10 See, e.g., Luskin, Fishkin and Jowell, ‘Considered Opinions’; Luskin, Robert C., Fishkin, James S., Jowell, Roger and Park, Allison, ‘Learning and Voting in Britain: Insights from the Deliberative Poll’ (paper presented at the annual meeting of the American Political Science Association, Atlanta, Ga. 1999); Luskin, Robert C., Fishkin, James S. and Plane, Dennis L., ‘Deliberative Polling and Policy Outcomes: Electric Utility Issues in Texas’ (paper presented at the annual meeting of the Association for Public Policy Analysis, 1999); Luskin, Fishkin, Higley and Ryan, ‘Information Effects in Referendum Voting’.

11 See, e.g., Luskin, , Fishkin, and Jowell, , ‘Considered Opinions’; and Luskin, , Fishkin, , Higley, and Ryan, , ‘Information Effects in Referendum Voting’.

12 See List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

13 Converse, Philip E., ‘The Nature of Belief Systems in Mass Publics’, in David E. Apter, ed., Ideology and Discontent (New York: Free Press, 1964). Cf. Hill, Jennifer and Kriesi, Hanspeter, ‘An Extension and Test of Converse’s “Black-and-White” Model of Response Stability’, American Political Science Review, 95 (2001), 397414.

14 To the extent that the first mechanism predominates, we should expect to see attitudes change so as to increase their predictability from socio-demographic variables, proxying interests; to the extent that the second predominates, they may change so as to decrease it. See Luskin, Fishkin and Jowell, ‘Considered Opinions’; Luskin, Robert C., ‘The Heavenly Public: What Would the Ideal Citizenry Be Like?’, in Michael B. MacKuen and George Rabinowitz, eds, Electoral Democracy (Ann Arbor: University of Michigan Press, 2003).

15 Across fifty policy attitude indices in seven DPs, the mean absolute net change of opinion ranges from 0.068 to 0.117, averaging 0.092, impressively large for net attitude change (for reasons sketched in Luskin 2002). Thirty-four (68 per cent) of the fifty indices show statistically significant net change. See Luskin, Fishkin and Hahn, ‘Deliberation and Net Attitude Change’.

16 For example, consider a population/sample of four individuals, with (declining) preference orderings (x, y, z), (z, y, x), (y, x, z), and (y, z, x). These preference orderings are single-peaked, with structuring dimension [x, y, z]. Each individual has a first preference, a second preference on one side of it, and a third preference no ‘closer’ (in terms of ordinal slots) to it. By contrast, if a fifth individual with the preference ordering (x, z, y) is added to this population/sample, the preferences are no longer single-peaked. They are not single-peaked with respect to the structuring dimension [x, y, z], because the fifth individual prefers x, then z, which is ‘further’ than y from x on this dimension. And there is no other possible structuring dimension (left–right ordering of x, y and z) with respect to which all five preference orderings are single-peaked. For a more precise definition, see List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

17 Black, Duncan, ‘On the Rationale of Group Decision-Making’, Journal of Political Economy, 56 (1948), 2334 and Arrow, Kenneth, Social Choice and Individual Values.

18 See Brams, Steven J., Jones, Michael A. and Kilgour, D. Marc, ‘Single-Peakedness and Disconnected Coalitions’, Journal of Theoretical Politics, 14 (2002), 359383. Cardinal single-peakedness is sufficient but unnecessary for ordinal single-peakedness if the space is one-dimensional, but neither sufficient nor necessary if the space is multi-dimensional.

19 Condorcet, , Essai sur l’Application de l’Analyse à la Probabilité des Décisions Rendues à la Pluralité des Voix.

20 For example, Arrow, Social Choice and Individual Values, and McKelvey, Richard, ‘General Conditions for Global Intransitivities in Formal Voting Models’, Econometrica, 47 (1979), 10851111.

21 Riker, William H., Liberalism against Populism (San Francisco: W. H. Freeman, 1982).

22 Black, , ‘On the Rationale of Group Decision-Making’.

23 List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

24 In the second example of fn. 16, the largest subset of the population/sample of five whose preferences are single-peaked contains four members (the four of the first example, with respect to the structuring dimension [x, y, z]). Thus S = 4/5 = 0.8. For a more precise definition and discussion, see List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’. See also Niemi, Richard G., ‘Majority Decision-Making with Partial Unidimensionality’, American Political Science Review, 63 (1969), 488497.

25 Note that S concerns the aggregate patterning of preferences across individuals, not necessarily the cognitive organization of individuals’ preferences (although it may partly reflect that). See the further discussion below and in List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

26 Niemi, , ‘Majority Decision-Making with Partial Unidimensionality’. This is also supported by computer simulations reported in List, , Luskin, , Fishkin, , and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

27 See Miller, David, ‘Deliberative Democracy and Social Choice’, Political Studies, 40 (1992: special issue):5467; Knight, Jack and Johnson, James, ‘Aggregation and Deliberation: On the Possibility of Democratic Legitimacy’, Political Theory, 22 (1994), 277296; List, Christian, ‘Two Concepts of Agreement’, PEGS: The Good Society, 11 (2003), 7279; List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’. Cf. Van Mill, David, ‘The Possibility of Rational Outcomes from Democratic Discourse and Procedures’, Journal of Politics, 58 (1996), 734752, countered by Dryzek, John and List, Christian, ‘Social Choice Theory and Deliberative Democracy: A Reconciliation’, British Journal of Political Science, 33 (2003), 128.

28 This does not necessarily mean that deliberation will lead people to converge on some particular ranking, what List calls ‘agreement at a substantive level’. In fact, the results in List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’, suggest that it may decrease agreement of this sort. Our only claim here is that deliberation increases proximity to single-peakedness, an instance of what List calls ‘agreement at a meta-level’. See List, ‘Two Concepts of Agreement’.

29 Across thirteen issues, the index of ‘proximity to single-peakedness’, defined below, increases by an average of 0.101, on a scale that runs from a minimum always > 0 to 1.0. Across the ten low to moderate salience issues, the average was 0.134 (computed from Table 2 in List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’).

30 Across seven DPs, the correlation between pre-deliberation knowledge, proxying salience, and the mean absolute net change for all the policy attitude indices in that DP is −.61. See Luskin, Fishkin, and Hahn, ‘Deliberation and Net Attitude Change’. Across thirteen DPs, the correlation between pre-deliberation knowledge and the increase in “proximity to single-peakedness” (defined below) is −.59 (computed from Tables 1 and 3 in List, Luskin, Fishkin, and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’).

31 There was much discussion of residential property taxes and property tax relief for the elderly, disabled and those on a low-income, but regional revenue sharing was only mentioned briefly in one editorial and as a low-profile issue in the platform of one unsuccessful mayoral candidate.

32 We shall thus be referring to two sorts of ‘groups’ – the small groups, within which the issues are discussed, and the treatment groups, each consisting of eight small groups, which tackle the issues in different sequences.

33 The A-first and R-first groups thus had different plenary sessions on Saturday but shared a plenary session on Sunday.

34 One might expect preferences to reflect attitudes – more after deliberation than before and more, at least initially, on the airport than on revenue-sharing – and the results bear this out. Maximum likelihood estimation of binomial logit models expressing preferences for expanding airport service over ending it, for non-mandatory revenue sharing over local control, and for non-mandatory over mandatory revenue sharing as functions of the respective attitude indices produce pre- and post-deliberation (McFadden’s) pseudo-R 2s of 0.479 and 0.680, 0.377 and 0.345, and 0.099 and 0.371, respectively. Preferences are indeed relatively well predicted by attitudes by the end of the process, are indeed initially better predicted on the airport than on revenue sharing, and are indeed better predicted after than before deliberation in two of the three cases. More detailed results are available on request.

35 Niemi, , ‘Majority Decision-Making with Partial Unidimensionality’.

36 A thousand random samples of the relevant subjects were drawn with replacement. The standard deviation of the resampled S provides the estimate of the standard error.

37 The bootstrapped standard errors, moreover, are for S, not the difference between the values of S at different times, which may be somewhat larger or (less likely) smaller, depending on the sign and magnitude of the covariance.

38 List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

39 On the airport, the dimension along which the largest sub-sample is single-peaked remains the same throughout, ordering the alternatives as [A2 A1 A3]. The Condorcet winner, throughout, is A2 (expanding service). On revenue sharing, the dimension along which the largest sub-sample is single-peaked remains the same throughout, ordering the alternatives as [R1 R2 R3 R4], although the Condorcet winner changes from one non-mandatory sharing option to the other – from R3 (state-encouraged sharing) at T1 to R2 (voluntary sharing) at T3.

40 In each treatment group, the Condorcet winner changes from R3 to R2 over the interval during which the group deliberates the issue – from T1 to T2 in the R-first group and from T2 to T3 in the A-first group.

41 In words, S′ is the extent to which S exceeds its minimum, expressed as a fraction of the extent to which it could do so. Note that S′ necessarily ⩽ S (indeed < S for S < 1) and that because Sm is only approximate, S′ should be taken as heuristic. For more on S and S′, see List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

42 The T1-T2 gains tend to exceed their T2-T3 counterparts, no doubt because T1-T2 is a much longer interval, bracketing the receipt of the briefing materials and the anticipatory, off-site deliberations, as well as the first on-site deliberative session.

43 By contrast, the general knowledge items show significant but roughly equal gains in both the A-first and R-first groups from T1 to T2 and thus from T1 to T3 (results not shown). On these more general matters, there is no reason to expect the members of either treatment group to learn more during either interval, and they do not.

* Farrar and Green: both at Yale University, Institution for Social and Policy Studies, and Department of Political Science, respectively; Fishkin: Department of Communication, Stanford University; List, Departments of Philosophy and Political Science, London School of Economics and Political Science; Luskin, Department of Government, University of Texas at Austin (email: ); Paluck: Department of Psychology and Woodrow Wilson School of Public and International Affairs, Princeton University. Earlier versions of this article were presented at the annual meeting of the International Society of Political Psychology, Boston, Mass., 2003, and at the ECPR General Conference, Marburg, Germany, 2003. The authors are grateful to Ethan Leib, the League of Women Voters of Connecticut Education Fund Inc.; The Guild Group and Regional Plan Association for assistance in organizing and carrying out the split-half Deliberative Poll; and to the Carnegie Corporation, the Ford Foundation, the Renee B. Fisher Foundation, Fannie Mae, William C. Graustein, the New Haven Savings Bank, the South Central Connecticut Regional Water Authority, the United Way of Greater New Haven and the United Illuminating Company for making the project possible. The Center for Deliberative Polling and the Public Policy Clinic at the University of Texas at Austin, the Community Foundation for Greater New Haven and Yale University’s Institution for Social and Policy Studies also provided financial and in-kind support. The research and some of the analysis and writing were conducted while Fishkin and Luskin were Fellows at the Center for Advanced Study in the Behavioral Sciences, supported by grants from the William and Flora Hewlett Foundation (Grant 2000-5633), the Center General Fund and the University Research Institute of the University of Texas. Alice Siu contributed invaluable research assistance. Deliberative Polling is a trade mark of James S. Fishkin. Any fees from the trade mark are used to support research at the Center for Deliberative Democracy at Stanford University.

Disaggregating Deliberation’s Effects: An Experiment within a Deliberative Poll

  • Cynthia Farrar, James S. Fishkin, Donald P. Green, Christian List, Robert C. Luskin and Elizabeth Levy Paluck...

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed