The strategic road to current methodologies

As Sanders, Snijders and Hallsworth (Reference Sanders, Snijders and Hallsworth2018) correctly note, “While behavioural science is much more widely used than it was, it has yet to sit alongside economics as a discipline dominant in the thinking of policy-makers… there is a danger that behavioural science is seen to offer merely technocratic tweaks, rather than the more wide-ranging reassessment of public administration that could be possible.” I propose that this is partly due to governmental institutional, and thus BIT's, pressure to prove the concept and provide quick wins by recruiting the best-evidenced interventions from the behavioural science literature to achieve revenue-producing and money-saving projects in the context of high-level governmental backup. The resulting ‘test, learn, adapt’ approach to BPP that has evolved as a result is one where a wide array of robust behavioural insights are selected and tested in randomised controlled trials (RCTs) on problems challenging traditional regulation (Haynes et al., Reference Haynes, Goldacre and Torgerson2012). This is also evidenced by BIT update reports being crowded with well-evidenced interventions based on behavioural insights, like ‘loss aversion’, ‘social proof’, ‘reciprocity’ and other MINDSPACE (and later EAST) tools, to deal with issues of non-conformity to existing regulatory issues (Dolan et al., Reference Dolan, Hallsworth and Halpern2010; Service et al., Reference Service, Hallsworth and Halpern2014).

The problem with this approach, as I see it, is that it ignores the thorough analysis of what constitutes the actual ‘thorny’ behavioural problems that traditional policies often seem to get wrong from the beginning.Footnote ¹ Instead, it becomes ‘attractive’ to ‘make it easy’ by opting for desktop abductions about ‘procrastination’, ‘inertia’, ‘status quo bias’ or the like. However, the activity of labelling behavioural phenomena with behavioural terminology from the desktop is not particularly true to the spirit of applied behavioural science. Rather, from an applied behavioural science perspective, one would usually carry out prolonged, real-world, structured field observations, explorative analyses of existing public datasets and tests of hypotheses to develop more profound diagnoses of the targeted behavioural problems before any type of intervention (Robson, Reference Robson2002). Of course, this does not seem to be a particularly effective strategy if one is trying to achieve ‘quick wins’ or a ‘tenfold return on cost’ within just a couple of years. Perhaps this is the reason why BIT has opted for the shotgun approach of ‘test, learn, adapt’ instead, where multiple interventions are fired into a large sample of citizens with the aim of “driving social change incrementally but quickly.”Footnote ²

The problem with where we are at

However, I have several doubts about this approach. While ‘fast and furious’, I am not sure that this strategy is particularly effective, instructive, or scientifically socially responsible.

For one, I believe that it is more effective, in the sense of both accuracy and long-term behaviour change, to base interventions on a profound understanding of the interaction between actual regulatory barriers and psychological components constituting behavioural problems. Like at the doctor's office, providing a sound and precise diagnosis of a behavioural problem before intervening is likely to increase the success rate, albeit at a slower pace. Hence, I believe that, rather than merely treating symptoms or producing short-term effects with potential side effects that would likely lead to a reversal to the status quo in the long run, we should pursue a more ‘diagnostic’ approach that may help solve actual problems.

Second, unlike the institutional context in which BIT originated, most existing behavioural efforts will not be able to obtain the high-level governmental backup to access the large sample sizes required for this approach or the freedom to choose areas of interest to test anything beyond technocratic tweaks. At least, it is my repeated experience that we can quite easily run a letter-tweaking experiment involving thousands of taxpayers, but only provoke strenuous smiles when we say, “We could also try to rethink the policy assumptions.” Of course, one could say that the shotgun approach could ultimately lead to a series of incremental changes that evolve the system fundamentally. However, I see no theoretical reason to believe why this should be the case. After all, evolutionary processes only improve on the existing objects of evolution and often leave it with superfluous traits. Additionally, institutional environments may co-evolve too fast for new traits to ever reach optimisation. This latter point may be relevant to consider when, for example, looking at the co-evolution of strategies in consumer markets with reluctant stakeholders. Take the Credit Card Accountability, Responsibility and Disclosure Act in the USA (Willis, Reference Willis2013) and the imposed cool-down period on payday loans introduced in Denmark in 2017 (Toft, Reference Toft2017) – in both cases, banks quickly found ways to circumvent the behavioural insights that policies implemented.

Third, I believe the shotgun approach exposes citizens to an unnecessarily wide array of interventions. This is especially the case if applied to problems that are more serious than a lack of regulatory conformity. For instance, running RCTs in health care, whether medical or behavioural, should only be done with a sound diagnosis at hand. That is, hypotheses should be grounded in theory that would explain how and why we expect the intervention to work relative to the targeted behavioural problem. Opting instead for testing on a black box of citizens with a shotgun approach takes us too close to behaviourism, fails to treat citizens as goals rather than means and provides few lessons for the future. Of course, the shotgun should be part of our armoury in BPP, but not the only one.

For these reasons, I believe that the application of behavioural science to public policy – as currently practiced – is yet to sit alongside economics as a discipline dominant in the thinking of policy-makers, as well as yet to explain why behavioural science is seen to offer merely technocratic tweaks rather than a more wide-ranging reassessment of public administration. Of course, this challenge to the operational strategies and principles only grows when we turn our sights beyond ‘WEIRD states’ (Western, educated, industrialised, rich and developed) to a more global application of behavioural insights. At a more global level, policy efforts often do not run on fine-grained regulatory systems, and the most pressing problems tend to go way beyond non-conformity in responding to government letters on time. However, whether looking at BPP in WEIRD or developing countries, the point is the same: that is, to go beyond merely technocratic tweaks and contribute to more profound change in public policy we need to recognise that the potential of behavioural insights transcends that of translating the best-evidenced interventions from the behavioural science literature to provide a proof of concept and some quick wins. Rather, we should begin focusing more on providing realistic theories of the forces that shape human behaviour in the real world and through which we should understand public policy challenges and opportunities. The real prospect of behavioural science is not just an add-on to public policy – it is a rethinking of its foundations.

From policy delivery to policy development

The observation that we must go beyond policy delivery and focus more on policy development leads me to a point that parallels Sander, Snijders and Hallsworth's (Reference Sanders, Snijders and Hallsworth2018) discussion of the ‘reverse impact’ of behavioural science when applied in public policy. As they point out, scientists may be accustomed to publishing with the hope of having an impact on policy. However, as they also point out, in BPP, behavioural scientists “working on policy issues may experience the situation in reverse. Rather than publishing peer-reviewed research that may then influence government action, they may alter government actions and then attempt to publish the results in peer-reviewed journals. In other words, the impact comes first.”

However, ‘reverse impact’ is not the only aspect where applied behavioural science differs from traditional behavioural science. Traditionally, behavioural scientists, like other scientists, have had the privilege of choosing their research question, theory and methodology according to their likings, and occasionally their approaches are picked up and applied in policy contexts. Yet, this path is not characteristic of successful BPP, such as Save More Tomorrow, nor of how public policy development works or should work. Rather, profound public policy development usually begins with a particular policy challenge. That is, policy problems come first. The pursuit of potential interventions follows later, hopefully based on a sound understanding of the nature of the challenge. However, many current applied behavioural insights work the other way around, in that we have a behavioural intervention for some toy problem and then look for problems that match that intervention. For this reason, we should be careful about drawing conclusions about the general effectiveness of behavioural insights.

New beginnings

To do this, we need better tools, models and theories to work from the policy problems towards relevant applications of behavioural insights. MINDSPACE and EAST are nice frameworks for “using the best-evidenced interventions from the behavioural science literature in order to provide a proof of concept and some ‘quick wins’” through technocratic tweaks, as Sanders, Snijders and Hallsworth (Reference Sanders, Snijders and Hallsworth2018) describe. Nevertheless, they are not fit for developing BPP, and neither is the shotgun approach that adopts the ‘test, learn, adapt’ methodology to incrementally drive social change by shooting at low-hanging fruit. We need tools, models and methodologies cast in the forge of behavioural science to better define behavioural challenges in policy and analyse their constituents. The current state of affairs at this end of the arms gallery of BPP is one of eclecticism, the use of which is extremely dependent on the tacit knowledge of experienced people. This is a problem because it does not change the fact that policy challenges currently enter public policy and administration as predefined entities formulated around traditional policy assumptions rather than behavioural terms.

Thus, for instance, traditional policy efforts often begin by casting policy challenges in broad policy categories, such as the need to ‘create increased competition’, ‘facilitate public innovation’, ‘increase public awareness’ and the like, rather than identifying the key behavioural patterns that are crucial to target in order to achieve those ends. Likewise, to understand the factors that shape crucial behaviours, traditional public policy often approaches the analysis by surveying the self-reports of the beliefs, opinions, preferences and even behaviours of the involved parties. This is despite the fact that these methodologies do not sit very well with behavioural insights about the context dependency of self-reporting, the intention–behaviour gap or the limited reliability of human introspection.

As a result, central policy challenges have already been digested and formulated based on theories and methods that are not easily reconciled with behavioural science. This means that BPP easily ends up not only using the shotgun approach, but also firing it into the dark after constructions that would not necessarily count as important from a behavioural perspective. When lucky enough to hit something, knowledge about why something worked is limited. I believe that putting our hopes in the idea that if we just ‘incrementally’ fire enough times in different directions we will ultimately learn something important is not credible – and the idea of inviting untrained public servants to copy that approach does not sit well with me either.

Hence, we should not forget what I believe is the primary challenge at this point in the history of the application of behavioural science to public policy: that is, to ensure an effective and responsible development of BPP, we now need to develop our theories, tools and models to deal with what Colin Robson has referred to as ‘real-world research’ (Robson, Reference Robson2002). The particular sub-challenges this presents to the community of behavioural scientists are not necessarily familiar to any of us. For instance, how do we systematically operationalise broad policy challenges in behavioural terms in such a way that we can identify what key behaviours to change? How do we describe and empirically analyse those behaviours in terms of behavioural science and behavioural insights rather than receiving descriptions, analyses and reports in terms of traditional policy conceptions? How do we systematically match behavioural problems with behavioural insights so that we can discard the shotgun approach in favour of a more responsible approach where we can more effectively pinpoint effective behavioural intervention concepts in ways that allow small teams, whether inside or outside of WEIRD countries, to run tests in contexts of limited resources, minimal institutional power and a lack of well-working administrational infrastructures?

Of course, it is not accidental that I hold this belief. I am biased. For the last six years, my colleagues at iNudgeyou and I have been working on the framework BASIC – A Practitioners Toolbox and Ethical Guidelines for Applying Behavioural Insights in Public Policy (OECD, 2018; also see Hansen & Schmidt, Reference Hansen and Schmidt2017). The framework is currently in its final phase of development and will be published later this year by the OECD as an accessible toolkit for behavioural scientists and policy-makers. The BASIC toolbox is ‘diagnostic’ in the sense that it provides a series of tools for reducing policy challenges to behavioural problems and for selecting key behaviours according to their potential of having profound policy impacts. It also provides tools rooted in applied behavioural science that allow us to analyse behaviour according to behavioural insights and avoid some common pitfalls. Finally, at its core, BASIC provides a model for matching empirical analyses of behavioural problems with the best-evidenced interventions from the behavioural science literature according to their type. Of course, the effort to develop theories, models and methods like this will be an ongoing project, of which BASIC is only one of the first-born. It is the path that we must not forget if behavioural science is to “sit alongside economics as a discipline dominant in the thinking of policy-makers.”