1. Introduction
Mechanistic information is important for assessing the effectiveness and normative acceptability of behavioural public policies (BPP) (see for example, Grüne-Yanoff Reference Grüne-Yanoff2016; Marchionni and Reijula Reference Marchionni and Reijula2019).Footnote 1 But how should accepting this argument affect the practices of evaluating the effectiveness of policies? Current practices typically proceeds by experimentally establishing the effect size of a treatment type (e.g. default setting, social comparison, self-commitment) and extrapolating the finding to the policy target. In contrast, we argue that treatment categorizations should be supplemented or replaced by mechanistic categorization. We develop this argument for behavioural policies aimed at household energy consumption. Nevertheless, as we argue later in the paper, our findings can be generalized to other domains of BPP such as recycling and health, where randomized controlled trials (RCTs) and meta-analyses are also common.
Our argument proceeds in three steps. First, we show that current categorizations of behavioural policies in the energy consumption domain are based on the type of treatment they implement. The context sensitivity of recorded effect sizes, however, makes it difficult to address questions about extrapolation and ethical evaluation based on these taxonomies alone; mechanistic information is needed too. Second, we argue that supplementing treatment categorizations with mechanistic information, for example by coming up with finer-grained categories, is insufficient when there is both variation of mechanisms within each treatment type (mechanistic heterogeneity) as well as overlap between mechanisms across treatment types (mechanistic overlap). By surveying treatment types predominant in the literature on energy consumption, we show that there is both substantial mechanistic heterogeneity and mechanistic overlap.Footnote 2 We conclude that policy extrapolation and evaluation based on treatment types is hard to justify under such conditions and suggest that current categorizations of behavioural interventions should be revised to better reflect mechanistic information.
The paper is structured as follows. Section 2 describes our notion of mechanism and explains why mechanistic information is important for behavioural policies. Section 3 extracts dominant categorizations from recent reviews of the energy literature and argues that the context sensitivity displayed in this domain puts in doubt that this way of categorizing interventions can handle problems of policy extrapolation and evaluation. Sections 4 and 5 show that there is both substantial mechanistic heterogeneity and overlap in the domain of household energy consumption. Section 6 concludes.
2. Mechanisms and their relevance for behavioural policy
Mechanisms have been characterized as systems of causally interacting parts and processes, which – under certain conditions – predictably produce one or more effects (e.g. Glennan Reference Glennan and Humphreys2016; Craver and Tabery Reference Craver and Tabery2019). For behavioural policies, mechanisms causally link the policy intervention to the agents’ behaviour, making explicit the intermediate stages between them. These intermediate steps, called ‘mediators’, consist of, for example, components of agents’ cognition, social interactions between agents, or tools with which agents take decisions.
Figure 1 represents a mechanism where a behavioural intervention consists of a treatment (T) that the policymaker implements with the goal of affecting a change in the behaviour of individuals (B). Each letter represents a variable, with mediators (Me i ) connecting B to T. The directed arrows represent the potential causal influence of one variable on another. A mediator can either pass on causal influence or block it: some variable outside the causal chain might affect a mediator so that it no longer changes in response to the influence of a predecessor in the causal chain (it is ‘switched off’) or it is less likely to be so affected (its transmission ability is reduced). Factors that affect a mediator’s transmission ability are called moderators (Mo i ). In addition to passing on causal influence to their successor in the causal chain, mediators might also have side effects (SE i ). Side effects are variables outside the causal chain that are affected by mediators but do not affect them.
Intervention mechanism.
Figure 1 is a representation of a mechanism. The same mechanism can be represented in different ways. For example, if Me i-1 -> Me i -> Me i+1 describes a mechanism, so does Me j -> Me i+1 , where Me j combines the mediators Me i-1 and Me i as well as their interactions. It is often unknown whether a mediator can be analysed into a sequence of more fine-grained mediators. Furthermore, it is often ontologically uncertain (especially in the social sciences) whether, for a given mechanism, there is an ultimate level of fine-grained mediators. Consequently, we speak of models M of a mechanism, where multiple models might be accurate representations of the same mechanism:
M(T,B) is an ordered, non-empty set of variables that represent mediators Me i , which causally connect T to B, and which also connect to both moderators Mo i and side effects SE i .
To illustrate, a policymaker sets a green energy provider default (T). Many consumers see both advantages and disadvantages of this provider in comparison to others, but feel it is too costly for them to form an overall judgement (they would if a more extreme option had been set as default); thus T produces the ‘too costly’ feeling Me 1 which in turn leads people to stick to the default option (B) (for a discussion of various mechanisms of default-setting interventions, see Grüne-Yanoff Reference Grüne-Yanoff2016). The provision of a simple and trustworthy comparison tool (Mo 1) might reduce this ‘too costly’ feeling to such an extent that the effect of T on B is blocked. Furthermore, if Me 1 is active, eliciting such a feeling might contribute to a general sense that bureaucratic communications are not worth serious consideration (SE1).
The philosophical literature offers two arguments for the importance of mechanistic information. First, it helps to make precise the conditions for successful policy extrapolation. An intervention might generate a given outcome in one context (e.g. in an experiment) but fail to do so in another (e.g. in a policy target), because background factors vary greatly between real-world and test conditions and across different contexts. This is why evidence of efficacy from RCTs is generally not sufficient for extrapolation by itself. Instead, justifying inferences of effectiveness from one context to another requires also showing that these contexts do not substantially differ in relevant background factors (Guala Reference Guala2010; Steel Reference Steel2010; Grüne-Yanoff Reference Grüne-Yanoff2016; Marchionni and Reijula Reference Marchionni and Reijula2019). How does one know what factors are relevant to justify extrapolation? This is where mechanistic information comes in: if M correctly represents the mechanism through which T affects B, then relevant background factors are the moderators Mo i, which either amplify or inhibit the mediators Me i of M. Mechanistic information thus places the policymaker in a better position to make inferences from the test to the target population: when operating through the mechanism described in M, T has a chance of success if Mo i is absent but less so if Mo i is present.
A second argument for the importance of mechanistic information concerns the normative evaluation of policy applications. A behavioural policy can be assessed either by exclusively focusing on its outcome or by also considering some of its procedural features. A purely outcome-oriented evaluation will need to consider (potentially unintended) side-effects. For example, an intervention that successfully gets residents to lower thermostat settings might still be judged a failure when residents compensate by using portable heating elements, driving up electricity consumption. How does one know that such compensation effects are indeed caused by the intervention rather than, say, by a sudden cold spell? Mechanistic information helps here: by detailing the causal chain from T not only to B but also to side effects SEi, knowledge of mechanisms allows the identification and measurement of those variables affected by the intervention, and thus supports a comprehensive evaluation of its outcomes. The importance of mechanisms is even more obvious with respect to evaluations incorporating procedural features. For example, even if an intervention that improves the subjects’ knowledge and one that works through social pressure had exactly the same outcome, they might still be evaluated differently. Leveraging social pressure might raise issues of autonomy and subjective well-being if the subjects change their behaviour due to fear of shame or social sanctions. Such procedural information is provided by the mediators Me i of M. Mechanistic information therefore helps evaluating policies from both a purely outcome-oriented and a procedural perspective (Smith et al. Reference Smith, Goldstein and Johnson2013; Grüne-Yanoff Reference Grüne-Yanoff2016, Reference Grüne-Yanoff2018).
3. Current practice: categorization by treatment
We now show that in the domain of energy consumption behavioural policies have been largely categorized and compared on the basis of the type of treatment they implement. For this purpose, we discuss five popular policy types, as summarized in Table 1.
Common categorizations of behavioural interventions. To identify the dominant categorizations, we reviewed seven meta-analyses and systematic reviews of multiple behavioural treatments on energy consumption (Row 1). We determined the categorizations used in headings, subheadings and summarizing tables of each article and grouped them based on terminology, similar description in text and literature references (represented in the cells under each article). We labelled each treatment type after the most frequently used term (Column 1). We excluded treatments that do not match the profile of behavioural policies in energy consumption (e.g. material incentives, labelling), and those mentioned in less than two-thirds of the articles
We claim that these categorizations focus on treatment rather than mechanism for two reasons. First, the terminology in almost all cases only indicates an action of the policymaker or experimenter: they do something (set a goal, elicit a commitment, give feedback, etc.) with the purpose of affecting behaviour change. Descriptions of actions do not, however, describe how (in other words, through which mediators) the subjects are affected. To illustrate what we mean here, consider a description that includes such a ‘how’:
The social comparison in the message affects behaviour by highlighting social norms rather than by sending signals about privately efficient behaviour (i.e. highlighting cost-savings opportunities). (Ferraro and Miranda Reference Ferraro and Miranda2013: 357)
In most cases, the literature only speaks about social comparisons, or similar interventive actions, without specifying how these affect the behaviour and makes such interventive actions the grounds for grouping different studies together. This way of proceeding has the advantage of making it possible to compare effect sizes of different treatments across studies and populations without worrying too much about how the treatment works at the mechanistic level. The assumption is that whatever heterogeneity there is at the mechanistic level, it will be replicated in the target if the effect size is big and robust enough.
Relatedly, the concepts are closely linked to experimental practices: they name treatments implemented in behavioural experiments. These experiments record effect sizes of these treatments, but they do not investigate mechanisms directly. Although the experiments control for some background conditions, they typically do not systematically investigate their influence on effect sizes, nor are there systematic attempts to categorize treatments according to these background conditions. And, as we will discuss in section 4, different types of mediating processes have been proposed as explanations of how the same interventions operate, but usually only informally – without the experiments being designed to test them. Consequently, due to their link with these experimental practices, the categories refer to types of treatment, disregarding procedural or mechanistic aspects.
It could be argued that if effect sizes are robustly associated with treatment types across relevant contexts, this way of categorizing interventions is unproblematic. Unfortunately, empirical evidence contradicts this: the effect size of treatment types are often context-sensitive. To give one example, a recent meta-analysis of feedback interventions aimed at reducing household energy consumption, covering 42 studies, found a mean effect size of 0.071 (Karlin et al. Reference Karlin, Zinger and Ford2015). However, the range of individual effect sizes varied from −0.080 to 0.480, i.e. not only did the effectiveness of feedback vary significantly between studies, but several studies even reported negative effect sizes.
What explains this wide range of results? One explanation might be study quality, as larger sample size was associated with more modest effects (Karlin et al. Reference Karlin, Zinger and Ford2015: 1215). Another might be that the treatment category itself is too coarse-grained. For example, effect sizes might differ depending on the specific properties of the feedback provided (Karlin et al. Reference Karlin, Zinger and Ford2015: 1220). But such differences can also be due to context-dependence, i.e. the influence on the behavioural effect of factors outside the intervention. For example, socially comparative feedback has been found to be more effective for politically liberal individuals than political conservatives in the USA (Costa and Kahn Reference Costa and Kahn2013), to have higher impact if subjects are compared with those in their peer network (Peschiera et al. Reference Peschiera, Taylor and Siegel2010) and to have drastically different effects for high- and low-consuming households (e.g. Allcott Reference Allcott2011; Ferraro and Price Reference Ferraro and Price2013), with the lowest baseline users even increasing their usage in some cases (e.g. Schultz et al. Reference Schultz, Nolan, Cialdini, Goldstein and Griskevicius2007). Footnote 3 These contextual factors are examples of mechanistic moderators: they affect the way in which mediators from intervention to behaviour operate and, through that, they have an influence on the effect size of the behavioural intervention. Such context dependence of effect sizes exhibits the limitations of trying to answer questions about policy extrapolation and evaluation on the basis of a treatment-based typology alone. At the very least, such categorization would have to be supplemented with mechanistic information. Doing so however is problematic when there is no one-to-one correspondence between treatment and mechanism. If the same treatment operates through different mediators on different occasions, it can also be subject to different moderators and involve side-effects and procedural features that should be evaluated differently. For example, whether social comparison works as optimality signals or through instigating competitiveness may matter to the policymaker beyond its pragmatic utility. The policymaker may be concerned with the ethical consequences of encouraging further competition, for example. More generally, nudges have been criticized for bypassing awareness, suggesting that the way they work matters to policy selection.
4. Mechanistic heterogeneity of treatments
For the five intervention types listed in Table 1, we identify mechanistic hypotheses and explanations proposed in the existing literature and show that there is considerable mechanistic heterogeneity in each category.
Recall that we define a mechanistic model M as follows: M(T,B) is a (non-empty) ordered set of mediators Me i , moderators Mo i and side effects SE i , which provides a connection from T to B. Based on this definition, we say that a treatment T is mechanistically heterogeneous iff (i) there are multiple models M i (T,B), M j (T,B) that truthfully represent the causal sequence from T to B, (ii) M i (T,B) ≠ M j (T,B) and (iii) M i (T,B) is not a more fine-grained version of M j (T,B), – one cannot truthfully replace a causal sequence Me i-1 -> Me i -> Me i+1 in model M i with a simpler sequence Me j -> Me i+1 , where Me j combines the mediators Me i-1 and Me i as well as their interactions, yielding M i *, such that M i *(T,B) = M j (T,B). In other words, there is no truthful representation of the two mechanisms that connect T to B under which they consist of the same mediators, moderators and side-effects. Consequently, if multiple, mutually non-reducible, models are offered as descriptions of the working of the same intervention, we consider it a prima facie reason that the type operates through multiple mechanisms.Footnote 4
4.1. Commitment
A commitment intervention consists in eliciting commitments either to a self-determined or externally imposed goal from subjects who might otherwise not have done so (Katzev and Wang Reference Katzev and Wang1994). In a private pledge, subjects formulate a pledge only to themselves (Jaeger and Schultz Reference Jaeger and Schultz2017); whereas a public pledge is published more widely (Abrahamse et al. Reference Abrahamse, Steg, Vlek and Rothengatter2005). In a 2013 meta-analysis, Lokhorst et al. (Reference Lokhorst, Werner, Staats, van Dijk and Gale2013) find that during the intervention period reported mean effect sizes during were medium both for commitment only (r = 0.27) and for commitment plus another treatment (r = 0.31), but reduced to small in the long run. However, a number of reviews find commitment interventions to be effective in some contexts, for some people and some types of behaviour, but not for others (Osbaldiston and Schott Reference Osbaldiston and Schott2012; Frederiks et al. Reference Frederiks, Stenner and Hobman2015; Šćepanović et al., Reference Šćepanović, Warnier and Nurminen2017).
In the literature, commitments have been hypothesized to work through the following alternative mechanisms:
-
(i) Commitment focuses an agent’s attention. According to Feedback Intervention Theory (Kluger and DeNisi Reference Kluger and DeNisi1996), attention is the drive underlying task performance and must therefore be directed to a specific task-oriented goal. By making subjects pledge to certain goals, the intervention directs their attention to the goals whose realization is being promoted (McCalley Reference McCalley2006).Footnote 5 Consequently, actors that affect attention are the most important moderators for commitment interventions that operate through this mechanism.
-
(ii) Commitment helps agents to create a new goal. According to Self-Perception Theory (SPT, Bem Reference Bem1972), people develop their attitudes by observing their own behaviour and by seeking to rationalize this behaviour by self-attributing these attitudes. Commitment interventions elicit a behaviour (the initial commitment to energy conservation) that, through self-perception, causes the self-attributions of energy-conservation attitudes, which subsequently induce further energy-conseration behaviour. In contrast to (i), the effect of commitment here operates through the creation of new goals, not by directing focus on existing ones. Because it relies on a form of self-rationalization, the most important moderator here is the availability of alternative rationalizations – e.g. coercion, strategic or facile commitment (Burger and Caldwell Reference Burger and Caldwell2003).
-
(iii) Commitment changes existing goals. According to Cognitive Dissonance Theory (CDT, Festinger Reference Festinger1957), people experience cognitive dissonance when holding contradictory attitudes simultaneously. Commitment interventions might create a new goal that generates such dissonance. To reduce the negative arousal of dissonance experience, people will change attitudes until they become consistent. This might lead to the elimination of previous attitudes and the retainment of the ones to which one is committed to. The CDT mechanism thus differs from the one hypothesized by SPT in at least two respects. First, CDT posits that subjects’ original attitudes are clear and important to them, so that they detect the inconsistency, while SPT posits that those original attitudes are relatively ambiguous and less important. Second, CDT posits arousal (‘cognitive dissonance’) as driving attitude change, while SPT does not (Fazio et al. Reference Fazio, Zanna and Cooper1977; Schultz Reference Schultz2014). Consequently, ambiguity and alternative sources of arousal are the most important moderators for commitment interventions that operate through the CDT mechanism.
-
(iv) Public commitment might affect behaviour by making the subject expect negative social sanctions from non-compliance with the commitment (Cialdini Reference Cialdini2001). According to such social pressure models, commitment interventions affect subjects’ behaviour but not their intrinsic attitudes (Abrahamse et al. Reference Abrahamse, Steg, Vlek and Rothengatter2005), which were the driving factors in the previous three cases. Consequently, the subject’s perceived publicity of the declaration and the perceived constancy of the witnessing group are the main moderators here – if the subject felt that their promise was no longer public in this sense, the commitment effect would presumably break down. In this view, public commitments work similarly to public social comparison.
4.2. Goal setting
In a goal setting intervention, the goal of saving on energy consumption is formulated explicitly and fixed at a certain level. The goal can be set either by the agents themselves or externally by the experimenter/policymaker. In general, the best performance has been found to take place when the goal is set at a realistic level. In a meta-analysis on goal setting and feedback in general, Neubert (Reference Neubert1998) estimates an effect size of 0.63 when feedback was added to goal setting. Similarly, Osbaldiston and Schott (Reference Osbaldiston and Schott2012) find that goal setting in combination with other treatments has a weighted average effects size ranging from 0.34–1.31. For energy consumption, McCalley and Midden (Reference McCalley and Midden2002) find that feedback together with goal setting is more effective than feedback alone (F = 3.6, P = 0.03) with a reduction in energy consumption between 21.9% and 19.5% depending on whether the goal was self-set or assigned.
At least two alternative theories and respective mechanistic hypotheses have been advanced to explain why setting goals can promote behavioural outcomes.
-
(i) According to Goal-Setting Theory (GST), a specific goal increases performance compared with a vaguely specified goal or no goal (Locke and Latham Reference Locke and Latham1990) through four complementary mechanisms: (a) a goal focuses attention and action toward the desired end and away from irrelevant ones; (b) a goal encourages agents to put an amount of effort proportional to the perceived requirements for attaining the goal (this is why more ambitious goals are ceteris paribus more motivating); (c) a goal motivates persistence; (d) a goal activates the skills, knowledge and strategies needed to achieve it, or it motivates people to learn them. For the respective complements, the relationship between goal and performance is claimed to be affected by these moderators: (a) ability, knowledge or skill; (b) commitment to the goal; (c) feedback on goal progress; (d) situational resources. This type of mechanism overlaps with commitment mechanism (i). According to this theory, feedback is a moderator: it motivates persistence and allows the evaluation of current strategies.
-
(ii) An alternative mechanistic hypothesis takes inspiration from Prospect theory. The idea is that a specific goal, say reducing energy consumption by 5%, serves as a reference point. Due to framing effects, agents categorize different outcomes as either successes or failures, hence getting closer to one’s goal involves positive feelings and vice versa (Heath et al. Reference Heath, Larrick and Wu1999; Harding and Hsiaw Reference Harding and Hsiaw2014). This theory adds a cognitive element to the motivational effect postulated by GST, but in contrast to that, here setting a goal cannot change behaviour without a way of knowing how far or close one is to the goal, namely, without some kind of feedback. The most important moderators for goal setting interventions operating through this mechanism are factors affecting the framing.
4.3. Informing
Informing interventions truthfully provide information with the explicit goal of improving people’s decision-making. They are distinct from propaganda or deceptive advertisement because they aim to disseminate accurate information only. They are different from education interventions that operate through experiential or affect-based channels – e.g. modelling or gamification – because they spread symbolically encoded information (Dietz and Stern Reference Dietz, Stern, Dietz and Stern2002).
A large number of energy conservation experiments use information strategies to reduce energy use (for recent overviews, see Abrahamse et al. Reference Abrahamse, Steg, Vlek and Rothengatter2005; Delmas et al. Reference Delmas, Fischlein and Asensio2013). Empirical evidence on the effect of information interventions is mixed. Some researchers conclude that more information has little or no effect on energy use (Abrahamse et al. Reference Abrahamse, Steg, Vlek and Rothengatter2005), while others estimate that information programmes could result in energy use reductions on the order of 22–30% over the next 5–8 years (Gardner and Stern Reference Gardner and Stern2008). However, these studies examine interventions with different dissemination and information properties, in different contexts, and are therefore difficult to compare. The most recent meta-analysis of informing interventions in this domain found a mean reduction of electricity consumption by 5.61% with a standard deviation of 6.84 (Khanna et al. Reference Khanna, Baiocchi, Callaghan, Creutzig, Guias, Haddaway, Hirth, Javaid, Koch, Laukemper, Löschel, del Mar Zamora Dominguez and Minx2021, cf. also Delmas et al. Reference Delmas, Fischlein and Asensio2013, who record a mean effect size of 7.4%). The relatively low effect size, as well as the substantial variations, suggest that any potential causal relationship between information provision and energy consumption is likely mediated by several factors, which in turn are subject to different contextual moderators.
Two alternative mechanistic hypotheses account for the effectiveness of informing interventions.
-
(i) Information interventions remedy an information deficit. The agent lacks information to form justified beliefs about relevant questions and therefore has to deal with uncertainty (Schultz Reference Schultz, Dietz and Stern2002; Darby Reference Darby2003; Hargreaves et al. Reference Hargreaves, Nye and Burgess2013) or lacks the skills to perform certain actions (e.g. competences to reduce room temperature while remaining comfortable, cf. Corral-Verdugo Reference Corral-Verdugo2002; Schultz et al. Reference Schultz, Nolan, Cialdini, Goldstein and Griskevicius2007). In this case the effectiveness of information interventions depends on three moderators. First, the existence of a relevant deficit: if the subjects already possess the information, the intervention will fail. Second, attention: without attention, information is not taken up and thus cannot affect behaviour (Clark and Dukas Reference Clark and Dukas2003). Third, epistemic trust in the medium or source of information: without it, the information will be discounted as evidence and thus will not result in belief expansion or revision (McCraw Reference McCraw2015).
-
(ii) Even when accurate beliefs and relevant skills are present, information provision can affect behaviour by increasing subjects’ motivation. According to Norm Activation Theory, for example, changes in behaviour occur when a person is aware of an issue and thinks they can influence it (Vining and Ebreo Reference Vining, Ebreo, Bechtel and Churchman2002; Fischer Reference Fischer2008). Informing interventions might raise awareness in norm-activating ways, even when subjects do not lack beliefs or skills. For example, the subjects might already have the information, but the intervention makes the issue salient in a way that motivates them to change their consumption behaviour. In this case, the intervention operates through focus and attuning (Kurz et al. Reference Kurz, Donaghue and Walker2005). Therefore, this mechanism overlaps with commitment mechanism (i) as well as with the first goal-setting mechanism. For this mechanism, attention is an even more important moderator than for deficit-remediation or skill increase, as the subject no longer has independent reason to seek out the information. Informational overload thus is likely to undermine any saliency and thus any effect on behaviour. However, deficit or epistemic trust do not moderate the working of this mechanism.
4.4. Feedback
In feedback interventions, individuals are provided with information about their own energy use that is intended to be more accurate, frequent or accessible than what they currently have. In most cases, feedback includes comparison to past energy usage. Feedback is often combined with other intervention strategies that require individuals to know how much energy they consume, such as social comparison and goal setting.
Two recent meta-analyses found feedback to be an effective intervention strategy in general (Karlin et al. Reference Karlin, Zinger and Ford2015; Buckley Reference Buckley2020; but cf. Delmas et al. Reference Delmas, Fischlein and Asensio2013), but they also found results and effect sizes to vary drastically between studies. Some of the differences in effectiveness are likely to be attributable to differences in how feedback is provided (Karlin et al. Reference Karlin, Zinger and Ford2015: 1220–1221). Between studies, feedback varies in terms of frequency, medium of delivery, unit of measurement and granularity of information. Accordingly, some meta-analyses do not treat feedback as a homogeneous category. For example, they separate periodical feedback from real-time feedback (e.g. Delmas et al. Reference Delmas, Fischlein and Asensio2013; Buckley Reference Buckley2020).
Feedback interventions can affect behaviour through at least three different pathways:
-
(i) Feedback corrects misperceptions about one’s own consumption level. Individuals generally have inaccurate or insufficient knowledge about how much energy they use, which causes them to consume more (or less) energy than they would prefer. Feedback can fill this ‘information vacuum’ by providing accurate information, enabling households to optimize their energy consumption (Buchanan et al. Reference Buchanan, Russo and Anderson2014; Matsukawa Reference Matsukawa2018; Lee et al. Reference Lee, Kang, Song and Kang2020). In this sense, feedback interventions do not seem to operate differently from informational intervention mechanisms of the first kind and are subject to the same moderators.
-
(ii) Feedback shows how everyday actions contribute to energy consumption. Feedback provides a link between behaviour and energy consumption and facilitates learning through experimentation (Fischer Reference Fischer2008; Buchanan et al. Reference Buchanan, Russo and Anderson2014; Jessoe and Rapson Reference Jessoe and Rapson2014; Lynham et al. Reference Lynham, Nitta, Saijo and Tarui2016; Matsukawa Reference Matsukawa2018). The capability of feedback to support learning through experimentation is moderated by how close to real-time and how appliance-specific feedback is, so that agents can draw a close enough link between actions and outcomes.
-
(iii) Feedback serves as a cue that focuses attention to energy consumption. Overconsumption results from a lack of awareness when engaging in everyday activities, and feedback reminds individuals to conserve by making the energy consuming nature of activities salient (Fischer Reference Fischer2008; Allcott and Rogers Reference Allcott and Rogers2014; Carroll et al. Reference Carroll, Lyons and Denny2014; Tiefenbeck et al. Reference Tiefenbeck, Goette, Degen, Tasic, Fleisch, Lalive and Staake2018). To the extent that feedback works by directing attention to energy consumption, its effectiveness is likely to depend on how it is delivered. Feedback sent to individuals periodically by mail or email may capture attention more easily than feedback provided through smart devices, since the latter requires individuals to engage with it proactively (Karlin et al. Reference Karlin, Zinger and Ford2015; Lynham et al. Reference Lynham, Nitta, Saijo and Tarui2016). When feedback is delivered may also play a role, as providing feedback on a specific energy consuming activity while it is being engaged in may further increase situational focus (Tiefenbeck et al. Reference Tiefenbeck, Goette, Degen, Tasic, Fleisch, Lalive and Staake2018).
4.5. Social comparison
Social comparison interventions provide households with feedback on their own energy consumption compared with that of others. Evaluative elements that signal the social desirability of the subject’s level of consumption are commonly included. Usually, subjects’ consumption is compared with average consumption of similar households in the neighbourhood, or with least consuming households. Other reference groups found to be effective include close acquaintances (Peschiera et al. Reference Peschiera, Taylor and Siegel2010) and residents of the same street (Shen et al. Reference Shen, Young and Cui2016). By providing information about the behaviour of others, social comparison is a form of social norm communication. These types of treatments can communicate either descriptive norms, which describe which behaviour is prevalent in a group, or injunctive norms, which describe the prevalence of pro-attitudes towards that behaviour in a group (Cialdini et al. Reference Cialdini, Kallgren and Reno1991). In a recent meta-analysis (Khanna et al. Reference Khanna, Baiocchi, Callaghan, Creutzig, Guias, Haddaway, Hirth, Javaid, Koch, Laukemper, Löschel, del Mar Zamora Dominguez and Minx2021), social comparison interventions were found to reduce energy consumption by 5.34% on average with a standard deviation of 7.62. Given the complexity of social norms, and the wealth of theories about how they work, it is not surprising that we could identify at least four different mechanisms through which social norms interventions are hypothesized to work.
-
(i) Information about descriptive norms affects behaviour by being interpreted as optimality signals. Observing difference to others’ consumption, individuals might conclude that their own consumption habits are suboptimal (Allcott Reference Allcott2011; but cf. Ferraro and Miranda Reference Ferraro and Miranda2013). For example, high consuming individuals might conclude from observing lower average consumption that consuming less is likely to carry some benefit, as suggested by it ‘being the norm’. This effect is moderated by the degree to which subjects see the average as establishing such a descriptive norm.
-
(ii) Comparison with others might motivate some individuals by appealing to the competitive desire to outdo others (Abrahamse et al. Reference Abrahamse, Steg, Vlek and Rothengatter2005; Andor and Fels Reference Andor and Fels2018). In contrast to the signalling mechanism just described, this competitive mechanism suggests that individuals will seek to move away from the average in whatever direction they deem better. Consequently, the most important moderators for this mechanism are factors affecting competitiveness.
-
(iii) Social comparison affects the ‘moral cost’ of energy consumption by eliciting feelings of guilt in those who perceive themselves as violating a social norm and feelings of ‘warm glow’ in those who conform (Allcott Reference Allcott2011; Ferraro and Miranda Reference Ferraro and Miranda2013; Delmas and Lessem Reference Delmas and Lessem2014). For individuals motivated by a concern for social appropriateness, the descriptive norm serves as a reference point that determines socially acceptable levels of energy use (Schultz et al. Reference Schultz, Nolan, Cialdini, Goldstein and Griskevicius2007; Andor and Fels Reference Andor and Fels2018). This explanation is supported by the findings that social comparison is more effective on individuals who believe that others care about conserving energy (Jachimowicz et al. Reference Jachimowicz, Hauser, O’Brien, Sherman and Galinsky2018), and on individuals highly influenceable by what others think they should do (Anderson et al. Reference Anderson, Song, Lee, Krupka, Lee and Park2017). Consequently, the dominant moderators of this mechanism are those that influence the perception of appropriateness of one’s own consumption. For example, communicating descriptive norms to low consuming households might signal to them that it would be reasonable or justified to increase consumption (Schultz et al. Reference Schultz, Nolan, Cialdini, Goldstein and Griskevicius2007; Allcott Reference Allcott2011).
-
(iv) Social comparison, if public, operates through social pressure. In most social comparison interventions, individuals’ energy consumption is not visible to others, so deviating from the norm has no social consequences. However, when energy consumption is visible to others, individuals can be motivated by the desire to avoid sanctions and maintain a good reputation (Delmas and Lessem Reference Delmas and Lessem2014). For example, Ferraro and Miranda (Reference Ferraro and Miranda2013) found social comparison to have a larger effect on outdoor than indoor water use, which could be explained by outdoor use being more likely to be observed by neighbours. This mechanism overlaps with commitment mechanism (iv). Consequently, factors concerning public observability are the main moderators of this mechanism.
To conclude, we have identified multiple non-reducible mechanistic hypotheses for each of our five selected treatment types. The relevance of the within-type variation was further underscored by identifying different moderators that may amplify or inhibit the effectiveness of the treatment. We summarize our findings in Table 2.
Mechanistic heterogeneity. Column 1 lists the treatment types identified in Table 1. For each type, column 2 lists mechanistic hypotheses offered in the literature. Column 3 gives examples of potential moderators for the respective mechanisms
5. Mechanistic overlap
In the previous section, we showed that behavioural policies categorized on the basis of the kind of treatment they implement display substantial mechanistic variation within each treatment type, which we refer to as mechanistic heterogeneity. An obvious way to address this heterogeneity is to distinguish further subtypes, identified by the mechanism involved. But simply insisting on more fine-graining often is not an adequate strategy for dealing with BPP categorization. The problem with this strategy is that treatment-based categorizations also display mechanistic overlap.
Mechanistic overlap describes a situation where treatments of different types cause their effects through a similar set of mediators. We focus on the simplest case, where distinct treatments produce behaviour through identical mechanisms. Recall that we define a mechanistic model M as follows: M(T,B) is a (non-empty) ordered set of mediators Me i , moderators Mo i and side-effects SE i , which provides a connection from T to B. Based on this definition, we say that two treatments T 1 , T 2 are mechanistically identical iff (i) they produce the same B with similar moderator values, and (ii) there are two models M(T 1 ,B), M(T 2 ,B) such that M(T 1 ,B) = M(T 2 ,B). Note that there might be many models of the same mechanism for T 1 , T 2 , respectively. Our definition only requires that some models that accurately describe this relation are identical. Mechanistic identity thus is simply the negation of mechanistic difference as defined in section 3. Figure 2 illustrates mechanistic identity so understood.
Illustration of mechanistic identity.
An example of mechanistic identity is found between public commitment (T 1 ) and public social comparison (T 2 ). As discussed above, both might operate through a social pressure mechanism. In response to a public commitment treatment (T 1 ) individuals identify a group of people taking note of their commitment (Me 1 ). If group members also follow the commitment, individuals interpret their behaviour as an injunctive norm (Me 2 ). According to this injunctive norm, they feel pressure to satisfy their own commitment (Me 3 ), adjusting their behaviour (B) accordingly. When energy consumption of a group of consumers is made public (T 2 ), consumers identify a group of peers to which they compare themselves (Me 1 ). If they observe peers sticking with a consumption regime, they interpret their behaviour as an injunctive norm (Me 2 ). According to this injunctive norm, they feel pressure to avoid sanctions and maintain a good reputation (Me 3 ), adjusting their behaviour (B) accordingly. Thus T 1 and T 2 operate through the same mechanism under description D.
Note that mechanistic identity does not mean that T 1 and T 2 always operate through the same mechanism. As shown in section 4, both interventions might operate through multiple mechanisms. Mechanistic overlap only means that two treatments share one or more mechanisms among those through which they might operate.
Amongst the mechanisms through which each intervention type might operate as described in the previous section, some are identical across types. Table 3 summarizes these mechanistic overlaps.
Examples of different treatment types that might operate through identical mechanisms
Different treatment types thus sometimes might operate through the same mechanism. For example, if feedback, goal setting and commitment all operate through attention-direction, then they all share the same mechanism and thus require similar moderator values to be instantiated in order to be effective. But they need not operate through attention-direction: as we show above, each treatment type can also operate through mechanisms that are not shared between them. Importantly for our argument, the presence of mechanistic overlap shows that not only does treatment type categorization lack detail to discern heterogeneous mechanisms operating within each type, but also fails to account for the same mechanisms operating across types, thus disregarding information relevant for extrapolation and evaluation. For example, if an intervention operating through a social pressure mechanism is found to cause negative side-effects under certain conditions, then this could speak against implementing any treatment that is likely to operate through the same mechanism in similar contexts. To recognize where extrapolative and evaluative inferences can be generalized across treatment types, a mechanism-based categorization is needed. Mechanistic overlap, therefore, supports the claim that behavioural policies – at least in the energy consumption domain – should be categorized according to mechanisms, not treatment types.
6. Conclusions
In this paper, we accomplished three things: First, we showed that current taxonomies of behavioural interventions aimed at reducing household energy consumption focus on types of treatment rather than on types of mechanisms. Second, we reviewed mechanistic hypotheses about treatment types offered in the literature and showed that interventions of the same type are subject to mechanistic heterogeneity: there is substantial variation of mechanisms within each treatment type. Third, we argued that treatment-based categorizations also display mechanistic overlap: there is substantial overlap between mechanisms across treatment types.
Although in this paper we only analysed energy consumption BPPs, we believe that our results apply more widely, for two reasons. First, our reading of research papers and reviews from other BPP domains reveals similar categorization practices as documented here. In the literature on health BPPs and on recycling BPPs, we find pretty much the same interventions grouped according to the same criteria as in household energy consumption.Footnote 6 Second, many authors in this literature favour RCTs, and consequently focus on measuring effect sizes – i.e. the relation between treatment and behavioural effect. Such a focus generally neglects the investigation of mechanisms, and in particular, it prevents the recognition of mechanistic heterogeneity and overlap.
Our findings weaken the rationale for categorizing behavioural interventions by treatment type. More importantly, they have consequences for the use of treatment categorizations for purposes of policy extrapolation. Because of both mechanistic heterogeneity and mechanistic overlap, the conditions of an intervention’s efficacy are to be found in the kind of mechanism through which that intervention operates, not in the kind of treatment it implements. This makes extrapolation based on treatment types hard to justify and suggests that current categorizations of behavioural interventions should be revised to better reflect mechanistic information.
Till Grüne-Yanoff is Professor of Philosophy at the Royal Institute of Technology (KTH) in Stockholm. His research focuses on the philosophy of science, the philosophy of economics and on decision theory. In particular, he investigates the practice of modelling in science and engineering, develops formal models of preference consistency and preference change and discusses the evaluation of evidence in policy decision making.
Caterina Marchionni is a researcher in Practical Philosophy at the University of Helsinki and member of TINT – Centre for Philosophy of Social Sciences. Her primary research areas are the philosophy of economics and the philosophy of the social and human sciences. In her work, Caterina has been paying special attention to issues of explanation, models, and evidence across disciplinary boundaries. Email: caterina.marchionni@helsinki.fi
Tatu Nuotio is a MSSc from the University of Helsinki and affiliated with TINT – Centre for Philosophy of Social Science. His research interests centre on the topics of social norms and norm-related policy. Email: tatu.nuotio@gmail.com
Open access
