QL research: fluid enquiry and the processual turn

QL research typically takes the form of small-scale, in-depth studies of individuals or collectives, generating rich, situated data about unfolding lives. With its dual identity (longitudinal and qualitative) it spans two established methodological traditions. Firstly, as a small but vital component of the longitudinal canon, it seeks to follow the same individuals or groups (e.g. a panel of welfare recipients, service providers and/or policy makers) prospectively, in real-time, to capture ‘change in the making’ (Wright Mills, Reference Wright Mills1959). QL research also incorporates a valuable retrospective lens. This enables participants and researchers to trace processes forward and backward in time. The process is iterative, involving the application of abductive logic (detailed in Neale, Reference Neale2021). We are afforded insights not simply into what happens over time, but how processes unfold, and the meaning they hold for those experiencing them. How questions are inherently processual. Moving beyond simple, clear-cut what questions, they are crucial in understanding why things unfold in particular ways.

Secondly, located within the interpretivist tradition of social research, and as part of the broad and porous canon of socio-historical studies (longitudinal ethnography, re-studies, biographical, life history and narrative research), QL research explores dynamic processes through an in-depth, qualitative lens. This is real-world research, which gives insights into how people narrate, understand, practice and shape their evolving lives, and the evolving world of which they are a part. QL research takes human agency and subjectivity seriously, enabling understandings of lived experiences, and gaining an intimate familiarity with lives as they are being lived.

The qualitative and longitudinal elements of QL methodology are entwined and captured in the notion of ‘walking alongside’ people. By turning a snap-shot of social life into an intimate ‘movie’, it becomes possible to uncover how lives are lived and experienced, as well as how they are narrated (Neale, Reference Neale2021). The flexibility of this methodology, and its responsiveness to change, gives QL research a unique capacity to mirror the flux of the world, to follow processes as they unfold (Neale, Reference Neale2021). This flexibility is vital to its methodological rigour, ensuring that a QL study is in tune with and able to reflect dynamic, real-world processes. It is through a deep and sustained engagement in the field that complex processes can be brought more readily to light (Langley et al., Reference Langley, Smallman, Tsoukas and Van de Ven2013).

QL enquiry has developed over recent decades as part of a widespread processual ‘turn’ in social enquiry. This has brought a shift in focus from the form that social events take, their structural components, to their relational and dynamic elements. As Bergson and E. P. Thompson saw, it is the dynamic relationship between things that matters, not the things themselves. This shift in thinking from structures to processes has seeped into the very foundations of interpretive enquiry, and is set to provide an important bridge between positivist, realist and interpretivist research traditions (Neale, Reference Neale2021). As part of this processual turn, QL enquiry has gradually consolidated its credentials as an established and creative methodology that straddles the longitudinal (real-time) and qualitative (real-world) canons. This is reflected in a rapidly expanding body of methodological literature, and a growing sophistication in how this approach is being conceptualised and applied (reviewed in Neale, Reference Neale2021).

While QL social policy research has a long history, in recent years there has been a rapid uptake in policy-related research across a range of fields: for example, in health, housing, education, social work, criminology and environmental issues (Corden and Millar, Reference Corden and Millar2007; Neale, Reference Neale2021). This growth is hardly surprising. An understanding of change, continuity and causality is vital in policy domains where people are required or encouraged to change their practices, to adapt to changing circumstances, or to develop their knowledge or learn new skills. It is no less vital where the effects of policy interventions need to be monitored and evaluated (Corden and Millar, Reference Corden and Millar2007).

Fluid time

The powerful features of QL enquiry outlined above reflect the underlying theoretical drivers for this mode of research: it is rooted in fluid, interpretive notions of time that go beyond the fixed regularities of the clock and calendar (Bergson, Reference Bergson1910 [1889]; Adam, Reference Adam1990). From an interpretive perspective, objective, constant, linear, one-dimensional time gives way to a plurality of times, held in a simultaneous relationship with each other, which flow and intersect in multiple ways. Fixed time is perceived as an overarching external force, a constant unvarying presence within which events take place and lives unfold (an ‘events in time’ perspective). Fluid time, in contrast, is experiential and socially constructed. It is embedded in who we are, inhering in our everyday sensibilities, events, practices and processes (a ‘time in events’ perspective, Bergson, Reference Bergson1910 [1889]; Adam, Reference Adam1990). While fluid time does not replace fixed, linear time, in this broader formulation, linear clock-based time loses its dominance. It becomes one among the many flows of time that constitute our temporal world, and that shape our lived experiences (Adam, Reference Adam1990; Neale, Reference Neale, Worth and Hardill2015, Reference Neale2021).

How events and processes are perceived in social enquiry depends on how time itself is perceived. For example, viewing change through the lens of fluid enquiry has a transformative effect. The ontological focus shifts from change as something that happens to things, to change as an inherent part of the nature of things. Social phenomena of all kinds take on a new meaning when viewed through the prism of fluid time. From a fixed, linear perspective, for example, a process is an orderly set of events that unfolds sequentially, with a clear purpose, discrete markers along the way, and a definitive end point (see Charmaz, Reference Charmaz2014: 17 for this orthodox view). From a fluid perspective, however, processes can be understood as rich streams of linked events, contexts, actions, reactions and interactions that are intertwined, and that unfold in non-linear and unpredictable ways (Neale, Reference Neale2021). As will be shown below, similar transformations occur in our understandings of causality when viewed through a fluid lens.

In sum, studying lives qualitatively through time is not a straightforward matter of creating a moving, chronological picture that charts observable changes and discerns what happens next. Lives do not necessarily unfold in chronological order, through discrete changes, in one linear direction, or at a uniform pace. Nor do people experience time in these fixed, linear ways (Neale, Reference Neale2021). QL research alerts us to an ever-changing kaleidoscope of lives in the making. In the process, the messy, complex, volatile, contradictory, shifting facets of human experience are brought to light (Farrall et al., Reference Farrall, Hunter, Sharpe and Calverley2014). It becomes possible to explore the stability or inertia of continued states; the triggers, driving forces and tipping points for change; and the creativity, resilience and/or fragility of individuals and groups in shaping or accommodating to these processes. It is this capacity for fluid enquiry, underpinned by a fluid understanding of time, which gives QL research its rich explanatory power (Neale, Reference Neale2021).

Causal processes

Having set out the broad contours of QL research, we now turn to the nature of causality. The focus here is on how causal processes may be understood from a fluid, qualitative perspective. The search for causal explanations, an understanding of cause and effect, is deeply rooted in social enquiry, and in the flux of everyday life (Maxwell, Reference Maxwell2012). Causality is our way of making sense of the world, and, as Le Guin notes (above), of giving shape to our moral choices and responsibilities. In the Aristotelian tradition, ‘causes’ are simply explanations, forms of reasoning that give meaning to our existence. In general, we want to know how things happen and why events and actions turn out in a particular way. Indeed, our credentials as researchers often depend upon our ability to explain these deeply embedded processes, although we may not use the language of causality (Lofland et al., Reference Lofland, Snow, Anderson and Lofland2006: 159; Hammersley, Reference Hammersley2008). Research findings are likely to contain implicit causal reasoning, for ‘causal statements are pretty much inevitable in any discussion of human social and cultural life’ (Seale, Reference Seale and Seale2018: 573). As Maxwell notes, ‘any argument that something ‘influences’ ‘impacts’, ‘shapes’ or ‘transforms’ something else is a causal claim’ (Maxwell, Reference Maxwell2012: 42). Moreover, ‘unless qualitative researchers are able to talk in cause/effect terms they will not have much to say to a policy community focused on the question, ‘what works?’ (Anderson and Scott, Reference Anderson and Scott2012: 680; Donmoyer, Reference Donmoyer2012).

Despite this, causality has long been a contested issue in the interpretive tradition of research. Positivists have traditionally held the view that qualitative researchers should stick to descriptions of the world, and leave causal explanations to those working with large datasets that lend themselves to measurement based, statistical modelling and large scale generalisations (Maxwell, Reference Maxwell2004). Until recently, this paradigmatic ‘stand off’ was also evident among qualitative researchers. The search for cause and effect was seen as a defining feature of predictive, measurement-oriented positivism, and not, therefore, a valid enterprise for interpretive enquiry: ‘the concept of causality is so beleaguered and in such serious disarray that it strains credulity to continue to entertain it in any form approximating to its present (poorly defined) one’ (Lincoln and Guba, Reference Lincoln and Guba1985: 141). Qualitative understandings of causal processes were set back for decades, through fears that the findings would be tainted by their positivist associations (Maxwell, Reference Maxwell2004; Hammersley, Reference Hammersley2008).

Simple causality

Part of the problem is that the language of causality and of causal mechanisms conjures images of conveyor belt precision and narrow, instrumental, mechanistic processes that may bear little relationship to the flux of the real world. The variance/inference causal logic used by positivists produces a simple understanding of causality that relies on a combination of two temporal flows. The first is chronology (A is followed by B). The second is causality (A causes B). The logic holds that if A is typically followed by B, then A must cause B, particularly where this pattern occurs regularly and repeatedly across large samples (Abbott, 2001; for a more detailed account see Lofland et al., Reference Lofland, Snow, Anderson and Lofland2006). Since causality cannot be directly observed, it can be inferred from two or more snap-shot visions of events, a string of discrete moments in time (temporal variables), which are held together (correlated) by their consistently observed sequence in a linear chain. Causality is implied in the linear, orderly progression from past to present to future; cause and effect are intimately tied to this sense of chronology. Viewed as a simple, predictive, instrumental relationship between cause and effect, causality is reduced here to regularity, repetition and sequence (Maxwell, Reference Maxwell2012: 37).

Some blistering attacks have been levelled at this inferential logic. Sayer (Reference Sayer2000: 13), for example, astutely observes that what causes something to happen may have very little to do with the number of times we observe it happening. The link between causality and repetition and regularity is far too superficial and tenuous. Moreover, on its own, the bald fact that A is followed by B may also tell us very little. In much formal sequential analysis there is too little room to explore the discontinuities, irregularities, sudden accelerations, trigger points and pivotal moments of change, or to aim for a more rounded and nuanced understanding of how A is linked to B. The limitations of simple, linear understandings of causality, which create a series of snapshots of change, have been extensively critiqued. QL organisational researchers, for example, point out the ‘difficulties we face when we try to understand change by breaking it down to stages. By doing so, change is reduced to a series of static positions – its distinguishing features are lost from view’ (Tsoukas and Chia, Reference Tsoukas and Chia2002: 571; Gerrits, Reference Gerrits2008; Langley et al., Reference Langley, Smallman, Tsoukas and Van de Ven2013; Langley and Tsoukas, Reference Langley and Tsoukas2017).

Similarly, Abbott (2001: 192) observes that events or occurrences have duration and may overlap, raising questions about the conceptual order or sequencing of things. This is related to the ‘time horizon’ problem (Abbott, 2001: 193): how do we account for processes that involve multiple, contingent sequences of events that are moving at different speeds, across different scales of the social fabric? In short, how far is it possible to tease out the multiple interlocking threads of a process, occurring at varied and overlapping tempos? While we can discern a great deal from the intricate patterning of events and interactions, with their different temporal rhythms, the sequence itself is likely to form just one element in a much more fluid picture.

Of course, this is not to deny the critical importance of studies that discern widespread correlations between discrete social phenomena. For example, the simple link between smoking and ill health, established in the 1950s, has led to a raft of in-depth investigations into how such correlations actually arise. Taken together, these studies reveal a host of factors (diet, life style choices and broader socio-economic and environmental inequalities) that impact on health, suggesting that a complex intertwining of individual, social and structural factors is at play. Mixed longitudinal studies, which combine numerical and narrative forms of explanation, can provide depth of understanding alongside a broader vision of socio-historical processes (Neale, Reference Neale2021).

Nevertheless, where variance/inferential causal logic is used as the sole source of evidence, there is a danger of producing flawed causal models that are too superficial, tenuous, narrow or spurious to be of value (Maxwell, Reference Maxwell2004; Anderson and Scott, Reference Anderson and Scott2012; Westhorp, Reference Westhorp, Emmel, Greenhaugh, Manzano, Monaghan and Dalkin2018). The drawback is that simple causal models are spatially and temporally adrift, ‘discovered from no-where’ (Hartsock in Frankham, Reference Frankham2009: 2). They are too abstracted from real-world processes, practices and interactions, and the meanings and motives of those involved.

Complex causality

In recent decades, a more fluid, processual understanding of causality has emerged as an alternative to the variance/inference approach outlined above (Maxwell, Reference Maxwell2004, Reference Maxwell2012). This new thinking has emerged as part of the processual turn in social enquiry. It recognises the ‘openness, contingency and contextually variable character of social change’ (Sayer, Reference Sayer2000), and seeks to discern how causal processes operate and unfold in real-world settings (Maxwell, Reference Maxwell2004; Donmoyer, Reference Donmoyer2012). In policy terms, a processual approach seeks to understand not simply what works? but what works for whom, under what circumstances and why’ (Pawson and Tilley, Reference Pawson and Tilley1997; Pawson, Reference Pawson2006). This more recent thinking has wrested causality from its positivist stronghold and made it a legitimate concern for qualitative enquiry (Maxwell, Reference Maxwell2012).

A processual approach to causal understanding has developed across numerous studies. It is a cornerstone of QL enquiry (Pettigrew, Reference Pettigrew, Huber and Van de Ven1995, Reference Pettigrew1997; Gerrits, Reference Gerrits2008; Langley et al., Reference Langley, Smallman, Tsoukas and Van de Ven2013; Bidart et al., Reference Bidart, Longo and Mendez2013; Langley and Tsoukas, Reference Langley and Tsoukas2017); and of realist research (Pawson and Tilley, Reference Pawson and Tilley1997; Sayer, Reference Sayer2000; Pawson, Reference Pawson2006; Maxwell, Reference Maxwell2004, Reference Maxwell2012; Byrne et al., Reference Byrne, Olsen, Duggan, Byrne and Ragin2009; Emmel et al., Reference Emmel, Greenhalgh, Manzano, Monaghan and Dalkin2018). It has also formed the foundation for complexity theory, a powerful amalgam of ideas drawn from systems theory, social network analysis and chaos theory (Cilliers, Reference Cilliers1998). Varied ways of understanding and investigating causal processes have developed within and across these different research traditions. While there is no one settled or integrated methodological approach, several important synergies and differences are worth teasing out here.

Firstly, QL researchers tend to produce detailed longitudinal case studies to show how causal processes unfold (Pettigrew, Reference Pettigrew, Huber and Van de Ven1995; Gerrits, Reference Gerrits2008; Bidart et al., Reference Bidart, Longo and Mendez2013). Realists, in contrast, tend to develop theoretical models in order to illuminate and predict the regularities and patterns of causal processes, and the underlying social mechanisms that drive them (Byrne and Callaghan, Reference Byrne and Callaghan2014). A standard formula is commonly applied, which seeks to model contexts, mechanisms and outcomes (Pawson and Tilley, Reference Pawson and Tilley1997; Pawson, Reference Pawson2006), although this formula is acknowledged to be overly rigid and ripe for modification (Emmel et al., Reference Emmel, Greenhalgh, Manzano, Monaghan and Dalkin2018). There is also the view that such predictive modelling runs the risk of becoming reductionist and hypothetical, of ironing out the complexities of processes in ways that are too abstract and simplistic (Hedström and Swedberg, Reference Hedström, Swedberg, Hedstrom and Swedberg1998; Hedström and Ylikoski, Reference Hedström and Ylikoski2010).

Secondly, alongside these investigative and analytical differences, there are variations in how processual researchers engage (or fail to engage) with time. Much like QL researchers, social complexity theorists recognise and work with fluid understandings of time and causality (Cilliers, Reference Cilliers1998; Gerrits, Reference Gerrits2008; Boulton et al., Reference Boulton, Allen and Bowman2015; Neale, Reference Neale2021). For Boulton et al. (Reference Boulton, Allen and Bowman2015), seeing the world in terms of complex fluidity is closer to lived experience than the Newtonian idea of the world as a machine. They suggest ways to research the complex world, in all its grittiness and granularity, ‘not through mathematical models and theories but ‘in the flesh’ through ‘being in it’ (Boulton et al., Reference Boulton, Allen and Bowman2015: 107). They advocate process tracing as a means to identify chronologies and ‘the causal links and pathways which lead actions through to impact’ (Boulton et al., Reference Boulton, Allen and Bowman2015: 190; Neale, Reference Neale2021). They also recommend working across fixed and fluid, sequential and non-sequential time, and working both prospectively and retrospectively to piece together a processual picture. Moreover, they promote qualitative, narrative, case history, action research and longitudinal methodologies as the prime means to achieve these goals. Their general advice is to be processual: to follow processes and anticipate tipping points, to trace and represent the development of situations over time; to allow for detours along the way, and to be open to multiple perspectives through a critical subjectivity (Boulton et al., Reference Boulton, Allen and Bowman2015: 112-14).

Realist researchers, in contrast, rarely conduct their studies through time, or engage with the fluidity of time. Models of causality are usually developed with no more than a passing reference to time, or, rather, with an implicit understanding of time as a fixed, linear entity (Westhorp, Reference Westhorp, Emmel, Greenhaugh, Manzano, Monaghan and Dalkin2018: 45). Yet when fluid time is brought into the picture, the nature of realist modelling is transformed. The notion that there are discrete mechanisms, contexts and outcomes that can be separated out sequentially for analysis becomes suspect. The temporal location of these causal threads is continually shifting, such that an effect (or outcome) operating one moment in time, may become a cause, a contextual factor or a mechanism at another (Westhorp, Reference Westhorp, Emmel, Greenhaugh, Manzano, Monaghan and Dalkin2018). Influences, then, may run in more than one direction, giving cause and effect a circular rather than chronological relationship; the formula A leads to B is transformed where A leads to B leads to A (Byrne and Callaghan, Reference Byrne and Callaghan2014). The very idea of discrete causes, contexts, mechanisms or outcomes becomes problematic in this shifting, fluid landscape.

Such differences, however, do not detract from the important commonalities that exist across these fields of enquiry, and the potential to draw on a rich amalgam of ideas to produce new processual insights. Taking up this opportunity, an attempt is made here to tease out and untangle the multiple, fluid and relational elements of complex causality.

The multiplicity of causal processes (thick causality)

Perhaps the most evident feature of complex causality is that it is driven by an intricate web of interacting elements that make up the social world. Multiple causal threads unfold in any one process, spreading across varied spatial and temporal settings, and operating at different scales of the social fabric (local to global, individual to social, micro to macro) (Pettigrew, Reference Pettigrew1997; Tsoukas and Chia, Reference Tsoukas and Chia2002; Langley and Tsoukas, Reference Langley and Tsoukas2017; Pawson and Tilley, Reference Pawson and Tilley1997; Sayer, Reference Sayer2000; Maxwell, Reference Maxwell2012). These multiple threads are perpetually flowing, oscillating and interacting through time; they are intertwined like a confluence of streams flowing through a river basin or across a flood plain (Pettigrew, Reference Pettigrew1997). They are shaped by driving forces and feedback or feedforward loops that may converge at key moments in time: to reaffirm, grow, adapt or otherwise transform the social fabric (Cilliers, Reference Cilliers1998; Gerrits, Reference Gerrits2008).

The influences that produce change, or lead to recurring states are not only multiple and flowing in all directions. They are also distributed (Cilliers, Reference Cilliers1998). In complex systems there will be many pockets of differential power and influence, but there are no fixed levels, control points, stable contexts, and no enduring external designers that can, with any certainty, determine the nature, direction and impact of change, or shape exactly how or when it will occur. This helps to explain why policy directives may fail or have unintended consequences. Given their distributed nature, complex processes are unpredictable: actors working in one part of the system will not be fully aware of the range or impact of their actions and interactions in another (Cilliers, Reference Cilliers1998; Gerrits, Reference Gerrits2008). And neither will researchers. Causality is recast here. In the maelstrom of complex social processes, there is no simple or unitary relationship between a cause and effect. Indeed, there are no discrete causes that lead to discrete effects, in clear cut directions, over discrete sequences of time. Tracing effects back to a single cause is far too simplistic when they are rooted in entangled webs of causality (Nowotny, Reference Nowotny2005).

The aim of QL research is to capture this dynamic reality in flight (Pettigrew, Reference Pettigrew1997): to follow processes where they lead, identify their constituent threads, and discern how these threads are connected and how they coalesce to create the momentum for change. This is ‘thick’ causality (Fleetwood, Reference Fleetwood, Adler, Du Gay, Morgan and Reed2014), which ‘does not follow some homogeneous if-when formula … but takes a multiplicity of different forms’ (Porpora, cited in Dall and Danneris, Reference Dall and Danneris2019: 589). In policy terms, it becomes clear that ‘what works’ is shaped by the particular circumstances and contexts in which change occurs. A failure to generate holistic, empirically-driven and locally situated understandings risks producing policy recommendations that present a narrow, partial or abstract picture, or that treat the symptoms rather than addressing root causes (Anderson and Scott, Reference Anderson and Scott2012: 679). These varied insights suggest the need for pluralism and empirically grounded sensitivity in how causal investigations and explanations are framed (Maxwell, Reference Maxwell2004).

The fluidity of causal processes

Processes convey a sense of flux and change: of becoming, development, emergence, progression, evolution, revolution, transformation, dissipation, decay and so on (Lofland et al., Reference Lofland, Snow, Anderson and Lofland2006: 152). If causal processes are multiple, operating through different settings and scales of the social fabric, they are also inherently dynamic. Moreover, the relationship between cause and effect is not a simple matter of chronology: processes emerge and are shaped through the fluidity of time. If tracing back to a single cause is too simplistic, so too is tracing back through a clear-cut, linear sequence of events: ‘Causality is neither singular nor linear. … Changes have multiple causes, and are to be explained more by loops than lines’ (Pettigrew, Reference Pettigrew, Huber and Van de Ven1995: 96).

The temporal dimensions of processes increases their multiplicity. Every event or experience has multiple antecedents, spiralling backwards through time, and multiple consequences, spiralling forwards into the future, creating vast and ever expanding constellations of causal flows (Abbott, 2001: 144, 192). Linearity here gives way to multi-directional zigzags, recursive loops, and processual patterns that may be cyclical, occur at random, spiral up and down, or otherwise grow and dissipate through time. This means that the course of a pathway, its trajectory, patterning and momentum, plays a part in shaping how events unfold. In other words, the nature of the journey assumes just as much importance as the starting point, or the destination reached (Neale, Reference Neale2021).

This fluidity brings to the fore the inherently contingent, unfinished and unpredictable nature of causal understandings. Causal processes are not located in static environments or contexts; there are no stable or external ‘wrappers’ within which change unfolds. Processes have no temporal fixity, no discrete start and end points, other than artificial ones that are created as an artefact of the research process and its window of observation. As Abbott (2001: 145) notes, when we think through time, the idea of beginnings, middles and ends quickly gives way to a sense of endless middles.

These insights into how causal processes unfold have significance for policy and professional practice. Rather than a focus on ‘what works?’, we need a fluid, processual understanding of how things work through the stream of time. As Stich and Cippolone (Reference Stich and Cippolone2017) observe in their QL study of urban school reforms in the US, ‘it is not just the passage of time that is important … but also the accumulations, recursions, connections, relations, stabilizations and destabilizations that develop … as time passes’ (Stich and Cippolone, Reference Stich and Cippolone2017: 5). Discerning this fluidity in unfolding processes is greatly enhanced through in-depth, real-time, sustained field investigations, of the sort afforded by QL research.

The relationality of causal processes

This is the third key feature of complex causality. Complex causal processes are not only multiple and fluid. They are inherently relational and infused with ethical sensibilities (Le Guin, Reference Le Guin2019 [1974]). There is a growing recognition that causal forces do not reside in social structures per se, but in the complex interaction between individuals and the collectives of which they are a part (Harrē, in Blaikie and Priest, Reference Blaikie and Priest2017). As Sayer observes, ‘… explanation requires mainly interpretive and qualitative research to discover actors’ reasoning and circumstances in specific contexts – not in abstraction from them’ (Sayer, Reference Sayer2000: 23). Human agency (the capacity to act, to interact, to influence the shape of one’s life and the lives of others), and subjectivity (the shifting meanings that events and processes hold for those who experience and craft them) are central to causal understandings (Hammersley, Reference Hammersley2008; Maxwell, Reference Maxwell2012; Byrne and Callaghan, Reference Byrne and Callaghan2014; Neale, Reference Neale2021). As Hammersley (Reference Hammersley2008) notes, the very notion of agency is founded on an understanding of the causal powers of individuals and groups.

These insights suggest the need for further reformulations of the ‘what works?’ mantra. Understanding how things work requires a relational understanding of what works for whom, bolstered by a prior understanding of what matters to people, and what helps them in particular circumstances. Gaining insights into lived experiences is vital in understanding causality, and in generating ‘know how’ knowledge (Langley et al., Reference Langley, Smallman, Tsoukas and Van de Ven2013: 4). These more nuanced, situated, consultative, ‘insider’ forms of causal knowledge avoid a narrowly prescriptive and instrumental focus on ‘what works?’. As a broader, overarching concept, complex causality encompasses simple causality, rather than replacing it. In fleshing out the picture, it provides essential insights into the causes and consequences of change.

The discussion above suggests the value of finely-grained longitudinal case data in generating insights into causal insights. To give just one example, the relational and fluid elements of complex causality have been explored recently in a two-year QL study of a Danish welfare-to-work programme and its impact on vulnerable clients (Danneris, Reference Danneris2018; Danneris and Caswell, Reference Danneris and Caswell2019). The researchers followed the clients’ journeys through the programme, documenting how it was delivered ‘on the ground’, and discerning its effects, including the mental health effects of a programme driven by a rigid and punitive welfare ideology. The researchers developed a typology of client trajectories: progressing, deteriorating, stagnating and derailing (which, in some cases, included suicide or attempted suicide). These different journeys were fluid and unpredictable (rather than linear or straightforward), marked by fits and starts, zigzag paths between different trajectories, and recurring spirals of hope and despair (Danneris and Caswell, Reference Danneris and Caswell2019). It was not possible to predict when a client might enter into a downward spiral, or reach a tipping point into derailment. But these patterns were most often linked to the punitive sanctions of the programme, or the ‘quick fix’, ‘one-size-fits-all’ nature of the work placements, which were commonly tenuous, unsuitable, unsustainable and demoralising (Dall and Danneris, Reference Dall and Danneris2019).

Retrospective evidence on what led to sustained employment for these clients revealed that programme strategies (structured activities, a rule-bound system and sanctions to enforce compliance) were less effective than the positive relational elements that some practitioners managed to build into the programme: sustained and flexible support, and tailored, responsive provision based on the values of respect, encouragement, and a sense of partnership in a collaborative project (Danneris and Caswell, Reference Danneris and Caswell2019). The authors also tease out the intertwining of individual and structural factors in these unfolding processes, showing how clients and service providers may be ‘pushed down particular streams’ by external forces beyond their control (Davidson, pers. comm.) Similar findings on relational causality, alongside fluid narratives of contingency, chaos and/or liminality, have been reported in a range of QL studies that explore the lived experiences of service users (Dearden et al., Reference Dearden, Good, Whitfield and Cox2010; Ridge and Millar, Reference Ridge and Millar2011; Pinnock et al., Reference Pinnock, Kendall, Murray, Worth, Levack, Porter, MacNee and Sheikh2011; Neale and Davies, Reference Neale and Davies2015; Patrick, Reference Patrick2017; Ferguson et al., Reference Ferguson, Warwick, Singh, Leigh, Beddow and Disney2020; Dwyer, Reference Dwyer2018).

Methodological challenges

Exploring causality ‘on the ground’ in the ways suggested above is not without its challenges. There is the complex task of uncovering the multiple threads that lead to causal effects, and discerning their relative salience in shaping lives. Since causal processes are contingent, unpredictable, and inherently unfinished, we must capture them ‘in flight’, through small windows in time, and in situated corners of the social fabric. The aim is to produce plausible interpretations that are acknowledged to be inherently provisional and in the making. This requires greater modesty in our claims to causal understanding, a need to tie causal explanations of actions and reactions, effects and counter-effects much more closely to grounded, subjective and retrospective accounts (Cilliers, Reference Cilliers1998).

A related challenge concerns the common misconception that only statistically based, or more theoretically driven studies have the capacity to generalise and ‘scale up’ their findings (Anderson and Scott, Reference Anderson and Scott2012; Donmoyer, Reference Donmoyer2012; Dall and Danneris, Reference Dall and Danneris2019). In fact, qualitative forms of generalisation are well established (Gobo, Reference Gobo, Seale, Gobo, Gubrium and Silverman2004; Neale, Reference Neale2021), although they could be more effectively utilised, packaged and promoted. Unlike systematic reviews of strictly comparable statistical data, qualitative generalisations allow for a mosaic of complementary evidence to be woven into a composite picture, including insights drawn from legacy datasets (Neale, Reference Neale2021). With each new addition to the analytical ‘pot’, the conceptual picture is enhanced or transformed. In this mosaic approach to evidence building, insights about patterns and processes of change are given added credence through the extent, variety and weight of the evidence base, and the strategic manner in which comparisons and connections are drawn out (Halford and Savage, Reference Halford and Savage2017; Anderson and Scott, Reference Anderson and Scott2012; Donmoyer, Reference Donmoyer2012; Monaghan and Boaz, Reference Monaghan, Boaz, Emmel, Greenhalgh, Manzano, Monaghan and Dalkin2018; Dall and Danneris, Reference Dall and Danneris2019; Wright and Patrick, Reference Wright and Patrick2019; Middlemiss et al., Reference Middlemiss, Ambrosio-Albalá, Emmel, Gillard and Gilbertson2019). For busy policy makers, such evidence has the potential to create conceptual impacts, to raise new questions and to change perceptions of how things work, as a necessary precursor to creating instrumental changes in policy and professional practice (Neale and Morton, Reference Neale and Morton2012; Monaghan and Boaz, Reference Monaghan, Boaz, Emmel, Greenhalgh, Manzano, Monaghan and Dalkin2018).

Evidence-based policy?

A further and equally pressing challenge also arises: how to make dynamic evidence about complex causal processes count in policy decision making? Currently, the research evidence base is used highly selectively to address ‘what works?’ questions. In part, the issue arises within the academy. The preference is for ‘hard’ economic or statistical evidence, such as randomised controlled trials, meta-evaluation, quasi-experiments and quantitative statistical testing, along with systematic reviews of such research (Dall and Danneris, Reference Dall and Danneris2019; Monaghan and Ingold, Reference Monaghan and Ingold2019). Such evidence is placed at the top of an evidence ‘hierarchy’ on the grounds that it is scientifically robust, while ‘soft’ qualitative and case study evidence, which forms the bulk of the research canon, is relegated to the bottom, where it can be effectively side-lined. This hierarchy is currently entrenched: ‘to challenge variable-centred approaches … is to challenge a whole power structure in the academy’ (Byrne et al., Reference Byrne, Olsen, Duggan, Byrne and Ragin2009: 519-20).

A preference for ‘hard’ data is also found within organisational cultures and capabilities, which are adapted to working with simple, variance-based causal models (Monaghan and Ingold, Reference Monaghan and Ingold2019). With their simple inputs and outputs, these data can be translated with relative ease into policy directives. Working with complexity is often anathema to the clear-cut needs of policy makers. This is the case despite the fact that evidence rooted in simple causal models is consistently reported to be ambiguous and inconclusive in meeting real-world policy objectives, or in identifying what actually works (Pawson, Reference Pawson2006; Byrne et al., Reference Byrne, Olsen, Duggan, Byrne and Ragin2009; Donmoyer, Reference Donmoyer2012; Millar and Bennett, Reference Millar and Bennett2017; Dall and Danneris, Reference Dall and Danneris2019; Monaghan and Ingold, Reference Monaghan and Ingold2019). As the discussion above shows, the ambiguity of such evidence is hardly surprising if it is based on the tenets of simple causality i.e. on assumptions that a system input will, in instrumental, mechanistic or deterministic ways, lead to a desired outcome; and that the presence of an input will be the necessary and sufficient condition to bring about change (Dall and Danneris, Reference Dall and Danneris2019).

Of equal concern is an evident dumbing down of expert research evidence as a valuable base upon which to shape policy. The whole edifice of UK evidence-based policy, which developed in the late 1990s under New Labour, appears to be crumbling, as ideological drivers (the austerity agenda, the conditionality of state aid and support, and centralised decision making and control) take precedence: ‘Rationality and logic are often trumped by other considerations … bigger forces can easily relegate research to a bit player (or no player at all)’ (Sainsbury, Reference Sainsbury, Becker, Bryman and Ferguson2012: 373). The result is a growing chasm between empirical evidence and policy-making (Millar and Bennett, Reference Millar and Bennett2017; Patrick, Reference Patrick2017; Dwyer and Patrick, Reference Dwyer and Patrick2020). What counts as evidence, then, is based on what is deemed permissible and helpful to policy. While, ostensibly, the evidence shapes policy, policy is just as likely to shape the evidence to its own ends (Millar and Bennett, Reference Millar and Bennett2017; Monaghan and Ingold, Reference Monaghan and Ingold2019).

Re-configuring the interface between research and policy

Whether QL research can contribute to understandings of ‘what works?’ in such a climate is an open question. It would require some subtle shifts in the way policy processes are configured, along with greater clarity and consensus about what constitutes robust evidence for policy, and how it may be obtained and translated (Dall and Danneris, Reference Dall and Danneris2019; Monaghan and Ingold, Reference Monaghan and Ingold2019). Whether complex causality is recognised and valued may depend, too, upon its conceptual impact; on how this alternative way of seeing things can be packaged and presented to policy makers as a formula that can be operationalised. This, in turn, depends on new thinking about ways to broker the interface between research evidence and complex policy processes (Kislov et al., Reference Kislov, Wilson and Boaden2017).

In recent years, researchers have made some strides in addressing this issue. The late twentieth century principles of knowledge transfer (a one-way, linear approach) and knowledge exchange (two-way, but equally linear) have gradually been incorporated into a more systemic, iterative and integrated model of knowledge co-production (Best and Holmes, Reference Best and Holmes2010). This fosters policy-informed research and research-based practice (Neale and Morton, Reference Neale and Morton2012). It is an approach predicated on collaborative modes of policy formulation and implementation that are less ‘top down’ and ideologically driven, and that allow for active partnerships between researchers, policy makers, and service providers, along with the participation of service users (Frankham, Reference Frankham2009).

The credentials of such an approach have been established through action research and participatory methodologies, which do not simply research change, but which seek to facilitate change (Bradbury Huang, Reference Bradbury-Huang2015). Action research is inherently dynamic, flexible and cumulative, a process involving iteration between the research process and practical action (Figure 1). In a parallel development, partnership and design-based models of research are beginning to appear, for example, in the fields of health and education (Penuel and Gallagher, Reference Penuel and Gallagher2017; McKenney and Reeves, Reference McKenney and Reeves2018; Nyström et al., Reference Nyström, Karltun, Keller and Andersson Gäre2018). These accord a more collaborative and integrated role for research in designing, co-ordinating and implementing policy programmes, as well as evaluating them.

Figure 1.

The Action Research Spiral (Kemmis and McTaggart, Reference Kemmis and McTaggart1988).

QL impact studies

As part of these developments, QL impact research offers innovative ways to broker the interface between research and policy/practice developments, based on collaborative real-time modes of knowledge production. With its flexible, grounded, responsive and interactive tempo, QL research can act as a design, co-ordinating and navigational device, which helps to shape policy interventions and then ‘walks alongside’ to uncover how these processes are unfolding. Indeed, the real-time iteration between research and policy processes enables stakeholders to take stock at key moments, try out new directions, and refine the elements of a programme as it unfolds, in ways that may create and stimulate change. An exploration of how things work is embedded within and emerges through the research process.

Drawing on design-based methodologies (McKenney and Reeves, Reference McKenney and Reeves2018) and their application in QL research (Kinsey, Reference Kinsey2020), a QL impact design might include the following:

1. 1. Stakeholder Consultations to identify gaps in knowledge, perceptions, current limitations, possible solutions among policy makers, analysts, and front line services; recruitment of policy/practice partners and client consultants;

2. 2. Integrative review and synthesis of complementary research evidence/ legacy data, developed in consultation with stakeholders;

3. 3. Collaborative Design of a new intervention; production of practice protocols;

4. 4. Pilot Programme implementation: using intensive longitudinal methods to track, monitor, review and evaluate the new initiative;

5. 5. Case Study reporting/review of potential modifications and further developments in different contexts/settings.

6. 6. In a two-phase or rolling research design, an initial tracking and evaluation of existing provision can become a spring board for the design, development and evaluation of a new initiative. Indeed, in line with the action research spiral (above). the process allows for successive waves of reflecting, planning, acting and observing, in a cumulative process of review and refinement.

The organisational and relational challenges of QL designs based on partnership working should not be underestimated. Creating a porous and collaborative interface between research and policy/practice requires brokering skills to secure and sustain ‘buy in’ from busy professionals (Monaghan and Boaz, Reference Monaghan, Boaz, Emmel, Greenhalgh, Manzano, Monaghan and Dalkin2018). It also requires clear terms of reference, diplomacy and agility to maintain a balance between the competing claims and perspectives of multiple stakeholders (Flinders et al., Reference Flinders, Wood and Cunningham2016; Penuel and Gallagher, Reference Penuel and Gallagher2017; Kislov et al., Reference Kislov, Wilson and Boaden2017; Nyström et al., Reference Nyström, Karltun, Keller and Andersson Gäre2018; Kislov, Reference Kislov2019). Nevertheless, these forms of research are being effectively used by QL researchers, and they offer considerable power and potential to make a difference in policy and practice settings (Kislov, Reference Kislov2019; Kinsey, Reference Kinsey2020, showcased in Neale, Reference Neale2021).