Skip to main content
    • Aa
    • Aa

The Multiple Purposes of Policy Piloting and Their Consequences: Three Examples from National Health and Social Care Policy in England


In England, policy piloting has become firmly established in almost all areas of public policy and is seen as good practice in establishing ‘what works’. However, equating piloting with evaluation can risk oversimplifying the relationship between piloting and policy-making.

Using three case studies from health and social care – the Partnerships for Older People Projects (POPP) pilots, the Individual Budgets pilots and the Whole System Demonstrators (WSD) – the paper identifies multiple purposes of piloting, of which piloting for generating evidence of effectiveness was only one. Importantly, piloting was also aimed at promoting policy change and driving implementation, both in pilot sites and nationally. Indeed, policy makers appeared to be using pilots mainly to promote government policy, using evaluation as a strategy to strengthen the legitimacy of their decisions and to convince critical audiences. These findings highlight the ambiguous nature of piloting and thus question the extent to which piloting contributes to the agenda of evidence-based policy-making.

Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

P. Bate and G. Robert (2003), ‘Where next for policy evaluation? Insights from researching National Health Service modernisation’, Policy and Politics, 31: 2, 249262.

D. M. Berwick (2008), ‘The science of improvement’, Journal of the American Medical Association, 299: 1182–4.

C. Bonell , A. Fletcher , M. Morton , T. Lorenc and L. Moore (2012), ‘Realist randomised controlled trials: a new approach to evaluating complex public health interventions’, Social Science and Medicine, 75: 2299–306.

P. Bower , M. Cartwright , S. P. Hirani , J. Barlow , J. Hendy , M. Knapp , C. Henderson , A. Rogers , C. Sanders and M. Bardsley (2011), ‘A comprehensive evaluation of the impact of telemonitoring in patients with long-term conditions and social care needs: protocol for the whole systems demonstrator cluster randomised trial’, BMC Health Services Research, 11: 184.

A. Cameron , C. Salisbury , R. Lart , K. Stewart , S. Peckham , M. Calnan , S. Purdy and H. Thorp (2011), ‘Policy makers’ perceptions on the use of evidence from evaluations’, Evidence and Policy: A Journal of Research, Debate and Practice, 7: 429–47.

D. T. Campbell 1969), ‘Reforms as experiments’, American Psychologist, 24: 409–29.

N. Cartwright and J. Hardie (2012), Evidence-Based Policy: Doing It Better. A Practical Guide to Predicting If a Policy Will Work for You, Oxford: Oxford University Press.

R. Common (2004), ‘Organisational learning in a political environment: improving policy-making in UK government’, Policy Studies, 25: 3549.

M. Exworthy and M. Powell (2004), ‘Big windows and little windows: implementation in the “congested stateˮ’, Public Administration, 82: 263–81.

T. Greenhalgh and J. Russell (2007), ‘Reframing evidence synthesis as rhetorical action in the policy making drama’, Politiques de Santé, 1: 3442.

J. Hendy , T. Chrysanthaki , J. Barlow , M. Knapp , A. Rogers , C. Sanders , P. Bower , R. Bowen , R. Fitzpatrick and M. Bardsley (2012), ‘An organisational analysis of the implementation of telecare and telehealth: the whole systems demonstrator’, BMC Health Services Research, 12: 403.

G. T. Henry and M. M. Mark (2003), ‘Beyond use: understanding evaluation's influence on attitudes and actions’, American Journal of Evaluation, 24: 293314.

S. Innvær , G. Vist , M. Trommald and A. Oxman (2002), ‘Health policy-makers’ perceptions of their use of evidence: a systematic review’, Journal of Health Services Research and Policy, 7: 239–44.

K. Johnson , L. O. Greenseid , S. A. Toal , J. A. King , F. Lawrenz and B. Volkov (2009), ‘Research on evaluation use a review of the empirical literature from 1986 to 2005’, American Journal of Evaluation, 30: 377410.

V. Lowndes and L. Pratchett (2012), ‘Local governance under the coalition government: austerity, localism and the “Big Societyˮ’, Local Government Studies, 38: 2140.

S. Martin and I. Sanderson 1999), ‘Evaluating public policy experiments measuring outcomes, monitoring processes or managing pilots?’, Evaluation, 5: 245–58.

J. McNulty (2012), ‘Symbolic uses of evaluation in the international aid sector: arguments for critical reflection’, Evidence and Policy: A Journal of Research, Debate and Practice, 8: 495509.

N. Moran , C. Glendinning , M. Stevens , J. Manthorpe , S. Jacobs , M. Wilberforce , M. Knapp , D. Challis , J.-L. Fernández and K. Jones (2011), ‘Joining up government by integrating funding streams? The experiences of the individual budget pilot projects for older and disabled people in England’, International Journal of Public Administration, 34: 232–43.

W. Parsons (2002), ‘From muddling through to muddling up-evidence based policy making and the modernisation of British Government’, Public Policy and Administration, 17: 4360.

M. Petticrew , M. Mckee , K. Lock , J. Green and G. Phillips (2013), ‘In search of social equipoise’, British Medical Journal, 347: 1820.

H. W. J. Rittel and M. M. Webber (1973), ‘Dilemmas in a general theory of planning’, Policy Sciences, 4: 155–69.

I. Sanderson (2002), ‘Evaluation, policy learning and evidence-based policy making’, Public Administration, 80: 122.

J. Schofield (2001), ‘Time for a revival? Public policy implementation: a review of the literature and an agenda for future research’, International Journal of Management Reviews, 3: 245–63.

L. M. Shulha and J. B. Cousins (1997), ‘Evaluation use: theory, research, and practice since 1986’, American Journal of Evaluation, 18: 195208.

C. H. Weiss (1979), ‘The many meanings of research utilization’, Public Administration Review, 39: 426–31.

C. H. Weiss (1998), ‘Have we learned anything new about the use of evaluation?’, American Journal of Evaluation, 19: 2133.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Social Policy
  • ISSN: 0047-2794
  • EISSN: 1469-7823
  • URL: /core/journals/journal-of-social-policy
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Altmetric attention score

Full text views

Total number of HTML views: 3
Total number of PDF views: 35 *
Loading metrics...

Abstract views

Total abstract views: 357 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 30th April 2017. This data will be updated every 24 hours.