Skip to main content Accessibility help
×
Home

What role should randomized control trials play in providing the evidence base for conservation?

  • Edwin L. Pynegar (a1), James M. Gibbons (a1), Nigel M. Asquith (a2) and Julia P. G. Jones (a1)

Abstract

The effectiveness of many widely used conservation interventions is poorly understood because of a lack of high-quality impact evaluations. Randomized control trials (RCTs), in which experimental units are randomly allocated to treatment or control groups, offer an intuitive way to calculate the impact of an intervention by establishing a reliable counterfactual scenario. As many conservation interventions depend on changing people's behaviour, conservation impact evaluation can learn a great deal from RCTs in fields such as development economics, where RCTs have become widely used but are controversial. We build on relevant literature from other fields to discuss how RCTs, despite their potential, are just one of a number of ways to evaluate impact, are not feasible in all circumstances, and how factors such as spillover between units and behavioural effects must be considered in their design. We offer guidance and a set of criteria for deciding when RCTs may be an appropriate approach for evaluating conservation interventions, and factors to consider to ensure an RCT is of high quality. We illustrate this with examples from one of the few concluded RCTs of a large-scale conservation intervention: an incentive-based conservation programme in the Bolivian Andes. We argue that conservation should aim to avoid a rerun of the polarized debate surrounding the use of RCTs in other fields. Randomized control trials will not be feasible or appropriate in many circumstances, but if used carefully they can be useful and could become a more widely used tool for the evaluation of conservation impact.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      What role should randomized control trials play in providing the evidence base for conservation?
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      What role should randomized control trials play in providing the evidence base for conservation?
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      What role should randomized control trials play in providing the evidence base for conservation?
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Corresponding author

(Corresponding author) E-mail edwin.pynegar@gmail.com

Footnotes

Hide All
*

Also at: Sustainability Science Program, Harvard Kennedy School, Cambridge, USA

Footnotes

References

Hide All
Abdul Latif Jameel Poverty Action Lab (2019) http://www.povertyactionlab.org [accessed 11 August 2019].
Adams, W.M. & Sandbrook, C. (2013) Conservation, evidence and policy. Oryx, 47, 329335.
Alpízar, F., Nordén, A., Pfaff, A. & Robalino, J. (2017) Spillovers from targeting of incentives: exploring responses to being excluded. Journal of Economic Psychology, 59, 8798.
Andam, K.S., Ferraro, P.J., Pfaff, A., Sanchez-Azofeifa, G.A. & Robalino, J.A. (2008) Measuring the effectiveness of protected area networks in reducing deforestation. Proceedings of the National Academy of Sciences of the United States of America, 105, 1608916094.
Angrist, J.D. & Pischke, J.-S. (2010) The credibility revolution in empirical economics: how better research design is taking the con out of econometrics. Journal of Economic Perspectives, 24, 330.
Asquith, N.M. (2016) Watershared: Adaptation, Mitigation, Watershed Protection and Economic Development in Latin America. Climate & Development Knowledge Network, London, UK.
Babad, E.Y., Inbar, J. & Rosenthal, R. (1982) Pygmalion, Galatea, and the Golem: investigations of biased and unbiased teachers. Journal of Educational Psychology, 74, 459474.
Banerjee, A., Chassang, S. & Snowberg, E. (2016) Decision Theoretic Approaches to Experiment Design and External Validity. NBER Working Paper 22167, Cambridge, USA.
Barton, S. (2000) Which clinical studies provide the best evidence? BMJ, 321, 255256.
Bausell, R.B. (2015) The Design and Conduct of Meaningful Experiments Involving Human Participants: 25 Scientific Principles. Oxford University Press, New York, USA.
Baylis, K., Honey-Rosés, J., Börner, J., Corbera, E., Ezzine-de-Blas, D., Ferraro, P.J. et al. (2016) Mainstreaming impact evaluation in nature conservation. Conservation Letters, 9, 5864.
Bloom, H.S. (2008) The core analytics of randomized experiments for social research. In The SAGE Handbook of Social Research Methods (eds Alasuutari, P., Bickman, L. & Brannen, J.), pp. 115133. SAGE Publications Ltd, London, UK.
Bonell, C., Fletcher, A., Morton, M., Lorenc, T. & Moore, L. (2012) Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Social Science & Medicine, 75, 22992306.
Börner, J., Baylis, K., Corbera, E., Ezzine-de-Blas, D., Ferraro, P.J., Honey-Rosés, J. et al. (2016) Emerging evidence on the effectiveness of tropical forest conservation. PLOS ONE, 11, e0159152.
Börner, J., Baylis, K., Corbera, E., Ezzine-de-Blas, D., Honey-Rosés, J., Persson, U.M. & Wunder, S. (2017) The effectiveness of payments for environmental services. World Development, 96, 359374.
Bottazzi, P., Wiik, E., Crespo, D. & Jones, J.P.G. (2018) Payment for environmental ‘self-service’: exploring the links between farmers’ motivation and additionality in a conservation incentive programme in the Bolivian Andes. Ecological Economics, 150, 1123.
Brody, H. (2012) A critique of clinical equipoise. In The Ethical Challenges of Human Research (ed. Miller, F.), pp. 199216. Oxford University Press, New York, USA.
Bulte, E., Beekman, G., Di Falco, S., Hella, J. & Lei, P. (2014) Behavioral responses and the impact of new agricultural technologies: evidence from a double-blind field experiment in Tanzania. American Journal of Agricultural Economics, 96, 813830.
Butsic, V., Lewis, D.J., Radeloff, V.C., Baumann, M. & Kuemmerle, T. (2017) Quasi-experimental methods enable stronger inferences from observational data in ecology. Basic and Applied Ecology, 19, 110.
Cartwright, N. (2010) What are randomised controlled trials good for? Philosophical Studies, 147, 5970.
Chassang, S., Padró i Miquel, G. & Snowberg, E. (2012) Selective trials: a principal-agent approach to randomized controlled experiments. American Economic Review, 102, 12791309.
Chaves, W.A., Valle, D.R., Monroe, M.C., Wilkie, D.S., Sieving, K.E. & Sadowsky, B. (2018) Changing wild meat consumption: an experiment in the Central Amazon, Brazil. Conservation Letters, 11, e12391.
Conservation Evidence (2019) https://www.conservationevidence.com [accessed 28 January 2019].
Council of Economic Advisers (2014) Evaluation as a tool for improving federal programs. In Economic Report of the President, Together with the Annual Report of the Council of Economic Advisors, pp. 269298. U.S. Government Printing Office, Washington, DC, USA.
Curzon, H.F. & Kontoleon, A. (2016) From ignorance to evidence? The use of programme evaluation in conservation: evidence from a Delphi survey of conservation experts. Journal of Environmental Management, 180, 466475.
Deaton, A. (2010) Instruments, randomization, and learning about development. Journal of Economic Literature, 48, 424455.
Deaton, A. & Cartwright, N. (2018) Understanding and misunderstanding randomized controlled trials. Social Science & Medicine, 210, 221.
Donnelly, C.A., Woodroffe, R., Cox, D.R., Bourne, F.J., Cheeseman, C.L., Clifton-Hadley, R.S. et al. (2005) Positive and negative effects of widespread badger culling on tuberculosis in cattle. Nature, 439, 843846.
Ewers, R.M., Didham, R.K., Fahrig, L., Ferraz, G., Hector, A., Holt, R.D. et al. (2011) A large-scale forest fragmentation experiment: the stability of altered forest ecosystems project. Philosophical Transactions of the Royal Society B: Biological Sciences, 366, 3,2923,302.
Ferraro, P.J. & Hanauer, M.M. (2014) Advances in measuring the environmental and social impacts of environmental programs. Annual Review of Environment and Resources, 39, 495517.
Ferraro, P.J. & Pattanayak, S.K. (2006) Money for nothing? A call for empirical evaluation of biodiversity conservation investments. PLOS Biology, 4, e105.
Gillson, L., Biggs, H., Smit, I.P.J., Virah-Sawmy, M. & Rogers, K. (2019) Finding common ground between adaptive management and evidence-based approaches to biodiversity conservation. Trends in Ecology & Evolution, 34, 3144.
Glennerster, R. & Takavarasha, K. (2013) Running Randomized Evaluations: a Practical Guide. Princeton University Press, Princeton, USA.
Haynes, L., Service, O., Goldacre, B. & Torgerson, D. (2012) Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials. UK Government Cabinet Office Behavioural Insights Team, London, USA.
Independent Evaluation Group (2012) World Bank Group Impact Evaluations: Relevance and Effectiveness. World Bank Group, Washington, DC, USA.
Innovations for Poverty Action (2019) http://www.poverty-action.org [accessed 11 August 2019].
International Initiative for Impact Evaluation (2019) http://www.3ieimpact.org [accessed 11 August 2019].
Jayachandran, S., de Laat, J., Lambin, E.F., Stanton, C.Y., Audy, R. & Thomas, N.E. (2017) Cash for carbon: a randomized trial of payments for ecosystem services to reduce deforestation. Science, 357, 267273.
Kontoleon, A., Conteh, B., Bulte, E., List, J.A., Mokuwa, E., Richards, P. et al. (2016) The Impact of Conditional and Unconditional Transfers on Livelihoods and Conservation in Sierra Leone. 3ie Impact Evaluation Report 46, New Delhi, India.
Leigh, A. (2018) Randomistas: How Radical Researchers Are Changing Our World. Yale University Press, New Haven, USA.
Levitt, S.D. & List, J.A. (2011) Was there really a Hawthorne effect at the Hawthorne plant? An analysis of the original illumination experiments. American Economic Journal: Applied Economics, 3, 224238.
List, J.A. & Rasul, I. (2011) Field experiments in labor economics. In Handbook of Labor Economics (eds Ashenfelter, O. & Card, D.), pp. 104228. North Holland, Amsterdam, Netherlands.
McCarthy, M.A. & Possingham, H.P. (2007) Active adaptive management for conservation. Conservation Biology, 21, 956963.
McIntosh, E.J., Chapman, S., Kearney, S.G., Williams, B., Althor, G., Thorn, J.P.R. et al. (2018) Absence of evidence for the conservation outcomes of systematic conservation planning around the globe: a systematic map. Environmental Evidence, 7, 22.
Meals, D.W., Dressing, S.A. & Davenport, T.E. (2010) Lag time in water quality response to best management practices: a review. Journal of Environmental Quality, 39, 8596.
Michalopoulos, C., Bloom, H.S. & Hill, C.J. (2004) Can propensity-score methods match the findings from a random assignment evaluation of mandatory Welfare-to-Work Programs? Review of Economics and Statistics, 86, 156179.
Miteva, D.A., Pattanayak, S.K. & Ferraro, P.J. (2012) Evaluation of biodiversity policy instruments: what works and what doesn't? Oxford Review of Economic Policy, 28, 6992.
Pattanayak, S.K. (2009) Rough Guide to Impact Evaluation of Environmental and Development Programs. South Asian Network for Development and Environmental Economics, Kathmandu, Nepal.
Pressey, R.L., Weeks, R. & Gurney, G.G. (2017) From displacement activities to evidence-informed decisions in conservation. Biological Conservation, 212, 337348.
Pullin, A.S., Knight, T.M., Stone, D.A. & Charman, K. (2004) Do conservation managers use scientific evidence to support their decision-making? Biological Conservation, 119, 245252.
Pynegar, E.L. (2018) The use of randomised control trials in evaluating conservation interventions: the case of Watershared in the Bolivian Andes. PhD thesis, Bangor University, Bangor, UK.
Pynegar, E.L., Jones, J.P.G., Gibbons, J.M. & Asquith, N.M. (2018) The effectiveness of payments for ecosystem services at delivering improvements in water quality: lessons for experiments at the landscape scale. PeerJ, 6, e5753.
Rafidimanantsoa, H.P., Poudyal, M., Ramamonjisoa, B.S. & Jones, J.P.G. (2018) Mind the gap: the use of research in protected area management in Madagascar. Madagascar Conservation and Development, 13, 1524.
Rasolofoson, R.A., Ferraro, P.J., Jenkins, C.N. & Jones, J.P.G. (2015) Effectiveness of community forest management at reducing deforestation in Madagascar. Biological Conservation, 184, 271277.
Ravallion, M. (2009) Should the randomistas rule? The Economists’ Voice, 6, 812.
Rose, D.C., Sutherland, W.J., Amano, T., González-Varo, J.P., Robertson, R.J., Simmons, B.I. et al. (2018) The major barriers to evidence-informed conservation policy and possible solutions. Conservation Letters, 11, e12564.
Rosenthal, R. & Jacobson, L. (1968) Pygmalion in the classroom. The Urban Review, 3, 1620.
Rossi, P., Lipsey, M. & Freeman, H. (2004) Evaluation: a Systematic Approach. SAGE Publications, Thousand Oaks, USA.
Rundlöf, M., Andersson, G.K.S., Bommarco, R., Fries, I., Hederström, V., Herbertsson, L. et al. (2015) Seed coating with a neonicotinoid insecticide negatively affects wild bees. Nature, 521, 7780.
Samii, C., Lisiecki, M., Kulkarni, P., Paler, L. & Chavis, L. (2014) Effects of payment for environmental services (PES) on deforestation and poverty in low and middle income countries: a systematic review. Campbell Systematic Reviews, 2014, 11.
Saretsky, G. (1972) The OEO PC experiment and the John Henry effect. Phi Delta Kappan, 53, 579581.
Scriven, M. (2008) A summative evaluation of RCT methodology: and an alternative approach to causal research. Journal of Multidisciplinary Evaluation, 5, 1124.
Segan, D.B., Bottrill, M.C., Baxter, P.W.J. & Possingham, H.P. (2011) Using conservation evidence to guide management. Conservation Biology, 25, 200202.
Sutherland, W.J. & Wordley, C.F.R. (2017) Evidence complacency hampers conservation. Nature Ecology & Evolution, 1, 12151216.
The Collaboration for Environmental Evidence (2019) http://www.environmentalevidence.org [accessed 28 January 2019].
UNEG Impact Evaluation Task Force (2013) Impact Evaluation in UN Agency Evaluation Systems: Guidance on Selection, Planning and Management. United Nations, New York, USA.
USAID (2016) Evaluation: Learning From Experience: USAID Evaluation Policy. United States Agency for International Development, Washington, DC, USA.
Walsh, J.C., Dicks, L. V. & Sutherland, W.J. (2015) The effect of scientific evidence on conservation practitioners’ management decisions. Conservation Biology, 29, 8898.
White, H. (2013a) An introduction to the use of randomised control trials to evaluate development interventions. Journal of Development Effectiveness, 5, 3049.
White, H. (2013b) The use of mixed methods in randomized control trials. New Directions for Evaluation, 2013, 6173.
White, H. & Phillips, D. (2012) Addressing Attribution of Cause and Effect in Small N Impact Evaluations: Towards an Integrated Framework. International Initiative for Impact Evaluation, New Delhi, India.
WHO (2013) WHO Evaluation Practice Handbook. World Health Organization, Geneva, Switzerland.
Wiik, E., D'Annunzio, R., Pynegar, E.L., Crespo, D., Asquith, N.M. & Jones, J.P.G. (2019) Experimental evaluation of the impact of a payment for environmental services program on deforestation. Conservation Science and Practice, e8.
Woodcock, B.A., Bullock, J.M., Shore, R.F., Heard, M.S., Pereira, M.G., Redhead, J. et al. (2017) Country-specific effects of neonicotinoid pesticides on honey bees and wild bees. Science, 356, 13931395.

Keywords

What role should randomized control trials play in providing the evidence base for conservation?

  • Edwin L. Pynegar (a1), James M. Gibbons (a1), Nigel M. Asquith (a2) and Julia P. G. Jones (a1)

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed