3.1 Providing comprehensive assessments of the cost of social programs

Accurate and comprehensive estimates of social program costs are foundational to high-quality BCAs, and they have practical importance in their own right. Information about resource requirements and related costs, as well as the timing of investments, can help ensure that effects documented in demonstration trials are realized in subsequent real world implementation. They can also help program implementers choose programs that fit within their budget constraints and match their capacity for implementation. In spite of their utility, as Long and coauthors point out (Long et al., Reference Long, Brown, Jones, Aber and Yates2015) in “Cost Analysis of a School-Based Social and Emotional Learning and Literacy Intervention,” widely accepted standards for conducting cost analyses of social programs do not exist (Vining & Weimer, Reference Vining and Weimer2010). Reliance on budgetary data in some studies and actual resource use and related costs in others, different assumptions about the opportunity cost of volunteer time and other donated resources, and whether training and technical assistance are included in the analysis are some of the sources of variability across studies.

Comprehensive understanding of the full economic costs of implementing a program generally involves going beyond budgetary data to actual expenditures as well as the opportunity cost of donated resources common to social programs, such as volunteer time and program space (Karoly, Reference Karoly2012). Earlier prevention and intervention cost analyses have helped illustrate a variety of issues that need to be considered in thorough, high-quality cost analyses. These include assessing how intervention activities and associated costs vary over the life of a program (Crowley, Jones, Greenberg, Feinberg & Spoth, Reference Crowley, Jones, Greenberg, Feinberg and Spoth2012), the costs of training and technical assistance that support program efficacy (Kuklinski, Briney, Hawkins & Catalano, Reference Kuklinski, Briney, Hawkins and Catalano2012), and site-level variability in costs in multisite trials (Corso et al., Reference Corso, Ingels, Kogan, Foster, Chen and Brody2013).

The paper by Long and colleagues (2015) presents a cost analysis of the 4Rs (Reading, Writing, Respect, and Resolution), a universal, school-based social and emotional learning (SEL) program delivered in nine elementary schools. The study, which is part of a larger impact analysis, shows how two major approaches to estimating social program costs, the cost–procedures–processes–outcomes analysis (Yates, Delany & Dillard, Reference Yates2009) and ingredients method (Levin & McEwan, Reference Levin and McEwan2001), can be jointly applied to develop accurate and informative program cost estimates. The authors provide methodological detail and a variety of cost information, including total and per-student costs and variability in costs across schools and over time. With the rise in SEL interventions, Long et al.’s (Reference Long, Brown, Jones, Aber and Yates2015) analysis is likely to have utility in guiding the estimation of the costs of these interventions and helping SEL program implementers choose among alternative programs.

3.2 Applying benefit-cost analysis from a societal perspective to prevention and intervention programs

The papers by Bowden and Belfield (Reference Bowden and Belfield2015) and Cohen and Piquero (Reference Cohen and Piquero2015) represent classic applications of BCA to social programs. Each frames BCA within a societal perspective, assessing whether investment in programs serving at-risk youth or young adults pays off in the form of economic benefits over time that exceed costs. In “Evaluating the Talent Search TRIO Program: A Benefit-Cost Analysis and Cost-Effectiveness Analysis,” Bowden and Belfield (Reference Bowden and Belfield2015) draw attention to Talent Search, one of three related TRIO programs (Talent Search, Upward Bound, and Student Support Services) established in the 1960s as part of the War on Poverty. As Bowden and Belfield (Reference Bowden and Belfield2015) report, the programs received $768 million in federal funding in 2013–14 to support over 760,000 low-income students with the goal of increasing college attendance. Although they have a long history, these programs have not been subjected to economic evaluation. Bowden and Belfield (Reference Bowden and Belfield2015) amend the situation with their analysis of Talent Search, providing timely information that bears on debates over funding for TRIO related to reauthorization of the Higher Education Act (Perna, Reference Perna2015).

In contrast to BCA analyses that draw on findings from efficacy trials (Herman et al., Reference Herman, Mahrer, Wolchik, Porter, Jones and Sandler2015; Kuklinski et al., Reference Kuklinski, Fagan, Hawkins, Briney and Catalano2015), the rich analysis of Talent Search is one of few to apply BCA to an intervention conducted at a much greater scale. The paper is instructive in addressing several issues attendant in using a sample of intervention sites from two different states to derive overall economic estimates, including differences in costs, intervention emphasis, and impact. Bowden and Belfield (Reference Bowden and Belfield2015) illustrate their application of the ingredients method to assess costs, and show how existing models of lifetime economic gains from increasing educational attainment (high school and college) can be harnessed for their analysis. Breakeven analysis as well as sensitivity of BCA results in relation to differences in labor productivity growth, ability, and the discount rate are additional strengths of this paper, as is the authors’ exploration of variability in cost effectiveness across sites.

In “Benefits and Costs of a Targeted Intervention Program for Youthful Offenders: The YouthBuild USA Offender Project,” Cohen and Piquero (Reference Cohen and Piquero2015) shift focus to an intervention for youthful offenders. Targeting recent juvenile or young adult offenders, YouthBuild USA Offender Project (YBOP) participants were diverted from incarceration or had been referred after serving time in jail or prison. Participants were integrated into the larger YouthBuild program, acommunity-based job training and education intervention for at-risk low-income young adults. Participants gain construction skills that are applied to low-income housing projects, focus on obtaining high school diplomas or General Educational Development (GED) certificates, receive counseling support, and become part of a larger YouthBuild community. Over its 20-year history, YouthBuild has served over 130,000 students in the United States and has expanded to 15 different countries (youthbuild.org/our-impact).

Like Bowden and Belfield (Reference Bowden and Belfield2015), Cohen and Piquero (Reference Cohen and Piquero2015) apply economic evaluation to a social program that has been offered at large scale in comparison to many other prevention and intervention programs. Using a sample of 388 participants, they conduct a careful assessment of the societal level costs and benefits of YBOP. The latter follow from reduced recidivism and increased educational attainment among participants in relation to relevant comparison cohorts. Cohen and Piquero’s analysis is strong on several fronts, including its comprehensive look at costs of the program as well as the long-term value of reducing crime for offenders with at least one prior conviction. Moreover, it illustrates how two comparison cohorts, the National Longitudinal Survey of Youth and the Philadelphia Birth Cohort, were effectively enlisted in the analysis because YBOP had not been evaluated in a randomized controlled trial. Cohen and Piquero’s (Reference Cohen and Piquero2015) reasoning about the relevance of these cohorts, and their incorporation of breakeven analysis, offset concerns about possible selection bias and support conclusions reached.

These two examples of social program BCA demonstrate the substantial economic gains that can be anticipated from programs that effectively increase educational attainment and/or reduce crime. They benefit from several prior studies indicating lifetime economic gains from increasing high school and college graduation and estimating savings over a lifetime from reducing offending (e.g., Cohen & Piquero, Reference Cohen and Piquero2009; Heckman, Humphries, Veramendi & Urzúa, Reference Heckman, Humphries, Veramendi and Urzúa2014; Trostel, Reference Trostel2010). Moreover, these papers highlight the use of a variety of statistical methods to conduct economic analyses when evidence from randomized controlled trials is not available. Finally, in their focus on programs that have been implemented at a larger scale, results of these analyses are relevant to decisions to invest broadly in these programs.

3.3 Expanding the frontiers of benefit-cost analysis of prevention and intervention programs: Monetizing SEL outcomes

Three of the six papers in this volume address SEL or noncognitive outcomes in some manner (Belfield et al., Reference Belfield, Bowden, Klapp, Levin, Shand and Zander2015; Jones et al., Reference Jones, Karoly, Crowley and Greenberg2015; Long et al., Reference Long, Brown, Jones, Aber and Yates2015), a testament to the growing recognition that these outcomes are important predictors of positive development. Whereas Long and colleagues focus on the cost of an SEL intervention, Jones et al. (Reference Jones, Karoly, Crowley and Greenberg2015) and Belfield et al. (Reference Belfield, Bowden, Klapp, Levin, Shand and Zander2015) directly tackle the need to expand benefits models to directly incorporate noncognitive outcomes.

In “Considering Valuation of Non-Cognitive Skills in Benefit-Cost Analysis of Programs for Children,” Jones and colleagues (Reference Long, Brown, Jones, Aber and Yates2015) argue convincingly that the present limitations in valuing the noncognitive skills that are the focus of many prevention and intervention programs for children and youth impede thorough economic evaluation of these programs. Although these outcomes can be subjected to cost-effectiveness analysis, programs may be at a disadvantage in policy decisions that seek support from monetized outcomes. Moreover, when noncognitive outcomes are among several areas of impact, BCA becomes a desirable method because of its capacity for summing across monetized outcomes to estimate cumulative intervention impact.

Jones et al.’s (Reference Jones, Karoly, Crowley and Greenberg2015) study provides an in-depth review of the state of valuation of noncognitive skills. The paper begins with a summary of frameworks for characterizing noncognitive skills, considers measurement-related issues, reviews BCAs that have incorporated shadow prices of noncognitive skills, and describes research needed to facilitate valuation. This paper provides a conceptual roadmap that will be useful to others doing economic evaluation in this rapidly advancing area of intervention development and research.

Belfield et al. (Reference Belfield, Bowden, Klapp, Levin, Shand and Zander2015) echo the call for benefit-cost analysis to be expanded to incorporate SEL outcomes in “The Economic Value of Social and Emotional Learning.” After proposing an approach for establishing the benefits of social and emotional (SE) skills, they demonstrate how the approach can be applied to BCAs of four SEL interventions. As Belfield and coauthors point out, their goal is not to rank-order the interventions in terms of their efficiency gains; the latter would not make sense as the interventions have different emphases and impacts and therefore different benefits streams (as well as different cost structures). Rather, their intriguing paper demonstrates that, although further research to develop and validate shadow prices for SE skills is needed, their incorporation into BCAs leads to more comprehensive estimates of benefits. Under certain assumptions, these benefits are substantial. Taken together, Belfield et al. and Jones et al.’s papers identify many important questions that need to be addressed to move research in this area forward, and they also provide direction on how such advances can be achieved.

3.4 Using benefit-cost analysis to inform PFS financing

The last paper in the series, “Using Cost-Benefit Analysis to Scale up Early Childhood Programs Through Pay for Success Financing,” by Temple and Reynolds (Reference Temple and Reynolds2015), describes the role of BCA in social impact investing, an innovative mechanism for raising capital to expand the reach of effective prevention programs. Even when BCAs demonstrate that social programs are favorable investments that generate net benefits to society, fiscally constrained governments are limited in the programs they can support. Social impact bonds (SIBs), also known as “pay for success” (PFS) contracts, offer an alternative source of funds. Under this arrangement, private investors provide the capital to scale prevention programs, and government cost savings from successful program outcomes are used to repay investors.

The first arrangement of this kind was made between the United Kingdom and a nonprofit organization known as Social Finance, which raised $5 million in private capital to provide start-up funding to prevent juvenile reoffending (Crowley, Reference Crowley2014). Investors were to be repaid from avoided governmental expenditures if recidivism declined by 7.5%, including a bonus payment if recidivism dropped by 8% or more. Since that time, a number of PFS contracts have been established in the United States to expand prevention programs at a time when, as Temple and Reynolds (Reference Temple and Reynolds2015) report, political and economic pressures inhibit the use of tax revenues for the initial investment. They include contracts to prevent asthma (Clay, Reference Clay2013), reduce preterm births, prevent diabetes (Galloway, Reference Galloway2014), reduce recidivism (Butler, Bloom & Rudd, Reference Butler, Bloom and Rudd2013), serve homeless mothers with children in the foster care system (Greenblatt & Donovan, Reference Greenblatt and Donovan2013), and prevent special education placement (Temple & Reynolds, Reference Temple and Reynolds2015; Warner, Reference Warner2013).

With the promise of reducing pressures on the government to fund new programs, PFS has gained the attention of the Obama Administration, the U.S. Congress, and several states. For example, in 2012, the Obama Administration announced PFS as a major part of its social innovation strategy (Greenblatt, Reference Greenblatt2014). In 2014, Congress passed legislation to increase funding for PFS contracts (https://www.govtrack.us/congress/bills/113/hr4660), and the Department of Housing and Urban Development currently has a request for grant applications for PFS permanent supportive housing projects (www.pingree.house.gov/grant-opportunities/pay-success-permanent-supportive-housing-demonstration-deadline-feb-12-2016). Temple and Reynolds (Reference Temple and Reynolds2015) describe efforts in Utah and Colorado to support PFS contracts aimed at reducing special education placement.

Though attractive to the public sector and to those who want to expand the reach of prevention, PFS is not yet a proven mechanism for funding social programs. However, as Temple and Reynolds (Reference Temple and Reynolds2015) argue, prevention efficacy research and economic evaluation, specifically BCA, can play crucial roles in identifying viable candidates for PFS contracts. Research providing strong evidence for prevention program effects on behavioral health outcomes can lay a foundation for economic analysis documenting the timing and extent of anticipated economic gains. Temple and Reynolds (Reference Temple and Reynolds2015) show how research establishing the efficacy of Chicago Child Parent Centers, which was evaluated in the Chicago Longitudinal Study, and related benefit-cost analysis (Reynolds, Temple, White, Ou & Robertson, Reference Reynolds, Temple, White, Ou and Robertson2011), contributed to the development of a social impact borrowing contract to scale the program. Their paper illuminates several important issues requiring careful consideration for PFS contracts to be successful, including strong economic models estimating the magnitude and timing of benefits anticipated from prevention, the possibility of perverse incentives, need for evaluation by a neutral third party, and the role of foundations in reducing private investors’ exposure to risk in these contracts.

Temple and Reynolds (Reference Temple and Reynolds2015) also illuminate important differences in BCA as applied to PFS, which focus on cost savings to the government sector, and BCA conducted from a broader, societal perspective. For example, a program that may not be attractive from a PFS perspective may be attractive when viewed from a societal perspective. Moreover, as BCAs performed by WSIPP suggest, governments may be interested in more than the benefits to taxpayers that are central to PFS contracts (Lee & Aos, Reference Lee and Aos2011); WSIPP models, for example, go beyond taxpayer impacts to include quality of life gains from reducing victimization when crime is reduced. Their work suggests that the interests of PFS investors and governments may not fully overlap. Temple and Reynolds’ (Reference Temple and Reynolds2015) in-depth look at the use of BCA to support funding at scale of the Chicago Child Parent Centers intervention illuminates strengths and concerns in the use of PFS to scale prevention.

4.1 Common themes

Beyond their individual contributions, the collection of papers in the special issue suggest three themes pertinent to the larger context of evidence-based practice and policy as informed by economic evaluation.

Comprehensive analyses, that is, societal perspectives on benefits and costs, provide the optimal set of information for program implementers and decision makers. Societal perspectives on both benefits and costs ensure that decisions are made from the most informed position possible. Although values, political pressures, and other considerations may mean that some subperspectives (e.g., taxpayers) are given more weight than others, the societal perspective helps ensure that all perspectives can enter the decision-making process. On the cost side, comprehensive assessments of resources and related costs facilitate understanding of what is needed for high-quality program implementation that makes achieving outcomes and realizing economic gains more likely. Comprehensive assessments of costs, outlining where the major cost drivers lie, can also stimulate research and program refinements to increase resource and economic efficiency. On the benefit side, failure to account for all perspectives or to consider possible economic gains from important outcomes can result in downward bias to net present values, underselling the value of prevention and intervention programs. Although attaching monetary values to all economic outcomes may be the goal, Bowden and Belfield (Reference Bowden and Belfield2015) take a viable, conceptual approach; having established positive net benefit from gains in educational attainment, they acknowledge but do not estimate the health gains and reductions in crime that would make their results even more favorable.

In order for analyses to be comprehensive, shadow prices for noncognitive, social and emotional, and other “soft” skills that are common outcomes of prevention and intervention programs for youth need to be developed; shadow prices can establish the unique economic contribution of these outcomes, beyond what cognitive outcomes suggest. Many social programs target young children, long before they are independent members of society and part of the workforce. These programs really are investments, made to enhance the well-being and positive development of these young people over time. Current economic models can account for some of the economic gains and avoided costs of impactful programs. As youth development research increasingly recognizes the importance of noncognitive factors to positive development, including developmental outcomes with economic impact (e.g., educational attainment, substance use, delinquency and crime, and health), social program benefits models will need to expand if they are to be truly comprehensive. Such efforts are being supported by the RAND Corporation’s Valuing Outcomes of Social Programs (VOSP) project that is archiving estimates of such value from across the literature.

Benefit-cost analyses of prevention and intervention programs can help stimulate investment in these programs in at least three important ways. First, results of benefit-cost analyses, particularly net present values and related confidence intervals, can help identify social programs that represent an overall welfare gain to society, that is, those that are good investments of limited resources. As analyses by WSIPP and others have shown (http://wsipp.wa.gov/BenefitCost?topicId=9), not all effective programs will pass the benefit-cost test; although they may be impactful, some programs are not necessarily good public investments. Second, from an implementation standpoint, detailed information about costs, and to some extent benefits, can help schools, social service organizations, and community-based agencies understand which programs are best suited to their goals for reaching youth and young adults, capacity to implement them well, and budgetary constraints. Third, information about anticipated benefits streams, particularly fiscal impacts to governments, can facilitate the development of public–private–foundation partnerships to finance the implementation of evidence-based programs at scale.

4.2 Existing efforts to advance social program benefit-cost analysis

These themes reverberate in efforts, several currently under way, to increase the quality, comparability, and relevance of benefit-cost analysis of social programs.

Setting standards. Standards guiding BCAs of prevention and intervention programs could support quality in analyses as well as comparisons across analyses. When investment decisions involve choosing among alternative programs, economic evidence is most helpful if it can be compared in meaningful ways (Olson & Bogard, Reference Olson and Bogard2014). Papers by Karoly (Reference Karoly2012) and Vining and Weimer (Reference Vining and Weimer2010), originally part of a broader set of standards papers commissioned by the Society for Benefit-Cost Analysis, address the need for quality and standards. The Society for Prevention Research has also commissioned a task force charged with setting standards for BCAs of preventive interventions (Society for Prevention Research, 2014). Their report is due to be published in 2016.

Enhancing quality, utility, and use. Although benefit-cost analysis and economic evaluation more generally have practical implications for program implementers and those who make investment decisions, the divide between the production of these analyses and their actual use can be great. As several studies have shown (Oliver, Innvar, Lorenc, Woodman & James, Reference Oliver, Innvar, Lorenc, Woodman and James2014; Prewitt, Schwandt & Straf, Reference Prewitt, Schwandt and Straf2012), policy makers may not have timely access to high-quality research evidence, may not see it as relevant, and may not have the collaborations or relationships with researchers that build confidence in the data. The economic analyses performed by WSIPP at the direction of the Washington State legislature and the work being undertaken through the Results First initiative (Lee & Aos, Reference Lee and Aos2011; White & VanLandingham, Reference White and VanLandingham2015) are notable exceptions. To address the need for high-quality and useful economic evidence, the National Academy of Medicine has convened a consensus panel charged with improving the use of economic analysis to inform policy and investments for children, youth, and families (Institute of Medicine, 2014). Its final report and recommendations are also due in 2016.

Stimulating investments. As the paper by Temple and Reynolds (Reference Temple and Reynolds2015) illustrates, fiscal pressures across all levels of government have motivated the development of creative ways to finance the scaling up of effective prevention and intervention programs. In addition to providing additional funds, these “pay for success” or social impact bond arrangements typically shift risk away from government toward private investors and foundations. Benefit-cost analysis outlining the magnitude and timing of benefits, particularly fiscal benefits, have an important role to play in informing the terms of these contracts. The interest in PFS has led to the development of resources supporting the forging of these contracts, including the Nonprofit Finance Fund (payforsuccess.org), Institute for Child Success (http://www.instituteforchildsuccess.org/), and the Government Performance Lab at Harvard University Kennedy School of Government (http://siblab.hks.harvard.edu/).

Forging partnerships to build and utilize the economic evidence base. As economic evaluation and BCAs of social programs continue to grow, so too does the range of questions and issues addressed as well as their application to decision making (Crowley et al., Reference Crowley, Jones, Greenberg, Feinberg and Spoth2012). The Prevention Economics Planning and Research Network supported by the National Institutes of Health brings together prevention and intervention researchers from across the country to increase the use of benefit-cost analyses and build the science of investing in healthy development. This open network of economists, prevention scientists, and policy analysts provides support and coordination for efforts to strengthen methodology and accelerate efforts to understand the economic and fiscal impact of social programs. This includes projects around valuing proximal program outcomes for long-term projection (e.g., shadow prices, monetary conversion factors), accessing administrative data systems, and supporting rigor in performance-based financing (e.g., PFS). Importantly, this network actively engages end users to increase the utility of estimates—briefing Congress and engaging federal agencies on best practice and new findings.