Three Criteria for Evaluating Social Programs

Key to determining the success of public policies is the ability to measure costs and benefits. Did a program achieve its goals? Do those goals capture the totality of the program’s effects? Should the program be scaled? Does it need adjustments? Or are its costs so steep relative to its benefits that the program should be scuttled? 

Answers to these and related questions depend on the method of evaluation. Currently, many criteria are used to determine a program’s effectiveness, and this new research examines three: (i) the benefit-cost ratio or BCR; (ii) the net social benefit or NSB; and (iii) the marginal value of public funds or MVPF.

Before describing the authors’ evaluation of these three methodologies, it is useful to review the basic principles of social policy evaluation. The goal of these evaluation exercises is to define the benefits of a given project, discounted over its life. These benefits could include productivity-enhancing gains to project beneficiaries, as well as productivity spillovers to society at large, including both the values that non-participants place on those gains associated with “welfare weights” (or the marginal value that society puts on investing in the program), and the psychic (or subjective/non-monetary) gains to beneficiaries. This means that a policy which only directly benefits some may be valued by many not directly receiving benefits.

The challenge, then, is how to measure all these benefits over time, which can also include gains generated by the program (e.g., projects in the private sector activated by the program), as well as other social gains, and increases in tax revenue, among other factors. Complicating this task are the numerous programs competing for limited resources, some of which may be mutually exclusive (e.g., the same program at different scales), complementary, or independent. 

As for the three criteria evaluated in this paper, some put more or less weight on these and other factors. The authors’ evaluations of these criteria are broadly described here: 

Benefit-Cost Ratio (BCR) counts the full social benefits of a project, including expansions of the potential tax base. This is useful for evaluating one project only; however, it does not consider that project against alternatives. In other words, just because one project has a positive BCR does not mean it is the best choice; another project may perform even better. Further, BCR ignores the scale of net benefits. 

Marginal Value of Public Funds (MVPF) focuses attention on the revenues of the government raised by the program, which depend on institutional details regarding taxation. Please see the working paper for more details, but the upshot is that MVPF ignores a program’s scale, much like BCR. MVPF further ignores the welfare costs of taxation, meaning that a large MVPF could justify expenditure on programs generating low benefits. Consider, for example, a program that produces large government revenue that, on the other hand, might not offset direct costs, or that may not generate the greatest social benefits, including the social opportunity costs of public funds.

Net Social Benefit (NSB) is a measure of the net expansion or contraction of the social possibility frontier, where projects may vary in scale and content and, thus, result in varying costs and benefits. This feature is rarely captured by BCR or MVPF. The authors offer examples of studies that properly define NSB, including evaluations of early childhood education programs that can increase labor income, reduce criminal activity, and promote behaviors that improve the long-term health of their participants. These estimates include such net effects as a reduction in welfare costs due to lower costs of the criminal justice system (e.g., costs of imprisonment), private gains due to better health (e.g., better quality of life), and reductions in welfare costs of taxation due to lower costs for the public health system. 

Bottom line: Neither BCR nor MVPF adequately characterize the appropriate social criterion for evaluating policies, as they do not account for a project’s scale; MVPF also ignores the social cost of raising revenue to fund projects. The appropriate criterion is NSB, but it is rarely used because of the challenges of evaluating the full benefits and the true social costs of funding government expenditures.