The case for evidence-based policymaking in Washington
The most radical and surprising development in Washington policymaking today is the use of scientific research to determine what does or doesn’t work in government spending.
Basing spending on rigorous evaluation of a program’s results may seem self-evident, but it has never been a significant part of Washington’s work — until now. The Obama administration placed evidence-based policymaking at the heart of six major social initiatives covering job training, career education, K-12 education, teen pregnancy, and maternal and infant well-being, and the effort is beginning to bear fruit.
That’s the view of Ron Haskins of the Brookings Institution, a leading advocate of the approach and coauthor of what may become its handbook, “Show Me the Evidence,” a 2014 book examining the six initiatives. Haskins has solid Republican credentials as a former GOP counsel to the House Ways and Means Committee, and expertise in welfare reform and child care. But he praises the Obama White House for “attempting to change the culture of the entire field of social intervention” to bring scientific evaluation to the forefront.
That’s a daunting challenge, because even before a government program is established it must be backed by powerful constituencies. “Many programs were started with good intentions and supported by compelling anecdotes, but don’t deliver results,” wrote Peter Orszag, Obama’s former budget advisor, in 2009. “And once programs have been in place for a while, evaluating them rigorously becomes difficult.”
Haskins offers a “whimsical” pie chart displaying the factors that he believes influence legislation, including: political parties 8%, lobby groups 6%, the media 6% — and research 1%. Haskins acknowledges that research may never be the prime mover in social programs, but it should have more weight than it does now.
Evidence-based policymaking requires that every program be evaluated via random controlled trials to determine if the target population does better than a control group. If it does, that program can be expanded and replicated; if not, the trial helps identify shortcomings.
“In most cases, Congress tells a federal agency to allocate money to local agencies,” says Jon Baron, president of the nonprofit Coalition for Evidence-Based Policy, which maintains a roster of the most successful evidence-based social programs nationwide. “But which approaches are effective or ineffective plays little role in what gets funded.”
Evidence-based policymaking has a Republican pedigree and bipartisan support in Congress. The principle first gained currency in the Office of Management and Budget of the George W. Bush White House. Last November, then-Senate Budget Committee Chair Patty Murray (D-Wash.) and House Budget Committee Chair Paul D. Ryan (R-Wis.) introduced a bill creating a bipartisan commission to promote evidence-based policymaking.
Yet Capitol Hill Republicans have painted bull’s-eyes on all the Obama initiatives. They killed the Workforce Innovation Fund, which aimed to identify the most effective retraining programs for laid-off, young, welfare-dependent and disabled workers — and are poised to cut funding for another initiative, which supports innovative approaches in K-12 education. The largest and most successful initiative, which supports home-visiting programs for new low-income parents, had a “near-death experience” last year, Haskins says. Congress renewed its funding, but only to March 1.
The cuts, he says, point to a squandered opportunity. “You’d think that our employment and training programs would be much more effective than they are,” Haskins says. “At some point we have to figure that out and try and do a better job.”
Resistance to empirical evidence is particularly rife in government, Haskins observes: The “shibboleths by which many members of Congress operate (such as ‘Small government is best,’ ‘The IRS is evil,’ ‘Welfare destroys the incentive to work,’ ‘A rich country should not tolerate child poverty’) trump social science evidence on all but rare occasions.”
But the challenge of appraising results, especially when the results may undermine cherished assumptions, isn’t confined to government. Microsoft and Google have both found that 80% to 90% of their own new products or strategies end up having no significant effects, according to Jim Manzi, a software engineer and libertarian business analyst who contributed the foreword to Haskins’ book.
A few years ago, when scientists at the Thousand Oaks biotech firm Amgen double-checked the results of 53 landmark papers on which billions of dollars of investment had been based, they discovered that 47 of them couldn’t be proved valid.
The goal is not to use evaluations as a tool to kill underperforming programs outright, but to isolate and expand what works while discarding what doesn’t. Take the example of the Nurse-Family Partnership, which pairs registered nurses with pregnant mothers in underprivileged communities through the child’s second birthday.
The program has been hugely successful — among its California clientele, 92% of babies are born full-term, 91% are fully vaccinated by 24 months, and two-thirds of the mothers without a diploma or GED have earned or are working toward one. But some local programs had disappointing results. Years of randomized trials showed the problem — paraprofessionals they were sending into homes had less credibility than RNs with the expectant mothers.
“It takes time and resources to do this work well,” says David Olds, the University of Colorado pediatrics expert who founded NFP. “In the meantime, government policy can be pointed in the wrong direction and money wasted.”
Evidence-based policymaking may never be an easy sell in the social service community. Program administrators fear that it can be a tool to rationalize budget-cutting. “That would be the kiss of death,” Haskins acknowledges. “The attitude of federal agencies has to be, ‘We’re going to stick with you as long as you keep trying — we’re not taking your money away.’”
Michael Hiltzik’s column appears Sundays and Wednesdays. Read his blog, the Economy Hub, at latimes.com/business/hiltzik, reach him at mhiltzik@latimes.com, check out facebook.com/hiltzik and follow @hiltzikm on Twitter.
