An Expert Cautions State to Evaluate Innovations Before Issuing Edicts

A proposal gaining fans in Sacramento seems, at first, to be so modest as to be unworthy of comment: Prior to mandating or promoting any educational innovation, the state ought to evaluate the scientific evidence to make sure it will benefit students.

Big deal, right? That’s the way things are anyway, aren’t they? Well, in fact, the answer is no. And University of Oregon educational psychologist Douglas Carnine is quietly campaigning to see that change.

Carnine’s suggestion: The state should convene a panel of scientists to objectively evaluate existing research before risking millions of dollars and the educational success of students promoting new programs.

“In education, innovation can be something totally wild, and there’s nothing to stop it from spreading like wildfire, and there are no consequences if it doesn’t work,” said Carnine, who directs the National Center to Improve the Tools of Educators.

*

In medicine or engineering, innovations must be tested to make sure they are beneficial and not merely changes made for the sake of change. Research is conducted, published in professional journals, debated at conferences and, ultimately, put into action by practitioners. Innovators who ignore current research flirt with malpractice lawsuits, denunciation by their peers and loss of their license to practice.

Education, Carnine said, fails to complete that loop. Researchers publish their work in journals and debate it at conferences. But often that research is of questionable value, relying on case studies or anecdotes aimed at promoting a point of view, rather than on hard evidence of improved student achievement.

When valid research does exist, education has no mechanism for ensuring that such information influences practice. In the art of teaching, philosophy, rather than science, dominates, leaving education without an agreed-upon body of knowledge that all teachers are required to know and use. Lacking that, courts have ruled, teachers and school systems cannot be sued for malpractice.

The failure to discriminate between mere innovation and true reform sets in motion extreme, pendulum-like swings in educational policy, Carnine said. Ever since 1900, for example, math education has vacillated between an emphasis on the rote learning of basic facts and a focus on higher-order thinking and comprehension. Other subjects, such as social studies, have alternately stressed, on the one hand, dates and facts, and on the other, values and moral issues.

As a result, teachers are jerked from one method to another and ordered by the reigning curriculum experts in each approach to get on board.

*

If education were rational, Carnine said, policymakers would look at schools that are working--where, for example, all children are learning to read--and authorize research to find out why. Approaches that worked would be replicated.

Setting such a course will not be easy. But Carnine has persuaded legislators in California, Texas and Wisconsin to at least consider his idea for how it might be attempted.

He is suggesting that the state set up an independent, credible institute of scientists, similar to the National Academy of Sciences, to review, for example, the evidence surrounding school reforms, such as the educational impact of smaller class sizes. The institute would act as a sort of Consumer Reports, evaluating the likely impact on students of proposed policy changes.

“There would be very clear rules about what constitutes evidence,” Carnine said. “In many areas of education we don’t know very much . . . but even that would be a huge step forward, to be clear about what we know and don’t know. And everyone should be very cautious about issuing mandates in areas about which we know very little.”

Carnine was appointed last year by state Supt. of Public Instruction Delaine Eastin to a panel studying ways to prevent students from failing in school. But rather than coming up with a list of programs, the group’s draft report recommended that the state slow down and establish criteria for determining which existing intervention programs are working. Its final report is not yet out.

*

“We admit that more often than not we have succumbed to the enchantment that accompanies any new strategies and approaches, and the enticement of educational gurus and publishers who have marketed quick fixes in fancy packages, which may or may not be based on validated research,” the report said.

State Sen. Bill Leonard (R-San Bernardino), for one, likes the idea of having the state focus on evaluating policies rather than mandating approaches, then seeking compliance, which it tends to do now.

“The state’s role should identify programs that hold promise, spend money to research problems . . . announce the results and then get out of the way,” Leonard said. “If we could get the state Department of Education to be more of a research laboratory and collect the data that’s so scarce, then help districts make choices with good information, we’d all be better off.”