Illinois Experiment Puts Teaching Methods to Test

As the nation’s public schools cast about for ways to bolster student achievement, a small group of affluent school districts in suburban Chicago is delivering some lessons on what it takes to improve.

The 18 districts on Chicago’s North Shore, already among the best in the country, have spent the last five years and well over $1 million trying to figure out how to get even better. And, if their experience is any guide, California may be heading down the wrong path by building its reform program around rapid improvements in test scores.

First, the consortium had its students in grades 4, 8 and 12 participate in an international math and science test against 41 countries. Then, educators spent years analyzing what, when and how they teach each topic and comparing what they do with the methods of those who bested them. Then they began looking last fall in painstaking detail at how their students answered hundreds of questions, combing the data for strengths, weaknesses and clues on how to improve.

Among their conclusions, so far: Assign less homework. Throw out the overhead projector. Teach lessons well the first time and stop reviewing them year after year. Give teachers more time away from the classroom to improve their skills. Videotape teachers’ lessons and critique their performance.

Many experts contend this is the sort of research and analysis that should underpin school reform across the country, at high performing rich schools as well as underachieving poor schools. But very few districts actually engage in serious research or are able to sustain such efforts.

The Illinois districts are learning that change is costly and time-consuming. And by focusing their efforts on long-term improvements, school officials risk testing the patience of their constituents.

Indeed, the man who has led the improvement effort over the last five years, Paul L. Kimmelman, is quitting at the end of this year. The reason, he says, is that he lost the support of the community.

“There are no quick fixes,” said David J. Kroeze, superintendent of Northbrook District 27, a member of what is now known as the “First in the World Consortium.” “You have to sit down with the data and talk about it and see what it tells you.”

The name for the consortium was not born out of arrogance, even though the schools--in Northbrook, Winnetka, Glenbrook and other North Shore communities--are among the best in the country. It began about six years ago when several superintendents from the area decided to pursue a goal set by the Bush administration: that U.S. schools be “first in the world” in math and science by 2000.

The group grew to 18 school districts and 37,000 students, and it used the Third International Math and Science Study to measure its success. The consortium had its students participate independent of the rest of the country, so it could gauge their performance separately.

To the chagrin of American educators, U.S. students in general did poorly, falling further in the standings the older they got.

Against that rather gloomy picture, though, the North Shore schools provided a tiny point of light. In fact, they outshone just about every country in the world in science at grades 4 and 8, and in math were behind only world-beating Singapore. The performance of 12th-graders was mixed, hot in math but cool in science.

Gratifying as that was, the school districts--under constant pressure from their communities to excel--did not intend to rest on their laurels.

“Who says you have to be doing poorly to want to improve?” said UCLA psychology professor James W. Stigler, who has been working with the schools.

In 1997, the consortium received computer disks containing thousands of statistical tables. The tables cross-referenced student performance, question by question, with dozens of factors.

“Now what do we do?” Kimmelman, superintendent of West Northfield District 31, recalled wondering.

What they did was create an unusual partnership with the North Central Regional Educational Lab, a federally funded research facility just down the road, in Oak Brook, Ill. Ever since, researchers there have been helping networks of teachers, as well as administrators, examine the data. They secured more than $1 million in federal grants to help them capitalize on the information they had.

A key element of the data is a question-by-question breakdown of how the performance of consortium students compares with that of students nationally and internationally.

The standardized tests used by California and other states produce far less specific information--how well a school’s students read compared with national averages, for example. The specificity is limited because the questions and answers are kept secret so the tests can be used for several years.

Teachers and parents cannot know how their children did on particular questions or even in subject areas such as multiplying fractions or working with perimeters.

As a result, the Stanford 9, the guidepost for California’s multibillion-dollar school reform drive, is not very useful in suggesting how to improve learning. Instead, it is used largely as a scorecard for charting progress. Critics say the emphasis on raw scores leads to superficial reforms and short-term gains.

Although many educators today talk about using evidence, rather than hunches, to inform their decisions, few have the capacity to do so.

Mike Schmoker, a leading advocate of the use of achievement data to guide improvement, has written that schools in general “have an almost cultural and ingrained aversion to reckoning with, much less living by, results.” Educators, he writes, suffer from a “deep and debilitating confusion about how means relate to ends.”

Researchers, meanwhile, often have little understanding of the realities of classrooms.

“We were saying to teachers, ‘What do you need to know to help your students do better?’ ” said Gina Burkhardt, the lab’s executive director. “Our goal was to tease apart what makes a difference in teaching and learning.”

Math teacher Sue Winski, a 22-year veteran, got involved with one of the teachers’ networks studying the results. The group viewed videotapes of teachers from Japan and Germany.

“We did really well on TIMSS [the international study] so it has to be that there were good things here,” said Winski, who teaches eighth-graders at Field Junior High School in Northbrook. “But there were things that we were not so good at. So, I had to think, ‘What could I do to improve?’ ”

That kind of reflection has changed her teaching in ways small and profound. She’s tossed out the overhead projector. Instead, she uses the whole chalkboard. That allows her lessons to unfold as one connected story, instead of being flashed a snippet at a time on a pull-down screen.

More important, she has de-emphasized the kind of repetitive homework assignments typical of the United States. Higher-scoring countries tend to assign less homework. That frees up classroom time previously spent correcting homework, time she uses to try to engage students in thinking.

But Winski said it’s hard to change, especially for veterans like her. That’s clear from a journal she kept for two years, during which she scrutinized her lessons for what worked and what didn’t.

Kimmelman, who heads the district where Winski teaches, exults in such stories. Not only is the consortium project leading to classroom changes, he said, but teachers are leading the way by using test data to study issues of content and teaching.

Making the leap from the conference room to the classroom has always been difficult for American public education. And never more so than today. Policymakers have laid out their expectations for what students should know, but many critics say not enough is being done to help teachers meet those expectations.

The research underway in Illinois is aimed at helping teachers connect those dots.

Kroeze, the Northbrook superintendent, recently spent the day with analysts from the lab mulling over fourth-grade math questions.

He found that consortium fourth-graders scored below the international average in subtracting 2,369 from 6,000. Consortium students also scored far below the best countries on a question requiring a knowledge of place value.

Teachers studying the results were puzzled. “Given that we have been teaching this concept since first grade, we should have done better,” they wrote in a commentary.

That type of detailed analysis will eventually be used by each of the member school districts to shape teacher training, textbook selection and lesson plans.

But teachers aren’t waiting. Jon Bogie and his fellow teachers at Winkelman Elementary in Northbrook began meeting weekly last fall with the goal of deeply examining how they taught fractions, decimals and percentages. The district pays them for their time.

They narrowed their focus to equivalent fractions--a rough spot for consortium students--recasting it as a mystery hunt.

In March, Bogie taught that lesson in front of two video cameras. A UCLA researcher who has studied videos from other nations analyzed his performance--giving him generally good marks but suggesting that he could have done a better job of soliciting student participation.

Bogie said the exercise “improved my teaching tremendously.”

Now, he tries to ask questions differently, to draw out students’ understandings and misunderstandings of what he’s trying to teach. He works to stay on message, to keep coming back to the theme of the day’s lesson.

William H. Schmidt, a Michigan State University professor who oversaw the U.S. participation in the Third International Math and Science Study, has worked closely with the consortium districts.

“They’ve really dug deeply into this,” he said. “They did not just do it for the horse race. They’re on a track that would serve as a model for the country as to how you use research to improve practice.”

Earlier this year, the eighth-grade portion of the international math and science test was re-administered, and consortium students once again took part. The group’s leaders are eager to find out if their efforts over the last four years are paying off.

But one person who won’t be around then is Kimmelman. He will retire soon at the age of 55.

Although the project has inspired educators across the nation, it failed to impress the local community.

Academic achievement is almost taken for granted by the community. But many became concerned about rising property taxes. Three times Kimmelman asked voters to approve a tax hike to pay for a new school to accommodate growth. And three times they turned him down.

“My heart and soul has been in this project for over five years now,” Kimmelman said. “It has basically consumed me.”

He said he was so involved that he didn’t spend enough time selling the community on its value.

“You just have to be willing to be focused and to take all the negatives that go with it,” he said. “But you don’t look back. If you believe American kids should have the best education in the world, you get that passion, you drive it and you don’t let it go.”

* SCHOOLS CHIEF SEARCH

Two candidates to lead the L.A. school district pulled themselves out of contention. B1