A New Formula for Teaching Math

This fall, the Third International Mathematics and Science Study--the largest comparison ever made of student performance, involving 750,000 pupils in 50 countries--will release results that experts expect to show, as earlier studies have, that U.S. pupils lag behind those of most other developed nations.

But this time, U.S. education officials are hoping the study leads to something more productive than collective self-flagellation and casting of blame.

“This is not just another Olympics to determine who is first in math or physics but more to understand what these cross-cultural comparisons are all about,” said William H. Schmidt, a Michigan State University professor who is directing the U.S. arm of the study.

As part of that effort, UCLA psychology professor James Stigler has received a $2-million contract from the National Center for Education Statistics to create something that has never been attempted--a video portrait of how eighth-grade math is taught in the United States, Japan and Germany.

Based on hundreds of hours of lessons delivered by randomly selected instructors, the videos capture the work of teachers brilliant and dull alike. The portraits are intended to make it possible to understand differences in the knowledge, behavior and expectations of teachers, which might help explain the variance in student performance in the three countries.

The student data, as well as the analysis of the teachers’ lessons, are being kept under tight security. But excitement is building over the potential value of the video portraits for researchers and classroom teachers.

“This study just seems to hold such tremendous promise for actually being able to see what you are talking about when you talk about differences in teaching, differences in questioning techniques, just a wealth of things,” said Alice Gill, associate director of education issues for the American Federation of Teachers and an advisor to Stigler’s study.

“In the past, we looked at outcomes, at what kids do . . . rather than at what the teaching looks like. This is a very different perspective.”

Before being able to do that, however, Stigler and his team had to devise a research tool. They had to develop a way to handle the videotapes that would allow the researchers to treat them as a richly textured, real-life database rather than a series of scenes.

They had to find a way to crunch the videotape just as computer accounting software crunches numbers, to allow them, for example, to quantify how often a teacher assigns a problem or hands out a work sheet of practice equations. More than that, they wanted to be able to call up video of each group problem-solving scene to see how different teachers handled it.

“In the past, if you had hundreds of hours of videotape you were sunk,” Stigler said, because the tapes were so unwieldy they were virtually useless.

To solve the problem, Stigler transferred the videotape onto CD-ROMs, in digital form. Then, with the help of a public-private software company created just for this purpose, researchers developed video analysis software that allows users to instantly call up snippets of action based on keyword searches or coding--just as librarians can now search text.

When the portraits are available--perhaps via the Internet--a researcher can see the classroom scene, accompanied by a real-time transcript of what is being said and what is occurring.

So, for example, a researcher could use the software to compare the frequency with which teachers are interrupted by disruptive students in the three countries. Or, they could examine how teachers start lessons or how often they give quizzes.

And the technology could have applications in areas outside education--for market researchers conducting focus groups, for example, or for lawyers using videotaped depositions instead of transcripts.

“It turns out to be a really revolutionary idea and I think it will change the way we look at video,” he said. “This allows a new kind of analysis that wasn’t possible before.”

Gaea Leinhardt, a senior scientist at the Learning Research and Development Center at the University of Pittsburgh, agreed that the software developed by Stigler’s team holds great promise.

Now, Leinhardt videotapes highly accomplished teachers for days or weeks at a time, laboriously transcribing and analyzing what they say. She said Stigler’s software had never been available and is “exactly the right and neat thing to do.”

But, she warned, even videotapes do not provide as much information as actually being in classrooms. The camera excludes what’s going on outside its frame. And when researchers try to categorize the multilayered scenes of a classroom in full swing, they may miss or misinterpret subtleties, she said. A teacher seeming to praise a student, for example, may actually be doing so sarcastically. But the interaction might be recorded for analysis purposes as true praise.

“The danger is, of course, that the very appearance of tremendous precision suggests that you really know what’s going on, when you really don’t,” she said.

But Stigler said the computer software actually reduces the possibility of misunderstandings. Whenever researchers working with the material in the future have any doubts, they can call up the actual scenes and look at them.

The portraits will be valuable for other reasons as well. They will show the degree to which math teachers have embraced the reforms espoused by the National Council of Teachers of Mathematics, reforms that emphasize problem-solving, group work and deep understanding.

Such ideas have been hotly debated in California and elsewhere on a theoretical basis--with some parents, teachers and academics saying that basic skills are being ignored.

But Stigler said the videotapes show that most teachers are not even venturing to incorporate the reforms into their classrooms--even if they say they are. “It’s going to be a real eye-opener to see that the NCTM standards . . . have had very little impact in classrooms at all,” he said.

A selection of the tapes will be released to the public, and the entire package will be made available to researchers. They will provide examples of good and bad lessons, and Stigler believes that will help the partisans in the math debates get beyond ideology.

“Having watched the videotapes I can say that there is a lot of bad instruction going on,” Stigler said. And most teachers, once they see it, will be able to agree on what is good instruction.

And therein, he said, lies the greatest potential of the video portraits--providing the raw material for a national Socratic discussion of teaching.

“Successful teacher development has to include a focus on actual classroom instruction,” he said, “and we have to come to some agreement about what it means and whether or not it is good.”