Culprit for Drop in Test Scores Sought

By DOUG SMITH

July 19, 1998 12 AM PT

TIMES EDUCATION WRITER

As she scanned her district’s much-anticipated test scores, the assistant superintendent for Santa Monica schools was pleased at first to see above-average reading scores, building to a better than respectable ranking in the 63rd percentile in the eighth grade.

Then came the precipice. By the second year of high school, the students were scoring in the 44th percentile, marking an astonishing drop of 19 points. “Everything looked so good on the way up,” said Sue Gee. “It’s surprising.”

A similar surprise was in store for nearly every high school administrator in Los Angeles County and hundreds more across the state as results of this year’s statewide standardized testing trickled out this month.

Now educators and testing experts are struggling to identify the culprit responsible for the weak performance of ninth- and 10th-graders, who consistently scored below all other grade levels.

Theories include inadequate teacher training, the failure of students to read and even poor physical fitness. Several school officials said they will take a close look at vocabulary instruction as a potential culprit.

It’s also possible the apparent collapse of student performance represents a fluke in the test itself, a theory that testing experts said they doubted, but could not yet disprove.

Harcourt Brace Educational Measurement, which faces a public relations nightmare if its Stanford 9 test administered at 8,000 California schools this spring proves flawed, discovered the phenomenon last week, said Tom Brooks, manager of research for the company. Brooks said it might take a month to determine a cause.

“You’ll probably find a whole lot of interesting hypotheses,” he said.

And he’s right. Los Angeles Board of Education member David Tokofsky said his experience as a former high school teacher suggests that a closer look at high school dropouts may help solve the puzzle.

Because the dropout rate increases in the first two years of high school, conventional wisdom would suggest that scores should improve as poorer students left the testing pool.

Tokofsky believes, though, that dropouts come not from the academic dregs, as often supposed, but from the layer just below average, students who lose interest upon “realizing there is a gap between their ability and what is required.”

Meanwhile, those who “stay on and just sit there like lumps waiting for their courtesy Ds” pull the school scores down, Tokofsky reasons.

Those who already believe the public school system is failing view the ninth- and 10th-grade reading collapse, which averaged 11 percentile points in Los Angeles County, as only one of many problems that the state’s generally low scores reveal.

“The scores are just scary, they’re so low,” said Bill Honig, the former state superintendent of instruction who now works for a consulting firm that trains teachers.

Honig said the high rate of schools earning percentile scores in the 20s and 30s is an indictment of school management, which has failed to insist on reading diagnostics in the early grades to ensure that children learn fundamentals before they move on to more complicated texts introduced in middle school and high school.

“They get out of upper elementary and nobody does anything about it. They keep getting farther behind. It catches up with you in the middle and upper grades.”

Samples from Harcourt Brace show that the eighth- and ninth- grade test are similar in form, each posing questions about a narrative that can be understood without any reference to literature or history. But the more advanced narrative presents far more complicated sentence structures and wider vocabulary, and the questions require more attention to detail.

Honig suggested that students need to learn 4,000 words a year in high school to keep pace with the national sample group that constitutes the norm, a goal that can only be attained by reading 20 minutes a day outside of school.

“In the schools you’re looking at in California, nobody reads that much,” Honig said.

Mardel Kolls, assistant superintendent with the Rowland Unified School District, said she has a couple of theories. Like Honig and several other educators, Kolls considers the state’s standard for teaching vocabulary a prime suspect.

“We’ve not had such an emphasis on vocabulary,” she said. “We’ve been told over the years that it’s not very helpful to give students isolated lists of words.”

She also thinks students may not have been prepared for the rigor of the Stanford 9 test, which had 84 reading questions, compared with 54 on the test the district administered last year.

“One factor for students is simply sticking with it,” Kolls said.

Because the high school dip was consistent across the state, though, the possibility arises that there was a flaw in the national sample of students against which scores are measured. If students in the sample were unusually proficient in high school reading, those measured against them would look poorer by comparison.

However, Harcourt Brace’s Brooks said he doubts that could be the problem because the sample group didn’t do any better on reading than math, and no dip showed up in California’s math scores.

A spokesman for a competing firm concurred that it is unlikely the problem will turn up in the test.

“There are always surprises when you’re analyzing large data sets,” said Michael Kean, a vice president of CTB/McGraw-Hill, publisher of the California Test of Basic Skills. “If you have a solid instrument, then in most instances, the seeming anomalies are not due to the test, but might be explainable by the pupil population.”

In Sacramento, administrators of the state testing program also are looking at the student population for possible answers.

Gerry Shelton, administrator for the state Department of Education’s testing program, said one theory being discussed is the possibility that one grade level may have benefited from better instruction than another. For instance, instructional methods changed dramatically with the recent decision to teach reading through phonics.

“It could be that sometime back in their educational career there had been a substantial change in instruction or curriculum such that the ninth-graders’ education was very much different from eighth-graders’ education,” Shelton said. The state will be analyzing the data for months to try to answer those questions, Shelton said.

Another attempt at explanation focuses on the testing environment.

Gee, the assistant superintendent in Santa Monica, said she worried that high school administrators may have squeezed the test into the regular school day, unlike middle school principals, who created special schedules so that students would be more relaxed.

But like several other school administrators, Gee was unwilling to conjecture on any potential lessons for her district’s instructional program.

“At this point, anything I say would be speculative,” Gee said. “It’s a big indicator of something. We have to figure out what.”

She said the instructional staff will pore over individual students’ answer sheets to determine which types of questions they missed and interview principals and teachers to link the weak areas with curriculum and classroom technique.

For most school districts, the question has arisen at an awkward time with school out of session and teachers, who know best what goes on in the classroom, out of touch for the summer.

“I think once we get everybody back on board in September, we’ll take a close look,” said Lois Green, assistant superintendent for educational services with the Glendora Unified School District.

At this point, there are no answers.

“We get high quality work from our students, and then we see this dip,” Green said. “We just can’t figure out what is going on.”

(BEGIN TEXT OF INFOBOX / INFOGRAPHIC)

Samples From the Test

On the Stanford 9 test taken by all of California’s 8,000 schools this spring, middle school and high school students were asked to answer questions about short narratives. Following are examples showing the increasing complexity of the texts and questions between the seventh- and eighth-grade test and the ninth-grade test.

Grades 7-8

In a 10-paragraph narrative, a journalist describes his interview with Emily Yellow Wolf, the oldest known Native American in the state of Washington, and a quilt maker.

A key paragraph quotes Emily:

“That red bandanna cloth you see over there,” Emily said, pointing to her sewing machine, “well, that is the last of the blouse I was wearing when my girl Hayley was born. That square is more than a piece of cotton, you see. It is memory too. I put those memories into each quilt I make.”’

These are among the questions students must answer:

7. What do Emily Yellow Wolf’s quilts symbolize to her?

A: Her life *

B: Her home

C: Her friends

D: Her goals

8. The theme of the story has to do with?

F: The qualities of friendship

G: The importance of honesty

H: The care of older people

J: Remembering the past*

Grade 9

The narrative read by ninth-graders tells how a spider weaves its web, as observed by a scientist at the University of New Hampshire.

“The construction process begins, he says, with a pattern of radial fibers, the spokes that extend from the middle of the web. The spider spins these strands from the same fiber it uses to produce its ‘drag line, the thread that keeps it attached to its web. Next, the spider lays down a loose temporary spiral, working its way from the center of the spokes to the outside edges. ‘Sort of like the scaffolding used to construct a building, Tillinghast says. . . .

“The spider, in fact, is a perfect example of nature’s message for mankind--a creature who consumes its own web and then recycles it without leaving a residue. ‘In nature, the most sophisticated things are done in a very gentle way, without hurting the environment, says Tillinghast. ‘If there is to be a human future, we too will have to act with efficiency and gentleness.

According to the author, spiders set a good example for humans because they--

F: Get along well in groups.

G: Do not hurt the environment.*

H: Work very hard.

J: Make a product that can be used many times.

The boxes show some important ideas from the article.

Box 1: Lays out radial fibers.

Box 2: Is blank.

Box 3: Makes spiral of “sticky thread.

Which of these belongs in the empty box?

A: Spins temporary spiral.*

B: Catches insects in the web.