Worst Schools List Flawed, Analysis Finds

They have been dubbed the bottom 30 schools in Los Angeles, a distinction that dragged them through two years of scrutiny and humiliation.

And now they face much worse: With the posting of statewide test scores next week, any of the schools that fail to improve are to be placed into receivership, with key budget and curriculum decisions taken over by bureaucrats downtown.

There is only one problem. Many of the so-called 30 worst schools are nothing of the sort, a Times computer analysis has shown. One of the schools has such high scores that it ranks in the top quarter of all district schools.

Three more are in the top half and another eight are above the cutoff for the worst 100.

The school district’s flawed method of identifying its low-performing schools is part of an accountability plan that Supt. Ruben Zacarias improvised two years ago and the district has since tried to fix it with a succession of adjustments.

The problems in ranking the 30 schools reflect deficiencies that persist in the accountability plan.

The schools were listed as the 30 worst because their scores on tests to measure proficiency in such basic subjects as reading and math did not improve over a two-year period. As currently crafted, the plan allows the district’s almost 550 schools to escape consequences if they show improvement just once every four years, even if the overall trend of their scores is downward. It prescribes the same treatment for schools that make no gain from year to year as for those that take a deep dive.

Schools scoring in the top 10% nationally must meet the same standard as schools in the bottom 10%: an improvement of one percentile point. That means a low school can get by with a minimal gain, even though it remains far below the proficiency level, while the high one will face sanctions if it only repeats its excellent scores.

Moreover, the system makes no provision for the continuation and opportunity schools that chronically register the lowest test scores.

After holding onto this method for two years, L.A. Unified officials now have acknowledged its shortcomings and are scrambling to come up with an overhaul. The revisions won’t be ready to show the Board of Education until mid-July, however, two weeks after critical Stanford 9 test scores hit the Internet.

“We’re evaluating how to be as fair as possible in deciding what schools will become receivership schools,” Zacarias said in a prepared statement responding to the Times analysis. “No system is perfect, but we can’t wait for the perfect system.”

Already, district officials have concluded that 12 schools should be taken off the probation hook. Other schools may be spared after further evaluation, said Roger Rasmussen, head of the district’s independent analysis unit, who is drawing up the new accountability proposal.

Morale Takes a Beating

“It’s very complicated,” Rasmussen said. “For those schools that showed another year of decline, we will look very closely at as much data as we can to try to figure out what is happening.”

For weary school staff, the test score game has been a roller-coaster ride.

“Morale hit rock-bottom here as a result of being identified as one of the 30 schools,” said Claudia Holmes, testing coordinator at LaSalle Elementary School. “It had a tremendous negative effect on a school family.”

Illustrating the apparent disparities in the accountability plan, LaSalle was one of the 12 schools taken off probation, due to changes in the scoring system, even though its scores are at about the 20th percentile.

“It has been a tremendously hard year, a lot of pressure on the staff, trying to look for answers, working super-hard to get the scores up,” said Betty Coleman, principal of Westminster Elementary School, which remains on the list although its scores are near the district average. “What can I say? It was impossible to get off unless we showed improvement.”

The problems with the accountability system can be traced to a bold plan Zacarias introduced in the spring of 1997, when he was competing to become superintendent. He said he would identify the lowest-performing 100 schools and devote his personal attention to improving them.

The list was hurriedly drawn up by school officials from an older test, even though data from the first year of the Stanford 9 exam was soon to be available.

Although the accountability plan drew wide praise for sending out a strong message that schools must improve student performance, the list was criticized as targeting primarily schools in the city’s poorest neighborhoods.

Responding to the complaints, Zacarias later decided to judge schools by whether their scores went up or down.

By this time, the district had two years of Stanford 9 scores, making a year-to-year comparison possible. Any school whose score failed to improve or went down would be put on alert, Zacarias said. Further, schools showing no gain for two years would be placed on probation, and those failing to gain by the third year would go into receivership. A fourth stagnant year would put the school in line for reconstitution, meaning its principal and staff could be replaced.

At the time, district officials failed to spot several weaknesses in the design. A school such as San Jose Highly Gifted Magnet, whose average score was at the 95th percentile, could soon face sanctions for not climbing to the absolute top, the 99th percentile. Conversely, Drew Middle School, at the 14th percentile, could cruise along for 10 years by making modest gains without reaching the level of minimum proficiency.

Also troubling was the fact that some schools, such as Vernon City Elementary, with a seven-percentile point drop, received no more serious a warning than dozens of schools that had no change at all.

The new plan brought harsh news for the 30 schools--those from the original list of 100 whose scores did not increase. Zacarias moved them to the second step of sanctions: probation.

A closer examination would have disclosed that the new measuring method--which used a combination of the Stanford 9 in English and the Aprenda in Spanish--generated dramatically different results than the older tests used to rank the 100 schools. As an example, Westminster Elementary, which was 49th on the 100 schools list, would have been 306th, or better than half of all schools. All told, a third of the 100 schools would never have been on the list.

That illustrates why statisticians warn against switching from one measure to another in any accountability test.

“Are you really confident these are the 100 schools that need the most attention?” asked Richard S. Brown, a project director for the UCLA Center for the Study of Evaluation. “If a third of them would have been replaced by a factor of the measure being used, interpreting growth is very difficult.”

The difficulty only increased this year when Proposition 227 caused the district to drop the Aprenda test from its measuring system. The district recalculated earlier scores without those test results.

Once again, there was a significant reshuffling. Seventy schools whose scores declined in 1998 based on the old measure had improvement under the new one. Twelve of those were the schools removed from the list of 30. Thirty-eight switched from better to worse.

Brown said the wide fluctuations could indicate that the measures may reflect something other than student performance.

A Better Rating System Is Promised

Principals interviewed by The Times suggested several possibilities. A school that practiced English immersion before Proposition 227 might have improved when the Spanish-language test was dropped, one said.

Others pointed to another change: Magnet programs on shared school campuses were counted as separate schools in 1997, but joined to their home campuses in the new measure, possibly boosting their scores.

Rasmussen said the proposed overhaul of the accountability plan will provide a far more sophisticated approach. For one thing, there will be a threshold above which high-performing schools will be free of sanctions, even if their scores dip slightly. And all low-performing schools will remain on alert until they reach a standard level of proficiency.

Rasmussen said he is also proposing a separate test to measure students’ mastery of curriculum and a test crafted specifically for students who are not proficient in English.

Finally, in what would be the most difficult undertaking, he wants to start following the same students from year to year to measure how much they learn.

These are groundbreaking proposals that won’t soon be ready for implementation.

So again this year, schools that make no gain on the Stanford 9 are expecting to face humiliation.

There are actually those who find a silver lining.

“In a way it was a good thing,” said Jacquelyn Carr, principal of Vernon City Elementary School, which was put on alert last year, but recently learned that its seven-point slide last year has turned into a nine-point gain this year.

After giving in to jubilation at the news, Carr decided not to pass the message to her staff until the last week of school because she didn’t want them to lose their intensity.

“Everything was there to make some changes,” Carr said. “I did not want to lose that. We were happy, but we realized that we still have a lot of work to do.”

(BEGIN TEXT OF INFOBOX / INFOGRAPHIC)

Different Rankings

The 30 L.A. Unified schools listed below were placed on probation at the beginning of this school year because they were on the list of 100 lowest-performing schools two years ago and did not improve on 1998 tests. But the rankings below show that the tests now being used give significantly different results than the outdated one used to create the original list. The middle column of numbers shows how the schools would have ranked among the almost 550 schools in the district using a combination of the 1997 Stanford 9 and Aprenda tests--last year’s accountability measure. The right-hand column shows how the schools would have ranked using only the Stanford 9, which is this year’s accountability measure. In all three listings, 1 indicates the lowest position.

*--*

Actual Rank Based Rank Based ’97 District on Stanford 9/ on Stanford 9 School Ranking Aprenda Results Results Only Jordan High 4 17 21 118th Street Elementary 11 32 4 Russell Elementary 14 37 49 Manual Arts High 17 48 49 Woodcrest Elementary 23 67 49 Dorsey High 27 32 81 Markham Middle 28 1 14 75th Street Elementary 29 48 14 Carver Middle 30 12 36 Muir Middle 34 23 49 Pacoima Elementary 37 100 14 Graham Elementary 42 48 49 Westminster Elementary 49 306 330 Olive Vista Middle 54 48 102 Bell High 57 145 235 Normandie Elementary 58 82 49 Budlong Elementary 60 100 81 Belmont High 62 100 181 Franklin High 63 116 307 Wadsworth Elementary 68 167 21 Parmelee Elementary 74 82 21 Los Angeles High 75 100 163 La Salle Elementary 78 37 69 McKinley Elementary 80 324 307 Audubon Middle 88 48 181 Garfield High 90 116 181 Banning High 91 145 259 Barton Hill Elementary 94 145 69 Ambler Elementary 98 223 444 Gardena High 99 167 235

*--*

Sources: Los Angeles Unified School District; analysis by Times Education Writer DOUG SMITH