Area Schools Upset by Low Rankings in ‘Effectiveness’ Book

A newly published book that numerically rates the effectiveness of school districts all across the nation--suggesting that those in the Los Angeles area are relatively ineffective--has San Gabriel Valley parents, teachers and administrators hopping mad.

Only one of 14 school districts in Los Angeles and Orange counties that were rated in “Public Schools USA” by education writer Charles H. Harrison scored in the top half of the 500 districts surveyed. San Marino rated a 60 out of a possible 100 on Harrison’s “effective schools index,” slightly above the median of 58.

Several other San Gabriel Valley school districts ranked near the bottom, including Duarte with 30, Pomona with 29 and Pasadena with 23.

Twenty of the 26 districts that scored 85 or higher in the book, which was released Wednesday, are in the Northeast, including seven in the New York metropolitan area.

‘Flat-Out Nonsense’

“Nonsense,” fumed William Bibbiani, director of research and testing for the Pasadena Unified School District. “Flat-out nonsense.” Harrison’s “effective schools index,” or ESI, is more a measure of relative amounts of state aid than of effectiveness, he insisted.

“It’s just shoddy research,” Bibbiani said. “The whole study is spurious.”

Some critics contend that the book’s broad-brush approach has “maligned” their school districts, publicizing districtwide statistics without acknowledging individual programs that have been singularly successful.

“It really upsets and angers me that they do this,” said Jeanette (Jay) Blackshaw, president of Pasadena’s Parent Teacher Council. “There’s a self-fulfilling prophecy in things like this.”

Emphasis Questioned

Harrison, reached at his home in New Jersey, said his critics are putting too much emphasis on the numbers. The initial press coverage of his book, which offers a one-page thumbnail description of each of the rated districts, mistakenly concentrated on ranking them, he said.

“This wasn’t meant to be a ranking system,” said Harrison, the author of four other books about education and a past president of the National Education Writers Assn. “The ESI was just one of three parts to each district’s entry.”

Besides the ESI, the book provides data on each district, such as total enrollment, teachers’ salaries and the percentage of school buildings built before 1955. It also provides “qualitative” observations on school leadership, instruction and school “environment.” The rated districts all had a minimum of 2,500 enrolled students and a full range of grades, and they were within 25 miles of a major city.

The ESI is derived from 10 statistics, including average daily attendance, spending per pupil, dropout rate, percentage of eligible students taking the Scholastic Aptitude Tests (SATs), average combined scores on the SATs, teacher-student ratio in elementary and secondary grades, counselor-student ratio in secondary grades and number of students per music specialist in elementary grades.

Rated 0 to 10

The results for each category are weighted from 0 to 10, depending on a district’s entry. For example, Pomona received 0 in one category, for its average combined SAT score of 760, while San Marino, with an average combined score of 1,012, received 10 in the same category. Pasadena, which did not submit data in that category, also received a 0.

“Some districts took it rather casually,” said Harrison. “The superintendent passed it off to a subordinate who filled it out rather casually, without anyone checking or double-checking. Some districts which got zero would have gotten higher scores if they had done it more carefully.”

But he insisted that the data gives parents who are moving from one metropolitan area to another a “legitimate guide” in selecting a district with an effective school system.

“It seems to me that if a parent were looking at various districts,” Harrison said, “and he saw one school district where the SAT scores were 200 points above the national average and another where the scores were 200 points below the national average, he might be interested in looking at one rather than the other.”

Until now, he said, parents were at a loss to find the best districts in a metropolitan area on their own. “It was shooting in the dark or else taking the word of a real estate agent.”

Bibbiani contends that nine out of 10 of the ESI statistics are related either to state aid or to the socioeconomic characteristics of the district, rather than to effectiveness. “Affluent kids tend to go to college more often than poor kids. So if you’re dealing with a diverse socioeconomic group (as in Pasadena), you tend to look bad,” Bibbiani said.

Income Comparison

“If he had just looked at per capita income in those 500 districts and lined them up from top to bottom, he would have gotten the same results,” added Saul Glickman, president of United Teachers of Pasadena and a Pasadena High School mathematics teacher.

Glickman also faulted Harrison for “sloppiness” in giving zeros to districts that did not provide sufficient statistical information. “Take the top district among the 500,” he said. “If it had not answered the questions or misinterpreted them, it would wind up on the bottom.”

Bibbiani said the issue of state aid explains why a disproportionate number of the top-rated districts are in New York. “Why are they in New York?” he asked. “Because New York outspends California by (almost 2 to 1).”

A spokeswoman for the state Education Department said that in 1985-86, the most recent school year for which comparative statistics are available, California, which ranked 25th in education spending, spent an average of $3,809 per pupil, while New York, ranked second after Alaska, spent $6,011.

The projected figure for the current school year in California is $4,199.

The emphasis on money neglects other educational values, said Blackshaw, whose seven children have all passed through the Pasadena school system. “There are other advantages to going to school with students from all different economic levels,” she said. “My kids are color-blind. They’re not going to perpetuate a lot of prejudices.”

SAT Scores ‘Inappropriate’

While Bibbiani dismisses nine of the 10 ESI statistics as irrelevant to effectiveness, the average SAT scores should also be eliminated as statistically inappropriate, he said.

“The Educational Testing Service (which created the SATs) insists every year that the test scores should not be used to rank schools,” Bibbiani said. “When was the last time a district tried to get into college? It is students who try to get into college. Rankings based on district averages are just foolish.”

High average SAT scores are as much a function of population trends (such as high educational aspirations) as of the scholastic achievements of a group of students, Bibbiani said. “You’re dealing with a self-selected population that can be radically different across the country and within cities. It’s wholly inappropriate to rank school districts by their scores.”

“Our school has high achievers and low achievers,” added Janice Urbina, president of the Pasadena High School PTA. “It’s a real melting pot. We try to encourage everybody, not just the high achievers. It’s a little discouraging to see (the district’s low ranking) plastered all over the front page.”

Gary Richards, superintendent of the San Marino Unified School District, also questioned the value of the ESI. “There are other qualitative indicators to look at,” he said. “They could have gotten into the level of training of staff or the numbers of kids involved in the total school program. In a lot of schools, kids essentially come to school and go home, without getting involved in extracurricular programs, like the band, the debating club or the newspaper.”

Harland Donahue, assistant superintendent for educational services in Pomona, said Harrison’s book failed to recognize districts that were overcoming great obstacles. “The dropout rate, for example,” he said. “You have to compare different populations when you do it.”

About half of the Pomona Unified School District’s 24,000 students have limited proficiency in English, Donahue said, and 55% are poor enough to qualify for the free lunch program. “I’d like to deal with that aspect,” Donahue said. “Given those figures, how is it that we’re near the top (of a group of districts with similar characteristics)?”

Additional Criteria

Donahue also suggested some additional criteria for judging effectiveness--”like having a clear school mission, a safe and orderly environment, high expectations among the students, strong instructional leadership in which the program is regularly monitored.”

Harrison said that where statistics did not show the entire picture, he provided qualitative observations. “Some of these are the best part of the book,” said Harrison, who spent a year and $10,000 compiling the data.

For example, the book notes that in the Glendora Unified School District, principals are hired mainly for their ability as educational leaders. “However, some principals who have been in the system a while were part of an ‘old boy’ network, and their leadership is not as good, according to observers,” the book says.

Harrison said he culled observations like that from a network of contacts all across the country--”the League of Women Voters, reporters from the local paper, the Chamber of Commerce, people who, though not employed by the district, are very close to it.”

Phil Wollam, administrator in charge of educational programs in Glendora, denied that there was an “old boy network” there. “We have a veteran staff, but it’s always been on the innovative side,” he said.

Not ‘Final Judgment’

Bibbiani added that the qualitative observations he had read tended to be generalizations that could apply anywhere. For example, the book suggests that “internal politics” is a factor in selecting principals in Pasadena. “There’s not one of the 16,000 school districts in the country about which that couldn’t be said,” he said.

Somewhat defensively, Harrison insists that his critics are over-interpreting his book, which can be ordered for $17.95 from Williamson Publishing Co., Box 185, Charlotte, Vt., 05445. “I’m not saying that this book is the final judgment on the schools or that these are the only measures of quality,” he said.

But he insisted that it was more authoritative than the kind of information that school districts give to prospective students’ families.

“If I go to a district and ask for information, they’ll give me their latest brochure, which probably says that it’s the best school district in America,” he said. “I’ve judged enough national contests for school publications to know that most of them are pretty vague and self-congratulatory.”

LOCAL RANKINGS Rankings of local school districts according to the book “Public Schools USA” on a scale of 0-100.

Arcadia 50 Charter Oak -- Claremont 54 Duarte 30 Glendora 47 Los Angeles 28 Pasadena 23 Pomona 29 San Marino 60