Getting Better Results From Standard Tests

July has arrived, and so have the much-anticipated Stanford 9 test results. Having spent $35 million testing 4.1 million California students in second through 11th grades, do we know anything new? Not really.

How can that be? The first hint comes from contradictions generated by the results themselves. For instance, Gov. Pete Wilson called the scores “deplorable,” and Supt. of Public Instruction Delaine Eastin said they’re “good news.”

Contradictions surround the language arts results. Several districts sued to block release of scores for limited-English students. While those districts were in court claiming unfairness, other districts boasted their English-as-a-second-language students outperformed their English-only ones.

A third contradiction comes by way of my two kids. As students, they are opposites. The older one does as little as possible. The younger one gets straight A’s and takes mostly AP courses. Yet, with the exception of math, their scores were fairly even.

While contradictions may cause us to raise our eyebrows, they don’t explain why we don’t know more. The chief reason is that the Stanford 9 is not tuned to California educational standards. Therefore, it can’t measure how well our students have learned what we expect them to know.

It’s not meant to. It is what’s known as a norm-referenced test. This type of test compares one group of students with other students. Here’s how a norm-referenced test works.

Let’s say we have 100 students climbing a 10,000-foot mountain. Of these, 90 are spread out between the base and 6,500 feet. The other 10 are spread out above 6,500 feet. This observer is at 7,500 feet, trailing just four other climbers.

On a norm-referenced basis, a climber at 6,500 feet would be in the 90th percentile because 90% of the climbers are at the same level or below. Yours truly would be in the 95th percentile.

Now let’s use the same mountain scene to represent the other major type of test, known as standards-based. The peak symbolizes mastery of the subject matter.

This type of test measures students in terms of how close they are to the peak. It doesn’t matter to us where the others are. Our only interest is where we are.

Relative to my classmates, I am among the leaders. However, I only know 75% of what I am expected to have learned. Without moving an inch, I just tumbled to “C”-level performance. The large group clustered around 5,000 feet dropped from “average” to failing.

The Stanford 9 tells us nothing about whether we’re near the peak or down in the foothills. (Other evidence leans to the latter.)

Does this mean that the Stanford 9 exercise has been a total waste of time and money? No.

While it shouldn’t have cost us $35 million to do so, the test format did set an important precedent. It established that the public has a right to easy access to detailed information about our schools.

What should we do with this test and its precedent? First, complete the new standards currently in progress. The State Board of Education has already adopted strong new math and language arts standards.

The new math standards are so strong, the Hudson Institute gave them a “perfect” score--besting Japan’s mere “A.” Domestically, a whopping third of all states flunked outright.

The draft version of the new science standards looks equally strong. It is premature to make a call on the history standards.

Armed with robust new standards, we should switch to a standards-based test tuned specifically to them. The test should be integrated into a system modeled after the Tennessee Value-Added Assessment System (TVAAS).

The TVAAS, developed by John Sanders at the University of Tennessee, has been producing data long enough that officials are now finding remarkable trends. For instance, students who have had a terrific teacher tend to do better for three years after. Conversely, some students who have had a lousy teacher never recover.

With the Sanders system, accountability quickly becomes a meaningful topic. With accountability, exciting possibilities arise everywhere.

Administrators would have data upon which to base management decisions. Parents could request particular teachers or refuse to let their children be subjected to others. Teachers would have a foundation for merit-based pay.

If it’s true that knowledge is power, then the knowledge of which districts, schools and teachers are getting the job done will make the entire public education system much more accountable and responsive to the public. The real value of this year’s statewide test may have been its role in moving toward greater accountability and responsiveness.