This Test Is the Best for Wielding Carrots and Sticks

Experience in other states shows that educational reform works best when all the elements of a new system--such as sound curriculum, higher standards, relevant tests, teacher development--are in sync. The California Board of Education is ignoring this lesson.

First, it adopted a state test, the Stanford 9, that does not fully reflect the state’s new standards and curriculum guidelines. To compound the problem, last month the board postponed the development of a new test, the Comprehensive Assessment of Applied Academic Skills, or CAAAS, that would have been tightly linked to those standards and that promised to be a better measure of school performance.

That decision should be reversed when the board meets today.

The conflicting signals are particularly troublesome right now because the state is about to inaugurate a new, high-stakes accountability system that features test scores as the key measure of school performance. Starting next year, schools can earn cash bonuses if students improve their scores on the Stanford 9 test. Conversely, schools that fail to demonstrate improvement could suffer severe sanctions.

It would be a great mistake to rely exclusively on the Stanford 9 test to measure school performance. True, it is probably as good as any other commercially available test for determining students’ relative skills on commonly taught curriculum. Yet it has many limitations as a tool for educational accountability. For example, it only includes multiple-choice questions; students never have to write essays or solve problems. In addition, the same version of the test is reused annually in California, so teachers become familiar with the questions that are asked.

A large body of evidence indicates that when the same test is reused year after year, teachers tend to focus their classroom efforts on those narrow questions rather than on the broad subject matter that the test is supposed to represent and that the state’s standards supposedly require.

More important, putting so much weight on a single accountability measure--and a flawed one at that--provides the wrong incentives. Feeling pressure to improve scores, teachers doubtless will have their students spend dozens of hours of valuable classroom time practicing to take this one test instead of learning to read books, write stories or solve real-world mathematics problems.

The proposed CAAAS test has many advantages as an accountability tool:

* It would reflect California’s own standards in a comprehensive manner.

* New items would be introduced each year to keep the test fresh and discourage teachers from focusing too narrowly on the test itself.

* It would include open formats as well as multiple-choice items. For example, student writing would be judged on the basis of how well students write, not how well they recognize grammatical errors.

* It would cover much more curriculum than the Stanford 9. It would accomplish this by giving different sets of questions to different students. This method--also used by the highly respected National Assessment of Educational Progress--yields valid measures of classroom and school performance while not unreasonably extending the length of the test.

Doubts about the CAAAS test center on the fact that it will not produce scores for each and every student. However, that is not its purpose. CAAAS is designed to test how well schools as a whole are meeting the state standards. It complements the Stanford 9 test, which does produce scores for individual students. CAAAS provides the comprehensive, standards-based perspectives that the Stanford 9 lacks.

We are spending billions of dollars to reform education in California. Without CAAAS, the accountability system may fail to detect the effects of our investment in standards, textbooks, reduced class-size and teacher development.

California will be best-served by a comprehensive, integrated accountability system in which the key elements are in alignment rather than working at cross-purposes.

The CAAAS test is a critical step in turning California’s education reform into an effective whole rather than a set of conflicting parts. It will greatly enhance the validity of school scores and hence of the carrots and sticks built into the state’s new accountability system.