New CAP Tests Stress Students’ Reasoning : Education: Exams measuring problem solving and writing to be emphasized over multiple-choice format.

One hundred and fifty California teachers and curriculum specialists gathered at a local hotel last week to pore over student papers, trying to figure out how to score the state’s trailblazing performance-based English/language arts tests.

The English tests, like all those in the new California Assessment Program, stress reasoning ability, writing skill and problem solving instead of the multiple-choice questions that have dominated testing for years.

For example, after reading a Kate Chopin story about a woman’s mixed feelings on learning of her longtime husband’s death, 10th-graders are asked not only to answer questions about the story but also to write an essay “reflecting on your concepts of marriage.”

The mathematics tests include open-ended questions like the following eighth-grade puzzler:

“The cycle for the traffic light on Main Street is green for 55 seconds, yellow for five seconds, and red for 30 seconds. What is the probability of having to stop at the traffic light? Explain your reasoning and include consideration of how drivers react to yellow lights.”

Future math tests will feature group “investigations,” involving several students in a math problem that might take two days to solve.

A 10th-grade history/social science question asks students whether they agree or disagree that the roots of World War II can be found in the Treaty of Versailles, which ended World War I, and to “explain your position using historical evidence.”

Science assessment will focus on problem solving and “hands-on” tasks such as asking sixth-graders to make an electrical circuit.

The new approach is called “performance-based testing” or “authentic assessment,” and it will be applied in all subjects and at all grade levels tested (English/language arts, mathematics, science and history/social science, in grades four, five, eight and 10), beginning next spring.

The tests will yield individual scores and allow for comparisons between schools and school districts but will not be used for students’ promotion or graduation.

“We’re aiming for tests that tell us what kids know and can do in a given subject matter,” said Allan R. Odden, professor of education at USC.

“Multiple-choice gives you a glimpse of a student’s content knowledge, usually at a superficial level,” he said, “but it’s almost impossible to tell what the student can do with that knowledge. With these new tests, in which kids do science experiments and write persuasive essays about a key historical situation, you get a deeper insight into their ability to reason.”

Eva Baker, director of the National Center for Research on Evaluation, Standards and Student Testing at UCLA, said the tests will “provide good information about the quality of school accomplishments and, more important, good information that can be used by teachers and school folks in making their decisions.”

In 1990, former Gov. George Deukmejian, bickering over the education budget with state schools Supt. Bill Honig, eliminated funding for CAP, one of Honig’s pet programs.

Pete Wilson, the new governor, reversed that decision in 1991, agreeing with Honig that statewide testing is essential. However, Wilson wanted the new system to measure individual students as well as schools and school districts.

Wilson signed legislation authored by state Sen. Gary Hart (D-Santa Barbara) to establish the new testing program and provide $10 million for 1991-92, its first year of operation.

Honig and Dale Carlson, testing manager for the state Department of Education, decided to take this opportunity to introduce performance-based assessment, an idea that was beginning to work its way out of the shadows of educational journals and into the light of everyday practice.

Working with the department’s specialists and with four of the nation’s largest testing companies, Carlson and his staff developed the first group of “authentic assessments” and field tested them in more than 1,300 schools last spring.

“People are crazy about them,” said Susan Bennett, an education department consultant. “Kids loved them and they were a great experience for the teachers.”

The first tests being developed include some multiple-choice questions, because, Carlson and Bennett said, they are still a good way to evaluate certain skills and are much less expensive to administer than performance-based assessments.

But the tests emphasize open-ended questions and “hands-on” problem solving in math and science and they stress reasoning, analysis and writing, even at the fourth-grade level.

For instance, fourth-graders are presented with this kind of math problem:

“A visitor from outer space has just arrived. It is confused about our number system. It asks you: ‘Is 5 plus 29 equal to 529?’ Answer the visitor’s question and explain your answer.”

Last spring’s field tests turned up problems, large and small.

A “trash test” in which fifth-graders were asked to distinguish between garbage that decomposes and garbage that does not, itself decomposed when carrots and potatoes spoiled because of improper handling.

“Those are the problems you run into with the ‘hands-on’ approach,” Carlson said.

Equity is a more serious problem. Do the new assessments fairly judge the abilities of affluent suburban children and those who attend school in poverty-scarred inner cities?

“We are concerned about what is called the ‘delivery problem,’ ” said Karen Sheingold, of Educational Testing Service, one of the companies that is helping the Department of Education develop the tests. “It’s not fair to hold students and teachers accountable for comparable performances on tests if the schools are not delivering comparable services.

“Can we create a system where there’s a fair amount of flexibility

in the forms of evidence that kids present but at the same time generate information that’s comparable and fair?” she asked. “That’s the challenge.”

Performance-based tests, which must be read individually, are much more expensive than multiple-choice tests, which can be scored rapidly by electronic scanners.

In its final year, 1990-91, the old CAP tests cost $9 million. The new tests are costing $10 million this year and the tab will reach $47 million in the program’s fifth year.

“A lot of our visitors say: ‘Yes, we love this performance assessment but it’s very expensive,’ ” said Carlson, the education department’s testing manager.

“The whole issue of cost vs. benefit is very important,” said Ray Reinhard, assistant secretary of child development and education in the Wilson Administration. “My sense is you can spend a lot more money and not necessarily wind up with better information.”

Finally, there is the problem that brought teachers from all over the state to Sacramento last week: Can hundreds of individual test readers agree on a scoring system that can be applied to students statewide?

As the teachers examined papers written by fourth-, eighth- and 10th-graders during last spring’s field testing, it was reasonably clear which papers merited a top score of 6 (exceptional performance) and which should be given the lowest rating of 1 (minimal performance).

But it was much harder to separate a 3 (minimally proficient) from a 4 (proficient) or a 4 from a 5 (commendable performance).

Nevertheless, by the end of the week, there was general agreement on a range of scores that this group of teacher-readers could pass on to others before testing begins in earnest in the spring.

Most in the group were enthusiastic about the new approach.

“There are different types of intelligence and this approach respects that fact,” one teacher said.

Another said: “This is so very, very exciting. This is a testing approach that will actually help us to be better teachers. It couldn’t come at a better time. I’m just so thrilled.”

How Smart Are They?

These are sample questions from the proposed “performance-based” statewide test. It seeks to assess students’ ability to write, read with comprehension, solve math problems and perform science experiments. An open-ended science question for Grade 6:

A small tree is planted in a meadow. After 20 years it has grown into a big tree, weighing 250 kg. more than when it was planted. Where do the extra 250 kg. come from? Explain your answer as fully as you can.

A math question for Grade 8 or 10:

Bob says that an increase from 10 to 40 is 300% and, therefore, a decrease from 40 to 10 must be a 300% decrease. Do you agree with Bob? Explain your reasoning.

A math question for Grade 8 or 10:

A family of six ordered 13 hot dogs. Nine hot dogs had mustard, 3 had ketchup, 8 had relish, 4 had both mustard and relish, and 3 had mustard, ketchup, and relish. Explain to the clerk how many hot dogs had no relish or ketchup. You may use drawings in your explanation.

A “short answer” history/social science question for Grade 10:

China, unlike Japan, was eventually dominated by the European imperialist powers. There were many reasons for this. Select two items from the list below and explain how they contributed to the failure of China to resist the West.

* Chinese world view

* Taiping Rebellion (1851-1864) or Boxer Rebellion (1890s)

* Confucian scholar-gentry class

* European technology