Commentary : Down Side of Rising SAT Scores : ‘Scholastic Aptitude’ Gets an Incomplete in Real World

SAT scores are up. That’s good news. But it is good news that needs to be carefully qualified, lest we end by confounding success with a new brand of failure.

The real problem with the Scholastic Aptitude Test isn’t any of those problems that have received the most publicity. No matter how much we may dislike the idea of standardized tests, the evidence is overwhelming that they’re the best instruments for making certain kinds of measurements. They are the best way to evaluate the relative performance of individuals in a limited range of knowledge or skills, and the best way to evaluate the trends in student performance over time in a limited range of knowledge and skills. The fly in the ointment is that all-important qualifier: a limited range of knowledge or skills.

The problem with objective tests, all objective tests, is that what they don’t evaluate is more important than what they do evaluate. Thus, if we take them too seriously, we risk trivializing the whole process of education.

Answers Are Doubly Subjective

The first thing we need to realize about objective tests is that they aren’t. From the point of view of the student taking the test, a multiple-choice or true-false question must be answered quite subjectively: She has only what’s in her own head to work with. Indeed, taking an “objective” test is doubly subjective; not only is the test-taker limited by her own subjectivity, she’s forced to guess at the subjective processes of the test writers: “Now what do they mean by that ?” The testee can’t ask questions, can’t explain answers, can’t stipulate definitions of key terms. These limitations make taking multiple-choice tests extremely subjective because they exclude all the traditional techniques for establishing common ground. This is one reason why the testee often gets testy.

The “objectivity” of “objective” tests comes in when they’re graded. And what that “objectivity” amounts to is that these tests are evaluated without any exercise of judgment or intelligence on the evaluator’s part. That’s why a machine or a child can do it. Indeed, Banesh Hoffman, the renowned nuclear physicist, suggested that instead of calling such tests “objective,” we should call them “child-gradable.”

That child-gradable tests really evaluate subjective skills is no criticism, because learning is always and everywhere a subjective process. The problem is that the child-gradable test calls forth only a very limited range of subjective skills, and those are probably the least important.

‘Correct’ Answers Cast in Concrete

What makes these tests easy to grade is that the correct answers are pre-defined and cast in concrete. The skill that these tests call for is the skill of picking from among alternatives. That’s a nice thing to be able to do, but there’s not much call for it except on multiple-choice tests and vending machines. Most of the substantial problems of life are the sort where we need to come up with our own alternatives. And any task in which we want the cooperation of other people requires us not only to invent our own options but to explain them, to explore other people’s perceptions of them and cast them in terms others will understand.

Whenever we face real problems in the real world, whenever we seek to work together on problems by gaining a shared understanding of them, we begin not by answering questions but by asking them. Almost all those individuals who changed the world in vast ways--Plato, Jesus, Copernicus, Thomas More, Isaac Newton, Martin Luther, John Locke and a thousand more--began not by giving answers but by rewriting the questions. What we often fail to realize is that those people all around us who change the world in small ways every day do essentially the same thing. A good programmer--or a good baseball player, a good plumber or good office manager--is always asking new questions. That’s the only way to get better at anything.

“Objective” tests tell students that there’s one right answer and that once you’ve got it, you’re done. There’s a word for that approach to things outside the classroom. The word is incompetence.