Method Is All but Lost in the Imagery of Social-Science Fiction

In the last several weeks Shere Hite’s latest opus, “Women and Love,” has been accorded the dignity of ground-breaking science: Serious-sounding statistics announce that such-and-such a number of all women are unhappy in their relationships; such-and-such a number of all women feel that men don’t listen to them, and so on.

Hite may be right about women, but not because of her numbers. The numbers are, to put it simply, a joke. Hite mailed out 100,000 “questionnaires”--like her other two surveys, this one was biased and unprofessional--and got a 4% return from a sample of women who were in no way representative of “all American women.” This return rate is what you would pray for if you were in the junk-mail business or if you were starting a magazine and wanted to ascertain public interest from a direct-mail campaign. If you were a survey researcher hoping for a credible sample, you would be obliged to start over, or perhaps consider another line of work.

Hite is very sensitive about the question of her “methodology.” She devotes a chapter to defending subjective routes to truth, and then tries to convince the reader that her work is objective and scientifically accurate as well. (When the manuscript of the book was sent to magazines around the country for possible excerpting, this chapter was missing. It appeared only in the final, bound books. This means that many reporters and editors had to make their decisions about the worthiness of the book without knowing how Hite got her results.)

Well, what is wrong with subjective routes to truth? Why not publish some articulate complaints by women?

The answer, I think, lies in the growing popularity of what Robert Asahina, a writer and editor, calls “social-science fiction”--books that are not really social science but “naive personal journalism.” This category includes such popular works as the previous two “Hite Reports,” fairy tales such as Colette Dowling’s “The Cinderella Complex” and Dan Kiley’s “Peter Pan Syndrome,” and Nancy Friday’s “Men in Love.” These books consist largely of anonymous letters or interviews, but the impresssionn that they are based on “research” adds a veneer of respectability and seriousness, and supposedly elevates them above the authors’ personal experiences.

Of course, many times social scientists conduct research on unrepresentative samples. Too many psychologists have drawn too many conclusions about all of us from studies of white male sophomores. Sex researchers, including Grandfather Kinsey, have always had to rely on the kindness of strangers who would be willing to answer impertinent questions. I myself ‘fess up to having conducted surveys for Psychology Today and Redbook, where we were lucky if respondents represented the readership, much less the whole country. (We always acknowledged this in reporting the results.) And all social scientists appreciate the value of subjective routes to truth--from personal experience, armchair observations of the world and dinner conversations with friends.

But there are several critical differences between social science and social-science fiction. Scientists understand that a study is only one fragment of the mosaic; this is why they are at pains to cite other research, both confirming and critical, along with their own. They are aware of the many sources of distortion in research: in the experimenter’s own expectations, in the biases of volunteers, in the way instructions and questions are worded. They understand that what people say is only tenuously related to what they do. They begin with a hypothesis that they seek to disprove.

The writers of social-science fiction do just the opposite. They rarely investigate or cite other research; they cheerfully ignore problems of bias in their own interviewing or questionnaire design; they tend to accept everything their volunteers say uncritically,and they set out with a hypothesis that they are determined to prove.

It doesn’t worry me if Shere Hite wants to make an argument (or write a book) by reprinting letters and drawing elaborate generalizations about all women. She may even be right in her analysis; to say she is writing social-science “fiction” doesn’t mean her argument is untrue.

What does worry me is the fact that so many media reports regard Hites’s “methods” as tangential or amusingly controversial, and that so many journalists reported her findings without knowing the methods. By obscuring the truth that what we know depends on how we know it, social-science fiction contributes to the uncritical and mindless attitude that one study is as good as another, that evidence is intellectually unnecessary for one’s argument. Most of all, I dislike the trend toward science-coated journalism because it represents yet another domain in this society in which image is everything and substance counts for little. If it looks like science, has numbers like science, and asserts that it is science, it must be science. Why, that’s enough these days to make the nightly news.