Constituent Polling: Is It Reliable or Just PR? : Congress: One lawmaker got a questionnaire return rate of 2.6%. Professional pollsters consider 50% rock bottom for a survey to have statistical validity.

Every year for the past decade, Rep. Anthony C. Beilenson (D-Los Angeles) has sent questionnaires to each household in his West San Fernando Valley and Westside Los Angeles district, soliciting constituent views on national defense, the deficit and social issues.

“It gives me a feel for how people out there feel,” Beilenson said, citing the thousands of responses his office receives. “It does give people an opportunity to tell their representative how they feel. People like that and they feel good about it.”

Many of Beilenson’s colleagues agree that such surveys help them keep tabs on the pulse of their far-flung districts.

In the Valley area, Rep. Elton Gallegly (R-Simi Valley) has referred to results from such mailings as reflecting the views of taxpayers in his district, in much the same way as a public opinion poll. And Rep. Carlos J. Moorhead (R-Glendale) has cited his constituent surveys to make points with colleagues, most recently his constituents’ perspective on air pollution problems.

Not everyone familiar with the questionnaires, however, embraces them as a time-honored rite of American democracy. Many observers and participants, in fact, are skeptical about what they represent and how they are used.

First, taxpayers do not appear terribly enthusiastic, based on the small percentage that participate. Gallegly received 7,500 responses to 285,000 questionnaires last year--a meager 2.6% return rate. Beilenson and Moorhead did only slightly better, with about 4%.

Professional pollsters, meanwhile, said that lawmakers who circulate such surveys are misguided at best and duplicitous at worst when they treat the responses as accurate representations of the views of all their constituents.

And election opponents assert that the surveys--each of which costs $6,000 to $10,000 to print and mail and takes many days of valuable staff time to tabulate--are a self-serving waste of taxpayers’ money intended primarily to score points with voters.

“I don’t think people in Congress are doing these surveys to find out how their constituents feel,” said Mervin Field, director of the California Poll and president of the nonpartisan, public affairs Field Institute in San Francisco.

“I think they’re doing it for two reasons,” Field said. “It’s publicity for the congressperson. And if you open it up and you see a survey and look at the questions, you say, ‘Here’s my representative who wants to know what I think. Isn’t that a good thing!’ ”

Scientific opinion polls--such as the Gallup and Harris polls and those commissioned by many members of Congress for their election campaigns--solicit the opinions of people who are carefully chosen to be representative of an entire group. In contrast, the constituents who return congressional surveys are self-selecting and, pollsters said, probably not typical.

“In many cases, they are people who tend to be closer to that elected official,” said Arnold Steinberg, a Republican pollster based in Sherman Oaks. “A lot of other people write the congressman off and say, ‘He doesn’t care about my views.’ ”

A related problem is the small percentage of people who respond. Professional pollsters seek a participation rate of at least 60% to 70%--with 50% considered rock bottom--for a survey to have statistical validity. As the reply rate drops, so does the probability that the sample accurately reflects the views of the whole group.

Thus, even if the number of responses in a congressional survey appears impressively large, the results still are considered of dubious validity when the response rate is less than 5% of those queried.

“At 5%, the sampling error is astronomical,” said I. A. Lewis, director of the Los Angeles Times Poll. “It’s probably incalculable.”

Further distortion occurs, pollsters say, if questions are framed in a manner that is likely to induce answers that support a lawmaker’s position.

In 1989, for instance, Gallegly stated his views on various issues and asked constituents whether they agreed, disagreed or had no opinion.

“Defense spending has decreased in real terms (after inflation) in each of the past three years. In real terms, the United States must, at a minimum, maintain current spending levels” was a typical item in Gallegly’s survey.

Field called this “the kind of loaded question you would put in a survey methods textbook to show how you can load an issue.

“It’s saying, ‘We’ve already cut the thing to the bone. We don’t want to cut it any further, do we?’ The public will always respond in the direction the question is worded. They’ll try to agree.”

In general, the veteran pollster said, by only giving one side of most issues, Gallegly’s mailing was “not a survey but a planned effort to get a sycophantic response.”

Gallegly was in Europe with a House Foreign Affairs Committee delegation and could not be reached last week for comment. His spokesman John Frith said the substantial number of respondents who differed with Gallegly’s statements in the 1989 survey refuted the pollster’s contention that the exercise was slanted to produce the desired results.

The lawmaker recorded the most disagreement with his defense spending statement. Overall, 38% differed and 57% concurred with Gallegly’s assertion that defense should not be cut.

“The results of the poll were pretty much in line with what he expected,” Frith said, referring to Gallegly. “It was gratifying to him to see the margins of those responding who agreed” with his positions.

Gallegly changed his format this year to pose actual questions rather than asserting his views. Beilenson and Moorhead take different--and, generally more even-handed--approaches than Gallegly’s 1989 mailing.

In his 1989 survey, Beilenson asked constituents to experience vicariously the choices a lawmaker faces in cutting the federal budget deficit. He offered a menu of spending cuts for social and defense programs and foreign aid and 10 options for increasing taxes. He also included figures for the respective savings or increased revenues each selection would bring so that respondents could determine their bottom-line success in deficit reduction.

“A lot of people had comments that this was a real eye-opener,” Beilenson said. “It was meant to educate and give people an understanding of the problems we’re up against.”

This format received relatively high marks from pollsters, including one Republican who called it “pretty impressive” because it offered a wide range of trade-offs instead of simply positive or negative replies to individual questions.

“I like the approach,” said Lewis of The Times Poll. “The person was trying to do the right thing, but didn’t know what he was doing.”

In addition to endemic problems with sampling and self-selection, Lewis said the format was extremely complex and demanding.

Jim Salomon, a Beverly Hills Republican who opposed Beilenson in 1988 and is running again, said the incumbent’s survey failed to include important options, such as eliminating farm subsidies or reducing Amtrak subsidies for long trips covered by airline routes.

But, ironically, the challenger said that Beilenson’s results “seem to be relatively consistent” with Salomon’s scientific polling of the 23rd District on issues included in both surveys.

Moorhead’s 1989 questionnaire also tended to offer constituents a range of choices, asking them to identify the most important problem facing the nation--from seven options--as well as the steps that should be taken to further stabilize international relations--from five options.

In contrast to Gallegly’s respondents, Moorhead’s 22nd District respondents differed from the lawmaker’s positions on some issues.

For instance, 57% said that having stricter gun control laws “would reduce the amount of violence in this country.” Another 39% said that it would not do so. Moorhead, who generally opposes stricter gun control laws, said this result “somewhat surprised” him.

David Joergenson, Moorhead’s press aide, said he picked the questions from professional surveys published in the National Journal and elsewhere.

“People love to have the opportunity to express their position, and it stimulates a lot of letter writing, so I get a better feel for how people are thinking,” Moorhead said.

“It is not a scientific survey, it is true,” he said. “Nothing you can do really is totally scientific. You can go to hundreds of meetings and talk to hundreds of people in the district, and that’s not a scientific survey either. You can get thousands of letters, and that’s not a scientific survey either.”

These lawmakers and their spokesmen said they view the surveys as part of the process of soliciting the views of their constituents. Other methods include town hall meetings, events in the district, sessions with constituents, and the mail they receive.

None, however, could recall ever switching or rethinking his position on an issue because of questionnaire results.

“If 90 to 95% of our constituents wanted a tax increase he would, I’m sure, look at it,” Frith said of Gallegly. The lawmaker generally opposes tax increases.

In contrast to her California colleagues, Linda DiVall, a Republican pollster based in Alexandria, Va., said the surveys hold some value for members of Congress.

“By the response they get back, they will begin to notice some intensity on certain issues,” she said. “And they begin to build a loyal audience that begins to give feedback.”

But Field said that if members of Congress really want to solicit their constituents’ sentiments on issues, they could abandon the mailings, replete with their pictures and names prominently displayed, and hire a professional polling firm to conduct a telephone survey of 1,000 representative respondents. He said this would be less costly as well.

Moorhead rejected that suggestion because it would deny all but a handful of constituents the chance to communicate with their representative. Moreover, he said it is not surprising that Field and his cohorts would debunk congressional surveys because “pollsters are in business. Naturally they don’t like anything that takes business away from them.”

The questionnaires are not the only form of “franked” mail that members of Congress send out at public expense. As part of a package of ethics reforms passed by Congress last year, each member is allowed to send only three district-wide mailings annually instead of six. This may reduce the use of surveys by some members.

Two other Valley-area lawmakers, Reps. Howard L. Berman (D-Panorama City) and Henry A. Waxman (D-Los Angeles), already had abandoned the questionnaires in recent years.

“It’s not really the most efficient way to get a sense of what constituents think,” said Philip M. Schiliro, Waxman’s administrative assistant. “What you get isn’t a really good reflection.”

Schiliro said that constituent letters, in which “people really flesh out their thoughts,” provide a better barometer.

But for those who continue to use them, constituent surveys sometimes provide a self-perpetuating rationale.

When Gallegly’s 1988 GOP primary opponent charged that he was abusing his publicly funded mailing privileges with self-aggrandizing newsletters, the Simi Valley lawmaker replied that his constituents wanted to continue receiving at least as many flyers as he was already sending them.

He cited the overwhelming agreement with his survey assertion: “Congressional newsletters sent over the past year have been helpful in understanding events taking place in Congress.”

In addition to loading the question with the words “helpful” and “understanding,” pollster Lewis said it played on people’s reluctance “to be insulting or unpleasant.”

But, most significantly, the favorable figure Gallegly cited was based on replies from a tiny percentage of the 21st District households that filled out the surveys--not a clear majority of his district, as the lawmaker suggested.

“I’m sure the ones who don’t like to see these threw them away or said they don’t know,” Lewis said. “Who would get to the very bottom and say they don’t want these things? He’s pre-selected his answers.”