Page One

Debunking the Pollster Myth: Biased Sources Skew Results

By MARTY SCHIFFENBAUER
Friday May 09, 2003

Why do we believe a large majority of the U.S. public approves of President Bush’s job performance? We believe it because that’s what the pollsters tell us. 

But what if the polls are rigged? Not deliberately rigged, with the numbers simply made up. But biased in a more subtle manner, the way SAT scores are biased. 

Polls don’t merely reflect public opinion. They’re often a significant factor in shaping public opinion. A poll pronouncing overwhelming support for the president will discourage some of his opposition and stifle their criticism. And news that Bush is exceptionally popular will sway many in the center, having a weak or neutral opinion, to join the majority in a kind of bandwagon effect. 

Consequently, public opinion polls can become a self-fulfilling prophesy. 

There are relatively few prominent U.S. pollsters whose findings are widely disseminated. Gallup is the most recognizable polling name, and polls conducted by media partnerships such as New York Times/CBS News and NBC News/Wall Street Journal are also influential. In California, the Field Poll is the best known. 

Since polls play an important role in shaping public opinion and there are only a handful of top polling organizations, it’s tempting to presume a conspiracy where opinion polls are manipulated to promote the message of the “ruling class.” It seems more plausible that rather than deliberate manipulation, the methods used by the pollsters typically guarantee that their findings confirm and advance the views of those running the country. Nonetheless, whether public opinion polls are manipulated or methodologically biased, their impact winds up being the same — to inhibit dissent and swing undecideds to the majority position. 

Is there a way to diminish the power of the polls? My personal reaction to a recent national poll by the New York Times/CBS News may be instructive. 

The poll, as reported in the April 15 New York Times, found that “73 percent of Americans approve” of President Bush’s job performance. This, said the Times, was a 14 percent jump from Bush’s 59 percent approval rating the week before the Iraq war started. What’s more, “61 percent” of Democrats, the poll indicated, were now in the Bush camp. 

Given my strong convictions that Bush’s presidency has been a disaster, domestically and internationally, when I saw his huge approval numbers in the Times, my heart sank. And I was sufficiently disgusted and dismayed by the poll results that a part of me just wanted to throw in the towel and move to Canada. 

However, after a little moaning and groaning, the “Question Authority” module in my brain began firing away. First off, I did a bit of mental arithmetic. If 73 percent of the people polled voiced approval of President Bush, this means 27 percent declined to declare such support. That’s a decent number, considering the public has been bombarded with a vast amount of pro-war propaganda and one-sided media coverage. Not to mention that even mild critics of Bush have been viciously attacked, ranging from Vietnam war hero Sen. John Kerry (D-Mass.) to the Dixie Chicks. 

Therefore, it’s impressive that 27 percent of those confronted by some stranger on the phone in the middle of dinner had the courage to withhold support of the Commander in Chief while U.S. troops were in combat. One wonders how many of the 73 percent who identified themselves as Dubya fans, but really weren’t, might be willing to express their true feelings in the secrecy of a voting booth. 

I next carefully reread the Times report of the poll results, paying particular attention to the section describing “How the Poll Was Conducted.” According to the Times, the pollsters only phoned “898 adults throughout the United States.” A quick Google search discovered the U.S. adult population is about 215 million. Plugging this number into a calculator revealed that each of the 898 people polled was supposedly a proxy for roughly 240,000 U.S. residents. 

To ensure these proxies were representative, the Times explained, the polling sample was weighted to take account of “geographic region, sex, race, age and education.” How was the weighting system determined? 

The Times doesn’t inform us, yet it does point out the potential for sampling error. And with a single poll respondent standing in for 240,000 other individuals, there wouldn’t need to be much error in the assumptions employed in devising the weighted sampling system to invalidate the poll’s findings. Along with sampling error, the Times further qualified the poll numbers by noting that the specific “wording and order of questions” asked by the pollsters could have compromised the results. Would it be a surprise if the guys writing the poll questions were burdened by the same racial and class biases as the guys writing the SAT questions? 

Deconstructing the details of the Times/CBS poll, my confidence in the results markedly dwindled. But my guess is most people, as I did initially, accepted the 73 percent approval margin at face value. And by creating the widespread impression that Bush’s popularity had soared, the poll likely helped shape the opinions and actions of a substantial segment of the U.S. public, whatever their political tendencies. 

What can opponents of Bushism do to limit the influence of slanted poll results? For starters, we can minimize their depressing effect on our own psyche by treating super-high Bush poll ratings with lots of skepticism. It’s worth remembering that Daddy Bush had great poll numbers following Iraq War I, yet still lost to Clinton. 

In addition, progressives need to challenge the media to evaluate public opinion polls more critically, scrutinizing the validity of the numbers and publicizing the biases inherent in the pollsters’ methodology. Perhaps, as well, progressive organizations should begin to do some counter-polling. For example, they could do a poll to gauge whether Americans have become more fearful to express certain opinions to pollsters; notably, their opinion of the president and his policies. 

 

Marty Schiffenbauer is a Berkeley resident.