Pollsters say they weren't wrong; voters just changed
Share with others:
Last week, it wasn't easy being a pollster.
In Tuesday's New Hampshire Democratic primary the polls, and thus, the news media, had the race clinched for Sen. Barack Obama of Illinois, with some predicting a double-digit margin of victory.
But when the votes were counted, Sen. Hillary Rodham Clinton of New York ended up with 39 percent to Mr. Obama's 37 percent -- giving the Clinton campaign new life and leading to angry cries from the news media and public asking how the polls could have been so wrong.
Pollsters argued that the polls weren't wrong -- there was just a change in the thinking of voters, especially women, who suddenly became eager to support an underdog, emotional Mrs. Clinton.
But a variety of factors has made polling more difficult now, and pollsters are struggling to adapt.
For starters, a growing proportion of households do not have a land line, using only cell phones. Since those people tend to be younger, pollsters, whose standard method of polling is cold-calling randomly selected home phones, give extra weight to young people who do respond.
"But at some point you begin to worry about the consequence of making one person stand in for two," said Scott Keeter, director of survey research at the Pew Research Center. "And you worry about whether the cell-phone-only young people are different from the ones you can reach."
Even those who do have land lines can be tough to corral, since call screening has increased.
"People just simply don't pick up the phone anymore," said Susan Hunter, an associate professor of political science at West Virginia University who has done polling since 1980. "I don't pick up the phone unless I know who's calling."
That means that pollsters get lumped in with organizations soliciting money, many of whom disguise their efforts by trying to look and sound like pollsters. Some phone "surveys" -- whether from commercial ventures or nonprofit organizations -- will conclude with a request for money.
"All these seemingly good, thoughtful, well-intentioned organizations are doing this," said Robert P. Daves, a market and opinion research consultant and past president of the American Association of Public Opinion Research.
"It creates confusion in the mind of the public as to who are the legitimate researchers."
The result has been steadily declining response rates. Even the best polls, Mr. Keeter said, get only a 20 to 25 percent response.
In an attempt to counteract this trend, the polling industry has proposed a variety of methods to increase response and get a more accurate view of the population.
Some polls are calling people on cell phones, which isn't considered as much of an intrusion anymore.
There also has been a sharp rise in Internet surveys. As Ms. Hunter described it, a group of people that has agreed to be contacted beforehand receives an e-mail with a link to a survey. They know it's coming so they don't think the polls are spam, and they have more time to think through their responses.
Though technological advances are an impetus for changes in polling, traditional polls have remained quite accurate.
In 2004, Mr. Keeter said, the average of all the final polls before the presidential election was only one-half of a percentage point from the actual vote. With more variation in the multitude of races, polling had similar success in predicting the 2006 midterm elections.
"We're having these problems, but it hasn't really led to the kind of spectacular failure we saw [Tuesday] night," Mr. Keeter said. "I think this situation [in New Hampshire] is somewhat unique."
Indeed, there were several factors that might have fouled up poll results and swung the Granite State race in the final hours.
First was the media frenzy around Mrs. Clinton in the closing days, starting with reports that a decisive loss would be a rebuke of the Clinton "dynasty" and could sink her candidacy. The coverage then shifted Monday when she seemed on the verge of tears at an event.
Both of those factors may have caused voters to turn between the weekend polls and Tuesday. In the CNN/WMUR primary tracking poll, conducted by the University of New Hampshire, only 53 percent of respondents said they had "definitely decided" on a candidate.
But in exit polls from the primary, among voters who had decided either "today" or "in the last three days" support for Mr. Obama and Mrs. Clinton was virtually even.
Polling likely Democratic primary voters also was tricky, because independents could choose to vote in either the Democratic or Republican primary, and both parties had candidates who appealed to independents -- Mr. Obama on the Democratic side and Sen. John McCain on the Republican side. Mr. McCain, of Arizona, won the Republican primary as the polls predicted.
Another theory for the polling problem was the so-called Bradley Effect, in which voters in biracial races won't say they don't support a black candidate because it's politically incorrect -- leaving the polls incorrect, politically.
This secret bigotry then manifests itself in the privacy of the voting booth. It is named for Tom Bradley, the black Democratic mayor of Los Angeles who ran for governor of California in 1982 against a white Republican, George Deukmejian. Though pre-election polls showed Mr. Bradley to have anywhere from a 9 to 22 percentage point lead, he narrowly lost on election day.
Mr. Keeter has studied the Bradley Effect for years and said that while it was apparent in the 1980s and early '90s, it appeared to have waned in several 2006 elections in which polling proved accurate when a black candidate ran against a white one.
"It looked like this phenomenon might be behind us, but the results [Tuesday] night raised the question of whether that was the case," Mr. Keeter said.
No matter the reason, pollsters took a beating last week. But one state's results are too small a sample size to judge the entire polling apparatus.
Mr. Keeter has an eye on Super Tuesday, in which the polls will be tested by caucuses and primaries in 24 states.
"Tuesday, Feb. 5," he said. "If the polls are good then, I'll feel much better."
First Published January 13, 2008 12:00 am