Of what use are opinion polls?

News media organizations mount a massive effort to gather exit poll data. In 2004, some 70,000 voters were interviewed at nearly 1,500 polling locations, while another 5,818 telephone interviews were conducted of absentee and early voters.

While an exit poll is a more accurate reading of the electorate than a pre-election poll, it, too, can be inaccurate. Case in point: In the 2004 presidential election, exit polling overstated Kerry’s share of the vote.

Kerry voters more willing to be interviewed
Edison-Mitofsky, the firm that conducted the exit poll, said after investigating its own methodology, the error in the estimate was primarily due to “Kerry voters participating in the exit polls at a higher rate than Bush voters.”

Edison-Mitofsky said, “it is difficult to pinpoint precisely the reasons” that Kerry voters took part in the exit polls at a higher rate than Bush voters, but pointed to one factor: “in this election voters were less likely to complete questionnaires from younger interviewers.”

If the voters avoiding the younger exit poll interviewers were Bush supporters, the exit poll sample would be skewed.

As Darrell Huff explained in his classic 1954 book, "How to Lie with Statistics," “The kind of people who make up the interviewing staff can shade the result in an interesting fashion.”

Interviewers do make a difference 
Huff cited the case of the National Opinion Research Center survey conducted during World War II.

Two sets of interviewers surveyed black residents of a Southern city. One set of interviewers was black, the other white.

One survey question: “Would Negroes be treated better or worse here if the Japanese conquered the United States?”

Black interviewers reported that nine percent of those they interviewed said the Japanese would treat black people better. But white interviewers found that only two percent of those they interviewed thought the Japanese would treat them better.

“The operation of a poll comes down in the end to a running battle against the sources of bias,” Huff said, such as interviewer effects and faulty samples. “The battle is never won.”

This is not to imply that polls are worthless. Combined with other data, polls can be useful — but they are not in themselves an infallible source of truth about why people vote they way they do.

Picking the president: The candidates Click to visit that candidate's MSNBC page or click the XML symbol for an RSS feed. John McCain                Barack Obama

MORE FROM

Section Front

Obama campaign head gets big book deal Officials turn down Franken request Joe the Plumber to become war correspondent Franken lead at 49 with absentees left to count Minn. Senate winner won't be known in 2008 First Read: 'Twas the seventh week of recount... Franken opens first lead in Minn. Senate race Mom of Palin daughter's boyfriend arrested Minn. court weighs rejected absentee ballots Minnesota panel reviews disputed ballots Section Front   Add headlines to your news reader: