2008 Pollster Report Card
The table below is an assessment of pollster performance in the 2008 Presidential election. The pollsters were graded on both the accuracy of their final poll (popular vote) and the consistency of their polling during the month of October.
The overall score is a weighted average of their Accuracy and Consistency numbers. The weighting is 75% accuracy and 25% consistency. Formula details are at the bottom of this page.
|NBC News / Wall St. Journal||76%||C||77%||75%|
|ABC News / Wash Post||67%||D+||62%||82%|
|CBS News / NYT||60%||D-||62%||56%|
The data for this chart come from the Real Clear Politics summary on 12 November 2008. Here is a local mirror of that data, with the final polls summarized at the top. Here is a quick summary of the final spread predicted by the pollsters, ranked from most accurate to least:
|Poll||Obama||McCain||Spread (actual=6.5)||Off by|
|NBC News / Wall St. Journal||51||43||8||1.5|
|ABC News / Wash Post||53||44||9||2.5|
|CBS News / NYT||51||42||9||2.5|
Every final poll showed Obama with a lead, so assessing their accuracy is just a matter of comparing their forecasted spread with the actual spread of the election which was Obama +6.5.
Final poll accuracy score = 100 – ( |(POLL SPREAD – 6.5)| / 6.5 )
All the pollsters graded had more than one poll during the month of October. If a poll reported a wide spread and then suddenly narrowed down right before the election, this formula produces a lower score. Likewise a pollster gets a lower score if they predicted a very tight race until the end and then widened up.
October Consistency = 100 – ( POLL STDEV / POLL AVERAGE )
The List: Which presidential polls were most accurate?
The Pew Research Center and Rasmussen Reports were the most accurate in predicting the results of the 2008 election, according to a new analysis by Fordham University political scientist Costas Panagopoulos.
The Fordham analysis ranks 23 survey research organizations on their final, national pre-election polls, as reported on pollster.com.
On average, the polls slightly overestimated Obama’s strength. The final polls showed the Democratic ahead by an average of 7.52 percentage points — 1.37 percentage points above his current 6.15-point popular vote lead. Seventeen of the 23 surveys overstated Obama’s final victory level, while four underestimated it. Only two — Rasmussen and Pew — were spot on.
Here is the list –
1T. Rasmussen (11/1-3)**
1T. Pew (10/29-11/1)**
3. YouGov/Polimetrix (10/18-11/1)
4. Harris Interactive (10/20-27)
5. GWU (Lake/Tarrance) (11/2-3)*
Gauging the Accuracy of Election Polls | Pew Research Center
Although polls have occasionally failed to predict who will win an election (most recently in the 2008 Democratic primary election in New Hampshire), polling’s track record is actually very good. The National Council for Public Polls has conducted analysis of presidential election polling accuracy from 1936 to the present and provides reports summarizing these results on their website.
The good track record of final pre-election polls does not mean that all pre-election polls are reliable. Polls conducted early in an election season should be taken as snapshots in time, and obviously cannot capture the impact of the campaign and events to come. This publication examines presidential election polls conducted well in advance of the election and attempts to gauge how predictive they are:
- How Reliable Are the Early Presidential Polls? February 14, 2007
These publications provide a few tips to help in reading polls and deciding how much weight to give them:
- The Bounce Effect September 11, 2008
- Beware of the Bounce August 4, 2000
- Another Bush Lead Vanishes? July 14, 2000
- So Who’s Ahead? April 14, 2000
Other Posters were rated on the Margin of the win, not the actual numbers. Here is a report that shows the most accurate by margin in 2004 and 2008. In this article, TIPP/IBD was claimed the winner. Additionally, in this measure of margin accuracy, the top pollsters in 2004 were TIPP, CBS, Pew and Rasmussen. Worst pollsters were Battleground, Fox News, and Marist. In 2008 the top pollsters were TIPP, Fix and 2 others. Worst: Battleground and Zogby.
Pollster Accuracy in the National Polls – Pollster.com
If we are going to try to compare pollsters, the wisest thing to do is to measure accuracy across as many polls as possible, because the role of random chance will gradually diminish as the number of polls examined increases.
Unfortunately, that observation is not stopping a lot of reporters and observers from scanning the final national polls and trying to identify winners and losers. So before moving on to more elaborate aggregations, let’s look at the list the final national poll conducted by 19 different organizations over the final week of the campaign. Looking first at the final survey results (as opposed to “projections” that allocated the undecided), we see that all of the polls had Obama leading by margins of 5 to 11 percentage points. A straight average of these surveys shows Obama leading by 7.6% (51.4% to 43.8% ).
How did these polls compare to the actual results? First, let’s keep in mind that provisional and late arriving mail-in ballots are still being counted in some places (and may not be reflected in the “99% of precincts counted” statistics typically provided by the Associated Press). The most current and complete national count I can find now shows Obama with a 6.6% lead in the national popular vote (52.7% to 46.1%). Obama’s margin has increased by about a half a percentage point over the last week and (if the pattern in 2004 is a guide) may increase slightly more as secretaries of state release their final certified results.
Hopefully these presentations from multiple sources will give you some insight into which polls you should pay attention to and which ones you may dismiss. While the media seems to grab their favorite polls that Present their desired outcome, you now have in your hand some data that will help you grasp the total view of the 2012 Presidential election.
We will begin our Politisite Political Projections for the election after the October 3rd Presidential Debate. We notice that pollsters get a bit more accurate one month before the election and most people make their decisions for November.
- Forecasting the Presidential Election: Other Crystal Balls (centerforpolitics.org)
- PRUDEN: Why the media spin on the Nov. 6 presidential election is wrong (washingtontimes.com)
- 2008 Vs. Actual Election Results – who were the most accurate pollsters in 2008 Presidential election