Saturday, November 08, 2008
The Problems With DJ Drummond's 2008 Election Analysis
DJ Drummond is a writer at WizBang. Before the 2008 election, he wrote several articles theorizing that the polls were wrong in showing Obama with a commanding lead for the election. Using his knowledge of statistics, he claimed several systematic flaws in the polls and a better way of analyzing the raw polls using historical weightings. One thing that was a bright spot for DJs articles was thye first paragraph of his Accountability article. There he made some very good points about being honest and being responsible for what he writes. Given his history, I was very interested to see if he would manage to live up to that after the election was completed and the results were known. Sadly, he did not (see Issue #1). So, I have taken it upon myself to document where he was wrong and why. I just have seven issues at the moment, but I expect to be adding more as time goes by and I have a chance to document them.
Issue #1:
DJ has cut off comments on that Wizbang after too many people called him on his multiple errors, logical flaws, and self-forgiving rhetoric. Eric and mantis already did a good job of bringing up points that I was going to make myself. DJ, being the accountable person that he is, decided to post a non-responsive reply, accusing mantis of lying without providing any evidence, claiming vindication, and having the last word. He did nothing to address the arguments mantis and Eric made, especially the one about using old polls for comparison (see Issue 2).
Issue #2:
After the election, DJ, arguing that his analysis should be considered correct, takes polls from two weeks before the election. Of course, the actual polls taken just before the election had converged to a 7.6% advantage for Obama, withing 1% of the actual margin. Naturally, there was more uncertainty and variance 2 weeks before the election. DJ completely ignores this fact. If his analysis about how the polls had systematic flaws was correct, then those flaws should have still been there on election day.
Issue #3:
A portion of this article is devoted to the premise that "turnout this year was down, not up. Down by more than five million votes from 2004. Somebody did not bother to vote this year." I questioned this premise, since DJ was comparing the final tallies of 2004 (123,535,883) with the current, unfinished tallies from this election. At the time, the spread was 5 million votes, as DJ said. According to DJ's CNN link, the current total for McCain and Obama, as of 11/8, stands at 122,852,251 with 99 percent of precincts reporting, so there are still some more votes yet to count. However, this still leaves out two things. First, the CNN site just has Obama and McCain vote tallies. If you add the votes for other candidates, the total is over 1 million votes higher. The total for just Bush and Kerry was 121,069,054. The other thing is that there are still absentee ballots that haven't been counted yet. Even if a precinct has reported, these can still be added to the final total. Here is an independent and much more detailed analysis of what is left outstanding. While it does seem very unlikely that the total will reach the 130 million that was being reported the morning after the electon, 125 million certainly seems within reach. There are already more total votes than in 2008, so at least numerically, this article’s premise on that point is not correct.
Issue #4:
I was speaking of the 2006 article and your reiteration of the claim here. I didn't mention poll analysis, so I'm not sure how DJ got that idea. As he says, this was a prediction based on opinion. However, he had very specific reasons that he believed this would happen. He believed that Republicans in general will put the country ahead of everything else, and the voters know this. Did McCain lose because Republicans don't put the country first or because the voters didn't realize it?
Of course he also believed that the Democrats would nominate a Senator, and the Republicans wouldn't. He further challenged another commenter:
Issue #5:
Let's get down to Historical Weighting, the basis for much of DJ's analysis over the past few weeks. Specifically, we'll turn to his Turners article. Most polls had several states solidly Obama and several solidly McCain, with fewer swing states. We'll use FiveThirtyEight for comparison. They had CA,CT,DC,DE,HI,IA,IL,MA,MD,ME,MI,NJ,NY,OR,RI,VT,WA,WI as "Safe Obama"; CO,MN,NH,NM,NV,PA,VA as "Likely Obama"; OH as "Lean Obama"; FL,IN,MO,NC as TossUp; GA,MT,ND as "Lean McCain"; AZ,LA,SC,SD as "Likely McCain"; and AK,AL,AR,ID,KS,KY, MS, NE,OK,TN,TX,UT,WV,WY as "Safe McCain". The election went almost perfectly this way. Their percentage for popular vote was also almost exactly correct. In DJ's article, historical weighting produced the following bizzare schisms with reality.
It showed CO a "lock" (75.0%) for McCain, as opposed to "Likely Obama".
It showed FL a "lock" (70.6%) for McCain, as opposed to "Tossup".
It showed IN a "lock" (80.9%) for McCain, as opposed to "Tossup".
It showed NH a "lock" (75.5%) for McCain, as opposed to "Likely Obama"
It showed OH a "lock" (71.7%) for McCain, as opposed to "Lean Obama"
It showed VA a "lock" (79.4%) for McCain, as opposed to "Likely Obama"
It showed ME a "toss-up" (51.6%) for Obama as opposed to "Safe Obama"
It showed MI a "toss-up"(53.3%) for McCain as opposed to "Safe Obama"
It showed PA a "toss-up"(51.5%) for Obama as opposed to "Likely Obama"
It showed WI a "toss-up"(52.3%) for Obama as opposed to "Safe Obama"
It showed OR a "toss-up" (50.2%) for Obama as opposed to "Safe Obama"
5 of DJ’s 6 toss-ups were double-digit wins for Obama. Only NC was an actual toss-up. Additionally, 6 or DJ’s 17 locks for McCain went to Obama. This analysis was quite far from reality, much farther than the national and state polls at that point.
Issue #6:
Of course, the actual numbers showed no such thing. Illinois was never in question for Obama. North Carolina was very close, and did go to Obama.
Issue #7:
The closest thing to a prediction based on your historical norm statistical analysis that I could find was:
If DJ was right, then McCain, who finished with 46% and 163 electoral votes barely got anything after that. Meanwhile Obama went from 47% to 53% and claimed around 300 Electoral Votes. That happened in one week. Is it easier to believe that almost all the undecided voters broke for Obama or that DJ’s analysis was very inaccurate? While he does not call this definitive, that’s just enough to give him bragging rights in the unlikely event he was right, but deniability if he was wrong.
Issue #1:
DJ has cut off comments on that Wizbang after too many people called him on his multiple errors, logical flaws, and self-forgiving rhetoric. Eric and mantis already did a good job of bringing up points that I was going to make myself. DJ, being the accountable person that he is, decided to post a non-responsive reply, accusing mantis of lying without providing any evidence, claiming vindication, and having the last word. He did nothing to address the arguments mantis and Eric made, especially the one about using old polls for comparison (see Issue 2).
Issue #2:
First, the election results. At this time, President-elect Obama has won 52% of the popular vote, while Senator McCain has received 46% of the popular vote. Against that, let's look at what the polls were saying on October 21...
After the election, DJ, arguing that his analysis should be considered correct, takes polls from two weeks before the election. Of course, the actual polls taken just before the election had converged to a 7.6% advantage for Obama, withing 1% of the actual margin. Naturally, there was more uncertainty and variance 2 weeks before the election. DJ completely ignores this fact. If his analysis about how the polls had systematic flaws was correct, then those flaws should have still been there on election day.
Issue #3:
A portion of this article is devoted to the premise that "turnout this year was down, not up. Down by more than five million votes from 2004. Somebody did not bother to vote this year." I questioned this premise, since DJ was comparing the final tallies of 2004 (123,535,883) with the current, unfinished tallies from this election. At the time, the spread was 5 million votes, as DJ said. According to DJ's CNN link, the current total for McCain and Obama, as of 11/8, stands at 122,852,251 with 99 percent of precincts reporting, so there are still some more votes yet to count. However, this still leaves out two things. First, the CNN site just has Obama and McCain vote tallies. If you add the votes for other candidates, the total is over 1 million votes higher. The total for just Bush and Kerry was 121,069,054. The other thing is that there are still absentee ballots that haven't been counted yet. Even if a precinct has reported, these can still be added to the final total. Here is an independent and much more detailed analysis of what is left outstanding. While it does seem very unlikely that the total will reach the 130 million that was being reported the morning after the electon, 125 million certainly seems within reach. There are already more total votes than in 2008, so at least numerically, this article’s premise on that point is not correct.
Issue #4:
If you mean the 2006 article, Joe, that was opinion, not analysis. If you are claiming that I predicted a McCain victory from poll analysis, you are claiming something other than what I said.
I was speaking of the 2006 article and your reiteration of the claim here. I didn't mention poll analysis, so I'm not sure how DJ got that idea. As he says, this was a prediction based on opinion. However, he had very specific reasons that he believed this would happen. He believed that Republicans in general will put the country ahead of everything else, and the voters know this. Did McCain lose because Republicans don't put the country first or because the voters didn't realize it?
Of course he also believed that the Democrats would nominate a Senator, and the Republicans wouldn't. He further challenged another commenter:
Show me where I specifically predicted a McCain win, Crusty. Link and specific quote.but cut off comments on that thread before he could be answered. Unless he want's to claim that McCain was not the Republican nominee, I believe that has been adequately answered.
Issue #5:
Let's get down to Historical Weighting, the basis for much of DJ's analysis over the past few weeks. Specifically, we'll turn to his Turners article. Most polls had several states solidly Obama and several solidly McCain, with fewer swing states. We'll use FiveThirtyEight for comparison. They had CA,CT,DC,DE,HI,IA,IL,MA,MD,ME,MI,NJ,NY,OR,RI,VT,WA,WI as "Safe Obama"; CO,MN,NH,NM,NV,PA,VA as "Likely Obama"; OH as "Lean Obama"; FL,IN,MO,NC as TossUp; GA,MT,ND as "Lean McCain"; AZ,LA,SC,SD as "Likely McCain"; and AK,AL,AR,ID,KS,KY, MS, NE,OK,TN,TX,UT,WV,WY as "Safe McCain". The election went almost perfectly this way. Their percentage for popular vote was also almost exactly correct. In DJ's article, historical weighting produced the following bizzare schisms with reality.
It showed CO a "lock" (75.0%) for McCain, as opposed to "Likely Obama".
It showed FL a "lock" (70.6%) for McCain, as opposed to "Tossup".
It showed IN a "lock" (80.9%) for McCain, as opposed to "Tossup".
It showed NH a "lock" (75.5%) for McCain, as opposed to "Likely Obama"
It showed OH a "lock" (71.7%) for McCain, as opposed to "Lean Obama"
It showed VA a "lock" (79.4%) for McCain, as opposed to "Likely Obama"
It showed ME a "toss-up" (51.6%) for Obama as opposed to "Safe Obama"
It showed MI a "toss-up"(53.3%) for McCain as opposed to "Safe Obama"
It showed PA a "toss-up"(51.5%) for Obama as opposed to "Likely Obama"
It showed WI a "toss-up"(52.3%) for Obama as opposed to "Safe Obama"
It showed OR a "toss-up" (50.2%) for Obama as opposed to "Safe Obama"
5 of DJ’s 6 toss-ups were double-digit wins for Obama. Only NC was an actual toss-up. Additionally, 6 or DJ’s 17 locks for McCain went to Obama. This analysis was quite far from reality, much farther than the national and state polls at that point.
Issue #6:
Chicago is getting really, really interesting. Yeah, it's strange that McCain should be close at all in Illinois, but there's some blue-collar backlash that turned Indiana around and has started moving some Illinois opinion. Like other states, I think the movement is temporary; Obama will no more lose Illinois than McCain will lose North Carolina, but I just report what the numbers show, and weird they are..."
Of course, the actual numbers showed no such thing. Illinois was never in question for Obama. North Carolina was very close, and did go to Obama.
Issue #7:
The closest thing to a prediction based on your historical norm statistical analysis that I could find was:
I will not call it definitive, but in my opinion if the demographic weighting is corrected the popular vote becomes Obama 46.9%, McCain 46.6%, but with McCain taking the electoral vote 278-260. When the shadow effect is applied, the electoral numbers change to 147-71 McCain, with 320 to be decided. The message is clear then, that the race remains to be decided.
If DJ was right, then McCain, who finished with 46% and 163 electoral votes barely got anything after that. Meanwhile Obama went from 47% to 53% and claimed around 300 Electoral Votes. That happened in one week. Is it easier to believe that almost all the undecided voters broke for Obama or that DJ’s analysis was very inaccurate? While he does not call this definitive, that’s just enough to give him bragging rights in the unlikely event he was right, but deniability if he was wrong.
