2004 United States presidential election controversy, exit polls

{{subst:#ifeq:a|b||{{subst:#ifexist:Wikipedia:Articles for deletion/{{subst:PAGENAME}}|{{subst:lessthan}}!-- The nomination page for this article already existed when this tag was added. If this was because the article had been nominated for deletion before, and you wish to renominate it, please replace "page={{subst:PAGENAME}}" with "page={{subst:PAGENAME}} (2nd nomination)" below before proceeding with the nomination. -->}}}}

{{subst:lessthan}}!-- Once discussion is closed, please place on talk page:

-->

Parent article: 2004 U.S. presidential election controversy

After the 2004 U.S. presidential election there were allegations of data irregularities and systematic flaws which may have affected the outcome of both the presidential and local elections. Unofficial results currently indicate a victory by George W. Bush over John Kerry. Allegations range from significant exit poll and other data irregularities potentially characteristic of fraud, to complaints voting was not conducted equally for all citizens, for example, uneven voting machine distribution which might lead to long voting lines and disenfranchisement.

Exit polls

Exit polls have been used in other countries to attempt to check for election fraud. For example, in the Venezuelan recall referendum, 2004 exit polls showed 59 percent in favor of the recall, while the official tally showed only 42 percent in favor. [1] This discrepancy of 17 percent is larger than claimed for the 2004 US election. International observers nonetheless endorsed the Venezuelan result. [2] The "parallel vote tabulation" [3] is also expected to be a key factor in resolving problems with the Ukrainian_presidential_election,_2004 in which the US and other nations are alleging serious election fraud..

Because final published exit polls in America are matched to vote counts, they cannot be used to determine election fraud. However, in the 2004 election, pre-matched exit polls were leaked onto the internet. The exit poll results of one major consortium of news organizations, the National Election Pool ("NEP"), were based on interviews with voters in 49 states (omitting Oregon because its system of voting by mail eliminated the traditional polling place). The NEP results available during the day on Election Day showed Kerry leading Bush. These discrepancies led to charges that the exit polls were more accurate than the official counts, for various reasons. The co-director of NEP, Warren Mitofsky, said he suspected that the difference arose because "the Kerry voters were more anxious to participate in our exit polls than the Bush voters." [4]. Other academic analysts conclude that such explanations are poorly supported.

(Reliability of Exit Polls as a predictor of election popular vote is discussed below #Reliability of Exit Polls. They are said to be consistently very accurate, often within fractions of a percentage point, across many elections and many countries, including USA)

Result plots - Florida

See http://www.ustogether.org/election04/florida_vote_patt.htm , http://ustogether.org/election04/mitteldorf/Liddle.htm , and http://ideamouth.com/voterfraud.htm

The Florida plots on ustogether.org are compiled from the following sources amongst others:

• Official Florida votes by county: [5]
• Registered Voter Affiliation by County: [6]
• Election machines in use by county: [7]
• Summary table used in analysis: [8]

Criticisms of data:

1. Minor differences are noted in the data versus the summary table, for example Alachua County is officially reported to have 47,762 REP // 62,504 DEM, wheeras in the summary table the figures are 47,615 REP // 62,348 DEM (equivalent to -0.3% REP // -0.25% DEM)
2. Expected Votes are calculated on the basis of Registered Voters and not native exit polls (ie unfixed to match official voting). (Native poll data has not in general been released by NOP). Registration may differ from voting across party lines.

Discrepancies map

Voting locations that used electronic or other types of voting machines that did not issue a paper receipt or offer auditability correlate geographically with areas that had discrepancies in Bush's favor between exit poll numbers and actual results. Exit polling data in these areas show significantly higher support for Kerry than actual results (potentially outside the margin of error). From a statistical perspective, this may be indicative of vote rigging, because the likelihood of this happening by chance is extremely low. A study of 16 states by a former MIT mathematics professor places the likelihood at 1 in 50,000. [9]

File:2004 us popular vote2.gif

Exit polls vs. machine tallies, by state (9 states)

Supporting the same conclusions of the maps above, here are bar graphs indicating the differentials between Exit Polls and Machine Tallies for nine e-voting and paper ballot states. The discrepancies appear to affect the e-voting states to a significantly greater degree than they affect the Paper Ballot states.

File:Exit poll small.jpg Source and background discussion are listed here: http://www.bradblog.com/archives/00000893.htm

Source data and analysis: http://ustogether.org/Florida_Election.htm http://ustogether.org/election04/mitteldorf/Liddle.htm

Corroborating data and analysis for download: Steve Freeman research, ZIP format

Retrospective adjustment of exit poll data

The following tables compare final exit poll data with penultimate exit poll data, note the large swing of support towards Bush, with Kerry losing votes, which is impossible if votes are only being added. National Election Pool, the consortium which conducts exit polls, has stated that the early data was inaccurate due to regulations preventing pollsters from approaching voters, legal barriers, and their belief that Democrats are more willing to answer exit polls. The consortium dismissed the possibility that their early exit poll was accurate and that vote counts were wrong, due to the reasons they provided. The early exit poll data was not meant to be released to the public. The data that was meant to be released to the public was intended to be weighted by the actual vote count. Exit polling companies claim this is standard procedure. Critics argue exit poll data should only ever be weighted by population samples to balance out any differences in sampling, and never normally made to match final results, and have requested access to the raw data.

• Direct links to screenshots and data: CNN website 12.21am CNN website 1.41am, or here.
• In-depth article analysing exit poll accuracy in the US, past track history, pollsters FAQs, and the like here.

At least one academic analysis of the exit polls which attempts to be rigorous, acts as a second source that unaltered data was released on CNN until approximately 1.30 am, as cited above:

"The data I use for this analysis was available apparently only because a computer glitch allowed apparently "uncalibrated" data (not yet "corrected" to conform to announced vote tallies) to remain on the CNN website until approximately 1:30 AM election night.⁵ At that time, CNN substituted data "corrected" to conform to reported tallies. I have attempted to obtain the raw exit poll data from AP, Edison Media Research, Mitofsky International, and the NY Times, but have as yet received no response ⁶" ([10])

It is also said that:

"...the raw data is under the control of the organizations -- ABC, CBS, CNN, FOX, NBC and the Associated Press – that collectively own it. In past years they have deposited the raw data at the archives of the Roper Center [AKA Roper Centre for Public Opinion Research], where it is available for analysis to the general public ..."

Secondary support of the claim that from around 1.30 am on election day, publicised numbers were covertly adjusted or replaced to match those expected or reported, comes from Alan Freeman, cited below, as well as being implied by CalTech/MIT's own statement that they had to estimate the exit poll responses due to lack of genuine raw data.

CNN screenshot #1:

12.21 am, 1963 respondents so far

Total vote: Male 47% , Female 53% of which:

	Male - Bush		47% x 49% x 1963		452
	Male - Kerry		47% x 51% x 1963		471
	Female - Bush		53% x 47% x 1963		489
	Female - Kerry		53% x 53% x 1963		551
	TOTAL - Bush				941
	TOTAL - Kerry				1022

(rounding: estimates of voters in each category accurate within +/- 10)

CNN screenshot #2:

1.41 am, 2020 respondents so far (57 more than above)

Total vote: Male 47% , Female 53% of which:

	Male - Bush		47% x 52% x 2020		499
	Male - Kerry		47% x 47% x 2020		451
	Female - Bush		53% x 50% x 2020		535
	Female - Kerry		53% x 50% x 2020		535
	TOTAL - Bush				1034
	TOTAL - Kerry				986

(rounding: estimates of voters in each category accurate within +/- 10)

The addition of an extra 57 voters at this station was therefore reported as +93 votes for Bush by AP and CNN at least, and voters monitoring the exit polls were told authoritatively that Bush had now taken a lead from Kerry.

Note that the counts for Kerry under Male voters changed in a negative direction after additional voters were included. The net subtraction of 20 votes from the Kerry total after adding new voters seems to reflect an adjustment process.

Reliability of exit polls

1. The same US online encyclopedia cited above ([11]) states that

   "Exit poll data - asking voters which way they voted as they leave the polls - are used around the world as excellent predictors of actual vote counts, usually accurate within a fraction of a point. Exit polls in this election seemed to match the vote tallies, as usual, except in those areas using touchscreen voting machines (like the Diebold Accuvote) or other software or modem-mediated electronic systems (like those from ES&S) with no paper trail - used by approximately one third of voters, many in swing states. 80% of all US voters [emphasis in original article] use some kind of voting machine from one of these two companies."

2. Dick Morris, a career pollster (Republican), states in the Hill News that the Election Night pattern of exit polls versus popular vote in six battleground states - Florida, Ohio, New Mexico, Colorado, Nevada and Iowa - was "virtually inconceivable":

   "Exit polls are almost never wrong ... So reliable are the surveys that actually tap voters as they leave the polling places that they are used as guides to the relative honesty of elections in Third World countries. … To screw up one exit poll is unheard of. To miss six of them is incredible."

   (Speculative material alert: The article goes on to state that these differences demonstrate that the differences were due to "more than honest error". However it then proceeds directly to assumptive hypothesis as follows: "...To miss six of them is incredible. It boggles the imagination how pollsters could be that incompetent." Readers should note that this further hypothesis of pollster error is not in fact supported by a quoted source, nor is any explanation given to justify it, and for Wikipedia purposes is presently an unverified statement. The article has not in fact made any attempt to analyse or justify this assumption as to where the errors lay, but presumes the machine votes were correct and therefore the exit polls necessarily "bogus". However this does not cast doubt over the statement as to general accuracy of exit polls per se, which agrees with information from other sources. [12]

3. Thom Hartmann states that in Germany, [13]

   "... people fill in hand-marked ballots, which are hand-counted by civil servants, watched over by volunteer representatives of the political parties. ... It's totally clean, and easily audited. And even though it takes a week or more to count the vote ... the German people know the election results the night the polls close because the news media’s exit polls, for two generations, have never been more than a tenth of a percent off."

4. Students at BYU have been conducting Utah exit polls (see "Navigation" links) since 1982. They write:

   "... [the] results are very precise; In the 2003 Salt Lake County mayoral race, the KBYU/Utah Colleges Exit Poll predicted 53.8 percent of the vote for Rocky Anderson and 46.2 percent for Frank Pignanelli. In the actual vote, Anderson carried 54 percent of the vote to Pignanelli’s 46 percent ... In the Utah presidential election, for example, they predicted Bush 70.8%, Kerry 26.5%. The actual was Bush 71.1%, Kerry 26.4%. Consistently accurate exit poll predictions from student volunteers, including in this presidential election, gives us good reason to presume valid data from the world’s most professional exit polling enterprise."

Academic analysis

CalTech/MIT analysis and rebuttal

Other sources have provided explanations for these discrepancies. See for example the Caltech/MIT Voting Technology Project analysis which can be found here:

CalTech/MIT state that in their view there is no significant exit poll difference, and that in summary, these discrepancies are "not new or unique," but are within the expected margin of error.

Others state that the CalTech paper, which is unsigned and lacks statistical rigor, is more of the nature of a position paper than an academic report, and that by its own admission it analyses data already known to be flawed and compromised.

One such study challenging their conclusion is an analysis which argues that the MIT-Caltech study contains several "serious methodological problems", including:

"[...] the CalTech/MIT report used the "corrected" exit poll data now available online at CNN.com [...]. 'They are NOT [the] unweighted, end-of-day poll results that have been the object of all the speculation, the ones that showed Kerry doing better in most states than he did in the actual count"

It also states that CalTech/MIT appear to have analysed and compared the corrected exit poll figures (not the raw ones), which CNN's website showed had been weighted to agree with official results, and that "the CalTech/MIT researchers probably grabbed results for [Rhode Island, Oklahoma and New York] before NEP got around to weighting them to match actual results"

The analysis concludes that Caltech/MIT were probably comparing exit data which was adjusted to fit with official figures, against the same official figures ... "Thus, the charts in the CalTech/MIT report don’t really tell us much. They are essentially analyzing rounding error ... More important, as noted here before, the analysis ... could be done with far more precision and power using the raw exit poll data. The exit polls track type of voting equipment down to the precinct level ... [so] this would be a very easy theory to 'debunk'. Unfortunately, NEP officials are so far reticent to discuss their data."

A further analysis of the reliability of the Caltech report together with sourced links for verifiability can be found here. It highlights the following concerns with the Caltech report:

1. CalTech used questionable source data adjusted ex-post facto (by their own statement they took evening data; ex post facto adjustments were noted (see above) as early as 1.30 am). A footnote states "The exit poll data were taken from the cnn.com web site ... because the web site does not report the bottom line candidate percentages directly, we had to calculate them from the demographic breakdowns. In this case, we estimated the Bush percentage of votes in the exit polls using the gender breakdown."
2. It alleges that Caltech/MIT also used "flawed" 2000 data.
3. The CalTech/MIT paper is not in fact a scientifically drafted analysis, it is more a "position paper", and is described as "anything but systematic". There are few substantiated attempts to analyse the matter as a formal statistical study, it contains many numbers but few calculations and a lot of reliance on graphs whose assumptions, weaknesses or significance are not carefully analysed.
4. The heads of the Voter Technology Project (VTP) have a variety of documented strong political ties to right wing thinktanks, voting machine companies and their owners, and the like, including the Hoover Institution (which has very strong links to the Bush administration), "right-wing evangelical Christian and Republican circles" and "the highly secretive far-Right Council for National Policy", and the Urosevich brothers referred to above under Diebold and ES&S.
5. They have already been implicated in scientific fraud elsewhere.
6. The paper is unsigned, unlike other CalTech papers, which is not in accordance with any usual professional scientific standard.
7. Behind the graphs, the paper is in fact extremely thin on genuine analytic value, attention to data issues, self-questioning, and the like, which a stats paper should be. (By contrast the paper by Dr Freeman is strong on these areas).
8. The methodology is believed flawed by some other statistical and analytical professionals.
9. The arguments in this unsigned paper conflict with core positions they've taken before on electronic voting, but no strong explanation or justification is given. "I thought it odd that the November 11, 2004 Caltech/MIT [paper] would focus so narrowly on justifying the discrepancy between the exit polls and the actual vote without ever mentioning that the openness to fraud of the electronic vote tabulation that they delineated in the July 2001 95-page report from the Caltech/MIT VTP entitled 'Voting: What Is, What Could Be'."
10. The brief paper ends with what some describe as spin quotes, as opposed to formal academic or professional conclusions.

Some sources also draw a connection between influential figures in the CalTech/MIT Voter Technology Project, namely David Baltimore and Howard Ahmanson Jr, and the unsigned unprofessional rebuttal of exit poll suspicions. These people note that Baltimore has been the subject of an investigation by a congressional enquiry into scientific fraud [14], and the Ahmanson family are involved in hard right politics and the ownership and funding of major voting machine companies, and has made large donations to CalTech. Whilst the latter is true, Baltimore was in fact cleared of any charges of data falsification. summary of story

(Sidenote - The original CalTech report is no longer accessible on their website; an amended "version 2" can be found here)

Professor Steven F. Freeman's study of the exit poll discrepancies

From University of Pennsylvania Professor Steven F Freeman's analysis, "The Unexplained Exit Poll Discrepancy: Part I"

Source of data:

"The data I use for this paper are those posted on the CNN website Election Night. CNN had the data by virtue of membership in the National Election Pool (NEP), a consortium of news organizations that had pooled resources to conduct a large-scale exit poll (as was done in the 2000 election). NEP, in turn, had contracted two respected firms, Edison Media Research and Mitofsky International, to conduct the polls."

"For this report, I use data that apparently are based solely on subjects surveyed leaving the polling place. These data were reportedly not meant to be released directly to the public, and were reportedly available to late evening Election Night viewers only because a computer glitch prevented NEP from making updates sometime around 8:30 p.m. that night."

"... these CNN data look good, and can be used to generate some highly suggestive findings."

Broader issues:

• Discusses reliability of exit polls - as covered elsewhere in this article.
• Debunks the circular reasoning employed in the Caltech/MIT study.

Results:

File:Kerry 2004, predicted vs actual.gif

The author states that, "Figure 2 [not shown here for copyright reasons] depicts the resulting distribution curve for samples of 1,936 randomly selected respondents from approximately 40 randomly selected precincts in a state [such as Ohio] in which 48.5% of the vote went for Kerry. ... It turns out that the likelihood that Kerry would poll 52.1% from a population in which he receives only 48.5% of the vote is less than one-in-one hundred (.0073)."

"Conducting the same analysis for Florida, we find that Kerry’s poll prediction of 49.7% of the vote is likewise outside the 95% confidence interval. Given a population in which he receives only 47.1% of the vote, the chances that he would poll 49.7% out of 2846 respondent in an exit poll with no systematic error is less than two-in-one-hundred (.0164). Kerry’s poll numbers are outside the 95% confidence interval as well in the third critical battleground state, Pennsylvania. Although he did carry the state, the likelihood that an exit poll would predict 54.1%, given 50.8% support of the electorate is just slightly more than one-in-one-hundred (.0126)."

"Assuming independent state polls with no systematic bias, the odds against any two of these statistical anomalies occurring together are more than 5,000:1 (five times more improbable than ten straight heads from a fair coin). The odds against all three occurring together are 662,000-to-one. As much as we can say in social science that something is impossible, it is impossible that the discrepancies between predicted and actual vote counts in the three critical battleground states of the 2004 election could have been due to chance or random error."

The latest version of the paper is the first of two parts; the second part was not available as of November 25, 2004. Some general conclusions from an earlier version of the paper have been omitted from this version. The findings are numerically weaker than before, now that precinct-level clustering has been accounted for, but the main finding that the discrepancies cannot be due to chance remains consistent.

UC Berkeley Data Archive

Home page [15] (report link [16] Summary page PDF)

A large academic paper, with full data provided for verification. Although not formally yet peer reviewed, it has been examined by 7 professors (Source [17])

"...researchers examined numerous variables that might have affected the vote outcome. These included the number of voters, their median income, racial and age makeup and the change in voter turnout between the 2000 and 2004 elections. Using this information, they examined election results for the Republican and Democratic presidential candidates in the state in 1996, 2000 and 2004 to see how support for those candidates and parties measured over eight years in Florida's 67 counties."

They discovered that in the 15 counties using touch-screen voting systems, the number of votes granted to Bush far exceeded the number of votes Bush should have received -- given all of the other variables -- while the number of votes that Bush received in counties using other types of voting equipment lined up perfectly with what the variables would have predicted for those counties. The total number of excessive votes ranged between 130,000 and 260,000, depending on what kind of problem caused the excess votes. The counties most affected by the anomaly were heavily Democratic. "

"Sociology professor Michael Hout, who chairs the university's graduate Sociology and Demography group, said the chance for such a discrepancy to occur was less than 1 in 1,000. 'No matter how many factors and variables we took into consideration, the significant correlation in the votes for President Bush and electronic voting cannot be explained...' "

Executive summary:

1. Because many factors impact voting results, statistical tools are necessary to see the effect of touch-screen voting. Multipleregression analysis is a statistical technique widely used in the social and physical sciences to distinguish the individual effects of many variables. This multiple-regression analysis takes account of the following variables by county: (1) number of voters, (2) median income, (3) Hispanic population, (4) change in voter turnout between 2000 and 2004, (5) support for President Bush in 2000 election, (6) support for Dole in 1996 election...
2. When one controls for these factors, the association between electronic voting and increased support for President Bush is impossible to overlook. The data show with 99.0% certainty that a county’s use of electronic voting is associated with a disproportionate increase in votes for President Bush.

(note - confidence levels are formally tested for significance at a predetermined level, typically 95% or 99%. So 99% would be quoted as the result of confidence testing, as a minimum. That said, the actual confidence figure can be calculated backwards, and when this is done turns out to be closer to 99.9%. Hence the two figures of 99.0% and 99.9% cited in the summary)

Key findings:

1. Irregularities associated with electronic voting machines may have awarded 130,000 excess votes or more to President George W. Bush in Florida.
2. Compared to counties with paper ballots, counties with electronic voting machines were significantly more likely to show increases in support for President Bush between 2000 and 2004. This effect cannot be explained by differences between counties in income, number of voters, change in voter turnout, or size of Hispanic/Latino population.
3. In Broward County alone, President Bush appears to have received approximately 72,000 excess votes.
4. We can be 99.9% sure that these effects are not attributable to chance.

Other organisations

NBC

Dec. 3 - NBC announces that it has studied the discrepancy between WCVI figures for Latino votes and NOP (Edison/Mitofsky). They now believe WCVI were more accurate and that NOP under-represented urban (democratic bias) areas and over-represented non-urban (republican bias) areas. NBC has therefore adjusted its figures and believes that in fact Kerry had 58% not 54% of the popular Latino vote and Bush has 40% and not 44% as previously stated. They also revised NBC's estimate for Hispanic support in Bush's home state of Texas, revising a reported 18-point lead for Bush to a 2-point win for Kerry among Hispanics, a "remarkable" 20-point turnaround from figures reported on election night.

"Since the Election Day numbers came out, a controversy has existed between WCVI and exit poll officials. Competing exit polls showed a significant gap in support among Latinos for President Bush and Senator Kerry ... NBC has set an example for network poll integrity by taking a giant step away from the Edison International/Mitofsky election results, and toward WCVI's findings. For example, today NBC stated that 70% of its respondents came from non-urban areas and 30% from urban areas, while acknowledging that 50% of Latino voters come from urban areas. This admission could explain the difference in their results and WCVI's. They under-represented Latino urban voters (who are more likely to vote democratic) and over-represented Latino non-urban votes (who are more likely to vote republican). We hope the other networks follow suit with more adjustments in their findings."

House Judiciary Committee Democrats

Dec. 3 - Judiciary committee members write to NOP requesting raw exit poll data. [18]

"...the only way to restore complete legitimacy on this election is to have an independent firm conduct a detailed analysis of the data, including the raw data taken at the polls early in the day ... I would also like to request your attendance at a forum ...the Judiciary Committee will be holding to discuss any issues and concerns reagrding the numerous voting irregularities that have been reported ..."