A ballot is cast in the second round of the French presidential election of 2007. (Photo: Wikimedia Commons/Rama)
As the 2016 Presidential Campaign finally enters primary season—and with no clear frontrunner for the nomination from either party—allegations of electoral misconduct and dirty tricks are sure to remain in the headlines for some time.
But there is hope for the election observers who have long sought out a way to guarantee the validity of election results. A test commonly used in forensic accounting called the Second-Digit Benford’s Law (2BL) just might make it possible to spot a numerical irregularity just by looking at it.
The story behind the test itself, favored by statisticians, is fascinating. According to TK, physicist Frank Benford was working at the GE Research Laboratories in the 1920s when he noticed that there seemed to be more numbers with low first digits in the data sets he worked with. After testing numerous geographic, scientific, and demographic distributions, Benford calculated the expected frequencies for particular digits appearing in a particular position.
For example, if we have a set of numbers that includes the number 154, Benford’s Law predicts a 30 percent chance that 1 would be the first digit, and about a 9.6 percent chance that 5 would be the second digit. Therefore, if you look at a distribution and see a lot of numbers with the digit 5 in the second position, it implies something strange is going on.
Plot of Benford's Law vs the first significant digit of a set of physical constants. Data from http://physics.nist.gov/constants. Created by Aaron Webster. (Source: Wikimedia Commons/Drnathanfurious)
If it sounds hard to believe that every set of numbers would adhere to this law, well, you’re right! Only certain distributions are known to obey Benford’s Law, including transaction level data like sales and numbers resulting from mathematical combinations. Benford’s Law definitely does not apply to numbers assigned in sequence (like invoices), numbers influenced by human thought (like prices), or distributions with a set minimum or maximum (like IRA contributions). But when it works, it really works: in his description of applying Benford’s Law to account auditing, Dr. Mark Nigrini provides a great do-it-yourself test of the law:
For a hands-on introduction to Benford's law, open the Wall Street Journal and pick a random starting point in the stock tables for the two major exchanges. Tabulate the first digits of the daily volume (in hundreds) for 100 stocks. About 50 of the numbers on the list should start with a 1 or a 2. Only about 5 numbers should start with a 9—just as Benford's law would predict.
But how does Benford’s Law help us detect election fraud? Some mathematicians argue that an application of Benford’s Law called the Second-Digit Benford’s Law (2BL) test can be applied to vote totals to detect irregularities that may indicate fraud. Dr. Walter Mebane, a professor at the University of Michigan, has done substantial work on the application of the 2BL test to vote counts, and argues that 2BL can be used to detect potential voting irregularities. In 2004, he applied 2BL to known fraudulent elections in Philadelphia, Chicago, Pittsburgh, and Cleveland. In three instances, the 2BL test detected the voting irregularities. He applied the test again in 2009 to the highly-contested Iranian presidential election with similar findings. Similarly, The Guardian applied the 2BL test to Vladimir Putin’s 2012 landslide victory in Russia’s presidential elections, and found numerical shenanigans.
So, if we’ve applied the test to elections that we’re pretty sure were less-than-democratic and had our suspicions confirmed, it must be a good test, right? Unfortunately, even proponents of the test like Dr. Mebane are quick to explain things are much more complicated. As he pointed out the US News & World Report in 2009, “[A]nomalous statistics could imaginably have an innocent explanation.” As for Putin’s victory, the Guardian explains that there’s a possibility that Benford’s Law doesn’t apply to the distribution at all:
Each polling station within this set of data has a relatively tight range of votes: between three and around 2,600. This makes it unlike most data in, say, financial accounts, due to lack of variance, and can mean data doesn't comply to the Benford pattern even when totally legitimate.
Furthermore, using the 2BL test to detect voting irregularities is far from accepted within the academic community. Joseph Deckert, Mikhail Myagkov, and Peter C. Ordeshook strongly challenged the viability of the 2BL test in 2011, applying the test to a number of simulated fair and fraudulent elections with inconsistent results. In response, Mebane quickly took issue the paper’s methodology. Similarly, Susumu Shikano and Verena Mack applied the 2BL test to the 2009 German Federal Parliamentary Election. They found that although no serious complaints of fraud had been registered, the test indicated fraud in a number of constituencies.
Ultimately, the 2BL test may not be a foolproof way of guaranteeing fair elections. Fortunately for the United States, at least, widespread voter fraud is merely a persistent myth. Elsewhere, UN Peacekeepers sometimes play a role in observing and validating elections, but more steps need to be taken to guarantee free and open elections worldwide. Statistics still may hold the key to quick, reliable analysis of election data.