Auditing VVPATs has been a headline issue in this election season. A delegation of 22 political parties requested the Election Commission to tally the EVM votes with the VVPAT slips for 50% of the booths in each constituency. The Election Commission rejected the request, leaving the delegation to move the Supreme Court with the same demand. The court ordered the counting of VVPAT slips across 5 randomly selected booths in each constituency. The mathematics used by Election Commission in calculating the sample size for audit can only be described as misleading.
In India, electronic voting was introduced in 2001, and all elections since 2004 have been conducted using electronic voting machines (EVM). The Supreme Court, in a 2013 judgment, ordered that “the ‘paper trail’ is an indispensable requirement of free and fair elections. The confidence of the voters in the EVMs can be achieved only with the introduction of the ‘paper trail’.” This judgment paved the way for the inclusion of the VVPAT unit along with the EVM in each polling booth. The paper slips generated by the VVPAT machine provide an opportunity to verify the votes polled through the EVM, but the Election Commission did not begin counting VVPAT slips until December 2017. Effectively, EVMs were in use from 2001 to 2017 without any audit of their functioning in real election environment. The demonstration in a closed room in the presence of representatives from different political parties cannot be considered as an audit because nobody gains by manipulating the votes in a demonstration; the real election scenario would have vastly different consequences from the controlled environment of the demonstration room.
In 2017, before the Gujarat and Himachal Pradesh assembly elections, the Election Commission announced that, thenceforth, VVPAT slips from one randomly selected booth per constituency would be counted as a confidence-building exercise. The sample size of one booth per constituency was not based on any mathematical calculation, and the data from counting VVPATs never made available to public. If the objective of the exercise was to build public confidence, EC should have voluntarily uploaded that data on their website. B B Swain, Chief Electoral Official of Gujarat accepted that the VVPAT count did not match the EVM count in 4 out of 182 booths but he conveniently assumed that the presiding officers must have forgotten to delete mock polls from the EVMs which led to the mismatch. In any case, there was no mechanism to detect such discrepancies until 2017. The sample of 1 booth per constituency is still not sufficient to understand the scale of these discrepancies. Another mismatch was found during the Karnataka assembly elections in 2018, with the Election Commission releasing a press note accepting the same. As the VVPAT counting data is not available in public, it is not possible to confirm or deny that there are no other cases of mismatch between EVM count and VVPAT count.
B B Swain’s statement, published by News18 on Dec 19, 2017. Full article.
Responding to increasing pressure for counting VVPAT slips to verify EVM results, the EC asked an Indian Statistical Institute (ISI) committee to suggest a sample size for VVPAT counting. Later, NewsClick found out that Election Commission did not write to ISI Director to form a committee, but rather wrote to Prof. Abhay G Bhatt, head of ISI’s Delhi Centre, who in turn headed the so-called ISI committee. The committee submitted its report in March 2019 and suggested a sample size of 479 for the general election. So far, the report has not been made public. Here is the final recommendation of the report:
(Defective EVM is defined as one for which EVM count does not match with VVPAT count)
This calculation has two serious issues:
- The population was defined as all the EVMs used in an entire election. Therefore, it was inherently assumed in the calculation that the defective EVMs are distributed uniformly, and the probability of someone manipulating the election through EVMs is same for each constituency. But, in an election, no one will try to manipulate the elections on all constituency. The sample size of 479 for 543 constituencies means that all constituencies will not be crosschecked in the audit (In random sampling, two booths can also be picked from same constituency). Remember, the objective was to find a sample size for each constituency, and, hence, EVMs used in each constituency should have been defined as population.
- The calculation presented in the report allowed for 2% defective EVMs, as mentioned in their recommendation. 2% defective EVMs is equivalent to uncertainty on 2% votes which is sufficient to make the entire result questionable. The sample size would have increased significantly if less number of defective EVMs were allowed.
Mainstream media reported the ISI committee’s recommendation but almost none of them mentioned the allowance of 2% defective EVMs in the calculation. The Election Commission claimed that the sample size of 479, as suggested by the committee, is in line with their practice of one booth per constituency. Such a claim is completely unscientific.
There were 10.35 lakh polling booths across 543 constituencies in this general election, which translate to around 1900 polling booths per constituency. If we do not allow 2% defective EVMs and calculate the sample size for each constituency, we will get the following answer (the calculation is provided at the end):
Simpler way to read above table: if a constituency has 2000 booths and EC wants to ensure that the number of defective EVMs are less than 0.5%, they should verify VVPATs of 1237 booths (which translates to 61.9% VVPAT counting). The opposition was ridiculed by media for demanding 50% VVPAT count in each constituency, but their demand has a solid statistical basis.
When the delegation of 22 parties went to Supreme Court with this demand, Election Commission showed reluctance in increasing the sample size and argued that 50% VVPAT counting will delay the election results by 6 days. The Election Commission told the court that one booth will be randomly selected for each assembly segment of each constituency; Supreme Court ordered to increase the number from one to five.
Assembly segment has no significance in parliamentary election. This new method of drawing sample (one booth from each assembly segment) further reduced the chances of detecting a mismatch between EVM count and VVPAT count. There are 4125 assembly segment, which means 251 booths per assembly segment. With this new method of drawing sample, a sample size of 5 can detect a mismatch with 99% confidence only if the numbers do not match for 60% of the booths. Such a sample strategy allowed 60% defective EVMs. In other words, if there were 0.5% defective EVMs, there was just a 2% chance of detecting a defective EVM through this kind of sampling. Instead of dividing the parliament constituency into assembly segment to draw sample, if a sample of size 35 (on average, there are 7 assembly segment in a constituency) was drawn from the population of EVMs used in each constituency, the chances would have increased from 2% to 17% (still a ridiculously low chance, but better than the method used in the election). Why did the Election Commission decide to introduce the concept of assembly segment in the parliamentary election? Your guess is as good as mine.
Another question which was debated during this election was: what should Election Commission do if the EVM count does not match the VVPAT count for any booth? The EC told that the VVPAT count will get the preference in such a case. In fact, when mismatches were found in Gujarat and Karnataka assembly elections, the VVPAT count was considered for those booths. But, this is a fraudulent approach to deal with mismatches. A mismatch in the sample imply that there are mismatches in the entire population. By just rectifying the sample data, it cannot be ensured that no mismatch exists in the entire population. (For analogy: if a box contains 2000 apples, and a sample of 5 was drawn from the box to check the quality of apples. One apple in the sample was found bad. By replacing that one bad apple with good apple does not mean that there is no bad apple in the entire box. In fact, it only means that there are 20% bad apples in the box.) The opposition parties had demanded counting all the VVPATs in case of any mismatch. S Y Quraishi, former Chief Election Commissioner of India, had made the same suggestion. But, the Election Commission rejected it.
In summary, the Election Commission has shown reluctance to count the VVPAT slips of any reasonable sample size. The “ISI committee” defined the EVMs used in entire election as population; while the EC defined EVMs used in an assembly segment as population. Both of them avoided the most obvious and scientifically accurate definition of population which would have been the EVMs used in a constituency. If a VVPAT is counted and mismatch is found, the VVPAT count got preference; but for the booths where VVPATs were not counted, the EVM count was assumed as accurate. It can be said without bias that all the steps EC took regarding the VVPAT counting issue reduced the chances of detecting EVM manipulation in election.
Calculation:
Population, N = 2000
Margin of error, P = 0.5%
Number of defective EVMs in the population, M = 2000 * 0.5/100 = 10
Confidence interval, CI = 99.993665752% (as used by “ISI committee”)
Let S be the sample size. Calculate S such that
C(M,0) * C(N-M,S)/C(N,S) ≤ 1-CI/100
Here C(a,b) refers to “a choose b”. The calculation is simple but the numbers involved are large, so it cannot be done with calculator. A small computer code can be written to estimate sample size. We arrive at S = 1237.