Why human error caused the INEC server to crash

INEC and PDP agree to a truce about the scrutiny of electoral materials
The several cases of glitches on the server of the Independent National Electoral Commission (INEC) that affected the electoral process during the February 25 polls were mainly a result of human interference, BusinessDay can report.

The electoral umpire would on Saturday make available the same server for the upload of the results of the governorship and House of Assembly elections. Festus Okoye, national commissioner on voter education at INEC, said in a statement that the commission was aware the INEC Result Viewing (IReV) portal had been slow and unsteady, however, its technical team was working to solve all the outstanding problems.

Many Nigerians have sought to know what was responsible for the glitches the IReV portal experienced which affected the upload of results, especially for the presidential election.

Voters had recounted how, after voting ended, they waited for INEC ad hoc staff to upload the results of the presidential election with minimal success. For some reason, the results of the National Assembly election were quicker to upload whereas those of presidential poll were delayed.

An ad hoc staff member who was in charge of a polling unit at Oshodi/Isolo told BusinessDay on condition of anonymity that the commission gave the officials a code that was able to upload the results of the National Assembly election but failed when it was used for the presidential election results.

But a report by Sunday Punch, citing sources in INEC, said a code error from the server application frustrated the upload process. Efforts by INEC’s technical team to address the glitch failed. Hence, six days after the elections, INEC was struggling to upload the complete results from the 177,000 polling units. On Monday, the commission finally granted approval to lawyers and tech experts from political parties contesting the outcome of the elections to access the electronic system.

A code error refers to a program function that is described in the specifications for the program but is omitted from the code or a program function or user interface that does not operate or that gives incorrect results when measured against its specifications.

While a code error is machine-generated, it is only possible due to human negligence resulting from not conducting proper testing on the server before deploying it for use. Lawrence Asekhamhe Onaulogho, an Amazon Web Service (AWS) expert, said every code error has an AWS log that shows how it is generated.

Jiro Ejobe, founder and CEO of VIISAUS, agrees that coding errors are caused by human beings. However, they may not be deliberate.

“An un-deliberate coding error is an error in software programming. In the industry, we call it a bug. A deliberate error is also done using software programming. In the industry, we call it a virus or a trojan horse (there are many names for this one). But the most common name is a virus. So INEC should answer if it’s a bug or a virus,” Ejobe said.

BusinessDay conducted a search of the INEC server, which showed that it is hosted on AWS. In essence, the results from the elections were supposed to be uploaded on the IReV portal, which server is hosted on AWS. This finding is also corroborated by tech experts who have conducted a similar search of the portal.

AWS has a huge infrastructure backing its hosting services, which translates into a strong performance (at least most of the time). BusinessDay found that the IReV portal runs on AWS serverless: Lamba, API-gateway, dynamodb, etc.

Serverless technologies feature automatic scaling, built-in high availability, and a pay-for-use billing model to increase agility and optimize costs. These technologies also eliminate infrastructure management tasks like capacity provisioning and patching,

Importantly, the server scales with the number of requests payload automatically. The implication is that for an organisation like INEC with huge requests, the serverless enables it to serve this purpose without experiencing any glitches.

“Let’s say a server can manage 10 requests and you’re getting 1 million requests. With serverless, the servers would scale to ~100000M servers automatically,” said Onaulogho.

The serverless is not self-managed. The cloud provider, AWS, manages the operations of the server. This is why power outage would not have been responsible for the glitch on the INEC server because AWS ensures an uninterrupted electricity supply.

However, the newly created AWS account comes with default root user privileges. The root user is also known as a super admin – for example, the INEC director of ICT or the lead engineer – who can assign privileges to other users.

This root user can also interrupt the server operations but cannot shut it down 100 percent successfully. In the case of INEC’s server, experts say the root user could have only paused or stopped the service. The instance where the server is paused can be verified because AWS logs all activities, including running or pausing a service. However, AWS releasing the log evidence to any party other than the root user is against their data privacy policy.

“But the root user of the INEC AWS account can share the logs with you if he or she wants,” one expert who chose anonymity said. The commission can also be compelled to share the logs with a court order according to an expert.

A poor Internet connection can also interrupt the operations of the server, but this has to happen for an extended period to have the impact experienced by the IReV after voting ended on Saturday and Sunday.

On March 9, INEC through its Chief Press Secretary, Rotimi Oyekanmi, admitted that the IReV experienced glitches for some hours on February 25.

“There’s also a provision for the BVAS to transmit the polling unit results in offline mode if there is no network at the polling unit at the time of the activity,” Oyekanmi said. “First, the IReV and BVAS are two separate entities, and they are independent of each other. The relationship between the two is that the BVAS is used to transmit the picture of Form EC8A, which is the polling unit result sheet in PDF format, as well as the Voters’ Accreditation Data to the IReV.”

Be the first to comment

Leave a Reply

Your email address will not be published.