The ‘Excel error’ that led to 16,000 missing coronavirus cases

A technical error with an Excel spreadsheet is believed to have caused 16,000 cases of coronavirus to be missed from national tallies, causing a “shambolic” delay to tracing efforts.

The issue occurred between September 25 and October 2 when testing data failed to transfer from the labs to the dashboards that report the numbers. 

Although the original numbers suggested the UK’s coronavirus cases were beginning to plateau, the missing cases instead confirmed the country’s number of positive test results is still on an upward trajectory. 

The error meant on September 30, although the official daily tally showed 7,109 positive results, the total should have been 3,049 higher. On October 1 the total number of missing cases climbed to 4,133 and, on October 2, that number was even higher, at 4,786. 

These 16,000 missing coronavirus extra cases have now been added to the official count. Although Public Health England said everyone who tested positive was informed in the normal way, those who came into contact with them were not.

Instead their details were passed to official contact tracers after the error was discovered, at 1am on Saturday, leaving the test and trace system facing a giant backlog of cases. 

What caused the IT error?

Public Health England’s interim chief executive Michael Brodie blamed the error on a “technical issue”, which he said was identified overnight on Friday, 2 October.

However, a report by PA suggests the fiasco was due to an Excel file containing lab results reaching its maximum file size, which stopped new names being added in an automated process. 

The files have now been split into smaller multiple files to prevent the issue happening again, the report claims. 

Matthew Parker, a mathematician and author of Humble Pi, tweeted:

“It sounds like there was an arbitrary cut-off point for files on the system,” said Michael Veale, Lecturer in Digital Rights & Regulation at University of College London. 

“It could’ve been, for example, that it did not accept spreadsheets that had a large number of rows. 

“While these kinds of errors can happen, data entry procedures should have checks to ensure that they don’t, or provide feedback to those uploading files that they have been truncated. Those are common practices in industry and it would be surprising if they were not present here.”

“It would be truly astonishing if the Government’s track and trace program failed to update new cases because it is run on an Excel spreadsheet which became ‘full’,” said Natalie Cramp, CEO of data science company Profusion.

“This would indicate that not only is the system of reporting using incredibly basic data management tools, it would also tell us that it had been set up poorly. The failure to realise that the stats had not updated correctly for some time, is a worrying sign that this data may not be being monitored and managed closely. If you use the right data management tools, there are simple automated alerts which can highlight this for you in almost real time.”

Public Health England is yet to respond to queries about the nature of the glitch and what type of system it was using. 

A note on the Government’s coronavirus data dashboard says that the issue has been “resolved” and PHE has said that “further robust measures have been put in place as a result”.

Could it have been prevented?

Although Public Health England has been keen to brand the problem as a “technical issue”, experts say the problem is more likely to be linked to human error. 

Technology strategist and consultant Rachel Coldicutt told the Telegraph: “I think it shows that data management and collection is subject to very human processes and errors, and that UK government’s capabilities – whether people or infrastructure – is still a long way from being able to drive the “quantitative data revolution” Dominic Cummings is hoping for.”

Jon Crowcroft, a Computer Science professor at the University of Cambridge, said: “It seems possible they exceeded the number of things Excel could deal with. If this was the case, it is also remiss of them, as the limits on what Excel can handle are pretty common knowledge and should also be part of any sensible testing.

He added: “When a glitch like this happens in a service, you have to ask what level of professional scrutiny was there over the system design choices and code? it sounds like “not much” would be a fair answer.” 

What happens now?

Health Secretary Matt Hancock will appear in the Commons this afternoon to make a statement at 4.35pm, according to BBC Parliament.

He is expected to face questions about the government’s habit of blaming computers or “mutant algorithms” – in the case of exams results – instead of acknowledging internal mistakes. 

Will this have contributed to the spread of the virus?

There is no way of knowing the precise ramifications of the error because of the way the new coronavirus spreads.

Some people are asymptomatic carriers and will be infected without knowing, while others will show symptoms, including a loss or change in sense of smell or taste, a new and persistent cough and a fever; others could be “pre-symptomatic”.

One could presume that all of those who become symptomatic immediately begin to self-isolate at home, but the problem lies with those who are infected but are not aware.

This could lead to them continuing their lives as normal in the community and potentially increasing the spread of the virus.