They say that a bad workman blames his tools but, anyone who has experience of a practical field, like woodwork, knows that making a poor choice can render a task difficult to do well or plain impossible. You can drive a Pozidriv screw in with a Phillips screwdriver but you are likely to do damage if you attempt the same thing with a flat-head driver. I think something of this ilk is behind the recently discovered computer glitch that caused a large number of COVID-19 cases to go unreported in the English track-and-trace system.
I understand that data was being fed into an XLS format file. That was a format native to older version of Microsoft’s Excel spreadsheet program. For years now, the new standard has been XLSX, which is capable of dealing with much higher volumes of data. However, I am left wondering how an XLS file got into the picture all. I don’t know that it was the native storage format on the source system (that should have been some kind of industrial grade database system) but I can’t think of any way in which an out of date, proprietary system would make sense as a data transfer mechanism.
Over the twenty or so years I spent in the field on the IT end, I have seen people involved in medical research and practice using what I’d regard as suboptimal mechanisms. Sometimes, indeed, Excel is a good choice. I’ve done projects where I have rendered extracts of data as Excel spreadsheets for distribution to senior staff because it was an appropriate way to communicate key information about the data they were based on. However, when going from system to system, you want a format that is easy for another computer to parse rather than one that is pretty for a human to look at and you need one that can cope easily with the data you expect to throw at it, leaving a good bit of head room on top for unexpected growth. If your data is hard to predict, you keep a close idea on it and devise ways to run regular checks. That’s common sense, good practice and, given the vital nature of the project in question, there shouldn’t have been a shortage of experienced and skilled IT professionals with their eyes on it.
Mind you, I was also surprised that PHE was allowed to fall on its sword for this one. I thought the news from August was that it was being gutted and being replaced tout suite by a new National Institute for Health Protection, led by Baroness Dido Harding (formerly of TalkTalk and a woman who should have had the basis for a world class understanding of data insecurity)? It’s not about how well we say we’re doing; its about whether competent people are put to work and allowed to use the right tools to do an important job properly.