The Importance of Data Governance 'The Horizon IT Scandal'

How data governance can protect you and your business

Tom Clements

24 Jan 2024

In an era where information is power, having the ability to turn data into information as seamlessly as possible has never been more important. It can provide sustained competitive advantage and propel business performance. However, without incorporating a data strategy alongside your broader corporate strategy, you risk causing irreparable damage to you, your company and your customers. This month @ Pentify, we are going back in time to discuss the Horizon IT scandal, and why in our view, much hurt and pain could have been avoided had those in power at Fujitsu (Horizon IT’s parent company) given more thought to data governance at the start of their engagement with the Post Office.

For those of you reading from the UK, you will be all too familiar with this story and the extremely sad circumstances many postmasters found, and continue to find themselves in. For our readers not from the UK, you may be less familiar with this ongoing saga. We are not going to delve into the intricacies of the story and the outcomes here because we simply would not do it justice, after reading this piece, we encourage you to read more about this story to fully understand the sequence of events, what happened to many postmasters, particularly those brave enough to speak out.

To start, we need to go back to 1999, when the Horizon IT system was rolled-out to the UK Post Office. This was essentially an initial attempt to modernise the way the Post Office operated, relying on this system to process payments digitally, with the added hope of driving customers back into Post Offices around the UK. Immediately after rollout, many subpostmasters (essentially franchisees of Post Office UK) began reporting a plethora of balancing errors in the system they were seeing at their end. However, Horizon IT rejected this many times, insisting on the robustness of its system. What resulted was, in many cases, subpostmasters being made to repay the system-identified shortfalls.

As the system was built to automate many tasks in different and inefficient ways, such was the nature of the poorly written underlying system code, there was no agreed and documented way for subpostmasters to perform reconciliation tasks from one Post Office to another, which leads us to our first data failing worth highlighting. No data dictionary! A data dictionary is a centralised repository that defines all the data elements in a system, a complete data lineage. It is a key pillar for modern data management. In this case, a data dictionary would have displayed where the data comes from, how it is transformed, and where it is used. This would have potentially identified system issues automatically, if not prevented them to begin with, and ensured transparency and accuracy in data processes.

The next failing, we want to highlight, related very much to the data fundamentals of a data dictionary was ignoring data literacy. Obviously, it is very difficult for us to fully assess, both technical and non-technical, the operational training subposmasters received on using the new system, not to mention that we have the benefit of 20+ years of witnessing how the data landscape has evolved and its importance to businesses of all sizes. We think it is still worth highlighting this as an example of what ignoring data literacy, across all stakeholders, can lead to. In this case, a bug-riddled system, with little security and little transparency would have made the element of data literacy a challenge. If we fast-forward back to today, it is absolutely vital that users of your data, at all levels, are cognisant of what they are looking at, what it means, and more importantly, how it will assist them to make better decisions.

Finally, but of no less importance, is data security failings. This is one of the more staggering items to come to light out of this saga. Fujitsu told the Post Office that “no-one apart from branch managers themselves could access or alter Horizon records”. This turned out to be false, with Horizon staff having the ability to access and alter system records directly. It may go without saying, but access of this nature is a complete and utter no-no in our world. Pentify is all for data democratisation, getting information to as many people as possible when and where they need it, but this is not that. This was a complete lack of foresight and accountability within the Horizon IT strategy. Only by administrators, and only in extremely rare circumstances, should records be amendable. Even then, these amendments need to be fully documented and explainable for transparency.

Data dictionary, data literacy, and data security. These are just three areas falling under the data governance banner that we have highlighted as issues in this case. Of course, a lot has changed in the data and analytics space over the past 20 years, predominantly for the better. We are fortunate now to have far more robust frameworks and methodologies to rely upon when embarking on a data and analytics challenge. We are using the Horizon IT case as an extremely clear example of why a data strategy is important, particularly data governance. We are certainly not suggesting that the Horizon scandal was purely a data problem. The events in this case over the last 20+ years have highlighted many other issues, mainly people and culture issues exemplified by extremely poor leadership by those who should have known better. However, it certainly emphasises the importance of having a sound data governance policy as part of a broader data strategy. Who knows where we will be in 20 years’ time, but there is one thing we can confidently predict. Having a robust and complete data strategy will only grow in importance.

To find out more about how Pentify can help you leverage your data assets, get in touch with us today!