Rolling out data lineage remains a challenge for financial institutions. How should data management teams approach lineage and put a framework in place to ensure it is complete and crosses business lines to meet the goals of data value?
A panel discussion during A-Team Group’s recent Data Management Summit held in the City of London on March 21, 2019, covered the key challenges in addressing data lineage and delved deep into the reasons why businesses need to urgently address data lineage strategies to improve efficiency.
The panel was moderated by Nicola Askham, The Data Governance Coach, and joined by Naomi Clarke, former head of data at GAM; Barry Green, former chief data officer at Bank of Ireland; Stephen Veasey, CEO at 3D Innovations; Ian Evans, managing director of EMEA at OneTrust; and Nimrod Vax, co-founder and head of product at Big ID.Discussion points included the impact of business drivers and regulations such as General Data Protection Regulation (GDPR) on data lineage; the question of how firms can scale up data lineage programmes; the journey from a manual to automated approach; the impact of new and emerging technologies; and best practices for sustaining data lineage once documented.
“It’s not just about the data, but about the vendors,” stressed one panellist. “How do we recruit them, how do we onboard them? Data lineage brings that journey to life.”
While the panel agreed that a coherent data lineage programme is critical, the concern was that a large number of organisations simply have not yet thought through or addressed the issue. “We still struggle with the basics,” said one panellist. “We urgently need to know where our data is coming from. That’s crucial if we are going to start using it to inform decisions in data analytics.”
Concerns include GDPR and data licensing, which has become increasingly complex, making data lineage processes even more important. “You need to know where your data is from in order to be able to comply. Surveillance and the ability to declare your data honestly is crucial to being able to comply with regulations,” said a panel member.
So what are the key challenges?“It’s not something you just install out of a box – it has to be a coherent solution,” said one panellist. “You have to know your business environment and understand what you are trying to achieve within the data universe. Most financial institutions do not have comprehensive enterprise-wide digital rights management capabilities the way they might have in other industries, like music or entertainment. Many companies are struggling with inherited legacy systems, multiple tech stacks and brownfield systems. That’s a real challenge, and it pushes you into addressing the issue on a case-by-case basis, which makes scalability a concern.”
It is a complex environment – and one solution is improved communication between business and IT functions. “There needs to be a combined approach where one enriches the other – that is the key challenge,” said a panellist.
As one panellist pointed out, data lineage shouldn’t be a regulatory solution: “It should be a solution for the business. Implementing good data management solutions allows the business to become more agile.” “There is no point doing data lineage if it’s static. You need some form of ownership, so that the person responsible makes sure it is fit for purpose,” noted another. “Data is dynamic, so you need to move towards a dynamic process where business users are informed along the way as needs, requirements and uses change. That will make the processes and data flows more relevant and continuously compliant, you can automatically identify changes and trigger interaction with business users to get a response.”
Keep it simple
“Data lineage has been around for a long time – it is nothing new,” noted a panel member. “Keep it simple. You don’t have to spend months and months documenting a 1,500 step process. The organisation needs to be able to look at the data, understand it, consume it and process it. Only add detail as and when you need it.”
“Don’t over complicate the data you capture,” agreed another. “In the event of a breach, lineage gives us the information about where the breach was, what caused it, how did it happen and how you can stop it happening again. Lineage is a visible diagram, it is a proactive tool that allows us to unlock questions about our business. That is its purpose.”
So where are we now in terms of automating data lineage? A survey undertaken during the panel found that while 37% of respondents at the Summit were planning data lineage automation, just 5% had made any significant progress and 21% had not made any progress at all. The biggest challenges were identified as data fragmentation (47%), no budget/lack of resources (22%), and a lack of business understanding (15%).
Subscribe to our newsletter
