About a-team Marketing Services
The knowledge platform for the financial technology industry
The knowledge platform for the financial technology industry

A-Team Insight Blogs

Using Data Lineage to Manage Regulatory Compliance and Financial Reporting

Subscribe to our newsletter

By Mark Hermeling, Chief Technology Officer at Asset Control.

Within data management, data lineage is increasingly becoming an important capability. From pinpointing the origins of data to how it evolves over time allows a wide range of sectors to achieve a clear view of their data, track the data exactly, and use it to assist with managing regulatory compliance and financial reporting.

Increasingly, data lineage is becoming hard-wired into regulations and data quality frameworks – ultimately the need is for ‘explainability’. If a bank, for example, values a position at $25 million, it might need to explain why it is valued at that amount, how it came to that decision and what data points it used in arriving at the valuation. All this context, and more, may need to be tracked.

Corporate treasurers and finance staff have a major role to play in ensuring that the process of reporting and consolidating financial data is consistently transparent and explainable. They can also use data lineage and the related concept of explainability in other contexts. When changes are made to existing processes, for example, data lineage can be used in diagnostics to improve data quality. It can also be key to data licensing. If an organisation is licensing third-party content and therefore has to abide by associated restrictions, it will need to know what data is a derivative of another piece of data.

Finally, data lineage can be key to achieving better management of client records and ensuring greater care is taken about where client data is moved to and used going forward. After all, if a business client under GDPR demands that their data be expunged from the records, the organisation will need to know where all of that customer’s details have ended up in order to be able to achieve this.

Horizontal and vertical lineage

To do all this efficiently and well, organisations will effectively need to implement two different kinds of data lineage: horizontal and vertical. Horizontal data lineage traces the journey of a piece of data as it moves through systems from source to destination. It effectively tracks the journey of a specific item of data – typically across systems and reports – and is focused on the process of keeping track of the data that the business has consumed, where it subsequently went, and who touched it on its journey. The objective is to trace the journey of the data upstream.

Vertical data lineage, in contrast, describes the transformations that happen to a piece of data on the journey. The data could be an element feeding into a calculation, one of the sources of a bond curve calculation, for example. The lineage in this case would be to ‘go back’ from the bond curve and see what individual bonds formed part of the input at a specific point in time.

In short, horizontal data lineage traces data back to the original source, while vertical data lineage reverse-engineers the transformations that happen along the way, whether they are simple processes like cross-referencing, or tracking the different taxonomies that exist for financial instruments or industry classifications. Often, in order to compare like for like, an organisation might want to express an issuer or counterparty within the same taxonomy. So, for example, if one taxonomy labels a segment ‘IT’ and another calls the same segment ‘computer systems’, the organisation may need to ensure that the same label is used for both.

Meeting the Challenge

The specific challenge in terms of an organisation’s ability to reverse engineer is that it will need to keep track of input data sources and their value at the time a transformation took place, all the calculation parameters that fed into the calculation and their value at the time it was done, and the algorithm that was used.

To address the data lineage challenge, firms need bi-temporality so they know the value of the data and associated business logic at the time the calculation took place. They need to be able to track metadata and keep cross-reference tables between different taxonomies and classification schemes up to date, and they need a clear administrative process detailing who can access data, where it goes, where did they get it from, and what their sources were. Moreover, they also need a sourcing hierarchy to be able to look at data sources that are clearly documented and provide access for everybody who needs the data.

This is a complex undertaking, but organisations, and more specifically treasury departments, that understand the requirement and can put the right combination of processes and technology in place will be best placed both to meet data lineage requirements and gain competitive edge.

Subscribe to our newsletter

Related content

WEBINAR

Recorded Webinar: Solving the operations talent crisis

With financial services in the grip of the Great Resignation, operations – a function which has always found recruitment and retention of talent difficult – is facing challenging times. Business growth is a must, but with scaling comes the cost and complexity of additional headcount. How can you ensure that these constraints don’t hold your...

BLOG

Challenges and Opportunities of Achieving a 360-Degree View of Risk

Achieving a 360-degree view of risk is a challenge, but can be achieved by taking a holistic approach to risk that embeds necessary data within a firm’s corporate structure and makes it accessible to all parts of the enterprise. Integrating data in this way enables all parts of the business to work in unison towards...

EVENT

RegTech Summit New York

Now in its 6th year, the RegTech Summit in New York will bring together the regtech ecosystem to explore how the North American capital markets financial industry can leverage technology to drive innovation, cut costs and support regulatory change.

GUIDE

Regulatory Data Handbook 2022/2023 – Tenth Edition

Welcome to the tenth edition of A-Team Group’s Regulatory Data Handbook, a publication that has tracked new regulations, amendments, implementation and data management requirements as regulatory change has impacted global capital markets participants over the past 10 years. This edition of the handbook includes new regulations and highlights some of the major regulatory interventions challenging...