About a-team Marketing Services
The knowledge platform for the financial technology industry
The knowledge platform for the financial technology industry

A-Team Insight Blogs

Using Data Lineage to Manage Regulatory Compliance and Financial Reporting

Subscribe to our newsletter

By Mark Hermeling, Chief Technology Officer at Asset Control.

Within data management, data lineage is increasingly becoming an important capability. From pinpointing the origins of data to how it evolves over time allows a wide range of sectors to achieve a clear view of their data, track the data exactly, and use it to assist with managing regulatory compliance and financial reporting.

Increasingly, data lineage is becoming hard-wired into regulations and data quality frameworks – ultimately the need is for ‘explainability’. If a bank, for example, values a position at $25 million, it might need to explain why it is valued at that amount, how it came to that decision and what data points it used in arriving at the valuation. All this context, and more, may need to be tracked.

Corporate treasurers and finance staff have a major role to play in ensuring that the process of reporting and consolidating financial data is consistently transparent and explainable. They can also use data lineage and the related concept of explainability in other contexts. When changes are made to existing processes, for example, data lineage can be used in diagnostics to improve data quality. It can also be key to data licensing. If an organisation is licensing third-party content and therefore has to abide by associated restrictions, it will need to know what data is a derivative of another piece of data.

Finally, data lineage can be key to achieving better management of client records and ensuring greater care is taken about where client data is moved to and used going forward. After all, if a business client under GDPR demands that their data be expunged from the records, the organisation will need to know where all of that customer’s details have ended up in order to be able to achieve this.

Horizontal and vertical lineage

To do all this efficiently and well, organisations will effectively need to implement two different kinds of data lineage: horizontal and vertical. Horizontal data lineage traces the journey of a piece of data as it moves through systems from source to destination. It effectively tracks the journey of a specific item of data – typically across systems and reports – and is focused on the process of keeping track of the data that the business has consumed, where it subsequently went, and who touched it on its journey. The objective is to trace the journey of the data upstream.

Vertical data lineage, in contrast, describes the transformations that happen to a piece of data on the journey. The data could be an element feeding into a calculation, one of the sources of a bond curve calculation, for example. The lineage in this case would be to ‘go back’ from the bond curve and see what individual bonds formed part of the input at a specific point in time.

In short, horizontal data lineage traces data back to the original source, while vertical data lineage reverse-engineers the transformations that happen along the way, whether they are simple processes like cross-referencing, or tracking the different taxonomies that exist for financial instruments or industry classifications. Often, in order to compare like for like, an organisation might want to express an issuer or counterparty within the same taxonomy. So, for example, if one taxonomy labels a segment ‘IT’ and another calls the same segment ‘computer systems’, the organisation may need to ensure that the same label is used for both.

Meeting the Challenge

The specific challenge in terms of an organisation’s ability to reverse engineer is that it will need to keep track of input data sources and their value at the time a transformation took place, all the calculation parameters that fed into the calculation and their value at the time it was done, and the algorithm that was used.

To address the data lineage challenge, firms need bi-temporality so they know the value of the data and associated business logic at the time the calculation took place. They need to be able to track metadata and keep cross-reference tables between different taxonomies and classification schemes up to date, and they need a clear administrative process detailing who can access data, where it goes, where did they get it from, and what their sources were. Moreover, they also need a sourcing hierarchy to be able to look at data sources that are clearly documented and provide access for everybody who needs the data.

This is a complex undertaking, but organisations, and more specifically treasury departments, that understand the requirement and can put the right combination of processes and technology in place will be best placed both to meet data lineage requirements and gain competitive edge.

Subscribe to our newsletter

Related content

WEBINAR

Recorded Webinar: How to leverage data lineage for actionable business insights

Data lineage is a must-have for financial institutions, but is it being used to its greatest extent, and is it delivering actionable business insights that inform product development and support competitive advantage? This webinar will address these questions, looking first at the status quo of data lineage across capital markets, and moving on to discuss...

BLOG

A Dive into the Detail of the Financial Data Transparency Act’s Data Standards Requirements

As the June 2024 deadline looms for US regulators within the scope of the Financial Data Transparency Act (FTDA) to provide joint rules on data standards and reporting formats, the question of what these standards and formats will be remains open. Front runners are the Legal Entity Identifier (LEI) and eXtensible Business Reporting Language (XBRL),...

EVENT

Data Management Summit New York City

Now in its 14th year the Data Management Summit NYC brings together the North American data management community to explore how data strategy is evolving to drive business outcomes and speed to market in changing times.

GUIDE

Regulatory Data Handbook 2023 – Eleventh Edition

Welcome to the eleventh edition of A-Team Group’s Regulatory Data Handbook, a popular publication that covers new regulations in capital markets, tracks regulatory change, and provides advice on the data, data management and implementation requirements of more than 30 regulations across UK, European, US and Asia-Pacific capital markets. This edition of the handbook includes new...