About a-team Marketing Services
The knowledge platform for the financial technology industry
The knowledge platform for the financial technology industry

A-Team Insight Blogs

Challenges and Opportunities of Data Lineage Implementation

Subscribe to our newsletter

Data lineage is critical to regulatory compliance, cost-effective data management, and arming the business with accurate and timely information on which to make decisions. It is also challenging to implement, difficult to sustain and often suffers from a lack of management buy-in and funding.

A recent webinar hosted by A-Team Group on how to get data lineage right, explored the challenges and described approaches to successful implementation. While an early audience poll suggested data lineage is not yet mature, with 38% of respondents starting to build a solution, 28% in the planning stage and just 13% with a complete solution, the benefits of data lineage are enticing, with 58% of respondents to another poll question expecting to gain significant business benefits and 42% significant operational benefits.

Commenting on the progress of data lineage adoption, Dan Bucosky, a data quality and governance consultant at a Tier 1 institution, said: “Large institutions are well aware of the need for data lineage and many are using metadata to collect required data. I have also seen firms using Excel, but this is not the best approach.”

Regulatory drivers behind adoption include BCBS 239, Dodd-Frank, Markets in Financial Instruments Directive II (MiFID II) and, to a limited extent, Fundamental Review of the Trading Book (FRTB) regulation. Data lineage can also reduce the risks of non-compliance with General Data Protection Regulation (GDPR). Considering these regulatory requirements, Philip Miller, co-founder of Solidatus, commented: “Not having data lineage for regulatory compliance could be hugely expensive.”

That said, getting data lineage right comes with numerous challenges. Stephen Veasey, CEO at 3d Innovations, noted problems around managing large volumes of data, legacy environments, disparate systems, mixed data formats, data quality, data ownership and extracting useful information. Responsibility for these challenges and solutions, he said, should lie with the chief data officer. Bucosky suggested that with components of lineage in various parts of large organisations, a centralised metadata group could provide expertise and guidance.

Vendor solutions are available to support data lineage – an audience poll showed 42% of respondents using a mix of in-house and vendor solutions – and provide automation. Miller noted: “Most firms are retrofitting data lineage, but with good data access they should be able to automate about 70% of the process.”

To achieve this, data governance is a necessary partner to lineage, providing an underlying understanding of data for both technology and business teams, and ensuring data is used appropriately. Business buy-in is best achieved by demonstrating the benefits of applying lineage to an important business project, gaining a beachhead and moving on to develop a broader and sustainable solution. As Bucosky concluded: “Understand business drivers and start small with a focussed dataset.”

Subscribe to our newsletter

Related content

WEBINAR

Recorded Webinar: Managing Off-Channel Communications Compliance

Managing off-channel communications – business interactions occurring outside of approved corporate systems – continues to challenge firms’ compliance efforts. The rise of personal messaging apps, social media, and other unmonitored channels – for example, messaging functionality embedded in an order management system – exposes firms to substantial regulatory risk. Enforcement actions by regulatory bodies, such...

BLOG

LSEG Wins Most Innovative Data Quality Initiative Award in A-Team Group Innovation Awards 2025

LSEG has won the Most Innovative Data Quality Initiative Award in A-Team Group’s Innovation Awards 2025 for its Tick History – PCAP, which was expanded this year to offer more than 400 feeds, with new coverage spanning 14 markets in the Americas, eight in the Asia-Pacific region and 76 in EMEA. These awards, now in...

EVENT

AI in Capital Markets Summit London

The AI in Capital Markets Summit will explore current and emerging trends in AI, the potential of Generative AI and LLMs and how AI can be applied for efficiencies and business value across a number of use cases, in the front and back office of financial institutions. The agenda will explore the risks and challenges of adopting AI and the foundational technologies and data management capabilities that underpin successful deployment.

GUIDE

AI in Capital Markets: Practical Insight for a Transforming Industry – Free Handbook

AI is no longer on the horizon – it’s embedded in the infrastructure of modern capital markets. But separating real impact from inflated promises requires a grounded, practical understanding. The AI in Capital Markets Handbook 2025 provides exactly that. Designed for data-driven professionals across the trade life-cycle, compliance, infrastructure, and strategy, this handbook goes beyond...