The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

How Financial Firms Can Close the Data Integration Gap

By Sourav Moitra, a director of technology at Sapient Global Markets

Driven by the need for faster access to data, along with consolidated data views across multiple platforms with greater consistency and transparency, financial firms are realising they must change their data integration process to gain efficiency and solve business needs.

The financial industry is evolving quickly. No longer can firms afford to wait for end-of-day data or work with different datasets in silos. Buy-side and sell-side firms want to react fast to business events with easy and near real-time access to data for better business decisions.

Too often, firms allocate a significant amount of time and money to updating existing data integration platforms when the foundations themselves aren’t strong enough. One key challenge is the sheer number of systems that create data inconsistencies, including legacy systems from mergers or acquisitions.

Further, these platforms limit transparency as they are based on mostly IT-driven processes. A data-centric approach grounded in data governance, along with business-driven data processes, can help create a more collaborative and flexible platform as data requirements develop further.

Evolution of data integration

Historically, data moved from one system to another based on a particular need. However, as data requirements evolved, complexity increased. Additional processes were added, leaving most organisations with an unstructured and complicated data architecture.

So how did we arrive here? Many firms implemented different layers of a data integration platform as extract, transmit and load (ETL) jobs. But business requirements didn’t just stop progressing. More requirements were added around audit and traceability, monitoring and issue resolution, intraday reporting, historical reporting and so on. The more requirements added on top of ETL-based approaches, the more complex the integration process became.

Many responded with silo-based models, implementing solutions each time a new requirement came into effect. For instance, audit and traceability requirements led to new data governance procedures. Intraday data consumption requirements led to new services and messaging. Historical reporting requirements led to the creation of data warehouses.

The industry developed these solutions in response to a specific issue, and that resulted in current-state models with retro-fitted data governance for specific requirements, inconsistent technology, and inefficient or redundant processes, such as issue resolution.

Closing the data integration gap

For starters, you can start improving your data integration strategy with an effectively governed data-centric approach. The following steps can help close the gap:

  1. Form a canonical data model. Canonical models standardise data processes and create consistent attributes for data in motion and data at rest to reduce conflicts. This model is also source agnostic as it looks at data as a whole, as opposed to multiple systems looking at data in different formats.
  2. Combine business and technical data processes. Rather than creating specific solutions for specific requirements, form a data platform that allows you to model business and technical processes, and fuse them together. Enforce a system of records, data element standards and data quality rules.
  3. Implement loosely coupled, event-driven architecture. Implement data acquisition, validation, usage and retention policies for efficient data access to address business events in real time rather than waiting on IT.
  4. Develop a business-driven approach. Measure and monitor operational efficiency, capture metrics for data use in business processes and centralise your operational and issue resolution strategy to ensure consistency and quality.

Start small

Moving to this future state won’t be easy. However, you don’t have to accomplish everything on day one. Start small and keep evolving your platform. For example, take a dataset like securities and move it in a direction where you at least correctly implement the governance aspect. Once that is complete, bring in trades, positions and other datasets one after the other.

Steady integration can help your firm move to a future state that is agile, responsive, governed and business driven.

Related content

WEBINAR

Recorded Webinar: Balancing Regulatory Transparency with Data Protection

You can listen to the recording of this webinar by registering on this page. Balancing the use of personal data required by Markets in Financial Instruments Directive II (MiFID) as part of its transparency regime with the personal data protection rules set out in General Data Protection Regulation (GDPR) is a tough task – tell...

BLOG

Reformis Adds Matrix IDM and NeoXam to Vendor Partner Programme

Reformis, a buy-side business transformation consultancy, continues to build out its vendor partnership programme with the addition of Matrix IDM’s cloud native data management software, and NeoXam’s Impress reporting solution. These join the company’s list of existing partners and products including IHS Markit’s EDM software, Snowflake’s cloud data platform, and Compliance Solution Strategies’ Accudelta product....

EVENT

TradingTech Summit London

TradingTech Summit London will explore how trading firms are innovating in today’s cloud and digital based environment to create flexible, scalable trading platforms to support speed to market and business agility. Leveraging the cloud, AI and ML technologies to get an edge, automate processes and simplify operations in a cost effective way is the name of the game and will share practical insight from practitioners and technology leaders who are innovating and driving forward change in trading operations.

GUIDE

Enterprise Data Management, 2010 Edition

The global regulatory community has become increasingly aware of the data management challenge within financial institutions, as it struggles with its own challenge of better tracking systemic risk across financial markets. The US regulator in particular is seemingly keen to kick off a standardisation process and also wants the regulatory community to begin collecting additional...