The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

Opinion: Data Management – Taking the Bus Without a Map?

Nick Jones, Senior Consultant, Citisoft PLC.

The first maps were probably scratched with sticks on the ground for people from the same locality who knew the surrounding area. Now when travelling we use satellite navigation. But when navigating data it feels as if we are still scratching around. We use electronic documents, not sticks, but we do still need data ‘locals’ to interpret the data geography. Isn’t it time that we had data maps to direct us?

Sticks in the Mud

Locals just know where things are; the landmarks and routes between them. The same is true with older systems that you work with. You know where the data you want is, and how to get to it. You also know that the direct route would be a tunnel through the data mountain; but that the slow narrow track around is the only current option.

Here Be Dragons

Outside your local area, in unfamiliar territory, you rely on previous visitors’ accounts. Regrettably, this is as far as data navigation gets many organisations. Like mediaeval maps, our own data maps show only the outlines of continents: entity and reference data surrounded by oceans of accounting and transaction flows. There are large areas of positional and valuation data where the shores are known, but the interiors are marked ‘Here be dragons’. And sadly, when expeditions are undertaken into these lesser-known areas the knowledge gained is not then added to the map. (The explorers are sent on another expedition, and sketches or notes they leave behind are lost or ignored!)

So, can data mapping move closer to the standardised mapping that emerged from the UK’s Ordnance Survey? If so, where could it go from there?

Ordnance Survey – the Paper Maps and the Parallels

When the UK Ordnance Survey (OS) started “…Europe was in turmoil, and there were real fears that the French Revolution might sweep across the English Channel. Realising the danger, the government ordered its defence ministry – the Board of Ordnance – to begin a survey of England’s vulnerable southern coasts.”

Post-2008, the financial services sector is under threat from the costs of regulation and downward pressure on margins. A rational response to this threat is to understand and document areas of vulnerability.

The OS’s mapping didn’t end at the south coast of England. Once maps had been produced they proved valuable for other purposes. Further mapping was driven first by taxation (FATCA now?) and then by the growth of the railways, whose engineers needed detailed maps (a need to access and integrate more volume and variety with more velocity today?).

The OS’s early work used new technology – a large theodolite capable of measuring angles and elevations very precisely. Now tools to provide data navigators with a clearer picture of their landscape are available. The OS also integrated existing and developing technologies. Photographic apparatus produced maps at different scales, and a method of producing printing plates (photo zincography) was developed. Now there is already scope for existing ETL frameworks and Model Driven Development tools to be used and there are BI and visualisation tools for exploring a data landscape.

However, there are too few detailed data maps around which these technologies can coalesce. Data groundwork, surveying, mapping and publication are needed to ensure that a whole organisation can refer to the same map. It is not necessary to survey the whole landscape to make progress. Properly surveying and mapping one or two ‘threat’ areas should be sufficient to show the benefits. Once the map exists data navigators can set off in the right direction and know roughly how long the journey will take.

Take the Bus?

Putting data on the Enterprise Service Bus (ESB) can make good sense. An ESB efficiently delivers its data passengers to all subscriber stops along its route; but without a good route map there is a risk that data passengers will be loaded onto the bus ‘just in case’. They will then be taken on unnecessary journeys. At each stop they will disembark, find they are lost, and travel on until reaching a destination where they are expected and welcomed.

The bus moves data passengers quickly and economically, but without a data map it can end up moving the wrong data to the wrong places (as well as the correct data to the correct places). With a data landscape map it becomes possible to plan data passenger journeys to maximise transport efficiency.

Related content

WEBINAR

Recorded Webinar: FRTB: The time to get your data in order is now

Fundamental Review of the Trading Book (FRTB) regulation requires firms within its scope to source significant amounts of data, some of which has not previously been required and is difficult to pin down. The data management challenges of the regulation’s Internal Model Approach (IMA) to market risk capital calculations include ongoing P&L attribution, back testing...

BLOG

Blackmore Capital’s Collaboration with OTCfin Completes Integration of ESG Factors into Investment Process

Blackmore Capital, a Melbourne-based asset manager set up in 2018, and New York-based OTCfin have completed the integration of ESG factors with financial data for all Blackmore portfolios. By incorporating ESG factors into Blackmore’s investment process, OTCfin’s risk and regulatory reporting solution will help the asset manager’s team improve portfolio monitoring from both a financial...

EVENT

ESG Data & Tech Summit 2022

This summit will explore challenges around assembling and evaluating ESG data for reporting and the impact of regulatory measures and industry collaboration on transparency and standardisation efforts. Expert speakers will address how the evolving market infrastructure is developing and the role of new technologies and alternative data in improving insight and filling data gaps.

GUIDE

Enterprise Data Management, 2009 Edition

This year has truly been a year of change for the data management community. Regulators and industry participants alike have been keenly focused on the importance of data with regards to compliance and risk management considerations. The UK Financial Services Authority’s fining of Barclays for transaction reporting failures as a result of inconsistent underlying reference...