The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

Opinion: Data Management – Taking the Bus Without a Map?

Nick Jones, Senior Consultant, Citisoft PLC.

The first maps were probably scratched with sticks on the ground for people from the same locality who knew the surrounding area. Now when travelling we use satellite navigation. But when navigating data it feels as if we are still scratching around. We use electronic documents, not sticks, but we do still need data ‘locals’ to interpret the data geography. Isn’t it time that we had data maps to direct us?

Sticks in the Mud

Locals just know where things are; the landmarks and routes between them. The same is true with older systems that you work with. You know where the data you want is, and how to get to it. You also know that the direct route would be a tunnel through the data mountain; but that the slow narrow track around is the only current option.

Here Be Dragons

Outside your local area, in unfamiliar territory, you rely on previous visitors’ accounts. Regrettably, this is as far as data navigation gets many organisations. Like mediaeval maps, our own data maps show only the outlines of continents: entity and reference data surrounded by oceans of accounting and transaction flows. There are large areas of positional and valuation data where the shores are known, but the interiors are marked ‘Here be dragons’. And sadly, when expeditions are undertaken into these lesser-known areas the knowledge gained is not then added to the map. (The explorers are sent on another expedition, and sketches or notes they leave behind are lost or ignored!)

So, can data mapping move closer to the standardised mapping that emerged from the UK’s Ordnance Survey? If so, where could it go from there?

Ordnance Survey – the Paper Maps and the Parallels

When the UK Ordnance Survey (OS) started “…Europe was in turmoil, and there were real fears that the French Revolution might sweep across the English Channel. Realising the danger, the government ordered its defence ministry – the Board of Ordnance – to begin a survey of England’s vulnerable southern coasts.”

Post-2008, the financial services sector is under threat from the costs of regulation and downward pressure on margins. A rational response to this threat is to understand and document areas of vulnerability.

The OS’s mapping didn’t end at the south coast of England. Once maps had been produced they proved valuable for other purposes. Further mapping was driven first by taxation (FATCA now?) and then by the growth of the railways, whose engineers needed detailed maps (a need to access and integrate more volume and variety with more velocity today?).

The OS’s early work used new technology – a large theodolite capable of measuring angles and elevations very precisely. Now tools to provide data navigators with a clearer picture of their landscape are available. The OS also integrated existing and developing technologies. Photographic apparatus produced maps at different scales, and a method of producing printing plates (photo zincography) was developed. Now there is already scope for existing ETL frameworks and Model Driven Development tools to be used and there are BI and visualisation tools for exploring a data landscape.

However, there are too few detailed data maps around which these technologies can coalesce. Data groundwork, surveying, mapping and publication are needed to ensure that a whole organisation can refer to the same map. It is not necessary to survey the whole landscape to make progress. Properly surveying and mapping one or two ‘threat’ areas should be sufficient to show the benefits. Once the map exists data navigators can set off in the right direction and know roughly how long the journey will take.

Take the Bus?

Putting data on the Enterprise Service Bus (ESB) can make good sense. An ESB efficiently delivers its data passengers to all subscriber stops along its route; but without a good route map there is a risk that data passengers will be loaded onto the bus ‘just in case’. They will then be taken on unnecessary journeys. At each stop they will disembark, find they are lost, and travel on until reaching a destination where they are expected and welcomed.

The bus moves data passengers quickly and economically, but without a data map it can end up moving the wrong data to the wrong places (as well as the correct data to the correct places). With a data landscape map it becomes possible to plan data passenger journeys to maximise transport efficiency.

Related content

WEBINAR

Recorded Webinar: What the LEI Means for your Firm

This webinar has passed, but you can view the recording here. This A-Team Webinar looks at the implications of the emerging legal entity identifier (LEI) for financial institutions as they address how to implement the new standard, and offers suggestions on best practices as the LEI becomes available. The industry initiative to develop and promote...

BLOG

A-Team Group Names Winners of Data Management Insight Awards 2020

A-Team Group has announced the winners of its prestigious Data Management Insight Awards 2020. The annual awards, now in their eighth year, recognise leading providers of data management solutions, services and consultancy to capital markets participants. The award winners were announced online across categories ranging from best sell-side and buy-side data management platforms to best...

EVENT

RegTech Summit Virtual

The RegTech Summit Virtual is a global online event that will be held in June 2021 with an exceptional guest speaker line up of RegTech practitioners, regulators, start-ups and solution providers to collaborate and discuss innovative and effective approaches for building a better regulatory environment.

GUIDE

Applications of Reference Data to the Middle Office

Increasing volumes and the complexity of reference data in the post-crisis environment have left the middle office struggling to meet the requirements of the current market order. Middle office functions must therefore be robust enough to be able to deal with the spectre of globalisation, an increase in the use of esoteric security types and...