The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

AI and Data: How Emerging Technologies Can Provide Value to Financial Firms and their Data Analysts

By Martijn Groot, Vice President, Strategy, Alveo Technology.

In previous decades, people could be forgiven for primarily associating terms such as machine learning and artificial intelligence (AI) with blockbuster films such as The Terminator and The Matrix, but it was, in fact, as long ago as 2017 when Amazon CEO Jeff Bezos described the technology as being in its ‘golden age’. For the financial sector, however, this renaissance has yet to fully take flight.

This is reflected by recent research conducted by Alveo, which found that only 37% of financial services organisations are currently able to incorporate solutions such as AI and machine learning into operational workflows, investment processes and market analysis. Logistical issues commonly await data scientists before they can access data to assist with decision making. With data management and analytics separated, quantitative analysts need to hunt around to gather the data required to develop, train and deploy their models.

The resources and time needed to collect this data means the focus for data scientists is unnecessarily shifted away from their core competency of data analysis. Often, these analysts still find they have to contact the IT department to write a query or set up a report. Even when quants access data, they may find the metadata that should provide insights into the quality and origins of data, and license permissions and approvals, are incomplete or even completely missing. Today, many financial services firms understand they need a better way to provision their data scientists and other key users with clean data sets on financial products and pricing histories.

Big data requires a suitable processing framework. Apache Spark, for example, used in conjunction with cloud-native database technology, allows organisations to move to the next level in scalable, highly performant and integrated data management and analytics. Open source NoSQL database technologies like MongoDB and Cassandra are highly scalable, flexible, and well suited for big data storage and processing.

Some 88% of respondents in the Alveo research reported having made active use of open source technologies in their data management analytics processes, while 36% cited open source as being beneficial in integrating AI and machine learning technologies.

Today, data is often still held in different department level data stores and within legacy applications where accessing it can be complex. The metadata surrounding the data is often not updated frequently, making lineage and understanding the relevant permissions and quality checks it passed difficult.

A lack of clear data catalogues and an increasing number of sources lead to time consuming searches to drive decision making processes. This was identified in the research findings as the top reason data scientists are struggling to find optimum data quality (28%). Using the latest technology to bring data management and analytics together enables them to treat the two as integrated disciplines and helps firms secure fast and flexible access to data with transparency for the preparation process and provenance behind it.

The advantages of integrated data management and analytics

For quants and data scientists, the benefits of integrated data management and analytics are numerous. Some 27% highlighted improved productivity in their daily tasks as the main benefit of integrating market and reference data with advanced data analytics. Better informed decision making requires easy access to clean and consistent data. Metadata information such as usage permissions and quality levels is required to ascertain fitness for purpose. With the help of popular quant languages such as Python and R, they can create a robust and scalable data meeting place, enabling users to share these analytics across their data supply chain and develop a common approach to risk management, performance management, and compliance.

This primarily facilitates increased productivity for quants and data scientists. We are seeing many data analysts today that are looking to dig into the data to find indicators that help them discover investment signals and returns in the market. Increasingly too, they are at least starting to incorporate innovative data science solutions, including AI and machine learning, into market analysis and investment processes.

Adopting this methodology allows proprietary analytics to be built with varying data types, helping to support different aspects of the supply chain and bring together views on the combined data. This approach to user enablement is also helping to democratise analytics, bringing them into the orbit of those who are not data experts.

Future potential

We are entering a new dawn of financial data management. AI and machine learning combined with increased market data provisioning technology are coming together to supply integrated analytics and data management. Financial data subject matter expertise and innovative data management solutions help firms lower the cost of data provisioning, improve decision making, and maximise the mileage they get out of their data.

Related content

WEBINAR

Recorded Webinar: How to establish data quality and data governance for analytics

Data quality has been a perennial problem for financial institutions for many years, but this needs to change as firms become increasingly reliant on accurate analytics to deliver business opportunity and competitive advantage. New approaches to data quality can help firms up their game and significantly improve their analytics capability. Adding the processes, controls and...

BLOG

GLEIF CEO Stephan Wolf Outlines LEI Strategy Based on Hindsight and Innovation

The Global LEI Foundation (GLEIF) has set strategy that will make the Legal Entity Identifier not only essential to smooth running of capital markets, but also the wider digital economy. Achieving these goals is a matter of hindsight and innovation based on the GLEIF’s decision to drive extended use of the LEI in the private...

EVENT

TradingTech Summit Virtual

TradingTech Summit (TTS) Virtual will look at how trading technology operations can capitalise on recent disruption and leverage technology to find efficiencies in the new normal environment. The crisis has highlighted that the future is digital and cloud based, and the ability to innovate faster and at scale has become critical. As we move into recovery and ‘business as usual’, what changes and technology innovations should the industry adopt to simplify operations and to support speed, agility and flexibility in trading operations.

GUIDE

Entity Data Management Handbook – Seventh Edition

Sourcing entity data and ensuring efficient and effective entity data management is a challenge for many financial institutions as volumes of data rise, more regulations require entity data in reporting, and the fight again financial crime is escalated by bad actors using increasingly sophisticated techniques to attack processes and systems. That said, based on best...