The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

AI and Data: How Emerging Technologies Can Provide Value to Financial Firms and their Data Analysts

By Martijn Groot, Vice President, Strategy, Alveo Technology.

In previous decades, people could be forgiven for primarily associating terms such as machine learning and artificial intelligence (AI) with blockbuster films such as The Terminator and The Matrix, but it was, in fact, as long ago as 2017 when Amazon CEO Jeff Bezos described the technology as being in its ‘golden age’. For the financial sector, however, this renaissance has yet to fully take flight.

This is reflected by recent research conducted by Alveo, which found that only 37% of financial services organisations are currently able to incorporate solutions such as AI and machine learning into operational workflows, investment processes and market analysis. Logistical issues commonly await data scientists before they can access data to assist with decision making. With data management and analytics separated, quantitative analysts need to hunt around to gather the data required to develop, train and deploy their models.

The resources and time needed to collect this data means the focus for data scientists is unnecessarily shifted away from their core competency of data analysis. Often, these analysts still find they have to contact the IT department to write a query or set up a report. Even when quants access data, they may find the metadata that should provide insights into the quality and origins of data, and license permissions and approvals, are incomplete or even completely missing. Today, many financial services firms understand they need a better way to provision their data scientists and other key users with clean data sets on financial products and pricing histories.

Big data requires a suitable processing framework. Apache Spark, for example, used in conjunction with cloud-native database technology, allows organisations to move to the next level in scalable, highly performant and integrated data management and analytics. Open source NoSQL database technologies like MongoDB and Cassandra are highly scalable, flexible, and well suited for big data storage and processing.

Some 88% of respondents in the Alveo research reported having made active use of open source technologies in their data management analytics processes, while 36% cited open source as being beneficial in integrating AI and machine learning technologies.

Today, data is often still held in different department level data stores and within legacy applications where accessing it can be complex. The metadata surrounding the data is often not updated frequently, making lineage and understanding the relevant permissions and quality checks it passed difficult.

A lack of clear data catalogues and an increasing number of sources lead to time consuming searches to drive decision making processes. This was identified in the research findings as the top reason data scientists are struggling to find optimum data quality (28%). Using the latest technology to bring data management and analytics together enables them to treat the two as integrated disciplines and helps firms secure fast and flexible access to data with transparency for the preparation process and provenance behind it.

The advantages of integrated data management and analytics

For quants and data scientists, the benefits of integrated data management and analytics are numerous. Some 27% highlighted improved productivity in their daily tasks as the main benefit of integrating market and reference data with advanced data analytics. Better informed decision making requires easy access to clean and consistent data. Metadata information such as usage permissions and quality levels is required to ascertain fitness for purpose. With the help of popular quant languages such as Python and R, they can create a robust and scalable data meeting place, enabling users to share these analytics across their data supply chain and develop a common approach to risk management, performance management, and compliance.

This primarily facilitates increased productivity for quants and data scientists. We are seeing many data analysts today that are looking to dig into the data to find indicators that help them discover investment signals and returns in the market. Increasingly too, they are at least starting to incorporate innovative data science solutions, including AI and machine learning, into market analysis and investment processes.

Adopting this methodology allows proprietary analytics to be built with varying data types, helping to support different aspects of the supply chain and bring together views on the combined data. This approach to user enablement is also helping to democratise analytics, bringing them into the orbit of those who are not data experts.

Future potential

We are entering a new dawn of financial data management. AI and machine learning combined with increased market data provisioning technology are coming together to supply integrated analytics and data management. Financial data subject matter expertise and innovative data management solutions help firms lower the cost of data provisioning, improve decision making, and maximise the mileage they get out of their data.

Related content

WEBINAR

Upcoming Webinar: How to get value from unstructured data: new approaches, tools and capabilities

Date: 19 January 2022 Time: 10:00am ET / 3:00pm London / 4:00pm CET Duration: 50 minutes Unstructured data volumes are huge, varied in type, and ready to be tapped to gain business value. While the theory is sound, the practicalities of capturing, managing, analysing and benefitting from unstructured data can be challenging, but there are...

BLOG

Substantive Research Notes Strength of Start-Ups in Review of ESG Data Providers

Substantive Research, which released an ESG Provider Dashboard back in June with a view to helping asset managers compare different aspects of ESG data providers and draw up shortlists of companies that meet their needs for ESG data and analytics, has published the findings of an independent survey of the evolving ESG data provider marketplace....

EVENT

RegTech Summit APAC Virtual

RegTech Summit APAC will explore the current regulatory environment in Asia Pacific, the impact of COVID on the RegTech industry and the extent to which the pandemic has acted a catalyst for RegTech adoption in financial markets.

GUIDE

Entity Data Management Handbook – Seventh Edition

Sourcing entity data and ensuring efficient and effective entity data management is a challenge for many financial institutions as volumes of data rise, more regulations require entity data in reporting, and the fight again financial crime is escalated by bad actors using increasingly sophisticated techniques to attack processes and systems. That said, based on best...