About a-team Marketing Services
The knowledge platform for the financial technology industry
The knowledge platform for the financial technology industry

A-Team Insight Blogs

AI and Data: How Emerging Technologies Can Provide Value to Financial Firms and their Data Analysts

Subscribe to our newsletter

By Martijn Groot, Vice President, Strategy, Alveo Technology.

In previous decades, people could be forgiven for primarily associating terms such as machine learning and artificial intelligence (AI) with blockbuster films such as The Terminator and The Matrix, but it was, in fact, as long ago as 2017 when Amazon CEO Jeff Bezos described the technology as being in its ‘golden age’. For the financial sector, however, this renaissance has yet to fully take flight.

This is reflected by recent research conducted by Alveo, which found that only 37% of financial services organisations are currently able to incorporate solutions such as AI and machine learning into operational workflows, investment processes and market analysis. Logistical issues commonly await data scientists before they can access data to assist with decision making. With data management and analytics separated, quantitative analysts need to hunt around to gather the data required to develop, train and deploy their models.

The resources and time needed to collect this data means the focus for data scientists is unnecessarily shifted away from their core competency of data analysis. Often, these analysts still find they have to contact the IT department to write a query or set up a report. Even when quants access data, they may find the metadata that should provide insights into the quality and origins of data, and license permissions and approvals, are incomplete or even completely missing. Today, many financial services firms understand they need a better way to provision their data scientists and other key users with clean data sets on financial products and pricing histories.

Big data requires a suitable processing framework. Apache Spark, for example, used in conjunction with cloud-native database technology, allows organisations to move to the next level in scalable, highly performant and integrated data management and analytics. Open source NoSQL database technologies like MongoDB and Cassandra are highly scalable, flexible, and well suited for big data storage and processing.

Some 88% of respondents in the Alveo research reported having made active use of open source technologies in their data management analytics processes, while 36% cited open source as being beneficial in integrating AI and machine learning technologies.

Today, data is often still held in different department level data stores and within legacy applications where accessing it can be complex. The metadata surrounding the data is often not updated frequently, making lineage and understanding the relevant permissions and quality checks it passed difficult.

A lack of clear data catalogues and an increasing number of sources lead to time consuming searches to drive decision making processes. This was identified in the research findings as the top reason data scientists are struggling to find optimum data quality (28%). Using the latest technology to bring data management and analytics together enables them to treat the two as integrated disciplines and helps firms secure fast and flexible access to data with transparency for the preparation process and provenance behind it.

The advantages of integrated data management and analytics

For quants and data scientists, the benefits of integrated data management and analytics are numerous. Some 27% highlighted improved productivity in their daily tasks as the main benefit of integrating market and reference data with advanced data analytics. Better informed decision making requires easy access to clean and consistent data. Metadata information such as usage permissions and quality levels is required to ascertain fitness for purpose. With the help of popular quant languages such as Python and R, they can create a robust and scalable data meeting place, enabling users to share these analytics across their data supply chain and develop a common approach to risk management, performance management, and compliance.

This primarily facilitates increased productivity for quants and data scientists. We are seeing many data analysts today that are looking to dig into the data to find indicators that help them discover investment signals and returns in the market. Increasingly too, they are at least starting to incorporate innovative data science solutions, including AI and machine learning, into market analysis and investment processes.

Adopting this methodology allows proprietary analytics to be built with varying data types, helping to support different aspects of the supply chain and bring together views on the combined data. This approach to user enablement is also helping to democratise analytics, bringing them into the orbit of those who are not data experts.

Future potential

We are entering a new dawn of financial data management. AI and machine learning combined with increased market data provisioning technology are coming together to supply integrated analytics and data management. Financial data subject matter expertise and innovative data management solutions help firms lower the cost of data provisioning, improve decision making, and maximise the mileage they get out of their data.

Subscribe to our newsletter

Related content

WEBINAR

Recorded Webinar: How to gain operational efficiency in corporate actions processing

While the risk associated with corporate actions is well established, many financial institutions continue with manual processing in the back office. More often than not, corporate actions information is manually keyed in and event processing lacks automation. This results in operational inefficiency, and financial risk due to missed events or simply getting it wrong. These...

BLOG

A-Team Group Announces Winners of its Data Management Insight Awards Europe 2023

A-Team Group has announced the winners of its Data Management Insight Awards Europe 2023. The awards recognise both established solution vendors and innovative newcomers providing leading data management solutions, services and consultancy to capital markets participants across Europe. This year’s awards included more than 30 categories of data management solutions and services ranging from Best...

EVENT

AI in Capital Markets Summit London

The AI in Capital Markets Summit will explore current and emerging trends in AI, the potential of Generative AI and LLMs and how AI can be applied for efficiencies and business value across a number of use cases, in the front and back office of financial institutions. The agenda will explore the risks and challenges of adopting AI and the foundational technologies and data management capabilities that underpin successful deployment.

GUIDE

Regulatory Data Handbook 2023 – Eleventh Edition

Welcome to the eleventh edition of A-Team Group’s Regulatory Data Handbook, a popular publication that covers new regulations in capital markets, tracks regulatory change, and provides advice on the data, data management and implementation requirements of more than 30 regulations across UK, European, US and Asia-Pacific capital markets. This edition of the handbook includes new...