About a-team Marketing Services
The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

AI and Data: How Emerging Technologies Can Provide Value to Financial Firms and their Data Analysts

Subscribe to our newsletter

By Martijn Groot, Vice President, Strategy, Alveo Technology.

In previous decades, people could be forgiven for primarily associating terms such as machine learning and artificial intelligence (AI) with blockbuster films such as The Terminator and The Matrix, but it was, in fact, as long ago as 2017 when Amazon CEO Jeff Bezos described the technology as being in its ‘golden age’. For the financial sector, however, this renaissance has yet to fully take flight.

This is reflected by recent research conducted by Alveo, which found that only 37% of financial services organisations are currently able to incorporate solutions such as AI and machine learning into operational workflows, investment processes and market analysis. Logistical issues commonly await data scientists before they can access data to assist with decision making. With data management and analytics separated, quantitative analysts need to hunt around to gather the data required to develop, train and deploy their models.

The resources and time needed to collect this data means the focus for data scientists is unnecessarily shifted away from their core competency of data analysis. Often, these analysts still find they have to contact the IT department to write a query or set up a report. Even when quants access data, they may find the metadata that should provide insights into the quality and origins of data, and license permissions and approvals, are incomplete or even completely missing. Today, many financial services firms understand they need a better way to provision their data scientists and other key users with clean data sets on financial products and pricing histories.

Big data requires a suitable processing framework. Apache Spark, for example, used in conjunction with cloud-native database technology, allows organisations to move to the next level in scalable, highly performant and integrated data management and analytics. Open source NoSQL database technologies like MongoDB and Cassandra are highly scalable, flexible, and well suited for big data storage and processing.

Some 88% of respondents in the Alveo research reported having made active use of open source technologies in their data management analytics processes, while 36% cited open source as being beneficial in integrating AI and machine learning technologies.

Today, data is often still held in different department level data stores and within legacy applications where accessing it can be complex. The metadata surrounding the data is often not updated frequently, making lineage and understanding the relevant permissions and quality checks it passed difficult.

A lack of clear data catalogues and an increasing number of sources lead to time consuming searches to drive decision making processes. This was identified in the research findings as the top reason data scientists are struggling to find optimum data quality (28%). Using the latest technology to bring data management and analytics together enables them to treat the two as integrated disciplines and helps firms secure fast and flexible access to data with transparency for the preparation process and provenance behind it.

The advantages of integrated data management and analytics

For quants and data scientists, the benefits of integrated data management and analytics are numerous. Some 27% highlighted improved productivity in their daily tasks as the main benefit of integrating market and reference data with advanced data analytics. Better informed decision making requires easy access to clean and consistent data. Metadata information such as usage permissions and quality levels is required to ascertain fitness for purpose. With the help of popular quant languages such as Python and R, they can create a robust and scalable data meeting place, enabling users to share these analytics across their data supply chain and develop a common approach to risk management, performance management, and compliance.

This primarily facilitates increased productivity for quants and data scientists. We are seeing many data analysts today that are looking to dig into the data to find indicators that help them discover investment signals and returns in the market. Increasingly too, they are at least starting to incorporate innovative data science solutions, including AI and machine learning, into market analysis and investment processes.

Adopting this methodology allows proprietary analytics to be built with varying data types, helping to support different aspects of the supply chain and bring together views on the combined data. This approach to user enablement is also helping to democratise analytics, bringing them into the orbit of those who are not data experts.

Future potential

We are entering a new dawn of financial data management. AI and machine learning combined with increased market data provisioning technology are coming together to supply integrated analytics and data management. Financial data subject matter expertise and innovative data management solutions help firms lower the cost of data provisioning, improve decision making, and maximise the mileage they get out of their data.

Subscribe to our newsletter

Related content


Recorded Webinar: How to optimise the business value of your data using agile data governance

Data governance is transforming from a risk management and compliance tool with limited and prescriptive controls, to a solution that can help you optimise the business value of your data. In this role, data governance must scale to manage rising volumes of data, more and different data types, and changing user requirements, while continuing to...


Implementation Analysis: The Catalyst for Rapid Analytics Success in Capital Markets

By Matthew Hodgson, CEO of Mosaic Smart Data. Harnessing the data explosion has become one of the biggest challenges facing capital markets participants today. With such massive amounts of data – all sourced by different methods and stored in different locations across the institution – the challenge can sometimes feel insurmountable. The benefits that come...


TradingTech Summit Virtual (Redirected)

Trading Tech Summit (TTS) Virtual will look at how trading technology operations can capitalise on recent disruption and leverage technology to find efficiencies in the new normal environment. The crisis has highlighted that the future is digital and cloud based, and the ability to innovate faster and at scale has become critical. As we move into recovery and ‘business as usual’, what changes and technology innovations should the industry adopt to simplify operations and to support speed, agility and flexibility in trading operations.


ESG Data Handbook 2022

The ESG landscape is changing faster than anyone could have imagined even five years ago. With tens of trillions of dollars expected to have been committed to sustainable assets by the end of the decade, it’s never been more important for financial institutions of all sizes to stay abreast of changes in the ESG data...