The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

Practitioners Discuss Use of Cloud Technologies for Signal Extraction

Capital markets firms’ appetite for cloud technologies is growing apace, driven by the promise of improved efficiency, reduced costs, faster development of new products and services, and the ability to do all of this at scale.

Cloud usage within the financial markets is growing across a wide range of business areas. Firms are exploring how they can utilise the cloud to ingest and analyse market data and new sources of information (including a fast-growing assortment of alternative data) to enhance the investment process and discover potential new sources of alpha.

This development – essentially the acceptance of cloud technologies by business areas long considered to be unsuitable – was the subject of a recent webinar hosted by ICE Data Services and featuring guest panellists from Amazon Web Services and SIGTech, a technology spinoff from the investment management firm Brevan Howard Asset management. The discussion offered good insights into how quant groups can use cloud-hosted infrastructure to access data and analytics, and process that data to produce trading and investment signals.

Current Trends

The webinar panellists – Balaji Gopalan, Principal Solution Architect at Amazon Web Services (AWS), Chao Yan, Chief Technology Officer at SIGTech and Satish Vedantam, Director of Quantitative Engineering at ICE Data Services- highlighted five key trends that are driving further adoption of cloud amongst quantitative investment firms.

First, is a marked increase in confidence in cloud security. Until relatively recently, there was a general lack of confidence in the security of the cloud. That is now changing, as capital markets firms across the industry recognise and trust the security and compliance of cloud solutions. Some regulators, including FINRA in the US, are even migrating their own workloads to the cloud.

The second observation was the need to scale. Last year saw unprecedented spikes in both volume and volatility across asset classes, as a result of the global pandemic and other world events. In this environment, firms using the cloud have been able to successfully scale and adapt to the ‘new normal’ of employees and traders working from home.

The third trend identified by panellists was the need for increased network connectivity and integration. As financial markets become more interdependent, market participants need to be able to connect with each other and with their customers rapidly and at scale. Cloud can help facilitate this.

Fourth, panellists noted an increasing scope for innovation. As firms look to develop and bring to market new products and services, the cloud gives them the ability to experiment and either fail fast, or quickly scale up if a new offering is successful.

Finally, there is growing use of AI and machine learning. The cloud presents a wide range of opportunities for data scientists to use AI and ML for analysing and processing large data sets, across many business functions.

Data Quality

With the expansion of new data sources, panellists discussed various approaches that firms can take to improve the quality of the data they are sourcing via the cloud. One is to measure the accuracy and timeliness of the data against recognised benchmarks. Another is to build a robust set of challenge criteria and challenge processes. Data can also be mapped against, and integrated with, existing reference data.

There are also a number of data validation tools that can be used to cleanse data as it is being ingested, and to categorise it with appropriate metadata. Cloud operator AWS, for example, works with various partners to offer solutions that can make data available to users in a cleaner and more accessible way. Its AWS PrivateLink service offers secure interconnectivity between data sources and data consumers via the AWS backbone, ensuring that data is not contaminated by being exposed to the public internet. And AWS Data Exchange allows users to discover and subscribe to ‘clean’ third-party data sets, which is particularly useful for alternative data.

These types of solutions, the audience heard, help address some of the challenges end-users face when using the cloud to integrate and unify data from multiple data vendors through a single platform or interface.

Use of new platforms and technologies

In recent years, there’s been a steady evolution from file-based to more real-time data access through the use of APIs. Increasingly, applications are accessing cloud data warehouses, which offer virtually unlimited scalability of data storage as opposed to in-house data warehouses, which are more difficult to scale.

This approach gives firms the ability to analyse petabytes of data using tools like Amazon Athena, Google Cloud BigQuery or Presto. It also enables firms to adopt a ‘no ops’ strategy, because they are not running their own data infrastructure. This then gives an added advantage of firms being able to operate in a cloud-agnostic way, selecting the appropriate cloud service provider to match the relevant workload.

Other cloud technologies that firms are increasingly using are things like GPU on demand, which enables massive parallel processing at scale and is essential for some ML applications. Streaming analytics via platforms like KX’s kdb+ are also now available on the cloud, via the AWS Marketplace, for example, providing firms with a clear migration path to the cloud without having to re-code applications.

From an AI and ML perspective, firms can use tools like Amazon SageMaker Autopilot to automatically build, train and tune their own machine learning models based upon multiple sources of data. This type of technology is increasingly being used by quantitative investment managers to identify trading signals.

It was clear from the webinar discussion that the public cloud continues to offer new and exciting opportunities for quant teams to develop, test and fine tune their trading and investment strategies.

Related content

WEBINAR

Upcoming Webinar: The evolution of market surveillance across sell-side and buy-side firms

Date: 21 September 2021 Time: 10:00am ET / 3:00pm London / 4:00pm CET Duration: 50 minutes Market surveillance is crucial, and in many cases a regulatory requirement, to ensuring orderly securities markets and sustaining confidence in trading. It can be breached and has become increasingly complex in the wake of the Covid pandemic, Brexit, and...

BLOG

CJC Goes ‘Cloud First’, with Google Cloud Migration

CJC, specialists in market data technology consulting and services, has completed a full migration of its managed service tooling and products to the public cloud, and will now operate a ‘cloud-first’ policy for all internal systems and products. In the process, CJC has also migrated its client base to the public cloud. Previously, the company’s...

EVENT

Virtual Briefing: ESG Data Management – A Strategic Imperative

This briefing will explore challenges around assembling and evaluating ESG data for reporting and the impact of regulatory measures and industry collaboration on transparency and standardisation efforts. Expert speakers will address how the evolving market infrastructure is developing and the role of new technologies and alternative data in improving insight and filling data gaps.

GUIDE

Trading Regulations Handbook 2021

In these unprecedented times, a carefully crafted trading infrastructure is crucial for capital markets participants. Yet, the impact of trading regulations on infrastructure can be difficult to manage. The Trading Regulations Handbook 2021 can help. It provides all the essentials you need to know about regulations impacting trading operations, data and technology. A-Team Group’s Trading...