About a-team Marketing Services
The knowledge platform for the financial technology industry
The knowledge platform for the financial technology industry

A-Team Insight Blogs

Intelligent Trading Summit: Making the Most of Big Data

Subscribe to our newsletter

Big Data is growing in both volume and adoption, but it needs to be clean, synchronised, analysed and used sensibly in tackling business problems if it is to reach its full potential in the trading environment.

Discussing Big Data for algo development and analytics at last week’s A-Team Group Intelligent Trading Summit, Peter Farley, director at A-Team Group, questioned an expert panel about the development of Big Data, what it comprises and how it is being used. Setting the scene, Roman Chwyl, worldwide financial services and insurance sales leader, IBM Platform Computing, said: “To make money, traders need to take on more risk and comply with more regulations, which means more computing. Today, big banks are taking on more data and processing, and they are leveraging Big Data tools.”

Nick Idelson, technical director at TraderServe, a provider of real-time trading applications, added: “Big Data is the underlying real-time data used by financial institutions. It grows all the time, but it is important to understand the quality of the data and whether it is contemporaneous. Banks are tackling the issues of Big Data and pulling together vast amounts of structured and unstructured data, including news information. They need to get it all into a system, process it and then use it, perhaps for automated trading or decision support, but they must be sure it is clean, synchronised and used sensibly.”

Picking up on this point, Peter Simpson, senior vice president of research and development at Datawatch, a provider of data visualisation software, said: “Successful implementations of Big Data are around business problems and consider where the data is coming from, how clean it is, how it can be analysed and the required business outcomes. Projects should focus on end business requirements and then work back to the technology stack.”

Turning to the issue of social media as a contributor to Big Data, Idelson explained: “The need is to take into account how low the signal to noise ratio is in financial markets and the potential correlation problems of social media. Social media can be used to generate sentiment numbers, but these may change quickly, raising the problem of whether the numbers had a real relationship to what was happening in the market when a trade was processed. In my view, this problem has not yet been adequately solved.”

Simpson continued: “When we think about big data, we think about unstructured data. We might process the data, put algos on a chip to optimise performance and run a machine next to an exchange, but things are always moving so someone needs to be looking at the data. When there is so much data no-one can look at every item, so someone must look for what is not expected to happen. Competitive advantage is in the human element of looking for unusual activity, finding the problem behind it and acting quickly to resolve it.”

These are relatively early days in the development of Big Data applications and tools, a fact borne out by differences of opinion among panellists about the benefits Big Data is delivering. Simpson said Big Data is becoming part of life and is already in use for algo testing and back testing trading realities, while Chwyl said Big Data is providing benefits to the extent that IBM is considering producing multi-tenanted, multiple application shared infrastructure for Big Data. Idelson was more sceptical about the benefits of Big Data, saying: “There is a lot of Big Data analysis going on and banks can make more money, but they seem to be gaining more advantage than customers.”

Subscribe to our newsletter

Related content


Recorded Webinar: High-performance, real-time multi-stream data processing for trading analytics and risk management

Financial analytics platforms in Hedge Funds are often segmented between real-time and back-office analytics systems fed by slower batch processes. To support winning analytics, hedge funds and other trading firms need to bring together real-time streams with historical analytic data in a single high-performance data store to refresh reports and risk models in near-real-time. Listen...


Taskize and Symphony Integrate to Streamline Financial Services Workflow

Taskize, a key provider of inter-company workflow in the financial services industry, has announced an integration with Symphony, the market infrastructure and technology platform, aiming to enhance real-time collaboration between operations teams and front-office users. This pivotal partnership was unveiled and demonstrated at the Symphony Innovate London event, 2023. The integration promises to modernise industry...


RegTech Summit London

Now in its 6th year, the RegTech Summit in London will bring together the RegTech ecosystem to explore how the European capital markets financial industry can leverage technology to drive innovation, cut costs and support regulatory change.


ESG Handbook 2023

The ESG Handbook 2023 edition is the essential guide to everything you need to know about ESG and how to manage requirements if you work in financial data and technology. Download your free copy to understand: What ESG Covers: The scope and definition of ESG Regulations: The evolution of global regulations, especially in the UK...