The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

STAC Benchmarks IBM’s Hadoop

STAC – aka the Securities Technology Analysis Center – has benchmarked IBM’s proprietary Platform Symphony implementation of Hadoop MapReduce, versus the standard open source offering, to compare their respective performance. On average, IBM’s implementation performed jobs 7.3 times faster than the standard, reducing total processing time by a factor of six.

Better known for its benchmarking of low-latency trading platforms, STAC leveraged the Statistical Workload Injector for MapReduce (SWIM), developed by the University of California at Berkeley. SWIM provides a large set of diverse MapReduce jobs based on production Hadoop traces obtained from Facebook, along with information to enable characterisation of each job. STAC says it undertook the benchmarking because many financial markets firms are deploying Hadoop.

The hardware environment for the testbed consisted of 17 IBM compute servers and one master server communicating over gigabit Ethernet. STAC compared Hadoop version 1.0.1 to Symphony version 5.2. Both systems ran Red Hat Linux and used largely default configurations.

IBM attributes the superior performance of its offering in part to its scheduling speed. IBM’s Hadoop is API-compatible with the open source offering but runs on the Symphony grid middleware that became IBM’s with its aquisition of Platform Computing, which closed in January of this year.

For more information on STAC’s IBM Hadoop benchmark, see here.

Related content

WEBINAR

Recorded Webinar: Embedding AI & Machine Learning into your trading operations

It’s no secret that AI and Machine Learning are changing the landscape of the trading industry. As these tools and techniques become more mainstream, their availability to even the smallest of financial firms is becoming a more viable proposition.  From speech recognition and natural language processing (NLP) to deep learning algorithms and predictive analytics, these...

BLOG

Refinitiv Teams with MarketPsych to Launch ESG Analytics

Continuing its focus on adding capabilities with an ESG theme, Refinitiv has teamed up with Los Angeles-based MarketPsych to launch Refinitiv MarketPsych ESG Analytics to monitor perceptions of sustainability and ESG risk. The analytical tool draws upon news and social media monitoring to offer analysts and portfolio managers numerical insights into companies and countries of...

EVENT

RegTech Summit Virtual

The RegTech Summit Virtual is a global online event that will be held in June 2021 with an exceptional guest speaker line up of RegTech practitioners, regulators, start-ups and solution providers to collaborate and discuss innovative and effective approaches for building a better regulatory environment.

GUIDE

The Trading Regulations Handbook

Need to know all the essentials about the regulations impacting trading infrastructure? Welcome to the first edition of our A-Team Trading Regulations Handbook which provides all the essentials about regulations impacting trading operations, data and technology. A-Team’s Trading Regulations Handbook is a great way to see at-a-glance: All the regulations that are impacting trading technology...