About a-team Marketing Services
The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

STAC Benchmarks IBM’s Hadoop

Subscribe to our newsletter

STAC – aka the Securities Technology Analysis Center – has benchmarked IBM’s proprietary Platform Symphony implementation of Hadoop MapReduce, versus the standard open source offering, to compare their respective performance. On average, IBM’s implementation performed jobs 7.3 times faster than the standard, reducing total processing time by a factor of six.

Better known for its benchmarking of low-latency trading platforms, STAC leveraged the Statistical Workload Injector for MapReduce (SWIM), developed by the University of California at Berkeley. SWIM provides a large set of diverse MapReduce jobs based on production Hadoop traces obtained from Facebook, along with information to enable characterisation of each job. STAC says it undertook the benchmarking because many financial markets firms are deploying Hadoop.

The hardware environment for the testbed consisted of 17 IBM compute servers and one master server communicating over gigabit Ethernet. STAC compared Hadoop version 1.0.1 to Symphony version 5.2. Both systems ran Red Hat Linux and used largely default configurations.

IBM attributes the superior performance of its offering in part to its scheduling speed. IBM’s Hadoop is API-compatible with the open source offering but runs on the Symphony grid middleware that became IBM’s with its aquisition of Platform Computing, which closed in January of this year.

For more information on STAC’s IBM Hadoop benchmark, see here.

Subscribe to our newsletter

Related content

WEBINAR

Recorded Webinar: Modernising Data Infrastructures: Challenges and opportunities of managing complex financial data and analytics in hybrid and multi-cloud environments

As they forge ahead with their digital transformation programs, financial institutions are finding that the internal platforms they use to manage complex data sets for trading, investment, risk, and compliance are no longer fit for purpose. The ongoing shift toward cloud hosting is forcing practitioners to manage the transition from deeply entrenched legacy platforms to...

BLOG

Singletrack Teams Up with Glue42 for Embedded Sell-Side CRM and Analytics

Singletrack has partnered with interoperability specialist Glue42 to integrate its client engagement and analytics platform into Glue42’s desktop integration platform. Through the initiative, which makes use of the FDC3 integration standard, Singletrack is seeking to offer hyper-personalised experiences to its sell-side clients. “As a company, we have been tracking the FDC3 standard for a while...

EVENT

Data Management Summit New York City (Redirected)

Now in its 8th year, the Data Management Summit (DMS) in NYC explores the shift to the new world where data is redefining the operating model and firms are seeking to unlock value via data transformation projects for enterprise gain and competitive edge.

GUIDE

Risk & Compliance

The current financial climate has meant that risk management and compliance requirements are never far from the minds of the boards of financial institutions. In order to meet the slew of regulations on the horizon, firms are being compelled to invest in their systems in order to cope with the new requirements. Data management is...