This week saw the publication of two “M3” benchmarks conducted by STAC, covering time-series management solutions. And while both relate to Kx Systems’ kdb+ database (apparently to date the only database to have published M3 benchmarks), comparing them would be an apples to oranges exercise.
STAC-M3 is focused on solutions that manage large amounts of market data. The benchmark tests have been defined by represenatives of trading firms that are members of STAC’s Benchmark Council, and have been designed to mimic real-life queries – incorporating both computationally-intensive and and I/O-oriented tests. Tests include calculating the NBBO (National Best Bid and Offer) for a specific day, calculating volume curves over 20 days and a theoretical P&L, among others.
While the two benchmarks published this week both include the kdb+ 2.7 release, running on Red Hat’s Linux 6 operating system, other components of the “stack under test” were very different:
1) HP ProLiant DL980 G7 server (with 8 * Intel X7560 8-core processors, running at 2.27GHz, and 1TB DRAM), 8 * HP StorageWorks Array D2700 (with a total of 144 hard drives), Serial Attached SCSI interface.
Presumably to keep the configuration to one that’s generally affordable, physical hard drives were deployed, instead of more costly, but higher performance solid state disks (SSDs).
2) Dell-Kove R910 Appliance (with 4 * 10-core Intel E7-4850 processors, running at 2GHz, and 1TB DRAM) , Kove XPD2 Storage System, Mellanox InfiniBand interface.
The Dell-Kove R910 Appliance is a purpose-built Dell PowerEdgeR910 server, while the Kove XPD2 contains 2TB of DRAM (and hard drives for persistence), and is capable of data reads in six , and writes in eight, microseconds.
Unsurprisingly, given one benchmark was against hard drives, and the other against DRAM, the results showed very different performance. As one example, the STAC market snapshot test resulted in a mean time to complete of nearly five seconds on the HP system, and just 360 milliseconds on the Dell/Kove system.
Commenting on the HP benchmarks, Kx’s chief strategist Simon Garland notes the current day norm of commodity servers with 64 cores and 1TB of DRAM, a configuration that would have been an expensive one even just a year ago. And with regard to the Dell/Kove benchmarks, he points to the lack of I/O bottleneck: “all the cores were able to work at maximum speed, rather than having to wait for the drives.”
Garland reckons that both benchmarks are useful as demonstrating the performance of different solutions designed to tackle different data requirements, in terms of realtime capture and volumes. To handle some real-life scenarios, such as receipt and storage of options data over a period, a hybrid DRAM, SSD and hard drive configuration might be required, he suggests.
For its part, Kove is pointing to its M3 benchmarks as representing the fastest results to date. The Chicago-based company founded in 2004 has established business in academic, HPC and bioinformatics markets, but has yet to land its first major account in the financial markets vertical. That should be soon, says the company’s CEO John Overton.
Subscribe to our newsletter
