A-Team Insight Blogs

Share article

Pivotal – the EMC/VMware analytics spinout – has released Pivotal Data Dispatch (PDD), a platform for discovery of and analytical processing on data stored in disparate repositories. The product was developed at NYSE Euronext for its internal surveillance and regulatory purposes and has been in use there since 2007.

According to product documentation, PDD provides “on-demand access to big data while supporting existing visualisation and analytical tools. installed as a middleware service on a commodity linux cluster, Pivotal Data Dispatch enables it administrators to set up data sources, analytical platforms, business metadata and access policies.” Data extracted from sources can then be fed into Hadoop or an MPP database for analytic processing.

Key features:

* Provisioning at Terabytes Per Hour – PDD gathers user-requested data from multiple sources, translates it to the destination format, applies user-defined and policy-driven transformations, and loads the data onto an analytical platform.

* Analytics Workflows – PDD includes a web-based workflow tool, so data workers can create multistep analytics workflows that include data movement, transformation and analytics.

* Heterogeneous Data Platform Support – PDD includes native adapters and resource monitoring for multiple platforms, including Pivotal HD and HAWQ, Pivotal greenplum Database (GPDB), Apache Hadoop, IBM Netezza, Oracle and SQL Server. It can also connect with any database through JDBC, and with most distributed file systems, such as NFS.

The NYSE/Pivotal collaboration came about when the exchange chose the Greenplum MPP database for its regulatory and surveillance needs, and built the middleware to populate the Greenplum analytics database from various internal data repositories. Greenplum was since aquired by EMC, and is now offered by Pivotal. EMC also provides storage for the Community Platform cloud offering, from NYSE Technologies, the exchange group’s commercial technology unit.

Says NYSE Euronext Chief Data Officer Steven Hirsch: “Pivotal Data Dispatch solves complex market surveillance and regulatory data needs, and is heavily utilised across NYSE Euronext production systems today. Together, we understand the pressures of managing big data, and the processing power behind Pivotal Data Dispatch technology allows us and their customers to store data reliably at significant scale and easily adapt to fast-paced changes in the big data universe.”

Leave a comment

Your email address will not be published. Required fields are marked *

*

Related content

WEBINAR

Recorded Webinar: Advances in Trade Infrastructure Monitoring

Date: 19 May 2020 Time: 10:00am ET / 3:00pm London / 4:00pm CET To wring greater performance from your trading infrastructure, it’s essential to be able to measure its performance. Leaders in the trading technology race are already using infrastructure monitoring capabilities to assess their trading set-ups and to derive value-added data and insights into...

BLOG

RJ O’Brien Deploys Eventus in the Cloud for Risk Monitoring and Trade Surveillance

Derivatives trader RJ O’Brien & Associates’ (ROJ) selection of Eventus Systems’ Validus as the global platform for its trade surveillance and risk monitoring requirements is aimed at meeting the firm’s regulatory obligations across a range of jurisdictions, including MAD/MAR, MiFID II RTS 6, the CFTC’s Tag 50 and regulations emanating from the Asia/Pacific region. According...

EVENT

RegTech Summit Virtual

We’re thrilled to introduce you to our new RegTech Summit Virtual event. Yes that’s right, all the fantastic content shared by A-Team’s unique community of practitioner experts that you’ve come to know and love from our RegTech Summit live events in London is now going to be made available to you online, so you can watch or listen at your leisure – whether that’s in your office, on your commute, or from the comfort of your own home.

GUIDE

Entity Data Management Handbook – Sixth Edition

High-profile and punitive penalties handed out to large financial institutions for non-compliance with Anti-Money Laundering (AML) and Know Your Customer (KYC) regulations have catapulted entity data management up the business agenda. So, too, have industry and government reports on the staggering sums of money laundered on a global basis. Less apparent, but equally important, are...