Amazon’s AWS (for Amazon Web Services) cloud platform has been around for a few years now – it was launched in 2006 by the online retailer then looking to make use of its spare IT capacity – and has been increasingly used by financial markets players looking for less expensive compute and storage resource. Recently, however, it has added a number of offerings that could make it very attractive to those in our space looking to manage big data sets and crunch analytics. It’s also begun to showcase third party solutions relevant to the financial services community.
For those new to AWS, it’s basically a way to tap into compute and storage in the cloud on a pay-per-use model. This can be attractive for companies both small and large. For smaller companies, it is possible to rent virtually as much compute power as needed (and only when needed) without investing in the infrastructure in house. Many smaller hedge funds and proprietary traders have taken advantage of this opportunity to run analytics and back testing, which are usually both compute and data intensive tasks that only need to be run perhaps once per day.
For larger companies, the cloud approach can save much time to market for individual business units, since deploying servers for line-of-business applications in an in-house data centre can often be a tedious and bureaucratic process – involving centralised internal IT groups often with many competing priorities – and one resulting in hefty internal charge backs. By contrast, AWS resources are available within minutes, with costs charged to a credit card.
Xignite is one company that has tapped AWS for its enterprise market data offering from the get go. As well as providing a number of public datasets via a web services interface, the company has since moved into offering private services for market participants, allowing them to also distribute internal data across their enterprises.
In fact, data management applications are increasingly being written with cloud deployment in mind, whether it be Hadoop implementations from the likes of Cloudera, to in-memory fabrics from ScaleOut Software.
AWS began life with a couple of core services: EC2 (Elastic Compute Cloud) providing compute power; and S3 (Simple Storage Service) offering block storage. Various options for those exist today, including a choice of Microsoft and Linux operating systems, and the ability to choose shared or dedicated instances (Amazon’s name for a virtual server).
Over the years, and especially in the past few months, AWS has rolled out a number of additional offerings, several with a view to management of big data and running of high performance computing (HPC) workloads. These include:
- Instances tuned for HPC workloads, with up to 32 processor cores for parallelisation, and GPU co-processors for floating point computation.
- Instances designed for in-memory and data-intensive applications, with solid state disks.
- Elastic Block Store persistent storage for applications, so data is retained when applications are no longer actively running.
- A number of hosted data stores, such as Relational Database Service (RDS), DynamoDB (NoSQL) and ElastiCache (in-memory).
- Redshift, a petabyte scale data warehouse, based on technology from ParAccel (acquired by Actian).
- Elastic MapReduce to offer Hadoop functionality in the cloud.
- Kinesis, providing event stream processing at scale for real-time analytics on so-called big data in motion.
Taken together, this set of value added offerings provide many of the features found in enterprise middleware stacks from the likes of IBM, SAP, Software AG and Tibco Software – companies that are well known in the financial markets space. As such, AWS is increasingly well positioned to benefit as financial enterprises look to leverage cloud services.
Making AWS more attractive for the financial markets is the recently launched financial services category in the Amazon Marketplace – a directory of applications that have been designed for, or ported to, AWS. Instances can be pre-configured with these applications so they are essentially turnkey to deploy.
Kensho Technologies is one recently launched company offering quantitative analysis tools based on AWS technology. In fact, Kensho is working with Nasdaq OMX to make use of FinQloud, the latter’s secure AWS-based cloud.
Other financial applications include: ChartIQ for financial visualisation; CloudPrime for trade confirmations; and Digital Reasoning’s machine learning analytics. And to keep all the instances in AWS in synch with one another, FSMLabs recently released its TimeKeeper product for the cloud.
For sure, AWS isn’t the only cloud game in town – there’s Azure from Microsoft, IBM’s SoftLayer offering, the Google Cloud Platform, services from Rackspace, and new developments from the EMC/VMware Pivotal spinout – what AWS provides today is probably ahead of its competition in terms of completeness of functionality.
While AWS isn’t likely to be seen in low-latency trading environments – offerings from the likes of the recently emerged Lucera are more geared to those extremes – as analytics on big data becomes more of the trading equation, so building and maintaining fast-to-deploy and cost effective infrastructure will become more important. And banks don’t really want to be in that business.
Pete Harris is Principal of Lighthouse Partners, an Austin, TX-based consulting company that helps innovative technology companies with their marketing endeavors. www.lighthouse-partners.com.
Subscribe to our newsletter
