About a-team Marketing Services
The knowledge platform for the financial technology industry
The knowledge platform for the financial technology industry

A-Team Insight Blogs

Octopai Supports Cloud Migration with Cloud Native Data Lineage for Business Intelligence

Subscribe to our newsletter

As data volumes grow, new sources emerge, and firms migrate to the cloud, data lineage has become essential to mapping, visualising and understanding data across the enterprise, and crucial to providing business users with trusted information. Octopai, which came to market in 2018, offers automated metadata-based data lineage with a focus on analysing and understanding an organisation’s business intelligence (BI) landscape.

The company is based in Rosh Haayin, Israel, raised initial funding in 2016, and spent the next two years developing an innovative product that would be ready when needed. Octopai CEO, Amnon Drori, explains: “Five years ago, we saw a problem that was small but would become large and need a technology solution. The amount of data firms were using was growing rapidly and it was increasingly accessed by business users after it had been though BI and analytics applications – this raises questions about where the data was actually coming from in the data warehouse and could it be trusted?”

Octopai answers these questions – and more – with an enterprise grade Software-as-a-Service (SaaS) automated data lineage platform running on Microsoft Azure. The platform includes machine learning, subtext decision tree analysis, and algos, and collects metadata from every BI tool, regardless of vendor or type, in order to visualise and analyse an organisation’s BI landscape. With automated data lineage in place, business users can trust the data they are using and make better decisions. Drori comments: “90% of the lineage done by Octopai cannot be done manually.”

Solutions supported by the Octopai platform come from traditional data warehouse and BI vendors such as Oracle and Business Objects, as well as from next-generation vendors such as Snowflake, Qlik and Tableau. The platform also includes automated metadata discovery that can map where data fields exist down to code line level in 20 seconds. It is accompanied by an automated business glossary to ensure data consistency across an organisation and ease compliance with regulations such as GDPR that require personally identifiable information (PII) to be located wherever it resides in an organisation.

As financial firms migrate to the cloud and adopt cloud native BI solutions, Octopai is ready with a cloud native version of its platform. The first cloud native solution on the platform is Microsoft Azure Data Factory. With Octopai’s support and analysis of Azure Data Factory, organisations can view end-to-end data lineage from the data factory to reporting. More cloud native applications will be added to the platform as the year progresses.

Drori says: “Evolution doesn’t replace tools, but creates more complex on premise and cloud environments. If BI is on premise, in the cloud or in a hybrid environment, Octopai provides a layer to analyse the complete landscape and provide intelligence on the BI. We can help firms trust their data, make money, save money, and keep their executives out of jail.”

Use cases of automated data lineage described by Octopai include identifying and understanding different answers given by multiple BI tools to the same question; finding the cause of reporting errors by pinpointing the data in question and explaining where it came from and any modifications it went through; supporting system migration and updates; identifying redundant systems and data; ensuring compliance with data privacy regulations; and executing impact analysis of any proposed changes to operational or business processes.

Subscribe to our newsletter

Related content

WEBINAR

Recorded Webinar: Best Practices for Building High-Performance Data Infrastructures

The requirement for high-performance data systems to support trading analytics for hedge funds, high-frequency trading firms and electronic liquidity providers is well established. But the explosion in Big Data over the past several years has expanded the scope of inputs being used by these firms. At the same time, cloud technologies have added complexity to...

BLOG

Exchange Data International Adds Irish Funds Data Service

Exchange Data International (EDI), a provider of global security corporate actions, pricing, and reference data services, has expanded its investment funds coverage with the addition of an Irish funds data service. The additional service covers more than 30,000 share class funds from over 200 fund management companies, providing comprehensive reference and corporate actions data. Tony...

EVENT

A-Team Innovation Briefing: Innovation in Cloud

This Innovation Briefing will explore approaches to data infrastructure transformation, technologies required and how to make sure processes are optimised to support real time data management. Hear from leading practitioners and innovative technology solution providers who will share insight into how to set up and leverage your data infrastructure to provide user access to consistent data and analytics, and companies the ability to monetise their data.

GUIDE

ESG Data Handbook 2022

The ESG landscape is changing faster than anyone could have imagined even five years ago. With tens of trillions of dollars expected to have been committed to sustainable assets by the end of the decade, it’s never been more important for financial institutions of all sizes to stay abreast of changes in the ESG data...