The leading knowledge platform for the financial technology industry
The leading knowledge platform for the financial technology industry

A-Team Insight Blogs

Octopai Supports Cloud Migration with Cloud Native Data Lineage for Business Intelligence

As data volumes grow, new sources emerge, and firms migrate to the cloud, data lineage has become essential to mapping, visualising and understanding data across the enterprise, and crucial to providing business users with trusted information. Octopai, which came to market in 2018, offers automated metadata-based data lineage with a focus on analysing and understanding an organisation’s business intelligence (BI) landscape.

The company is based in Rosh Haayin, Israel, raised initial funding in 2016, and spent the next two years developing an innovative product that would be ready when needed. Octopai CEO, Amnon Drori, explains: “Five years ago, we saw a problem that was small but would become large and need a technology solution. The amount of data firms were using was growing rapidly and it was increasingly accessed by business users after it had been though BI and analytics applications – this raises questions about where the data was actually coming from in the data warehouse and could it be trusted?”

Octopai answers these questions – and more – with an enterprise grade Software-as-a-Service (SaaS) automated data lineage platform running on Microsoft Azure. The platform includes machine learning, subtext decision tree analysis, and algos, and collects metadata from every BI tool, regardless of vendor or type, in order to visualise and analyse an organisation’s BI landscape. With automated data lineage in place, business users can trust the data they are using and make better decisions. Drori comments: “90% of the lineage done by Octopai cannot be done manually.”

Solutions supported by the Octopai platform come from traditional data warehouse and BI vendors such as Oracle and Business Objects, as well as from next-generation vendors such as Snowflake, Qlik and Tableau. The platform also includes automated metadata discovery that can map where data fields exist down to code line level in 20 seconds. It is accompanied by an automated business glossary to ensure data consistency across an organisation and ease compliance with regulations such as GDPR that require personally identifiable information (PII) to be located wherever it resides in an organisation.

As financial firms migrate to the cloud and adopt cloud native BI solutions, Octopai is ready with a cloud native version of its platform. The first cloud native solution on the platform is Microsoft Azure Data Factory. With Octopai’s support and analysis of Azure Data Factory, organisations can view end-to-end data lineage from the data factory to reporting. More cloud native applications will be added to the platform as the year progresses.

Drori says: “Evolution doesn’t replace tools, but creates more complex on premise and cloud environments. If BI is on premise, in the cloud or in a hybrid environment, Octopai provides a layer to analyse the complete landscape and provide intelligence on the BI. We can help firms trust their data, make money, save money, and keep their executives out of jail.”

Use cases of automated data lineage described by Octopai include identifying and understanding different answers given by multiple BI tools to the same question; finding the cause of reporting errors by pinpointing the data in question and explaining where it came from and any modifications it went through; supporting system migration and updates; identifying redundant systems and data; ensuring compliance with data privacy regulations; and executing impact analysis of any proposed changes to operational or business processes.

Related content

WEBINAR

Upcoming Webinar: Data management for ESG requirements

Date: 13 May 2021 Time: 10:00am ET / 3:00pm London / 4:00pm CET Duration: 50 minutes Environmental, Social and Governance (ESG) investing is moving into the mainstream, requiring asset managers to develop ESG strategies that deliver for both the firm and its investors. While these strategies can outperform those that do not include ESG factors,...

BLOG

Canoe Intelligence Expands into EMEA with Alternative Investment Solutions

New York-based Canoe Intelligence has fulfilled its ambition to expand into EMEA with an office in London and the appointment of Toby Bailey as vice president of sales in the region. The data ops vendor that uses AI to automate the digitisation of unstructured PDF reporting documents and provide actional data for institutional investors, allocators...

EVENT

TradingTech Summit London

The TradingTech Summit in London brings together European senior-level decision makers in trading technology, electronic execution and trading architecture to discuss how firms can use high performance technologies to optimise trading in the new regulatory environment.

GUIDE

Regulatory Data Handbook 2020/2021 – Eighth Edition

This eighth edition of A-Team Group’s Regulatory Data Handbook is a ‘must-have’ for capital markets participants during this period of unprecedented change. Available free of charge, it profiles every regulation that impacts capital markets data management practices giving you: A detailed overview of each regulation with key dates, data and data management implications, links to...