Datactics has enhanced the Augmented Data Quality Solution (ADQ) it brought to market in November 2023 with the addition of an AI magic wand, Snowflake connectivity and an SQL rule wizard in ADQ v1.4. The company is also working towards the release of ADQ v1.5 that will include generative AI (GenAI) rules and predictive remediation suggestions based on machine learning (ML).
ADQ was born out of market interest in a data quality solution designed for not only dedicated data quality experts and data stewards, but also non-tech users who could write their own data quality roles. A user-friendly experience, automation and a reduction in manual processes were also top of mind.
Kieran Seaward, head of sales and business development at Datactics, explains: “Customers said their challenges with data quality were the time it took to stand up solutions and enable users to manage data quality across various use cases. There were also motivational challenges around tasks associated with data ownership and data quality. We took all this on board and built ADQ.”
ADQ v1.4
ADQ made a strong start with v1.4 also a response to customer interests, this time in automation, reduced manual intervention, improved data profiling and exception management, increased connectivity, predictive data quality analytics, and more.
Accelerating automation, ADQ v1.4 offers enhanced out-of-the box data quality rules that ease the burden for non-tech users. The AI magic wand includes reworked AI and ML features and an icon showing where users can benefit from Datactics ML in ADQ. Data quality process automation also accelerates the assignment of issues to nominated data users.
Increased connectivity features the ability to configure a Snowflake connection straight through the ADQ user interface, eliminating the need to set this up in the backend. The company is working on additional integrations as it moves towards v1.5.
Predictive data quality analytics monitor data quality and alert data stewards of breaks and other issues. Stewards can then view the problems and ADQ v1.4 will suggest solutions. Based on a breakage table of historical data from data quality rules, ADQ v1.4 can also predict why data quality will fail in the future. Seaward comments: “Data quality is usually reactive but now we can put preventative processes in place. Predictive data quality is very safe to use as the ML does not change the data, instead providing helpful suggestions based on pattern recognition.”
The SQL rule wizard allows data quality authors to build SQL rules in ADQ, performing data quality checks in-situ to optimise processing time.
ADQ v1.5
Moving on to ADQ v1.5 and the integration of GenAI, users will be able to query the model, write a rule for business logic specific to their domain and test the rule to see if it produces desired results. Datactics is currently using OpenAI ChatGPT to look at the potential of GenAI, but acknowledges that financial institutuiosn are likely to have their own take on LLMs and will point its solution to these internal models.
Other developments include a data readiness solution including preconfigured rules that can check data quality and allow any remedial action before regulatory data submissions are made for regulations including EMIR Refit, MiFID III and MiFIR II, and the US Data Transparency Act and SEC rule 10c-1.
Criticality rules that will help data stewards prioritise data problems and solutions are also being prototyped, along with improved dashboards and permissioning, and as it started, next stage development will continue to make ADQ more friendly for business users.
Subscribe to our newsletter
