The International Swaps and Derivatives Association (ISDA) has found that a range of generative artificial intelligence models can achieve a very high level of accuracy in extracting and standardising contract details into digital form.
The findings suggest that AI can be deployed to reduce time and resources as well as risks when processing data within the asset class, which is growing in importance to financial institutions.Nevertheless, the success of large language models (LLMs) in mining the information is dependent on the quality of data they are fed, said ISDA Senior Director Digital Transformation–Analytics David Lee.
“It’s not necessarily the models, it’s really the data,” Lee told Data Management Insight. “We don’t see most financial institutions creating foundational models from scratch. Having the data that the AI can effectively use is the most important thing.”
Risk Mitigation
ISDA wrote in a white paper that a study of eight LLMs found they had a 90 per cent-plus success rate in extracting, interpreting and digitising five key legal clauses from credit support annexes (CSAs). These documents outline the means of mitigating credit risk in derivatives contracts and are a key piece of information for each party in a trade.
The study looked at how AI distilled that information into Common Domain Models (CDMs), standardised machine-readable models that enable the easy processing of trades.
Digitised Records
CDMs are detailed models that trace the contracts of financial products and how they are managed and traded across their lifecycle. Traditionally, organisations have done this in their own way. But, arguing that this provides no competitive advantage to firms, ISDA supports the digital model, which it says saves organisations substantial time and resources in the generation of CDMs.
The CDM, it says, brings a host of efficiencies and interoperability benefits and promotes straight-through processing. Automating the CDM generation processes through AI would offer further streamlining opportunities.
That’s especially important because automation has become a key focus of the derivatives market as more investors turn to the once niche instruments as part of a broader diversification of their portfolios. The global OTC derivatives notional outstanding – the measure used to calculate payments made of derivatives contracts – rose 5 per cent year-on-year to almost $700 trillion at the end of 2024, according to the Bank of International Settlements.
High Accuracy
The benchmark study found that the LLMs achieved accuracy levels of more than 90 per cent when prompted with CSA-specific information. For simpler clauses, the accuracy levels climbed to almost 100 per cent, the report said.
ISDA chief executive Scott O’Malia said the report’s conclusion shows that GenAI can be a powerful tool in creating CDMs.
“Our benchmarking study shows how generative AI can be used to accurately extract relevant contract information and digitise it into a standardised CDM format, reducing resource requirements and the potential for errors versus traditional contract data extraction,” O’Malia said in a statement.
He said the organisation could “conduct additional research to determine what steps are needed to further improve accuracy levels”.
Among its conclusions the report said that LLMs that had been trained on CSA-specific information had given the most accurate responses; that standardised phrasing within CSAs were easiest to extract; that larger proprietary LLMs worked better; and that the more nuanced a clause, the harder it is to extract and standardise.
“To enable generative AI to produce more accurate interpretations and structured outputs from CSAs, high-quality domain-specific data, such as the ISDA Clause Library5 and the ISDA Documentation Taxonomy6, needs to be incorporated,” the report stated. “When backed by relevant domain data, generative AI can better understand subtle variations in provision wording and uncover interdependencies among clauses.”
Getting Prepared
Lee added that the type of data fed into the models would make a substantial difference to their output.
“If you have data in databases, Word documents and PDFs, it’s not necessarily the most useful or efficient format for AI to use,” he said. “Are you getting prepared with your data for the time when AI becomes more integral and needs to retrieve information that’s specific to you?”
ISDA said the study was intended to benchmark the performance of LLMs. Industry feedback has been positive about the application of models. Even so, the report isn’t meant to be seen as a guide and the organisation won’t be making recommendations to members on the best models to use.
Subscribe to our newsletter
