By Carl Thornberg, Head of Optimisation and Analytics Technology at OSTTRA.
Integrating AI into the finance sector requires navigating a robust regulatory landscape designed to ensure stability and security, a process that demands a deliberate and meticulous approach to technology adoption. To build confidence for experimentation with this not-yet-fully-understood technology, essential safety nets must be established.
AI should be thought of as a catalyst for improving processes. Within post-trade services it should be focused on reducing operational risk and enhancing efficiency.
Weak engineering foundations, such as poor data quality or a lack of deployment automation can amplify existing dysfunctions, turning AI into new and unpredictable operational risk. Mature development practices, including automated testing, working with small code batches, and maintaining strong version control are therefore critical.
AI introduces a new spectrum of technical and security risks that must be proactively mitigated such as data leakage which can occur when sensitive information is accidentally revealed through the model’s output or by transmitting proprietary data to unapproved external services.
Prompt injection attacks, made using malicious inputs, could lead to unauthorised data access or system misuse and the rise of autonomous AI agents introduces the risk of uncontrolled outputs and cascading errors.
Building safeguards
To mitigate these technical and security challenges, a layered approach rooted in governance and control should be foundational. Only the largest financial firms can build and train their own LLMs internally, but firms who use existing models can ground their projects using different methods.
A main one is the use of Retrieval Augmented Generation (RAG), which serves as a secure, searchable repository for proprietary documentation and other types of data, retrieving specific data chunks to ground the LLM. By compartmentalising internal data and applying strict control over what data is used for each LLM use, for example using only data from one specific client, the risk of internal data leaks that can occur from LLM training and tuning is eliminated. This also aligns with existing financial security, compliance, and governance frameworks.
Extensive pre-baked instructions within the prompt dictate exactly how the LLM processes the data – to enable granular control over tasks – and further inject specific and sensitive data for use by the LLM. These primary methods can achieve precision without the complexity and cost of model training.
A combination of strategies enables firms to utilise state-of-the-art models within a controlled sandbox of internal facts and rigid rules.
AI operates on statistical probability but at the same time can be indeterministic. When developing AI agents, firms need to establish workflows incorporating stringent, context-specific controls. For high-stakes decisions, the error tolerance must be zero, but in less critical areas, occasional, rare errors may be permissible within controlled processes, creating two distinct models for oversight.
Full human authority: For critical or complex workflows, AI agents should function to accelerate and synthesise work up to a decision point. The output generated by the agent is treated as a highly informed recommendation. The final decision on the suggested action must be made by a human who acts as a pilot, where every single step of the agent’s proposed execution path is verified and approved before proceeding. All high-risk actions retain full human accountability and control, preventing the possibility of an uncontrolled error.
Framed autonomy: For more contained, high-volume internal tasks with a well-defined process, an agent can operate with a greater degree of autonomy, albeit still with some human oversight. A subject-matter expert defines an “operating frame,” which consists of acceptable parameters and thresholds. As long as an agent’s work and its results remain within this defined frame, autonomous action is permitted. Any minimal possibility of error is treated as an acceptable business-as-usual operational risk, comparable to human entry error.
Crucially, the agent’s execution is immediately paused the moment a parameter falls outside the defined frame. This mandatory intervention point is required before the agent can restart. This mechanism balances efficiency with safety by converting unpredictable risks into predetermined hold points.
AI in practice
Consider the highly manual and time-sensitive task of processing a counterparty margin call received through an unstructured email.
Currently, this process demands a human operator to manually extract critical data such as the call amount, settlement date, and collateral type from the notification (e.g. a PDF or a simple “OK to proceed” message) and input it into the system. This manual entry is slow, open to human error, and poses a direct operational risk.
Here an autonomous AI agent could be deployed to extract, process and sanity-check the raw data. If all extracted parameters (such as amount, date, and collateral) fall within defined limits, the agent is authorised to automatically generate the draft margin call record within the system.
If something falls outside the parameters, the process stops and a human being alerted.
Finalising a process, such as issuing a legally binding margin call or approving a payment, leaves no room for mistakes. While the AI agent might generate the order, a human pilot becomes accountable and must take the final, critical action – reviewing the structured data, the original source email, and the agent’s summary before it is sent.
Successful AI implementation requires firms to have both clean, sanitised inputs and an audited framework that identifies where errors are permissible. The guiding principle for minimising risk and optimising outputs is to ensure human oversight is deliberate, and intervention is made at predetermined milestones.
About the Author
Carl Thornberg is the Head of Optimisation and Analytics Technology at OSTTRA. He focuses on applying robust software development principles and risk mitigation frameworks to safely unlock the power of AI and enhance efficiency across post-trade services.
Subscribe to our newsletter
