By Thomas McHugh, Co-Founder and Chief Executive, FINBOURNE Technology.
In Financial Services (FS), data management has long been centred around traditional database storage. However, this approach is fundamentally misaligned with the nature of FS data, which is process-driven rather than static. The industry needs a shift in perspective – one that prioritises virtualisation over rigid data storage.
The Non-Linear Nature of FS Data
Unlike many other industries, financial datasets do not behave in a straightforward manner.
Trades do not simply add to positions. Their impact depends on multiple factors such as settlement processes, regulatory considerations, and corporate actions.
Trades are not always processed in order of receipt. Instead, various operational processes dictate when they are settled and reconciled.
And, yields and financial metrics are not additive. They rely on complex calculations influenced by market conditions, interest rates and compounding effects.
These characteristics make traditional storage models ill-suited for FS data processing. Attempts to fit non-linear data into structured databases create significant challenges.
The Challenges of Traditional Database-Centric Approaches
1. Brittle SQL Logic and Stored Procedures
To accommodate FS data in static storage, businesses often resort to writing complex SQL logic, embedding business rules in stored procedures (SPs). This leads to systems that are hard to maintain, difficult to debug and optimise, and prone to breaking with even minor changes.
The industry learned over two decades ago that such an approach is inflexible and unsustainable.
2. Dependency Management becomes Unmanageable
With SQL-driven logic, datasets become interdependent in unpredictable ways. A change to one dataset in a data warehouse table might create unintended consequences across the system, for example unexpectedly altering transaction dates, introducing new fields or causing missing data to break critical processes — often without anyone realising that a dependency existed.
The typical response is to set up a list or notification email that eventually will go out of date and the system ends up breaking again without warning. This common scenario leads to dataset stagnation as developers hesitate to modify data structures for fear of breaking downstream processes.
3. Redundant Controls and Reconciliations
SQL-based logic often duplicates business rules already embedded in operational systems. As a result, companies must implement additional controls and reconciliations to ensure consistency— contradicting the very purpose of reporting layers, which were designed to simplify and centralise data integrity management.
The Solution: Virtualising Not Just Data, But Functions
Instead of forcing FS data into traditional storage, organisations should embrace virtualisation at a deeper level.
1. Virtualising Business Logic, Not Just Data
Rather than storing data in rigid tables with predefined relationships, financial institutions should consider:
- Event-driven architectures that process trades and transactions in real-time based on operational triggers
- API-based access to data, allowing systems to dynamically retrieve and process information as needed
- Function-as-a-Service (FaaS) models where calculations and processing logic are executed dynamically rather than being embedded in SQL.
2. Metadata-Driven Governance and Dependency Management
By shifting to metadata-driven management, firms can:
- Establish clear ownership and accountability for data elements
- Improve traceability and impact analysis, reducing the risk of unintended consequences when modifying datasets
- Automate dependency tracking, ensuring seamless integration of changes.
3. Reducing Redundant Controls
By embedding business rules within operational functions rather than duplicating them in SQL layers, organisations can eliminate many of the reconciliations and manual controls currently required. The best control mechanisms are the ones that become unnecessary.
The Future of FS Data Management
Financial institutions must stop treating data as static assets stored in rigid databases. Instead, they should design architectures that reflect operational realities – dynamic, process-driven, and adaptable.
By virtualising not just data, but the functions and logic that govern it, firms can achieve greater agility in responding to market and regulatory changes, see a significant reduction in redundant processing and reconciliation efforts and create a more scalable and maintainable data ecosystem
It’s time for the industry to move beyond outdated database-centric approaches and embrace a more flexible, virtualisation-driven future.
Subscribe to our newsletter
