“Co-authored by Adaptive | Aeron and QuestDB”
Authors:
-
Ralph Swann, Strategy Group, Adaptive
-
Nicolas Hourcard, CFA, QuestDB
Three questions every trading-tech leader is being asked right now. When the matching engine fails over, can we prove every order was handled in the order it arrived, with no losses and no surprises for the regulator? And when compliance asks us to reconstruct the book at 09:47 on a Tuesday eight months ago, or a quant team asks whether a new execution policy would have done better that morning than the one that actually ran, can we answer in seconds without standing up a parallel data team?
Across capital markets, a pattern is emerging in how engineering teams answer those questions. Firms are picking specialised open-source tools layer by layer rather than extending a single vendor’s stack:
-
Aeron® for mission-critical, low-latency applications.
-
QuestDB for real-time market data and time-series analytics with lakehouse storage; Polars at the dataframe layer, Grafana for real-time dashboards.
AI has made the shift urgent because models, agents, and LLM-assisted engineering all assume open formats and machine-readable schemas; closed query languages and proprietary binary formats don’t fit into that picture.
Where Aeron and QuestDB sit in the low-latency trading stack
Aeron, maintained by Adaptive, is the high-performance stack running under matching engines, single-dealer platforms, exchanges, and order management systems at many capital markets firms. QuestDB sits on the other side of the wire, handling persistence and real-time analytics for pre-trade, post-trade, and surveillance work, with time-series SQL primitives and storage on open table formats including Parquet and Apache Iceberg.
The same store covers the queries trading and ops teams run intraday, and the regulatory archive that has to be retained for a decade or more, on object storage the firm already pays for.
The architectural split: Hot Path vs. History Path
The reason the split exists is that the two halves address different problems. The hot path must process millions of messages with microsecond latency while guaranteeing they have been replicated to enough nodes to survive node failure. The trade-off the matching engine and OMS pay for those guarantees is that the working set is bounded: the amount of state and history the hot path holds in memory is finite.
The history path has to accept everything, forever, and let anyone ask any question of it later, which breaks sub-millisecond write-path guarantees. Trying to do both with one tool is what produces the latency spikes and reconstructed-history problems many firms recognise from their own incident reviews.
The same split is what an ML or agentic system needs from the data layer underneath it. The bounded side holds the replayable log of every input the system saw. Re-run it, and you regenerate every downstream state. A quant team uses that to evaluate a candidate policy against the same conditions. A regulator uses it to ask why the policy made that decision at 14:32:11.040. The unbounded side gives the queryable history those systems need to evaluate against, train on, and present to a compliance team. It’s the same architectural split either way.
Integrating Aeron and QuestDB for exactly-once semantics
Take a firm running an exchange or an internal matching engine. Order flow and market data move over Aeron, with Aeron Cluster offering 99.999% availability. Events that need to be persisted and analysed (fills, quotes, book updates, order state) get written into QuestDB, where they’re immediately queryable in standard SQL.
The ordered, replayable log Aeron Cluster produces is the same log that feeds QuestDB. QuestDB deduplicates on log position, so failovers and replays re-deliver messages safely without double-counting. The result is exactly-once semantics from the live core into the historical store. Market data goes onto Aeron once and gets read in two places: QuestDB for tick capture, and Aeron Cluster for anything that has to be processed on the trading system hot path. No second source of truth to reconcile.
The integration lets the firm ask “what would have happened if we’d run a different policy last Tuesday?” and get a real answer, not a simulation. Take last Tuesday’s command log, swap the policy under test, and run it deterministically: every fill, every market-data tick, every timing relationship is identical to what actually happened, because the log is what actually happened. Write the output to QuestDB as a separate table or partition, and every existing TCA, surveillance, and risk query runs unchanged against the new policy.
From the same dataset, surveillance flags executions that diverge from prevailing top of book, TCA measures slippage against benchmarks, and risk runs rolling exposures over the whole book, without copies or extra pipelines. Researchers backtest against the actual data production saw, not a curated subset in a separate store. Markouts, point-in-time book reconstructions, and similar workflows now sit in SQL the engineering team can read and AI assistants can write.
AI as a property of the architecture, not a feature on top
Two architectural properties matter when an AI policy is in the loop. The Aeron and QuestDB pairing makes each of them structural, not bolted on.
-
Bit-by-bit reproducibility for AI: Regulators are converging on the same question for ML-driven and agentic execution that they ask of human-written code: what version was running at 14:32:11.040, why did it make that decision, and can you reproduce it bit-for-bit? Aeron‘s ordered command log records every input the policy saw. QuestDB stores every output it produced. Each policy version, including a hash of its weights, is announced to the system through a control message in the same log that carries the trades. When the regulator asks what was running at 14:32:11.040, the answer is in the log at a known position before that trade, along with the rest of the audit trail. Reproducibility is structural, not a claim.
-
Unified training data identical to production by construction: Aeron Cluster’s sequenced command log is not a byproduct of the trading system. It is the system. Every input, every state transition, in a single total order, replayable to any prior moment, encoded in Simple Binary Encoding (SBE) so an agent or a pipeline can read it without a proprietary decoder in the way. QuestDB holds the indexed, queryable projection of that log. ML teams have spent twenty years reconstructing approximations of this dataset from disagreeing application logs; on this stack, the training set is the production system, with no reconciliation step in between.
Why open-source infrastructure matters for DORA and MiFID II
Capital markets firms have wanted to own their tech for years because that’s where the differentiation lives. Three forces have turned the want into a procurement timetable:
-
DORA is enforceable now and audits are starting.
-
MiFID II/CAT continues to demand reproducible, sequenced audit trails.
-
EU T+1 lands in October 2027 and tightens the window in which post-trade reconciliation has to clear.
Each of those is a regulatory ask that the legacy single-vendor stack answers with a story rather than a property.
The analytics side is moving the same way. LLMs and agents read SQL and open formats. They don’t read proprietary ones. Firms with their history sitting in closed stores have a longer build ahead than firms whose data is already in well-understood open formats like Apache Parquet and standard SQL.
Aeron and QuestDB show up regularly in this kind of rebuild. Aeron handles transport in the open with the latency and resilience trading infrastructure needs. QuestDB handles persistence and time-series analytics in the open with the SQL primitives those workloads need. Anything downstream, including AI agents, inherits that openness.
Both projects are Apache 2.0, which is the answer to the procurement question of who owns the code if the vendor relationship ends. The commercial tiers (Aeron Premium and QuestDB Enterprise) cover the high availability, resilience, security, and enterprise support a mission-critical workload needs in production, which is the answer to the procurement question of who to call at 3am when it breaks.
Where this goes next in capital markets engineering
Adaptive and QuestDB are both working with capital markets firms that are rebuilding parts of their tech on open foundations. The Aeron and QuestDB pairing has been a useful reference architecture for that kind of work, especially for teams that want to move faster without giving up the latency and analytical depth they had before.
The pairing is already running in production at firms across exchanges, Tier 1 banks, and hedge funds, and QuestDB picked up the TTI Europe 2026 Best Trading Analytics Platform award off the back of that work. We’ll be writing more on specific architectures and customer use cases. If you’re working on something similar, get in touch.
Further Reading
Aeron
-
Aeron Cookbook – The technical reference for developers building with Aeron.
-
Aeron Performance Testing & Benchmarks
QuestDB
Subscribe to our newsletter
