For years, the evolution of low-latency trading infrastructure has largely been measured in one direction: faster. Microseconds became nanoseconds as trading firms invested in colocation, network optimisation, specialist hardware and FPGA acceleration to gain an edge in markets where position in the queue can determine whether an order is filled.
That race hasn’t gone away. But as participants in a recent TradingTech Insight webinar, Reviewing the Latency Landscape and the Next Generation of Ultra-Low Latency Infrastructure, discussed, the engineering challenge is expanding. Firms increasingly need to determine where extreme speed genuinely changes trading outcomes, while making that performance deterministic, observable and repeatable across a much larger and more complex trading estate.
A stratified latency landscape
Robert Tommaselli, former IT architect for trading infrastructure, risk management and market data at RBC Capital Markets and currently consulting at CIBC, described today’s latency landscape as increasingly “stratified,” with the importance of latency varying considerably by trading strategy and asset class.
At one end are proprietary trading firms and market makers operating in markets such as listed futures, options and US equities, where nanoseconds can determine queue position. FPGA and other hardware-based approaches are consequently heavily used by those firms for functions such as processing market data and building order books.
Beyond that ultra-low latency tier is a much broader group of sell-side electronic trading businesses operating in the microsecond range, where relatively small latency differences in areas such as smart order routing and ETF market making can still materially affect execution. Further along the spectrum, in markets such as FX and fixed income, breadth of venue coverage and connectivity may carry greater weight than achieving the absolute lowest latency.
Volatile markets strengthen the case for predictable performance. Vishal Gupta, Head of Equity Derivatives Technology at Mizuho Americas Services, highlighted the risk facing market makers when prices move rapidly and quotes become stale. “Even a few microseconds or nanoseconds can cause a big loss to market makers, so it is increasingly important for them to save as many nanoseconds or microseconds as possible.”
Industrialising low latency
Achieving low latency in one environment is one thing. Reproducing it consistently across venues, asset classes and geographies presents a different challenge.
“Linear scaling as messages increase in your platform is expected, but very difficult to achieve,” observed Deepak Dhayatker, CTO of Rapid Addition.
A low-latency application deployed in London, for example, should exhibit the same performance characteristics when deployed in Tokyo. But relatively small differences in BIOS settings, network cards, operating systems, kernels and other configuration details can introduce variations that undermine deterministic performance.
Dhayatker advocated automated deployment pipelines built around standardised, immutable images incorporating a consistent operating system, network configuration and hardware baseline alongside version-controlled applications. That provides an auditable configuration that can be reproduced across regions.
The operational burden can nevertheless be considerable. Tommaselli pointed to the continual exchange-driven changes facing firms operating across large numbers of venues. Maintaining FPGA firmware and associated infrastructure while responding to protocol changes can require substantial specialist resources.
That also feeds into the economics of build versus buy. Dhayatker argued that firms should buy commoditised infrastructure and concentrate their internal engineering resources on areas that provide genuine differentiation, while ensuring that vendor choices don’t create technical or commercial lock-in.
Drawing the hardware-software line
Firms also need to decide which functions are best suited to hardware acceleration. FPGA remains attractive because it can deliver highly deterministic performance with minimal jitter. Market-data processing, protocol handling, order-book processing, timestamping and certain pre-trade controls were identified by the panel as particularly suitable candidates.
But hardware comes with limitations. FPGA skills are scarce, development can be expensive and changes can take considerably longer to deploy than in software. The rate of change therefore becomes an important dividing line between hardware and software: relatively stable, performance-critical functions lend themselves to hardware, while pricing, routing, state management and other frequently changing business logic remain better suited to software.
Software can also be engineered for greater determinism. Dhayatker pointed to techniques including CPU isolation, thread pinning, kernel bypass and non-blocking zero-copy data structures as ways of reducing variability.
As Gupta put it: “Hardware provides determinism while software provides adaptability and flexibility.”
Observability and precision timing
As architectures become more complex, knowing where latency occurs becomes increasingly important.
Dhayatker argued that observability needs to be built as “a first-class citizen” of the application rather than added afterwards. Firms need visibility not only into headline latency but into the complete execution path, including whether performance remains consistent as systems scale and deployments change.
Tommaselli similarly highlighted the role of out-of-band monitoring at the market-data and execution layers. That becomes particularly important when diagnosing whether latency originates in an application, network component, hardware layer or external venue.
Precise time synchronisation is closely connected. The panel discussed technologies including White Rabbit, which can provide sub-microsecond and nanosecond-level synchronisation between infrastructure components. As trading latency falls, increasingly accurate clocks are needed to measure events, analyse performance and maintain an auditable record of what happened and when.
A changing infrastructure model
The boundaries around low-latency infrastructure are also starting to shift. Tommaselli pointed to partnerships between exchanges and hyperscalers, including CME Group with Google Cloud and Nasdaq with AWS, as evidence of cloud infrastructure moving closer to markets traditionally dominated by dedicated hardware and colocation.
That doesn’t mean cloud is about to replace the specialised infrastructure supporting the lowest-latency trading strategies. It does, however, point towards a more hybrid environment in which firms can combine dedicated hardware, software and cloud-based capabilities according to their performance requirements.
At the same time, longer trading hours are reducing the maintenance windows on which technology teams have traditionally relied. Moving towards 24/5 and potentially 24/7 markets means upgrades, patching and other operational changes increasingly need to be carried out while trading infrastructure remains available.
Gupta suggested that future-proofing in this environment rests on three principles: optionality, observability and governance. Rather than attempting to predict which technologies will dominate several years from now, firms need architectures capable of incorporating new hardware, software and infrastructure models as they emerge.
For firms competing in the nanosecond and low-microsecond tiers, raw speed isn’t becoming less important. The engineering challenge is expanding around it. Performance increasingly has to be deterministic, observable and reproducible across venues and regions, while the underlying infrastructure remains flexible enough to accommodate exchange changes, longer trading hours and the next generation of hardware and software.
A recording of this webinar is available at https://a-teaminsight.com/webinars/reviewing-the-latency-landscape-and-the-next-generation-of-ultra-low-latency-infrastructure/?brand=tti
Subscribe to our newsletter
