By Iosif Itkin, CEO and co-founder, Exactpro, and Daria Degtyarenko, Researcher, Exactpro.
With regulatory requirements and business pressures putting an ever-tightening squeeze on financial services, firms tend to look for the most efficient ways of ensuring the quality and stability of their platforms, with some turning to quality assurance outsourcing. However, outsourcing might not always be the best plan, for four key reasons: responsibility, capability, agility and quality.
Responsibility. In its SYSC 8.1 General outsourcing requirements, the Financial Conduct Authority (FCA) states that a regulated entity must not ‘delegate responsibility’ while outsourcing ‘critical or important operational functions or any relevant services and activities’. The quality of software systems and platforms underpinning the financial market infrastructure’s (FMI’s) operations and services is certainly important and arguably, critical. Therefore, it is a regulatory directive that quality assurance must not be delegated.
Capability. To stay competitive, any company must be technology-focused. No company would want its core competency to be building low-quality products, so quality assurance is not a peripheral function. Software testing cannot be viewed as an ancillary activity for a capable, competitive firm.
Agility. Relying on external vendors could hinder the flow of information necessary for companies focused on reducing time-to-market and undergoing agile transformations. Flexible software development and acceptance-testing methods require synchronisation among all team members, which can become difficult when outsourcing, because the teams are divided by firewalls, time zones, distances, and cultures.
Quality. ‘Quality cannot be tested into a product’, so goes the aphorism, meaning that product quality is not solely determined via testing. Firms focused on quality work on holistic approaches to software development continuously analyse testing outcomes, measure attributes related to product quality and invest in improving risk management tools with the aim to prevent rather than cure. ‘Quality is everyone’s responsibility’, as another old saying goes. Why would anyone want to rip out an integral piece of the intertwined quality puzzle by outsourcing software testing?
Exemplary organisations take their regulatory responsibilities and technology capabilities very seriously. They relentlessly strive to achieve quality and agility in their delivery. So surely the case for DIY software testing is watertight. But, true software testing is not there to confirm perfection. Software testing exists to uncover something that somebody doesn’t want to exist. It is about providing objective information about the system under test in a form beneficial for the stakeholders.
To paraphrase James Madison, one of the founding fathers of the US, if companies were angels, independent software testing would be unnecessary. If angels were to develop software, neither external nor internal controls would be necessary. Software testing, if independent, does not set managers free from responsibility. Rather, it pressures them toward higher levels of regulatory compliance.
When Tolstoy wrote in Anna Karenina: “Happy families are all alike; every unhappy family is unhappy in its own way”, he meant that in order for a family to be happy, it must succeed with respect to a specific range of criteria, and failure on any single count leads to unhappiness. Similarly, to be successful, all production systems must work well within a narrow range of criteria that assures their users’ and stakeholders’ satisfaction. In order to consistently reach that benchmark, it is of paramount importance for a technology firm to have a strong in-house capability to build, check and monitor systems to ensure that the family stays happy.
Software testing is focused on exploring the darker side of systems. Research confirms that developers are less likely to deploy advanced software testing techniques, such as passive testing, in part due to the congruence bias, i.e. overreliance on proving a specific hypothesis that directly results in neglect of indirect testing. Thus, leaving software testing to developers or cross functional teams could mean prioritising the component level and ‘happy path’ checking over proper risk assessment of an interconnected system as a whole.
Fundamentally, biases are not the enemy, because they exist for a reason: to increase efficiency. Superiority requires focus, and a good strategy always requires making a choice. In this particular case, it is whether to prioritise improving the core product versus building a sophisticated test harness. The creation of test harnesses is the core capability of an independent software testing business, so it doesn’t make sense to hire smart testers and then tell them what to do. By the same token, outsourcing does not guarantee software testers’ unbiased independence.
Many articles have been written about cross-functional, self-organising teams and confining software testing to sprints. Genuine agility is antifragile. Iterative processes considerably benefit from prompt independent feedback. In implementations that are agile in name only, there will always be a way to blame inefficiency on faulty communication. Reliable systems are not built on trust, rather, they are built on the absence of trust, which is why independent software testing is fundamental to well-functioning financial firms, similarly to the importance of free press to a well-functioning democratic system.
Independent testing alone is not enough to achieve quality, especially when introduced at a late stage of software delivery. The best way to limit the total cost of the project is to incorporate independent testers into self-organising teams as observers and contributors, so they can detect problems as soon as possible. Whether the issue is a software defect or a process deficiency, it is never too early to embrace reality and deal with it.
If outsourcing software testing is a necessary evil, the shame of delegation reflects a gap between the status quo and the ideal world. Relying on outsourcing is not perfect however, if applied properly, outsourced testing can at least create awareness that a gap exists and possibly help to narrow it.
Subscribe to our newsletter
