The potential real-world impacts of hastily deployed artificial intelligence rollouts have been highlighted in new reports that underscore the need for better-quality data and greater literacy in the technology.
Financial firms that don’t invest in creating greater workforce awareness of how AI tools can be used are at risk not only of failing to optimise the utility of their applications, but are also exposing themselves to the the deleterious consequences of misuse.
One study, published by Harvard Business Review, indicated that AI’s appeal and engagement by end users was creating unintended booby traps. IT employees at a company studied by the publication were found to be so excited by the productivity gains of their AI applications that they had begun working overtime and more intensively on them. They had become addicted to AI’s efficiency.Experimentation May be Undermining Your Business
Greg Hanson, Group Vice President and Head of EMEA North at data management and integration specialist Informatica by Salesforce, warned that such a situation would be unsustainable. It would result in what he calls an AI verification tax – a burden placed on employees to apply additional checks on the AI outputs produced by exuberant experimenters.
“AI intensifies workloads when companies fall foul to the AI verification tax,” Hanson said. “If AI can’t be trusted to work unsupervised, the productivity promise collapses and instead adds time to the task. This leaves employees spending more time checking and correcting AI outputs as they would if they were doing the task themselves.”
Hanson spoke to Data Management Insight earlier this year about the risks of improperly managing AI rollouts and how low levels of AI literacy within companies put them in jeopardy. It exposes them to over-reliance on outputs from models fed by poor-quality data. His AI verification tax concept is born of the same pain point.“This verification burden is compounded by a skills gap,” he said, adding that “75 per cent of data leaders tell us their workforce lacks data literacy, and 74 per cent say more AI literacy training is needed to use AI responsibly. But it isn’t inevitable.
“Where data is well governed and employees have the skills to challenge AI outputs, verification drops, decisions scale more safely and productivity gains become real rather than theoretical.”
The Harvard report also warned that the intensification of work caused by the novelty of AI was unsustainable. It argued that once the excitement of experimentation has worn off, the same workers who had eagerly embraced AI could find workload creep stretches their abilities.
From Excited to Exhausted
The impacts on organisations could be extensive.
“Because the extra effort is voluntary and often framed as enjoyable experimentation, it is easy for leaders to overlook how much additional load workers are carrying,” the report stated. “Over time, overwork can impair judgment, increase the likelihood of errors and make it harder for organisations to distinguish genuine productivity gains from unsustainable intensity. For workers, the cumulative effect is fatigue, burnout, and a growing sense that work is harder to step away from, especially as organisational expectations for speed and responsiveness rise.”
While experimentation appears to be a characteristic of financial firms’ use of AI, a separate report suggested that the industry is taking a cautious approach. Based on a survey of 250 financial services from seven countries by observability and AIOps provider Riverbed, the report found that firms are growing more confident in their artificial intelligence strategies and are increasingly seeing a return on their investment.
Almost two-thirds said they are confident in their AI programmes and 89 per cent said they were seeing ROI from their AIOps that exceeded expectations.
The study made some unsettling discoveries too.
Only 43 per cent of the respondents said they were fully confident in the accuracy and completeness of all their organisations’ data. That was the lowest confidence rating among industries surveyed in a broader study of 1,200 decision makers.
Just 33 per cent rated their data as excellent for relevance and suitability, even though more than nine on 10 said they recognised the that improving data was critical to success.
Inaccurate Data Remains AI’s Key Challenge
Pressure on financial firms to optimise operations through AI has meant many have adopted applications without the data readiness in place to properly utilise the tools. Consequently, only 12 per cent said their AI initiatives had been fully rolled out across the full enterprise.
The findings point to AI implementation gaps that could hinder the further scaling of AI beyond pilot programmes.
“Financial services organisations are among the most sophisticated and disciplined adopters of AI, and our research shows they’re already seeing strong returns,” said Jim Gargan, chief marketing officer, at Riverbed. “However, the sector operates under unique pressures, including rigorous regulatory scrutiny, zero tolerance for downtime and a critical need for data accuracy.”
Subscribe to our newsletter
