As generative AI (GenAI) continues to revolutionize various industries, the financial sector finds itself at a critical juncture. The potential benefits of GenAI are vast and far reaching, from enhancing productivity to improving regulatory compliance. However, alongside these advantages comes a significant and growing threat: the use of GenAI for sophisticated fraud, particularly through deepfake technology. Recent research from Gartner® and a harrowing example reported by The Wall Street Journal highlight the urgent need for financial institutions to bolster their defenses against this emerging risk.

The Dual-Edged Sword of Generative AI in Finance

According to the 2024 Gartner® The Current State of Generative AI Use and Adoption in Finance, “ While many finance leaders expect and see increases in productivity as a result of GenAI implementation, our research found a significant mismatch between expectations and realization of many other GenAI benefits.” This new and powerful technology also introduces new vulnerabilities. CFOs are increasingly aware that the benefits of GenAI, such as improved employee experience and reduced business risk, are accompanied by unforeseen challenges, particularly in fraud prevention. The January 2024 incident in Hong Kong, where a deepfake video was used to defraud a company of $25 million, underscores the scale and sophistication of these threats. This is not an isolated incident and as GenAI continually becomes more accessible, it is inevitable that the financial sector will see a surge in such attacks.

The Growing Threat of AI-Driven Fraud

Generative AI is making fraud easier and cheaper to perpetrate. Deepfake technology, once the domain of high-budget operations, is now widely available for free on the open internet enabling even amateur criminals to execute highly convincing scams at scale. Deloitte’s Center for Financial Services predicts that fraud losses enabled by GenAI could reach $40 billion in the United States by 2027, up from $12.3 billion in 2023—a staggering 32% compound annual growth rate.

The rise in deepfake incidents, which reportedly increased by 700% in the fintech sector in 2023, is a clear indicator of the threat level. Audio deepfakes, in particular, pose a significant challenge, as current detection tools are often inadequate. The financial sector’s reliance on trust and secure communications makes it especially vulnerable to such sophisticated forms of deception.

Preparing for the Future: Strategies for Financial Institutions

Given the rapid advancements in GenAI and its implications for fraud, financial institutions must act swiftly and decisively. Banks and financial firms can no longer rely solely on traditional fraud prevention methods, which were not designed to counter the dynamic and evolving nature of AI-driven threats. Here are several strategies that CFOs and financial leaders should consider:

1. Enhance AI and Fraud Detection Capabilities: Banks should invest in developing and acquiring advanced AI tools specifically designed to detect deepfakes and other AI-driven frauds. This includes partnering with third-party vendors specializing in media intelligence and disinformation security, such as Deep Media.

2. Strengthen Collaboration and Governance: Given the industry-wide implications of GenAI fraud, financial institutions must collaborate more closely with each other and with regulators. Sharing intelligence and best practices can help create a more unified defense against these threats. Additionally, involving compliance teams early in the development of GenAI initiatives can help establish robust governance frameworks.

3. Invest in Talent and Training: The fight against AI-driven fraud will require a new set of skills and expertise. Financial institutions must prioritize hiring and training employees who can identify, stop, and report AI-assisted fraud. This investment in human capital is crucial, even in an era where cost management is a priority for many banks.

4. Educate and Engage Customers: Financial institutions must also focus on educating their customers about the risks associated with GenAI. Building awareness through regular communication, such as push notifications about potential threats, can help customers become more vigilant and reduce the likelihood of falling victim to scams.

Conclusion: The Urgency of Deepfake Detection

With the advent of new and sophisticated GenAI technologies, one thing is abundantly clear: The financial sector is at a crossroads. While GenAI offers immense potential for enhancing operational efficiency and compliance, it also opens the door to unprecedented levels of fraud. As incidents like the Hong Kong deepfake scam demonstrate, the stakes are incredibly high. Financial institutions must recognize the urgency of developing and implementing robust deepfake detection and disinformation security strategies. Only by doing so, can they hope to protect their assets, their customers, and their reputations in an increasingly complex digital landscape.

For CFOs and financial leaders, the message is clear: the time to act is now. The future of finance depends not just on leveraging the benefits of GenAI, but also on safeguarding against its risks.

Gartner, The Current State of Generative AI Use and Adoption in Finance, By Lakshya Gupta and Marco steecker, 31 May 2024.

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.