Fintech AI Review #13
July 4th, AI mega-adoption, knowledge-system advances, a ton of takes, and news around risk and fraud
Greetings from beautiful Wellfleet, MA, where my family and I are spending the July 4th weekend. I’m grateful to have the good fortune of being born in the United States, the world’s greatest example of a society built on religious freedom, economic opportunity, the rule of law, and rights that flow from the people rather than from a sovereign. We may have our problems, but we remain a beacon of light in a complex world, and our inventiveness, openness, and fundamental optimism mean that there is no nation on earth better positioned to win the future.
Today’s newsletter covers a variety of topics sure to be of interest to builders, operators, and investors in artificial intelligence and financial services. These include a window into large-scale adoption by the largest institutions, technical innovations to build higher-performing knowledge systems, optimistic and pessimistic takes on the state of AI, and news around analytics, risk, and fraud use cases.
It can be difficult to distinguish between inputs and outcomes when reading accounts of AI adoption among the largest financial services institutions. It’s one thing to hire a ton of people, spend a lot of money, and issue press releases. It’s another to report on outcomes - products and services that create demonstrable value for firms and their customers. A report from Constellation Research provides much more detail than usual into J.P. Morgan’s data and AI strategy. nCino also released a “Banking Advisor” product that can be used to drive automation and efficiency among its many bank clients. Of course, building in AI is not as simple as throwing a bunch of data at an LLM and asking it some questions. Real-world applications in a highly-sensitive domain require well-tuned and thoughtfully-deployed models and high-performing end-to-end data pipelines. Predibase released an index of fine-tuned models and benchmarks to demonstrate the viability of task-specific fine-tuning, and Contextual.ai launched its concept of “RAG 2.0”. When I first learned about RAG a while back, my reaction was: “This is cool, but it also seems like kind of a hack. I wonder what will replace it.” I’m glad to see it as an active area of research and development.
At this stage in the AI hype cycle, there are a lot of takes. Even avoiding the unhinged (“AI will kill us all”) or the simplistic (“AI will solve every problem immediately”), it’s quite easy to find a range of opinions and predictions at varying levels of optimism and pessimism. Chris Mims wrote in the WSJ that AI is already losing steam. Ben Evans wrote a solid essay on the differences between general-purpose technologies and specifically useful products, wondering whether AI can actually be both. Retool released a survey and report on how companies (presumably including their many clients) are using AI today. Matt Harris of Bain shared a presentation on the fintech “hero’s journey”, and how things can only go up from where we are right now as an industry.
So many of the practical applications of AI in financial services are, from a user perspective, behind the scenes, and there’s a lot happening here. Taktile and Ocrolus (where I work) announced a partnership to bring high-quality data to lenders via a modern decisioning platform. Sentilink shared thoughts on the reality of GenAI-based fraud risks. Coris introduced an “agent” to help automate risk workflows. I also recorded a webinar with 2 industry colleagues that ended up being a really fun and wide-ranging conversation on the potential for AI in risk management.
As always, please share your thoughts, ideas, comments, and any interesting content. If you like this newsletter, please consider sharing it with your friends and colleagues. Happy reading!
Recent News and Commentary
Windows into adoption in banking
While size and incumbency may have been barriers to adoption in earlier waves of technology (e.g. mobile, open source, cloud), there’s a solid argument that the largest, most well-capitalized organizations are best positioned to benefit from the potential of modern AI, given the importance of large datasets, substantial capital investments, and talent. Of course, realizing the potential of AI in a massive organization isn’t easy; it requires a thoughtful data strategy and real dedication. This piece from Constellation Research is nearly a year old, but it provides some valuable insight into the data strategy of J.P. Morgan Chase, which does seem to be at the forefront of megabanks in the productive utilization of new technology. The report mentions how JPM is adopting a multi–public-cloud approach, even while modernizing its own data centers. The bank’s head of technology strategy stated that given their scale, having access to massive public cloud compute capacity is essential: "If you look at our size and scale, the only way to deploy at scale is to do it through platforms.” The report also provides some insight into the bank’s data mesh architecture, which in theory allows for the development of many data products across the bank while preserving control and lineage. Of course, what would be even more compelling is information on outcomes rather than inputs (i.e. not how much money they are spending or people they have working on it, but the magnitude of outcomes they have been able to achieve for their customers). Nevertheless, it is clear that JPM is making a massive bet on AI technologies, and it’s valuable to get a window into how this is being pursued.
Cloud-based banking software platform nCino announced the release of Banking Advisor, a co-pilot and set of automation tools that allows its bank clients to more easily make sense of client data, streamline workflows, and spend more time on uniquely human tasks such as managing client relationships. The release and accompanying demo video suggest a fairly broad set of capabilities, and it would be interesting to get more information on what technologies are used under the hood. However it works, these new tools appear to be quite useful for organizations engaged in relationship-based banking at scale, and nCino, with its large dataset and long client list, is well-positioned to develop and commercialize such an offering.
Technical innovations in knowledge systems
The Fine-tuning Index - Predibase
As builders and companies race to take advantage of the latest AI use cases, an important factor is the choice between commercial and open source models. While GPT-4o is nearly always an out-of-the-box leader on many benchmarks, the development of high-performance open source models has led many companies to consider whether they could use these as a lower-cost, higher-control option for more specific use cases. As with anything in engineering, there are a set of tradeoffs, the answers to which depend on the goal and definition of success. Predibase, a fine-tuning and inference platform for LLMs, has published a leaderboard of fine-tuned small and/or open-source models, comparing them to GPT-4 and GPT-3.5 Turbo on various benchmarks. Some of the open-source models show impressive results, particularly the Llama3 family, open-sourced by Meta. Of course, fine-tuning a model requires a large-enough repository of high-quality data, making such approaches viable for narrow, specialized tasks, but less likely for cases requiring reasoning or some form of generalized “thinking”. In financial services applications, there are certainly solid arguments to both approaches, depending on the criticality and generality of the use case. The most sophisticated and technically capable firms often build on high-quality, commercial foundation models to prove a use case, but many may shift to a hybrid approach over the long term.
Introducing RAG 2.0 - Contextual.ai
Retrieval-augmented generation (i.e. “RAG”) is a widely used technique for extending the capabilities and accuracy of an LLM-based system using a specific corpus of knowledge. This is particularly attractive in industry-specific or company-specific use cases, where there is value in allowing an LLM to access a large proprietary knowledge base in a highly-orchestrated and purposeful way. In a relatively short period of time, RAG has become so broadly adopted that several companies have emerged to focus on the facilitation of building and deploying RAG systems. One such company, Contextual.ai, has introduced a system they refer to as “RAG 2.0”. To draw contrast, Contextual in this blog post describes typical RAG systems as “frozen”, meaning that each component (embedding model, retriever, LLM, etc.) is static and that the full system is stitched together in a brittle way. RAG 2.0, in contrast, integrates its components end-to-end specifically for the purpose. Based on the benchmarks they show in the post, this approach appears effective. This type of evolution has the potential for real impact in financial services, where industry players are eager to take advantage of AI but are also highly focused on building systems that are relevant to the task, grounded in real-world (and often non-public) data, and have a lower risk of hallucination. My take is that we’re incredibly early in the journey toward fully-integrated, high-accuracy knowledge platforms, and it’s good to see so many people working on novel approaches.
The State of AI - Takes ranging from pessimistic to optimistic
The AI Revolution Is Already Losing Steam - WSJ
In forecasting the future, it is far easier to predict the what than the when. Technologies that appear novel and truly capture peoples’ imagination tend to spur wild visions of rapid change. Then inevitably, certain things happen slower than expected, and many changes take a form far different from when was originally imagined. Gartner famously calls this the hype cycle. Bill Gates famously said: “Most people overestimate what they can do in one year and underestimate what they can do in ten years”. Carl Sagan famously said “It was easy to predict mass car ownership but hard to predict Walmart”. It’s just incredibly early. In this WSJ article, Chris Mims points out the ways that the AI revolution is “losing steam”, referencing slower-than-expected progress in many areas and the unpleasant surprises many businesses have had regarding the cost, speed, and value of read-world AI deployments. It’s a worthwhile read, though I have a different perspective. It’s not really that AI is losing steam, and in fact many of the technical advances that are not as visible to the lay public are happening at an astonishing pace. It’s just that big things do take time - particularly when they involve questions of broad importance to society - and the impacts are often non-obvious.
Building AI products - Ben Evans
It’s one thing to have a set of new and impressive technologies. It’s another thing to build these into useful products that gain serious adoption. Perhaps even harder is when the technology in question is a new probabilistic computing paradigm that often “gets things wrong” (i.e. ‘hallucinates’). These are the topics that Ben Evans discusses in this characteristically well-thought-out essay. One particularly compelling point is his comparison of AI to the invention of the electric motor, which enabled a ton of products that otherwise would not be possible. Of course, as he says, “Electric motors are a general-purpose technology, but you don’t buy a box of electric motors from Home Depot - you buy a drill, a washing machine and a blender.” He wonders if some of the excitement about AI is its potential to actually break that pattern, to be a general-purpose technology that is also, itself, specifically useful. A good read for those building products and companies on AI building blocks.
The State of AI H1 2024 - Retool
Retool - the internal business tool development platform - conducted a new version of its periodic “state of AI” survey and released a comprehensive report on the findings. The sample is fairly comprehensive, comprising around 750 people across multiple industries, roles, and levels of seniority. The report - which, by the way, is presented with an admirably unique and captivating visual design - reveals sentiment across many different areas of AI adoption. For example, there are data on levels of adoption, specific technologies, ROI, areas of usefulness, developer tools, and predictions of the future. It’s a valuable read but also comes with a handy table of contents for anyone wishing to jump to particular areas.
Only Up From Here: 2024’s State of Fintech and the Hero’s Journey - Matt Harris @ BCV
Matt Harris and his colleagues at Bain Capital Ventures are fintech veterans who enjoy a comprehensive view of the financial services and technology ecosystems. It’s always valuable to read or hear what they have to say. In this presentation, given at a joint Fintech CEO Summit with NYCA Partners, Matt explained his thesis for why we are at a “bottom” in the fintech markets and painted an optimistic case for the future. Using the conceit of a “hero’s journey”, this presentation reviews some of the lessons from our collective path to the bottom, including the end of “middleware BaaS” and regulatory arbitrage. A core belief looking forward is that the traditional model of banking (i.e. net interest margin) is facing an existential crisis over the next several years, and that this will provide opportunity for resourceful and creative founders. It’s a thoughtful and forward-thinking analysis, and it’s worth the time to watch the video or read the transcript.
Analytics, risk, and fraud
Taktile and Ocrolus partner to unlock real-time underwriting for small business lenders
Lenders rely on high quality data, insightful analytics, and repeatable automation to make good credit decisions at scale. Yet, while there exist numerous data sources for understanding the financial health of people and business, many lenders lack the infrastructure to rapidly integrate, test, simulate, and deploy workflows using the entirety of available information. Ocrolus (where I work) is an AI-based document automation platform that provides highly-accurate data to lenders in small business, consumer, and mortgage lending (among other use cases), and Taktile is a modern and highly capable decision engine for lenders. I’m excited about this partnership given the clear compatibility of both companies’ technologies and the potential to streamline the integration and useful application of data in the underwriting processes of literally hundreds of lenders.
SentiLink’s Thoughts on GenAI Fraud
Anyone who’s explored the capabilities of state-of-the-art AI to generate realistic images, audio, video, and text has probably considered the risk of a greater incidence of fraud made possible by the technology. Of course, the reality is quite a bit more nuanced. This blog post from Naftali Harris at Sentilink, a leader in synthetic fraud prevention, considers the specific areas where genAI poses a greater or lesser risk and provides valuable insight on how to think about it. Interestingly, he does not believe genAI will result in an explosion of synthetic identity fraud, as the bottlenecks to the creation of credit-worthy synthetic identities aren’t easily alleviated by this technology. He does, however, mention several common fraud controls that could be severely threatened, including those involving video, voice, and behavior monitoring. While it’s reasonable to conclude that the solution to bad guys with AI is for the good guys to have better AI - and this is true in many cases - he also points out some reasons why this is not entirely straightforward, and the longer-term is unclear. Anyone in fintech should read this post, as if you’re in the business of money, you’re in the business of fraud prevention whether you know it or not!
Introducing Risk AI, your agent for risk management workflows - Coris.ai
In any risk decision process, there are always cases that are an easy yes or an easy no. Unfortunately, plenty of cases end up in a gray zone of uncertainty, where a quantitative prediction model or deterministic ruleset does not perform reliably. Decisions made in this zone often massively determine the fate of a lender, payment facilitator, or other provider. The solution in such situations has traditionally involved heaps of manual review, often with inconsistent decision quality, spotty results, and poor customer experience. Many hope that agents built on generative AI technology have the potential to enhance decision-making of this sort. Coris, a small-business-focused data intelligence platform, released RiskAI, its version of such an agent. This blog post explains the applications where such a tool is useful and includes details on its features, including the ability to slack with an AI agent that has access to merchant profile data and can be configured to assist in decision automation for particular use cases, specifically endeavoring to reduce false-positive errors.
What’s next for AI in risk management? Webinar recap and on-demand playback
All financial services businesses are fundamentally about risk management, and the advancement of AI technology provides optimism that we might be able to (metaphorically-speaking) catch more tuna while ensnaring fewer dolphins, all the while providing better customer experiences at a lower cost and passing on the savings to end users. I recently recorded a webinar with my friends Vinodh Poyyapakkam of Coris.ai and Ryan Hildebrand of Bankwell. It was a wide-ranging and intellectually-stimulating discussion of AI applications we’ve presently observed, the greatest potential problems that AI could solve, and what we may see over the next several years. You can watch the webinar in full or read a summary in this blog post from Coris.
Thanks for another thoughtful review. It is very helpful having you curate a range of perspectives on developments across Gen AI.