Fintech AI Review #6
Fintech AI Review #6
Upending the banking system with GenAI, expanding access to financial advice, open-source LLMs sparking permissionless innovation, and the critical role of quality
Happy summer to all of our readers and especially our new subscribers. The pace of AI in fintech does not seem to be taking a summer vacation! Those observing the space closely are witnessing meaningful progress on a daily basis, whether that takes the form of major developments in AI research, companies announcing specific product releases, or new thinking on generative AI’s ability to reshape parts of our banking system and culture.
Today’s newsletter contains a wide variety of content, along with my context and commentary for each link. Generative AI has the potential to massively disrupt the structure of the U.S. banking system, and CEOs of big businesses also need to consider how to lead their organizations through AI-driven transformation. Innovations like OpenLLaMA extend the power of highly-capable foundation models to additional use cases, potentially unlocking a huge wave of bottoms-up innovation the same way open source software did earlier.
Several financial services providers are debuting AI-powered chatbots, and a few examples are covered below. While we may wonder whether a chatbot would be worse or better than human assistants, perhaps that is the wrong question. There is a massive range of variation in human skill, task-related performance, and the quality of service provided by a human employee of a client-facing business. AI chatbots may not now or for quite some time be comparable to the best human reps, but they might perform better than some percentage of reps, even today. That number will certainly increase over time, and if done in a responsible manner with high attention paid to accuracy, it will expand access to good financial products and advice at more favorable costs.
Speaking of accuracy, all models require data, and generative AI models require data at massive scale. A couple of the pieces below consider the data that underlies all these technical advancements, including a deep dive into the world of human data labeling. In a world awash in data, much of it is still messy, hard to access, poorly labeled, inaccurate, or not legally viable. Those developing AI-powered applications for financial services particularly need to have data availability, quality, and accuracy as top priorities. This is easier said than done!
As always, please share your thoughts, ideas, comments, and any interesting content. If you like this newsletter, please consider sharing it with your friends and colleagues. Happy reading!
Latest News & Commentary
In this boldly-titled piece, Matt Harris of Bain Capital Ventures makes the argument that generative AI will drastically reshape the U.S. banking system, making the regional bank failures of the last several months look small in comparison. The United States is highly atypical among advanced, Western economies, with thousands of banks, many of them being small local ones. Attributing small-bank customer stickiness to inertia rather than loyalty - a point I’ve made for some time - he states that generative AI will overcome this inertia by making financially-optimizing recommendations to customers and lowering the cognitive load of switching institutions. He predicts the transition from 4,500 to ~100 banks will be ‘ugly’ but recommends that regulators ease the process by facilitating mergers and acquisitions to land at an equilibrium of 10 or so ‘too big to fail’ banks, along with 50-100 more specialized institutions, all well-capitalized. Change in the structure of the banking system is clearly going to take place, and the prospect of GenAI as the catalyst makes sense. It’s pretty clear that today’s system is sub-optimal, and it will be up to banks, their customers, and policymakers to consider options for the best possible future state.
Two of the most accomplished American business leaders make the case that CEOs must actively lead their companies through AI-driven transformation. In doing so, they reference past transformative technologies, the impact they had on business, and the consequences for those who were caught flat-footed. AI requires not just technology investment but a full-on management approach, and they cite 3 specific recommendations: 1) expand the scope of data governance to include AI; 2) train employees in all functions (not only technical ones) to be AI-literate; and 3) evolve corporate culture for the age of AI. It will be interesting to see how large companies, particularly those in financial services, are able to transform their cultures and businesses for the AI era. While there’s probably not one Fortune 500 that isn’t talking about its ongoing application of AI, there’s likely a substantial gulf between superficial toe-dipping and serious transformation. Consider how long it has taken so many banking institutions to undergo “digital transformation”. Perhaps today’s big companies will learn the lessons of past technological shifts and move quickly and decisively. In a couple years, it will be very clear who took which approach.
OpenLLaMA: An Open Reproduction of LLaMA
In February 2023, Meta AI released LLaMA, a powerful large language model (LLM) designed to achieve state of the art results while requiring less computing power than other comparable models (e.g. GPT-4). Thanks to Meta’s public release of LLaMA, many researchers have been able to use it as a foundational model to fine-tune for particular purposes. Indeed, many interesting AI papers in recent months - including some covered in this newsletter - have used LLaMA as their base model. While LLaMA is a fantastic achievement and powerful tool for the research community, it is not licensed for commercial use. Now, Xinyang Geng and Hao Liu from Berkeley AI Research have released an open-source reproduction of LLaMA, called OpenLLaMA. The model is built using the same procedure described in Meta’s original paper but is trained on a different, open-source dataset known as RedPajama (itself designed to be a reproduction of the LLaMA training set). According to the evaluation metrics published, OpenLLaMA performs comparably to LLaMA on a variety of benchmarks. Most importantly, this version is released under the Apache-2.0 license, which permits commercial use. If AI is indeed “having its Linux moment”, as many have stated, open-source foundation models like this will enable people and companies to fine-tune and build for their own specific applications. It will be interesting to see how models like this get adopted in fintech and financial services, particularly with the ability to run such a model without sending any data to 3rd parties. It’s not too hard to imagine the combination of a powerful, open-source base model and a massive proprietary dataset resulting in some truly compelling applications.
Digital bank One Zero to debut generative AI chatbot - Finextra
Almost every financial institution has some form of automated question-and-answer system. These come in the form of “interactive voice response” (IVR) phone menus or web-based chatbots and are typically powered by some form of pre-coded decision tree. One thing they all have in common: customers hate them! If you frequently find yourself shouting at the anthropomorphized IVR-bot, “I just want to speak to a human!!!”, banks are hoping that generative AI chatbots may solve the problem. One Zero, a digital bank in Israel, is piloting a GenAI-based chatbot with a small cohort of customers in advance of a widespread general release in Q4 2023. In doing so, it hopes to provide: “...an exceptional customer experience that closely resembles one-on-one human interaction, with zero wait time.” The bank partnered with AI21 Labs to build a chatbot that would not need to send data to a public cloud service and would be able to competently handle sensitive information. My assumption is that every single bank is looking at and considering this type of technology, yearning for its benefits while simultaneously grappling with important concerns, including data privacy but also how to mitigate the risk of a bot providing incorrect or misleading information to a customer.
Monarch, an online consumer financial management platform, released an AI-powered personal financial assistant. The demo and examples on the website look interesting and informative and presumably rely on several already-existing capabilities of the Monarch software, such as transaction categorization, investment tracking, and goal-setting. According to the FAQs, Monarch uses GPT-4 to take a user’s questions and provide relevant answers in the context of the data in their account. For example, “how does our restaurant spending compare to others like us” or “what are our top recurring subscriptions”. While I don’t think an AI financial assistant is ready to give truly unique advice or handle particularly complex portfolios, innovations like this are all about access. If Monarch and services like it can use a generative AI application to provide relatively standard prudent financial management, budgeting, and investment advice to consumers with uncomplicated portfolios, that is likely superior to what a non-trivial portion of American households could otherwise obtain.
In another example of generative AI’s potential impact on intellectual property law, a new subscription for AI-based services in Adobe products will come with protection against copyright infringement claims. As discussed in a previous issue of this newsletter, there is mounting concern about intellectual property issues around the use of large language and computer vision models, particularly if the models were trained on data that included content subject to copyright. Adobe has been a leader so far in this arena, training their “Firefly” image generation model only on images to which they own the rights or on those that are in the public domain. They are so confident in their approach that for enterprise Firefly users, they will now “...compensate businesses if they’re sued for copyright infringement over any images its tool creates.” This is an interesting approach to potentially-unclear or fast-changing areas of law, whether in design software, financial services, or otherwise: doing what you believe is the right thing and then offering to indemnify users against claims to the contrary.
AI Startups Have Tons of Cash, but Not Enough Data. That’s a Problem. - WSJ
While generative AI startups have raised billions in funding to pursue seemingly massive and multiple opportunities across a wide range of industries, many are struggling due to a lack of unique or proprietary data. This article in the WSJ tells the story of startups who are building AI-based products that target a particular use case but who face difficulties gathering the data they need to train task-aligned models or build a competitive moat. Some have tried to partner with more established firms for access to this information, but often, the large companies who possess the largest and likely most useful datasets are reluctant to provide access. Security and privacy concerns are top of mind, as are ownership issues around the resulting models and their eventual output. One investor compared the drive to obtain access to useful data to an “arms race”. This is clearly an important issue and likely familiar to anyone who has attempted to build a business that relies on data. It’s difficult to obtain data by partnering or convincing other companies to share what they perceive as a precious asset. It’s highly preferable to be organically in the flow of data for a product-related, operationally-critical reason. Models are easy; the hard part is data. If you’re building a product based on generative AI, or investing in someone who is, it’s important to consider if there is a differentiated ability to obtain high-quality, relevant, and usable training data.
Are we there yet? - Amias Gerety, QED
In this essay, Amias at QED explains the “hackathon mode” that is so common around novel uses for LLMs, why this is exciting, but why the best AI-driven solutions require going beyond “hackathon mode”, particularly in areas like financial services where accuracy is paramount. “Hackathon mode” is fantastic, particularly when it allows a single developer, or even someone non-technical, to use LLMs to quickly build an experience that might have otherwise taken weeks or months, and more than 1 person. While the ability to enable creative and productive work with greater efficiency is an impressive and useful capability of LLMs, there are industries, financial services in particular, where most of the relevant training data are not publicly available, and the particular business logic required is not accessible to general-purpose LLMs. Amias gives the example of 2 companies where he is an investor and board member, Ntropy and Ocrolus (where I work). These companies recently announced a partnership that is, in his words, “not only a triumph over ‘hackathon mode’ for each of them, but also allows their customers to get the benefits of proprietary data and fit for purpose models.” With highly skilled machine learning and data science teams, both companies could perhaps build “slide-ware” versions of each other’s products fairly rapidly, but lending and other financial use cases require a much higher standard. Ocrolus and Ntropy have prioritized accuracy and real-world relevance over “hackathon mode”.
AI Is a Lot of Work - The Verge
Models require data, and the latest, most impressive generative AI models require incredible quantities of it. This long and detailed article explains the world of data labeling and annotation - in other words, the work humans do to create the data used to train AI. Tasks performed by human annotators range from the simple and repetitive (e.g. “label the elbows in this photo of people”) to the complex (e.g. “talk with this chatbot about science fiction and mathematical paradoxes all day, and rate the quality of its responses”). This type of work is done by many companies, across many geographies, for a wide range of hourly compensation that tends to be a function of skill required and geography. Models are only as good as the data on which they are trained, and this is a good read for anyone seeking a greater understanding of some of the (often surprising) work that goes into developing today’s most impressive generative AI systems.
Mike Senecal at Cardrates interviewed me about how lenders are using Ocrolus’ AI-driven document automation platform to make high-quality decisions with trusted data, unparalleled efficiency, and a top-notch customer experience. I enjoyed speaking with Mike about a wide variety of topics on this theme, including how Ocrolus has combined best-of-breed AI products from Google, OpenAI, and Amazon with its own in-house models trained on proprietary data. We discussed how small business, mortgage, and consumer lenders rely on accurate data and insightful analytics to make important credit and fraud risk decisions.