Fintech AI Review #10
The coming AI revolution, AI for vertical SaaS, LLMs as a new type of computer, and some interesting real-world applications
Happy 2024 to all my fellow fintech and AI enthusiasts! While 2023 was a challenging year for many areas of fintech (e.g. neobanks, BaaS, monoline lenders, etc.), the acceleration of AI and its easily accessible proof points including chatGPT provided hope and inspiration. While it is much easier to forecast the ‘what’ than the ‘when’, it feels like 2024 is a year where part of this hope begins to transform into reality, with real-world applications in financial services.
When businesses first started to adopt electricity in their operations, beginning in the 1880s, those who did so were sometimes called ‘electric companies’, as the use of this new technology represented a meaningful distinction. Eventually though, electricity became widespread, and the only businesses that still called themselves ‘electric companies’ were those actually in the business of producing power. The same thing happened in the late 1990s with the internet’. Many businesses who had a website when it was not the norm started to be called ‘internet companies’, even if the technology was not their core product. Today, it’s just as odd to call Ikea an ‘internet company’ just because it does a lot of business online as it is to call your local coffee shop an ‘electric company’ just because it uses electric bulbs rather than whale oil lamps for lighting.
In 2023, we saw many companies rush to call themselves ‘AI companies’ if they were using AI in their products (and sometimes if they weren’t). This positioning seems to add some distinguishing value for many constituents (e.g customers, investors, employees). At some point, however, the integration of AI capabilities into every product and service will become so widespread that it ceases to be independently distinguishing. It’s hard to say whether this will happen in 2024 or 2028, but I do think we’re likely to move along that axis substantially this year, and it will happen faster than with prior technologies1.
Today’s newsletter covers a variety of topics, from investor research and thought leadership to highly technical explainer videos to chronicles of real-world applications for AI in financial services.
As always, please share your thoughts, ideas, comments, and any interesting content. If you like this newsletter, please consider sharing it with your friends and colleagues. Happy reading!
Latest News & Commentary
AI: The Coming Revolution - Coatue
The team at Coatue put together a long, well-researched, and opinionated deck on the AI revolution and its likely impacts over the next couple years. It’s extremely wide-ranging, covering everything from adoption velocity, pace of investment, and where value accrues in the stack, to developer culture, regulation, hardware wars, open source, the future of work, and AI’s impact on the power grid. There are a ton of interesting graphs and numbers, all with sources footnoted. Overall, the report takes some positions, but it asks even more open-ended questions. If you’re in the space, you owe it a read.
Vertical AI: Why a Vertical Approach is Key to Building Enduring AI Applications - Greylock
While vertical-specific SaaS companies have created large and impactful businesses in several industries, there are plenty of industries with far shallower SaaS penetration. In this thoughtful and detailed post by Christine Kim at Greylock, she outlines the very large opportunity in using AI technologies to bring next-generation vertical software to these industries. One of the key factors here is the ability of LLMs to work with and add structure to unstructured or messy data. This, in theory, would allow companies to build products for markets that don’t have highly-consistent systems of record. She describes 6 pillars to her vertical AI investing framework and then also gives examples of its potential application in professional services, financial services, and healthcare. These of course are not small industries! In fact, if we understand the market to be ‘industries with insufficient data structure/consistency/quality’, my view is that this describes….every real-world market! The opportunity here is very large, particularly for founders with deep industry expertise and the technical acumen to bring creative product solutions to life.
Intro to Large Language Models - Andrej Karpathy on Youtube
In this 1 hour video, Andrej Karpathy, an elite AI developer and OpenAI employee, describes how LLMs work in a sophisticated yet extremely accessible way. What’s great is the way that he is able to speak about an incredibly complex subject and make it understandable to a broad audience without dumbing it down. He explains how LLMs are built from the bottom up, how they are tuned to be useful, and some of their more recent new capabilities. He also shares some frameworks for reasoning about how these models ‘think’. For instance, they generally have a ‘system 1’ (instinctive thinking) but not a ‘system 2’ (slower, logical thinking), although this is developing (FYI my copy of Thinking Fast and Slow is on the bookshelf behind me as I write this). In addition, Karpathy explains how an LLM is more than a ‘chatbot’ and can actually be thought of as a new type of computer or new type of operating system, an analogy that seems to work quite well (e.g. an LLM’s context window is like a computer’s RAM). And, just as security matters in traditional computing, there are many rapidly developing security issues in the LLM world, many of which are demonstrated here. LLMs will be such a foundational building block of technology over the next few decades that they are important to understand, even if you’re not a hands-on practitioner. Just as broadly understanding relational databases has been massively useful over the past few decades (and will continue to be), understanding LLMs will be incredibly useful over the next few decades.
Building a financial large-language model in-house - NTropy @ Fintech DevCon
Ilia Zintchenko of NTropy gave a presentation on the very cool technical progress they have made building specialized LLMs for transaction classification. In the presentation, he demonstrates how and why, all other things being equal, high levels of accuracy would require a longer prompt, which of course involves higher cost and higher latency. Then, he discusses a few novel techniques they have used to achieve high accuracy at a reasonable cost and low latency. These involve fine-tuning, using models of different parameter size, entity recognition, and caching. He also describes some caveats for the use of such approaches. It’s interesting to watch Ilia discuss these challenges and solutions, as they will likely apply to many use cases, particularly in financial services. Companies will want to develop AI capabilities that deliver high accuracy on specialized tasks in a particular domain. Accomplishing this at a high level of quality while controlling costs and maintaining a performant system is a real technical challenge, and it’s great to think about all the ways it can potentially be overcome.
Coris AI launches Merchant Real Industry using GPT-4
The ability to classify the particular industry of a business is highly relevant in financial services, particularly in the domains of lending and merchant onboarding. Often, a small business lender will apply different decisioning frameworks to different types of businesses, and they may avoid certain industries entirely. For example, it could make sense to apply a particular cash flow analysis and underwriting framework to service businesses and a very different one to retail businesses with inventory. The most common taxonomy for business classification is NAICS. Despite the importance of this attribute, there is no perfect source of truth, and lenders often struggle to find a solution to obtain it with accuracy and coverage. While various data vendors attempt to fill this gap, the availability of high quality LLMs has given many people hope of a better solution. In this blog post from last April, Coris announced the availability of a GPT-based solution to determine the industry code of a business with high accuracy. It works very similarly to how a human might approach the task: find the merchant’s website, read the site, analyze the site’s content alongside the NAICS taxonomy guide to determine which category is the best fit. Classification of a business into a defined taxonomy of categories based on publicly available text seems like a great use case for LLM-based approaches. This matches what I’ve heard from several clients and friends in the space, who have adopted this method and achieved high accuracy at lower cost. Of course, an MCC or NAICS code is a fairly blunt categorization, so this is the tip of the iceberg. It will be interesting to consider other ways that language models could be used to turn large amounts of unstructured text into finite, structured data that could be used to better understand the financial dynamics of a business.
How Ally Financial Is Delivering Business Value From Generative AI - Forbes
Over the past decade, the century-old General Motors Acceptance Corporation (aka GMAC) has transformed itself into Ally, the nation’s largest all-digital bank, offering a wide variety of consumer financial products, in addition to auto loans. Ally has received accolades for its high-interest savings accounts as well as emphasis on digital customer service. It’s no surprise that Ally would make efforts towards the use of generative AI to enhance customer service and operational efficiency. This piece from Forbes discusses several of the bank’s early experiments, led by CIO Sathish Muthukrishnan. Ally has built an in-house platform to serve as a ‘secure bridge’ between external LLMs and Ally’s customer data, enabling employees to test use cases while retaining compliance with data security controls. Marketing is among the first production use cases for the technology, where generative AI is being used in campaign creative development, search engine optimization, and content drafting. In addition, Ally has run a pilot program for generative AI in customer service, using the technology for live transcription and summarization of phone calls, which reportedly reduced the time needed to handle customer inquiries. Ally’s Muthukrishnan notes that they have developed an AI Playbook and AI Governance Group, also integrating ‘Human in the Loop’ (aka HITL) processes to ensure accuracy.
The corporate bond market is nearly $11 trillion in the U.S. alone. Despite the size and importance of this industry, most corporate debt transactions are based on incredibly complex, lengthy, and arcane agreements. This makes deals difficult to understand and hard to benchmark, creating barriers to information access, risk to bond investors, and ultimately, a less efficient market. Noetica develops its own AI to understand and benchmark corporate debt transactions, apparently without the use of external LLMs. I’m curious whether this means self-hosting a fork of an open-source LLM or actually developing its own models from scratch. Either way, training and tuning an LLM-based learning engine to understand the intricacies of complex financial and legal structures seems like a productive application of the technology and something that could save a lot of time, money, and risk. Of course, in any field with massive precedent and entrenched ways of doing business, accomplishing innovation requires not only technological but also cultural adaptation. It will be interesting to see how Noetica is able to grow using this round of funding and what it will ultimately take to drive meaningful change in credit markets.
Perhaps at some point, companies that don’t use AI will be the exception and will inspire a ‘retronym’ to distinguish themselves (like for example an ‘acoustic guitar’ or ‘conventional produce’). I wrote about this concept back in 2016 in reference to online/marketplace lending.
Hey David, really useful blog for FinTech AI insight.