Why Vertical LLM Agents are the new Billion$ SaaS Opportunities
How CaseText went from a normal startup to..a massive exit story in 2 months.
Imagine this:
You spend 10+ years building a company which finally hits $100mn valuation.
And then something happens and in 2 months, you get an exit of..$650 mn!
‘that something’ is the power of vertical LLMs and we are discussing the story of CaseText, which went all-in on the AI legal assistant (one of the first vertical AI agents to be deployed at scale, used by thousands of lawyers) and exited to Thomson Reuters for $650mn.
Here are the 5 most important takeaways from the CaseText founder (as shared with YC partners recently) on what it takes to building a vertical LLM business.
=========
Ad: Who’s hiring in tech?
Zeta is hiring Lead AI Implementation Engineer
OpenAI is hiring Research Scientist
=========
Note: Beyond obvious advice like domain knowledge, one of the big things that came out is the red-eyed focus teams need to have on..testing.
The Power of Early Adoption
Heller's team gained early access to GPT-4, allowing them to pivot their entire company focus within 48 hours. This first-mover advantage was crucial in developing Co-Counsel, an AI legal assistant that outperformed previous models.
The future of work is not about job displacement, but job enhancement. AI tools like Co-Counsel are making legal work more efficient and allowing professionals to focus on higher-level strategic thinking.
From Incremental to Revolutionary
Case Text's journey illustrates the shift from incremental improvements to revolutionary change in legal tech. While earlier versions of AI struggled with legal tasks, GPT-4 marked a turning point, demonstrating human-level performance on complex legal assessments.
The leap from GPT-3.5 to GPT-4 was staggering. GPT-3.5 scored in the 10th percentile on the bar exam, while GPT-4 outperformed 90% of test takers.
Building AI Products: Beyond GPT Wrappers
Heller emphasizes that creating successful AI products involves much more than simply wrapping an API around a language model. It requires:
1. Deep domain expertise
2. Custom integrations with industry-specific systems
3. Handling edge cases and data preprocessing
4. Developing sophisticated prompting strategies
5. Rigorous testing and quality assurance
Test-Driven Development for AI
I was never a big test-driven development fan in the regular software world, but it's 10x more important in the LLM world.
Heller advocates for a test-driven approach to AI development, particularly in high-stakes fields like law. This involves:
1. Creating a comprehensive test suite
2. Iteratively refining prompts based on test results
3. Aiming for near-perfect accuracy before deployment
In legal tech, even small errors can erode user trust. Heller's team focused on achieving 100% accuracy to meet the exacting standards of legal professionals.
The Importance of First Impressions
For AI tools in professional settings, the first user experience is critical. Heller notes that busy professionals may dismiss AI solutions after a single poor experience, emphasizing the need for polished, reliable products from day one.
Lawyers are particularly demanding users. They're accustomed to billing by the hour and are skeptical of technology that could disrupt their workflow. Winning their trust requires demonstrating clear value and impeccable accuracy.
Challenges and Opportunities with GPT-4 and Beyond:
While GPT-4 represented a quantum leap in capabilities, Heller sees even greater potential in models like GPT-4 Turbo (1106) and beyond. These models offer:
1. Improved precision and attention to detail
2. Enhanced ability to follow complex instructions
3. Potential for injecting domain-specific expertise into the thinking process
The future may involve prompting AI not just with what to do, but how to think about solving problems, drawing on the expertise of top professionals in each field.
The AI revolution, particularly in professional services, is just beginning. There's enormous potential for entrepreneurs who can effectively harness these technologies to solve real-world problems.