The Challenges of large language models (LLMs), such as OpenAI’s GPT-4-Turbo, into sectors like finance has been met with high expectations. However, recent research by startup Patronus AI sheds light on the significant challenges these models face when analyzing Securities and Exchange Commission (SEC) filings. Despite advancements, even the best-performing models struggle to provide accurate and reliable answers based on SEC filings
raising concerns about their applicability in regulated industries.
The Test: Unveiling the Limitations
Patronus AI conducted an extensive test using over 10,000 questions and answers derived from SEC filings
creating a dataset named FinanceBench. The test included four language models: OpenAI’s GPT-4 and GPT-4-Turbo, Anthropic’s Claude2, and Meta’s Llama 2. The results were revealing and indicated the inherent challenges faced by LLMs in the finance sector.
GPT-4-Turbo’s Performance
Even with access to nearly an entire filing and specific source text, GPT-4-Turbo, considered a top-performing model, achieved only a 79% accuracy rate. In a “closed book” test without access to SEC documents
its accuracy plummeted to 12%, emphasizing the difficulty in extracting precise information autonomously.
Hallucinations and Refusals
One striking issue was the models’ tendency to “hallucinate” information not present in SEC filings, leading to inaccuracies. Additionally, the models frequently refused to answer questions, highlighting a significant gap in their reliability. Patronus AI co-founder Anand Kannappan emphasized the unacceptability of such performance rates, especially in automated and production-ready scenarios.
Non-Determinism and Rigorous Testing
A crucial aspect contributing to the challenges is the non-deterministic nature of LLMs—they don’t guarantee the same output for the same input. This unpredictability necessitates rigorous testing to ensure correct operation, prevent off-topic responses, and deliver reliable results. Companies integrating LLMs into their processes must acknowledge this and invest in comprehensive testing methodologies.
The Potential for Improvement
While the current results may indicate significant shortcomings, Patronus AI co-founders Rebecca Qian and Anand Kannappan believe in the potential of language models to revolutionize the finance industry. Acknowledging the need for continuous improvement, they express optimism about the long-term automation possibilities. However, they caution that, for now, having a human in the loop is essential to guide workflows and ensure accuracy.
Conclusion: Navigating the Complex Landscape
The challenges faced by LLMs in analyzing SEC filings underscore the complexities of applying cutting-edge technology in regulated industries. As companies strive to leverage AI for customer service, research, and analysis, a cautious approach is crucial. The finance sector, in particular, requires robust testing mechanisms and a commitment to addressing the limitations of existing models. While the road ahead may be challenging
the potential benefits of refined language models in finance remain a promising avenue for exploration.
FAQs
- Why do language models frequently refuse to answer questions based on SEC filings?Language models’ refusal to answer questions may stem from the non-deterministic nature of their output. Rigorous testing is necessary to understand and mitigate such behavior.
- How do hallucinations impact the accuracy of language models in finance?Hallucinations
where models generate incorrect information not present in SEC filings, significantly impact accuracy. This emphasizes the need for models to produce reliable and factual responses. - What role does non-determinism play in the challenges faced by language models?Non-determinism contributes to the unpredictability of language models. Companies incorporating these models must conduct thorough testing to ensure consistent and correct performance.
- Can language models be relied upon for autonomous financial analysis?The current limitations highlight the necessity of having a human in the loop to guide workflows and ensure accuracy in financial analysis. Continuous improvement is crucial for future autonomous applications.
- How can companies navigate the complexities of integrating language models into regulated industries like finance?Companies must invest in rigorous testing methodologies
acknowledge the non-deterministic nature of language models, and collaborate with human experts to navigate the complexities of regulated industries.