How to choose between ChatGPT and Open Source LLMs in Finance

Step-by-step guide to choosing between ChatGPT and open source LLMs in finance for your competitive advantage

Generated by the author with ChatGPT

Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning - Albert Einstein

Many consulting companies present LLMs and GenAI products to CEOs, CFOs, COOs, and CTOs. While these products may seem appealing, companies should remember the distinction between using ChatGPT with non-privileged information in daily tasks versus proprietary company information fed to such a GenAI model. It is essential to be cautious before investing in these products and to consider if they are essential for the company's needs and in which circumstances.

This article explains the following concise but essential information to help you understand the risks, benefits, and obstacles that can arise when introducing such models (Proprietary or Open-Source), focusing on financial institutions, which are generally relevant to any other sector.

In the end, I should have been able to provide insights that will guide financial professionals and institutions in making informed decisions about which technology best suits their needs.

---

1. Introduction

As financial institutions increasingly recognize the potential of AI to reduce costs, automate processes, and improve productivity, companies are exploring new innovative possibilities to gain these benefits. ChatGPT and Open-Source LLMs play a crucial role in this game - but is it so easy to adopt? No Risks? Only Benefits? If you ask yourself those questions - stay tuned until the end!

Large Language Models (LLMs) are revolutionizing the finance industry, offering unprecedented capabilities in processing and analyzing vast amounts of text-based data.

In general, we need to distinguish between two types of LLM or GenAI models: proprietary models such as ChatGPT and Open-source ones such as LLama2.

ChatGPT, developed by OpenAI, stands out as a prominent example, known for its conversational AI and deep learning capabilities. It has been widely adopted for various tasks, including data analysis, market trend prediction, and customer service automation.

Conversely, open-source LLMs are gaining traction due to their flexibility and adaptability. Unlike proprietary models like ChatGPT, they offer a more customizable approach, allowing users to tailor the models to specific financial contexts.

Both types of LLMs have their unique strengths and weaknesses. ChatGPT offers quick and efficient data processing with its advanced algorithms and ease of use. However, it may lack the customizability, privacy, and security that specialized financial tasks require. On the other hand, open-source LLMs offer greater flexibility, security, and privacy (If deployed on-premise) but can be more complex to deploy and manage.

Understanding these nuances is vital for finance professionals navigating the AI landscape. The following chapters will delve deeper into the applications, benefits, and challenges of ChatGPT and open-source LLMs in finance.

2. Comparative Analysis

When choosing between ChatGPT and open-source LLMs for financial applications, several factors come into play. Cost is a significant consideration, but not the most important later to that more.

ChatGPT, with its usage-based pricing, is more economical for lower volumes of data processing. However, the cost can become prohibitive as the scale increases to millions of requests. In contrast, while free to use, open-source LLMs incur substantial cloud hosting and deployment costs, especially for high-volume processing.

Performance and customizability also differ markedly, still not the most important, but ChatGPT, being a generalized model, offers broad applicability and ease of use but may fall short in specialized financial tasks. Open-source models, however, can be fine-tuned for specific financial data and scenarios, providing more targeted insights.

The most essential and foremost question you need to ask yourself is:

“Do we want to be in the Cloud or not? Are you willing to put our data into the internet, or do we need to have it behind Chinese walls?”

This question is crucial, as it determines the course of the discussion.

Let's run through a small scenario and imagine what it would cost us to deploy everything on-site.

Disclaimer: I am not an expert in hardware ordering and estimation, but it should give direction as to how much it would need based on different research I’ve conducted.

Calcualtion Scenario:

To calculate the price for deploying a Large Language Model (LLM) with new servers and GPUs, we need to consider several key components:

1. Servers: High-end servers capable of handling LLM workloads can range from a few thousand to tens of thousands of dollars each.

2. GPUs: State-of-the-art GPUs for deep learning (like NVIDIA's latest series) can cost several thousand dollars each. For a robust LLM deployment, multiple GPUs may be required.

3. Storage: SSDs or other high-speed storage solutions are necessary for handling large datasets and model weights.

4. Cooling and Power: Efficient cooling systems and power supply to support continuous high-load operation.

5. Software and Licenses: Additional costs may be incurred depending on the software stack and operating systems.

6. Installation and Setup: Professional setup and configuration of the hardware and software.

For example, for Falcon-7B or Llama2 -7B, you would require some A10 GPUs with 40GB of VRAM.

Given these factors, a basic setup with one high-end server and a couple of top-tier GPUs could start around $100,000 upwards. However, more extensive setups with multiple servers and GPUs could easily exceed this number, potentially reaching several hundred thousand dollars. This estimate does not include ongoing costs like electricity, cooling, maintenance, and possible software licenses.

3. How to choose between ChatGPT and Open Source LLMs in Finance

To be able to choose between propriety LLM models such as ChatGPT or Open-source LLM finance you need to ask the following questions, visualised in the flowchart below.

Flowchart to choose between proprietary and open-source LLMs

Besides this flowchart an equally crucial aspect is:

Do we have the money to create a LLM infrastructure on-prem and the intellectual capacity to use it correctly? Is the answer “YES” → Go for it! Is the answer “NO” → Ask yourself if you cannot go for online models such as ChatGPT for some less privacy risk first use cases.

Another essential factor you should not forget, especially if you deploy on-premise, is the number of parallel users of your LLM application, as this depends on several factors:

1. Model Complexity: More complex models require more computational power per request.

2. Size of Requests: Longer text inputs and outputs consume more resources.

3. Server Specifications: The power of the CPUs, the number of GPUs, and the available RAM.

4. Network Infrastructure: Bandwidth and network setup can also limit the number of concurrent users.

A recap in the form of a table:

Various feature comparisons between ChatGPT and Open-source LLMs when deployed offline

4. Future Outlook and Conclusion

The future of LLMs in finance points towards greater integration and sophistication. Advancements in AI and machine learning algorithms will likely enhance the accuracy and efficiency of both ChatGPT and open-source models.

We expect more powerful models to appear on the market, which will be less calculation-intensive and resource-intensive but still very powerful. An excellent example is Zephyr-7b-beta. It outperforms huge models such as Llama-2-70B and is very close to GPT-3.5 Turbo. However, one of the downsides is that it is only suitable for the English language. I’ve tried it myself, it does understand other languages such as French, but the answers are only good in English.

Source: https://huggingface.co/HuggingFaceH4/zephyr-7b-beta

Choosing between ChatGPT and open-source LLMs in finance hinges on specific needs, technical capabilities, and the scale of the application. While ChatGPT offers ease and broad applicability, open-source models provide flexibility and customization. A thorough evaluation of these aspects is crucial in leveraging the right LLM for optimal financial outcomes.