Bloomberg has developed BloombergGPT, the first large language model tailored specifically for the finance industry. Mark Dredze, an associate professor of computer science at Johns Hopkins University, sheds light on the project’s objectives and broader implications. While general-purpose language models like ChatGPT have showcased impressive language processing capabilities, domain-specific models are needed to understand the complexities and nuances of specific industries.
To address this, the Bloomberg team built a massive dataset combining financial-related text and general-purpose text, training the BloombergGPT model on this comprehensive dataset. The model demonstrated superior performance on financial tasks and remained competitive on general-purpose benchmarks, surpassing existing models of similar size.
Finance requires its own language model due to its unique terminology and intricate nature. Just like specialized domains such as medicine, finance possesses vocabulary that is not commonly found in general text. A finance-specific language model has the potential to enhance existing financial natural language processing tasks, including sentiment analysis, named entity recognition, news classification, and question answering. Additionally, it can unlock new possibilities, such as transforming natural language queries from financial professionals into valid Bloomberg Query Language (BQL) code. This would enable professionals to quickly and accurately retrieve specific data about different securities, facilitating seamless integration into data science and portfolio management tools.
Developing these models is a complex endeavor that requires meticulous attention to detail. Dredze’s team extensively studied papers from other research groups to gain insights and knowledge about language model construction. In an effort to contribute to the research community, they authored a detailed paper and released “training chronicles” to provide a comprehensive account of the model-building process. The collaboration between Bloomberg’s AI Engineering team and the ML Product and Research group, which includes Dredze as a visiting researcher, was crucial in making data and model decisions and conducting thorough evaluations. The project also showcases the value of a Johns Hopkins education, as graduates from the university continue to push the boundaries of scientific advancements even after their formal education is complete.
