Fine-tuned language models for intelligent aggregation systems in the banking sector

Abstract

This paper presents the design and implementation of a practical news aggregation system for decision making and risk management in the banking sector. The system gathers financial news from multiple sources and mines texts to provide insight into banks’ professional activities such as policies, products, and financial performance. Multiple natural language processing (NLP) modules are integrated to address tasks such as topic classification and sentiment analysis. To identify the most suitable technologies for each component, we systematically evaluated a wide range of NLP techniques, including large language models (LLMs) and domain-specific pre-trained models. A Vietnamese financial corpus of 12,000 annotated samples was constructed to fine-tune models such as PhoBERT, ViT5, and BARTPho. Experimental results show that fine-tuned models significantly outperform general-purpose LLMs (e.g., LLaMA-3.1-8B, Vistral-7B) in both accuracy and computational efficiency. The fine-tuned models achieve a 7.15% accuracy improvement and reduce resource requirements. The study demonstrates a scalable and adaptable framework for building multi-source, text-based intelligent systems in the financial domain.