2000 character limit reached
Enhancing Language Models for Financial Relation Extraction with Named Entities and Part-of-Speech
Published 2 May 2024 in cs.CL, cs.IR, and cs.LG | (2405.06665v1)
Abstract: The Financial Relation Extraction (FinRE) task involves identifying the entities and their relation, given a piece of financial statement/text. To solve this FinRE problem, we propose a simple but effective strategy that improves the performance of pre-trained LLMs by augmenting them with Named Entity Recognition (NER) and Part-Of-Speech (POS), as well as different approaches to combine these information. Experiments on a financial relations dataset show promising results and highlights the benefits of incorporating NER and POS in existing models. Our dataset and codes are available at https://github.com/kwanhui/FinRelExtract.
- Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8440–8451, 2020.
- BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186, 2019.
- A survey on personalized itinerary recommendation: From optimisation to deep learning. Applied Soft Computing, pp. 111200, 2024.
- Kpi-bert: A joint named entity recognition and relation extraction model for financial reports. In Proceedings of the 26th International Conference on Pattern Recognition (ICPR), pp. 606–612. IEEE, 2022.
- A french corpus and annotation schema for named entity recognition and relation extraction of financial news. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 2293–2299, 2020.
- Spanbert: Improving pre-training by representing and predicting spans. Transactions of the association for computational linguistics, 8:64–77, 2020.
- Refind: Relation extraction financial dataset. arXiv preprint arXiv:2305.18322, 2023.
- Albert: A lite bert for self-supervised learning of language representations. In Proceedings of the International Conference on Learning Representations, 2019.
- A transformer-based framework for poi-level social post geolocation. In European Conference on Information Retrieval, pp. 588–604. Springer, 2023.
- Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.
- Deep neural approaches to relation triplets extraction: A comprehensive survey. Cognitive Computation, 13:1215–1232, 2021.
- A simple approach to financial relation classification with pre-trained language models. In Proceedings of the 4th Workshop on Knowledge Discovery from Unstructured Data in Financial Services (KDF), 2023.
- Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.
- Finred: A dataset for relation extraction in financial domain. In Companion Proceedings of the Web Conference 2022, pp. 595–597, 2022.
- imetre: Incorporating markers of entity types for relation extraction. arXiv preprint arXiv:2307.00132, 2023.
- Deep neural network-based relation extraction: an overview. Neural Computing and Applications, pp. 1–21, 2022.
- Enriching pre-trained language model with entity information for relation classification. In Proceedings of the 28th ACM international conference on information and knowledge management, pp. 2361–2364, 2019.
- Finbert: A pretrained language model for financial communications. arXiv preprint arXiv:2006.08097, 2020.
- Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32, 2019.
- Relation classification via recurrent neural network. arXiv preprint arXiv:1508.01006, 2015.
- Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 35–45, 2017.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.