Small Language Models as Effective Guides for Large Language Models in Chinese Relation Extraction
Abstract: Recently, LLMs have been successful in relational extraction (RE) tasks, especially in the few-shot learning. An important problem in the field of RE is long-tailed data, while not much attention is paid to this problem using LLM approaches. Therefore, in this paper, we propose SLCoLM, a model collaboration framework, to mitigate the data long-tail problem. In our framework, we use the ``\textit{Training-Guide-Predict}'' strategy to combine the strengths of small pre-trained LLMs (SLMs) and LLMs, where a task-specific SLM framework acts as a guider, transfers task knowledge to the LLM and guides the LLM in performing RE tasks. Our experiments on an ancient Chinese RE dataset rich in relation types show that the approach facilitates RE of long-tail relation types.
- Scale: Synergized collaboration of asymmetric language translation engines. (arXiv:2309.17061). ArXiv:2309.17061 [cs].
- Markus Eberts and Adrian Ulges. 2021. Span-based joint entity and relation extraction with transformer pre-training. ArXiv:1909.07755 [cs].
- How good are gpt models at machine translation? a comprehensive evaluation. (arXiv:2302.09210). ArXiv:2302.09210 [cs].
- Bad actor, good advisor: Exploring the role of large language models in fake news detection. (arXiv:2309.12247). ArXiv:2309.12247 [cs].
- Lora: Low-rank adaptation of large language models. (arXiv:2106.09685). ArXiv:2106.09685 [cs].
- Span-based joint entity and relation extraction with attention-based span-specific and contextual semantic representations. In Proceedings of the 28th International Conference on Computational Linguistics, page 88–99, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Contrastive decoding: Open-ended text generation as optimization.
- Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318–327.
- P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), page 61–68, Dublin, Ireland. Association for Computational Linguistics.
- Unlocking the potential of large language models for explainable recommendations. (arXiv:2312.15661). ArXiv:2312.15661 [cs].
- Augmented large language models with parametric knowledge guiding. (arXiv:2305.04757). ArXiv:2305.04757 [cs].
- Large language model is not a good few-shot information extractor, but a good reranker for hard samples! (arXiv:2303.08559). ArXiv:2303.08559 [cs].
- Onerel: Joint entity and relation extraction with one module in one step. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):11285–11293.
- Joint entity and relation extraction with set prediction networks. arXiv preprint arXiv:2011.01675.
- A novel cascade binary tagging framework for relational triple extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, page 1476–1488, Online. Association for Computational Linguistics.
- Visual chatgpt: Talking, drawing and editing with visual foundation models. (arXiv:2303.04671). ArXiv:2303.04671 [cs].
- Small models are valuable plug-ins for large language models. (arXiv:2305.08848). ArXiv:2305.08848 [cs].
- Supervised knowledge makes large language models better in-context learners. (arXiv:2312.15918). ArXiv:2312.15918 [cs].
- Automl-gpt: Automatic machine learning with gpt. (arXiv:2305.02499). ArXiv:2305.02499 [cs, stat].
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.