LLMR: Knowledge Distillation with a Large Language Model-Induced Reward

Abstract: LLMs have become increasingly popular and demonstrated remarkable performance in various NLP tasks. However, these models are typically computationally expensive and difficult to be deployed in resource-constrained environments. In this paper, we propose LLMR, a novel knowledge distillation (KD) method based on a reward function induced from LLMs. We conducted experiments on multiple datasets in the dialogue generation and summarization tasks. Empirical results demonstrate that our LLMR approach consistently outperforms traditional KD methods in different tasks and datasets.