Evolution without Large Models: Training Language Model with Task Principles

Abstract: A common training approach for LLMs involves using a large-scale LLM to expand a human-provided dataset, which is subsequently used for model training.This method significantly reduces training costs by eliminating the need for extensive human data annotation. However, it still faces challenges such as high carbon emissions during data augmentation and the risk of data leakage when we use closed-source LLMs. To address these issues, we propose a self-evolution method for LLMs. First, we introduce the Multi-level Principle Generation, which enables a large-scale model to summarize task-completion principles based on a small amount of task data. Then, we propose the Principle-based Instance Generation, in which a smaller-scale LLM uses these task principles to generate a large amount of data. This data is then used for model training. Experimental results show that our proposed method significantly improves model performance compared to directly using a smaller-scale LLM to generate data. Additionally, since we only use the large-scale LLM to generate the task-completion principles, the carbon emissions associated with training the model are greatly reduced.