Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor

Abstract: Instruction tuning enables pretrained LLMs to perform new tasks from inference-time natural language descriptions. These approaches rely on vast amounts of human supervision in the form of crowdsourced datasets or user interactions. In this work, we introduce Unnatural Instructions: a large dataset of creative and diverse instructions, collected with virtually no human labor. We collect 64,000 examples by prompting a LLM with three seed examples of instructions and eliciting a fourth. This set is then expanded by prompting the model to rephrase each instruction, creating a total of approximately 240,000 examples of instructions, inputs, and outputs. Experiments show that despite containing a fair amount of noise, training on Unnatural Instructions rivals the effectiveness of training on open-source manually-curated datasets, surpassing the performance of models such as T0++ and Tk-Instruct across various benchmarks. These results demonstrate the potential of model-generated data as a cost-effective alternative to crowdsourcing for dataset expansion and diversification.

• The paper demonstrates that automated instruction tuning with 240k generated examples can rival human-curated datasets in performance.
• The two-step method uses a pretrained model with seed examples and paraphrasing to create diverse, scalable, and cost-effective training data.
• Experiments with T5-11B reveal significant sample efficiency, underscoring the potential to democratize high-quality data generation in NLP.

An Overview of "Unnatural Instructions: Tuning LLMs with (Almost) No Human Labor"

The paper "Unnatural Instructions: Tuning LLMs with (Almost) No Human Labor" presents a novel approach to instruction tuning of pretrained LLMs, bypassing the conventional reliance on extensive human-annotated data. This study introduces a pipeline for generating a comprehensive dataset of instruction-input-output triplets, called Unnatural Instructions, which consists of approximately 240,000 examples created with minimal human involvement. The methodology, experimental results, and implications for future research stand at the core of this work.

The authors devised a two-step, fully automatic process for data generation using a pretrained LLM as the primary tool. The first phase involves generating a core dataset by prompting the LLM with a few carefully crafted seed examples, resulting in 64,000 initial examples that include diverse tasks and formats. The process is enhanced by prompting the model to generate additional paraphrases of the instructions, significantly expanding the dataset. This innovation directly addresses several limitations of manual data collection, including cost, time, and creativity constraints associated with crowdsourcing.

A key strength of the Unnatural Instructions dataset is its demonstrated utility for instruction-based generalization. The authors showcased its effectiveness through comprehensive experiments using a LLM, T5-11B, and benchmarked its performance against several state-of-the-art models fine-tuned with human-created datasets, such as T0++ and Tk-Instruct. Notably, models trained with the Unnatural Instructions dataset achieved competitive performance—surpassing some manually curated datasets—on a range of evaluation tasks including Super-Natural Instructions, BIG-bench Hard, and LMentry benchmarks. This suggests that automatically generated data can not only rival but potentially exceed the quality of human-generated data.

One particularly noteworthy aspect is the sample efficiency achieved with Unnatural Instructions. The study highlights the dataset's robustness and scalability by using cost-effective and diverse model-generated training data. These advantages are emphasized by comparisons of performance scaling with dataset size and costs, showing significant improvements per cost unit in real-world scenarios.

The implications of this work are considerable, both practically and theoretically. Practically, Unnatural Instructions provides a cost-effective alternative to conventional dataset creation, potentially democratizing access to high-quality training data across institutions with varying resources. Theoretically, it opens up new investigations into the creative capabilities of LLMs, particularly their ability to generate diverse and linguistically rich instructions without direct human input. Furthermore, the findings suggest fertile ground for hybrid approaches that utilize model-driven creativity combined with selective human oversight to refine and validate outputs.

In conclusion, "Unnatural Instructions" offers compelling evidence for the effectiveness of automated data generation in instruction-tuning tasks, presenting a scalable and resource-efficient alternative to human-produced datasets. It paves the way for future research to enhance and leverage LLM-based data generation, thus impacting the development of increasingly sophisticated AI systems. Future directions could explore further refining the generated data quality, integrating additional LLMs, and expanding the range of NLP tasks addressed by such datasets.