2000 character limit reached
SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification
Published 27 Nov 2023 in cs.LG, cs.AI, and cs.CL | (2311.15983v2)
Abstract: Among the many tasks that LLMs have revolutionized is text classification. Current text classification paradigms, however, rely solely on the output of the final layer in the LLM, with the rich information contained in internal neurons largely untapped. In this study, we present SPIN: a model-agnostic framework that sparsifies and integrates internal neurons of intermediate layers of LLMs for text classification. Specifically, SPIN sparsifies internal neurons by linear probing-based salient neuron selection layer by layer, avoiding noise from unrelated neurons and ensuring efficiency. The cross-layer salient neurons are then integrated to serve as multi-layered features for the classification head. Extensive experimental results show our proposed SPIN significantly improves text classification accuracy, efficiency, and interpretability.
- Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016.
- Identifying and controlling important neurons in neural machine translation. arXiv preprint arXiv:1811.01157, 2018.
- Language models can explain neurons in language models. URL https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html. (Date accessed: 14.05. 2023), 2023.
- Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
- Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827, 2022.
- Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696, 2021.
- What is one grain of sand in the desert? analyzing individual neurons in deep nlp models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6309–6317, 2019.
- Discovering latent concepts learned in BERT. arXiv preprint arXiv:2205.07237, 2022.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
- Analyzing individual neurons in pre-trained language models. arXiv preprint arXiv:2010.02695, 2020.
- Linguistic correlation analysis: Discovering salient neurons in deepnlp models. arXiv preprint arXiv:2206.13288, 2022.
- Softmax linear units. Transformer Circuits Thread, 2022.
- Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913, 2020.
- Language Models Represent Space and Time. arXiv preprint arXiv:2310.02207, 2023.
- Finding Neurons in a Haystack: Case Studies with Sparse Probing. arXiv preprint arXiv:2305.01610, 2023.
- An introduction to variable and feature selection. Journal of machine learning research, 3(Mar):1157–1182, 2003.
- The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009.
- Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146, 2018.
- Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
- SemEval-2023 Task 10: Explainable Detection of Online Sexism. arXiv preprint arXiv:2303.04222, 2023.
- Distributed representations of sentences and documents. In International conference on machine learning, pages 1188–1196. PMLR, 2014.
- Optimal brain damage. Advances in neural information processing systems, 2, 1989.
- Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35, 2023.
- Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786, 2021.
- Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142–150, 2011.
- Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
- Andrew Y Ng. Feature selection, L 1 vs. L 2 regularization, and rotational invariance. In Proceedings of the twenty-first international conference on Machine learning, page 78, 2004.
- Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444, 2017.
- Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.
- Analyzing encoded concepts in transformer language models. arXiv preprint arXiv:2206.13289, 2022.
- Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615, 2022.
- Finding experts in transformer models. arXiv preprint arXiv:2005.07647, 2020.
- How to fine-tune bert for text classification? In Chinese Computational Linguistics: 18th China National Conference, CCL 2019, Kunming, China, October 18–20, 2019, Proceedings 18, pages 194–206. Springer, 2019.
- Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267–288, 1996.
- Attention is all you need. Advances in neural information processing systems, 30, 2017.
- Neurons in large language models: Dead, n-gram, positional. arXiv preprint arXiv:2309.04827, 2023.
- GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018.
- Finding skill neurons in pre-trained transformer-based language models. arXiv preprint arXiv:2211.07349, 2022.
- Improving bert-based text classification with auxiliary sentence and domain knowledge. IEEE Access, 7:176600–176612, 2019.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.