Deep Learning-based approaches for automatic detection of shell nouns and evaluation on WikiText-2

Abstract: In some areas, such as Cognitive Linguistics, researchers are still using traditional techniques based on manual rules and patterns. Since the definition of shell noun is rather subjective and there are many exceptions, this time-consuming work had to be done by hand in the past when Deep Learning techniques were not mature enough. With the increasing number of networked languages, these rules are becoming less useful. However, there is a better alternative now. With the development of Deep Learning, pre-trained LLMs have provided a good technical basis for Natural Language Processing. Automated processes based on Deep Learning approaches are more in line with modern needs. This paper collaborates across borders to propose two Neural Network models for the automatic detection of shell nouns and experiment on the WikiText-2 dataset. The proposed approaches not only allow the entire process to be automated, but the precision has reached 94% even on completely unseen articles, comparable to that of human annotators. This shows that the performance and generalization ability of the model is good enough to be used for research purposes. Many new nouns are found that fit the definition of shell noun very well. All discovered shell nouns as well as pre-trained models and code are available on GitHub.