Porous Lattice-based Transformer Encoder for Chinese NER

Abstract: Incorporating lattices into character-level Chinese named entity recognition is an effective method to exploit explicit word information. Recent works extend recurrent and convolutional neural networks to model lattice inputs. However, due to the DAG structure or the variable-sized potential word set for lattice inputs, these models prevent the convenient use of batched computation, resulting in serious inefficient. In this paper, we propose a porous lattice-based transformer encoder for Chinese named entity recognition, which is capable to better exploit the GPU parallelism and batch the computation owing to the mask mechanism in transformer. We first investigate the lattice-aware self-attention coupled with relative position representations to explore effective word information in the lattice structure. Besides, to strengthen the local dependencies among neighboring tokens, we propose a novel porous structure during self-attentional computation processing, in which every two non-neighboring tokens are connected through a shared pivot node. Experimental results on four datasets show that our model performs up to 9.47 times faster than state-of-the-art models, while is roughly on a par with its performance. The source code of this paper can be obtained from https://github.com/xxx/xxx.