Position-aware self-attention based neural sequence labeling

•This paper identifies the problem of modeling discrete context dependencies in sequence labeling tasks.•This paper develops a well-designed self-attentional context fusion network to provide complementary context information on the basis of Bi-LSTM.•This paper proposes a novel position-aware self-attention to incorporate three different positional factors for exploring the relative position information among token.•The proposed model achieves state-of-the-arts performance on part-of-speech (POS) tagging, named entity recognition (NER) and phrase chunking tasks.