Pattern Filtering Attention For Distant Supervised Relation Extraction Via Online Clustering

Distant supervised relation extraction has been widely used to extract relational facts in large-scale corpus but inevitably suffers from the wrong label problem. Many methods use attention mechanisms to address this issue. However, the attention weights in these models are not discriminative and precise enough to fully filter out noise. In this paper, we propose a novel Pattern Filtering Attention (PFA), which can filter noise effectively. Firstly, we adopt an online clustering algorithm on the instances labeled with the same relation to extract potential semantic centers (positive patterns) of each relation, and these patterns have less noise statistically. Then, we build a sentence-level attention based on the similarities of instances and positive patterns. Due to the large differences between these similarities, our model can assign more discriminative weights to instances to reduce the influence of noisy data. Experimental results on the New York Times (NYT) dataset show that our model can effectively improve the performance of relation extraction compared with state-of-the-art methods.