AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
Our work focuses on modeling multi-relational data from knowledge bases, with the goal of providing an efficient tool to complete them by automatically adding new facts, without requiring extra knowledge
Translating Embeddings for Modeling Multi-relational Data.
NIPS, pp.2787-2795, (2013)
We consider the problem of embedding entities and relationships of multirelational data in low-dimensional vector spaces. Our objective is to propose a canonical model which is easy to train, contains a reduced number of parameters and can scale up to very large databases. Hence, we propose TransE, a method which models relationships by i...More
PPT (Upload PPT)
- Multi-relational data refers to directed graphs whose nodes correspond to entities and edges of the form (denoted (h, , t)), each of which indicates that there exists a relationship of name label between the entities head and tail.
- The authors' work focuses on modeling multi-relational data from KBs (Wordnet  and Freebase  in this paper), with the goal of providing an efficient tool to complete them by automatically adding new facts, without requiring extra knowledge.
- The notion of locality for a single relationship may be purely structural, such as the friend of the author's friend is the author's friend in
- Multi-relational data refers to directed graphs whose nodes correspond to entities and edges of the form (denoted (h, , t)), each of which indicates that there exists a relationship of name label between the entities head and tail
- Our work focuses on modeling multi-relational data from knowledge bases (Wordnet  and Freebase  in this paper), with the goal of providing an efficient tool to complete them by automatically adding new facts, without requiring extra knowledge
- We proposed a new approach to learn embeddings of knowledge bases, focusing on the minimal parametrization of the model to primarily represent hierarchical relationships
- We showed that it works very well compared to competing methods on two different knowledge bases, and is a highly scalable model, whereby we applied it to a very large-scale chunk of Freebase data
- Unstructured  RESCAL  SE  SME(LINEAR)  SME(BILINEAR)  LFM  TransE
- Unstructured  RESCAL  SE  SME(LINEAR)  SME(BILINEAR)  LFM  TransE.
- O O O O O O O ON FB15K 0.75 DATA SET WN ENTITIES RELATIONSHIPS TRAIN.
- VALID EX
- Results for
Unstructured, SE, SME, SME and TransE are presented in Figure 1.
- The performance of Unstructured is the best when no example of the unknown relationship is provided, because it does not use this information to predict.
- This performance does not improve while providing labeled examples.
- TransE is the fastest method to learn: with only 10 examples of a new relationship, the hits@10 is already 18% and it improves monotonically with the number of provided samples.
- The authors believe the simplicity of the TransE model makes it able to generalize well, without having to modify any of the already trained embeddings
- The authors proposed a new approach to learn embeddings of KBs, focusing on the minimal parametrization of the model to primarily represent hierarchical relationships.
- It remains unclear to them if all relationship types can be modeled adequately by the approach, by breaking down the evaluation into categories (1-to-1, 1-to-Many, .
- Combining KBs with text as in  is another important direction where the approach could prove useful.
- The authors recently fruitfully inserted TransE into a framework for relation extraction from text 
- Table1: Numbers of parameters and their values
- Table2: Statistics of the data sets used for FB15k (in millions). ne and nr are the nb. of en- in this paper and extracted from the two tities and relationships; k the embeddings dimension. knowledge bases, Wordnet and Freebase
- Table3: Link prediction results. Test performance of the different methods
- Table4: Detailed results by category of relationship. We compare Hits@10 (in %) on FB15k in the filtered evaluation setting for our model, TransE and baselines. (M. stands for MANY)
- Table5: Example predictions on the FB15k test set using TransE. Bold indicates the test triplet’s true tail and italics other true tails present in the training set
- Section 1 described a large body of work on embedding KBs. We detail here the links between our model and those of  (Structured Embeddings or SE) and .
- This work was carried out in the framework of the Labex MS2T (ANR-11-IDEX-0004-02), and funded by the French National Agency for Research (EVEREST-12-JS02-005-01)
- K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, 2008.
- A. Bordes, X. Glorot, J. Weston, and Y. Bengio. A semantic matching energy function for learning with multi-relational data. Machine Learning, 2013.
- A. Bordes, J. Weston, R. Collobert, and Y. Bengio. Learning structured embeddings of knowledge bases. In Proceedings of the 25th Annual Conference on Artificial Intelligence (AAAI), 2011.
- X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS)., 2010.
- R. A. Harshman and M. E. Lundy. Parafac: parallel factor analysis. Computational Statistics & Data Analysis, 18(1):39–72, Aug. 1994.
- R. Jenatton, N. Le Roux, A. Bordes, G. Obozinski, et al. A latent factor model for highly multi-relational data. In Advances in Neural Information Processing Systems (NIPS 25), 2012.
- C. Kemp, J. B. Tenenbaum, T. L. Griffiths, T. Yamada, and N. Ueda. Learning systems of concepts with an infinite relational model. In Proceedings of the 21st Annual Conference on Artificial Intelligence (AAAI), 2006.
- T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NIPS 26), 2013.
- G. Miller. WordNet: a Lexical Database for English. Communications of the ACM, 38(11):39– 41, 1995.
- K. Miller, T. Griffiths, and M. Jordan. Nonparametric latent feature models for link prediction. In Advances in Neural Information Processing Systems (NIPS 22), 2009.
- M. Nickel, V. Tresp, and H.-P. Kriegel. A three-way model for collective learning on multirelational data. In Proceedings of the 28th International Conference on Machine Learning (ICML), 2011.
- M. Nickel, V. Tresp, and H.-P. Kriegel. Factorizing YAGO: scalable machine learning for linked data. In Proceedings of the 21st international conference on World Wide Web (WWW), 2012.
- A. P. Singh and G. J. Gordon. Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2008.
- R. Socher, D. Chen, C. D. Manning, and A. Y. Ng. Learning new facts from knowledge bases with neural tensor networks and semantic word vectors. In Advances in Neural Information Processing Systems (NIPS 26), 2013.
- I. Sutskever, R. Salakhutdinov, and J. Tenenbaum. Modelling relational data using bayesian clustered tensor factorization. In Advances in Neural Information Processing Systems (NIPS 22), 2009.
- J. Weston, A. Bordes, O. Yakhnenko, and N. Usunier. Connecting language and knowledge bases with embedding models for relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2013.
- J. Zhu. Max-margin nonparametric latent feature models for link prediction. In Proceedings of the 29th International Conference on Machine Learning (ICML), 2012.