TWIG: Towards pre-hoc Hyperparameter Optimisation and Cross-Graph Generalisation via Simulated KGE Models.

Knowledge Graphs (KGs) have become ever-more important for modelling biomedical information, as their intrinsic graph structure matches the structure of many biological interaction networks. Together with KGs, Knowledge Graph Embeddings (KGEs) have shown immense potential to learn biological data and predict new, in-band facts about the data the KG describes. However, recent literature has suggested several major deficits to KGEs: that they have an extremely short ‘receptive field’ of data they use to make predictions and that their learning is guided by memorising graph structure, not learning latent semantics. Moreover, while several studies have suggested that graph structure and KGE model choice affect optimal hyperparameters, the exact relationship of hyperparameters to learning remains unknown and is instead solved using a computationally intensive hyperparameter search.In this paper we introduce TWIG (Topologically-Weighted Intelligence Generation), a novel, embedding-free paradigm for simulating the output of KGEs that uses a tiny fraction of the parameters. TWIG learns weights from inputs that consist of topological features of the graph data, with no coding for latent representations of entities or edges. Our experiments on the UMLS dataset show that a single TWIG neural network can predict the results of state-of-the-art ComplEx-N3 KGE model nearly exactly on across all hyperparameter configurations. To do this it uses a total of 2590 learnable parameters, but accurately predicts the results of 1215 different hyperparameter combinations with a combined cost of 29,322,000 parameters. Based on these results, we make two claims: 1) that KGEs do not learn latent semantics, but only latent representations of structural patterns; 2) that hyperparameter choice in KGEs is a deterministic function of the KGE model and graph structure. We further hypothesise that, as TWIG can simulate KGEs without embeddings, that node and edge embeddings are not needed to learn to accurately predict new facts in KGs. Finally, we formulate all of our findings under the umbrella of the “Structural Generalisation Hypothesis”, which suggests that “twiggy” embedding-free / data-structure-based learning methods can allow a single neural network to simulate KGE performance, and perhaps solve the Link Prediction task, across many KGs from diverse domains and with different semantics.