Time Series Attention Based Transformer Neural Turing Machines for Diachronic Graph Embedding in Cyber Threat Intelligence

The cyber threats are often found to threaten individuals, organizations and countries at different levels and evolve continuously over time. Cyber Threat Intelligence (CTI) is an effective approach to solve cyber security problems. However, existing processes are considered inherent responses to known threats. CTI experts recommend proactively checking for emerging threats in existing knowledge. In addition, most researches focus on static snapshots of the CTI knowledge graph, while ignoring the temporal dynamics. To this end, we create a novel framework TSA-TNTM (Time Series Attention based Transformer Neural Turing Machines) for diachronic graph embedding framework, which uses time series self-attention mechanism to capture the non-linearly evolving entity representations over time. We demonstrate significantly improved performance over various approaches. A series of benchmark experiments illustrate that TSA-TNTM could generate higher quality than the state-of-the-art word embedding models in tasks pertaining to semantic analogy, clustering, threat classification and proactively identify emerging threats in CTI fields.