Dialog generation model based on variational Bayesian knowledge retrieval method

For dialog generation models that introduce external knowledge, the key challenge lies in how to select the relevant knowledge. The existing common method is to directly use a retriever to fetch knowledge according to the prior distribution that is only conditioned on dialog history. In fact, the response also contains crucial information that is helpful for knowledge retrieval. If we can make use of it and conduct posterior retrieval of relevant knowledge according to both dialog history and response, the selected knowledge will be more relevant to the dialog. Therefore, this paper proposes a dialog generation model (DG-VBKR) based on variational Bayesian knowledge retrieval method, which consists of two modules: knowledge retrieval and dialog generation. During the training process, both a prior knowledge retriever and a posterior knowledge retriever are trained simultaneously, and the prior knowledge retriever is guided by the posterior knowledge retriever. Specifically, the KL Divergence is used to measure the difference between the retrieval results of the two retrievers and make the trained prior knowledge retriever achieve a similar effect to the posterior knowledge retriever. In addition, this paper constructs four versions of the proposed model by using GPT-2, BART, TransformerDecoder, and TransformerEnc-Dec in dialog generation module. The experimental results show that the DG-VBKR model achieves certain improvements on several metrics compared to the baseline models. Among them, the GPT-2 version model with the best performance has improved by 7.0% on the PPL metric compared with the model using only prior knowledge retrieval. This indicates that the DG-VBKR model can effectively select knowledge related to the dialog to generate high-quality responses.