Topic modeling of Chinese language beyond a bag-of-words.

HighlightsTopic modeling on Chinese words and characters are investigated experimentally.A model CWTM is proposed by embedding character-word relation into topic modeling.Asymmetric Dirichlet prior over topic-word distribution leads to better results.CWTM model obtain better performance on classification and is more robust. The topic model is one of best known hierarchical Bayesian models for language modeling and document analysis. It has achieved a great success in text classification, in which a text is represented as a big of its words, disregarding grammar and even word order, that is referred to as the bag-of-words assumption. In this paper, we investigate topic modeling of the Chinese language, which has different morphology from alphabetical western languages like English. The Chinese characters, but not the Chinese words, are the basic structural units in Chinese. In previous empirical studies, it shows that the character-based topic model performs better than the word-based topic model. In this research, we propose the character-word topic model (CWTM) to consider the character-word relation in topic modeling. Two types of experiments are designed to test the performance of the new proposed model: topic extraction and text classification. By empirical studies, we demonstrate the superiority of the new proposed model comparing to both word and character based topic models.