Web Forum Thread Summarization Based on Dynamic Topic Modeling

Because there is no effective document summarization method for Web forum threads currently, this paper proposes a Web forums thread summarization method based on a latent Dirichlet allocation (LDA) topic model. To handle the topic dependencies and the drifting problem, we consider the reply-relations among posts in topic modeling, and set the distribution of each topic as a dynamic process with the change of the thread discussion. We utilize the Gibbs EM algorithm to get parameters of the proposed dynamic topic model and determine the importance of each topic according to the sum of the topic weight over all sentences. Finally we calculate the scores of sentences based on probability distribution of topics and then generate the summarization of each thread. The experimental results on the two different forum data sets show that the new method outperforms several widely used summarization methods in terms of ROUGE metrics.