ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2020, Vol. 57 ›› Issue (1): 136-144.doi: 10.7544/issn1000-1239.2020.20190204

Previous Articles     Next Articles

Mixture of Variational Autoencoder

Chen Yarui, Jiang Shuoran, Yang Jucheng, Zhao Tingting, Zhang Chuanlei   

  1. (College of Computer Science and Information Engineering, Tianjin University of Science & Technology, Tianjin 300457)
  • Online:2020-01-01
  • Supported by: 
    This work was supported by the National Natural Science Foundation of China (61402332, 61402331,11803022), the Natural Science Foundation of Tianjin City (17JCQNJC00400, 18JCQNJC69800), the Science and Technology Development Foundation of Higher Education Institutions of Tianjin (2017KJ034, 2017KJ035, 2018KJ106), and the Foundation for Young Teachers of Tianjin University of Science and Technology (2017LG10).

Abstract: Variational autoencoder (VAE) is a generated model with continuous latent variables, where the objective function is constructed by the variational approximation, and both the generated part and the inference part are based on neural networks. The traditional variational autoencoder assumes that the multi-dimensional latent variables of inference model are independent, which simplifies the inference process, but makes poor lower bound of the objective function and also limits the representation ability of the latent space. In this paper, we propose a mixture of variational autoencoder (MVAE), which generates the data through the variational autoencoder components. For this model, we first take the continuous latent vector with a Gaussian prior as the hidden layer, and take the discrete latent vector with a polynomial prior as the indicator vector of the component. Then we use the reparameterization trick and stick-breaking parameterization to rewrite the variational optimization objective, and train the model parameters through stochastic gradient descent. The mixture of variational autoencoder improves the inference accuracy using the complex model structure, expands the representation space of latent vector by the mixture of components, and effectively solves the corresponding optimization problems through the reparameterization and the stick-breaking parameterization technologies. Finally, we design the comparison experiments on datasets to demonstrate the higher inference accuracy and stronger representation ability of the latent variables.

Key words: variational autoencoders, mixture models, variational inference, stick-breaking parame-terization, generative model

CLC Number: