ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2017, Vol. 54 ›› Issue (5): 1036-1044.doi: 10.7544/issn1000-1239.2017.20160031

• 人工智能 • 上一篇    下一篇



  1. (解放军信息工程大学信息系统工程学院 郑州 450002) (
  • 出版日期: 2017-05-01
  • 基金资助: 

Joint Acoustic Modeling of Multi-Features Based on Deep Neural Networks

Fan Zhengguang, Qu Dan, Yan Honggang, Zhang Wenlin   

  1. (Institute of Information System Engineering, PLA Information Engineering University, Zhengzhou 450002)
  • Online: 2017-05-01

摘要: 针对不同声学特征之间的信息互补性以及声学建模中各任务间的关联性,提出了一种多特征关联的深层神经网络声学建模方法,该方法首先借鉴深层神经网络(deep neural network, DNN)多模态以及多任务学习思想,通过共享DNN部分隐含层为不同特征声学模型间建立关联,从而挖掘不同学习任务间隐含的共同解释性因素,实现知识迁移以及性能的相互促进;其次利用低秩矩阵分解方法减少模型估计参数的数量,加快模型训练速度,并对不同特征的识别结果采用ROVER(recognizer output voting error reduction)融合算法进行融合,进一步提高系统识别性能.基于TIMIT的连续语音识别实验表明,采用关联声学建模方法,不同特征的识别性能均要优于独立建模时的识别性能.在音素错误率(phone error rates, PER)指标上,关联声学建模下的ROVER融合结果要比独立建模下的ROVER融合结果相对降低约4.6%.

关键词: 语音识别, 深层神经网络, 声学模型, 低秩矩阵分解, 融合

Abstract: In view of the complementary information and the relevance when training acoustic modes of different acoustic features, a joint acoustic modeling method of multi-features based on deep neural networks is proposed. In this method, similar to DNN multimodal and multitask learning, part of the DNN hidden layers are shared to make the association among the DNN acoustic models built with different features. Through training the acoustic models together, the common hidden explanatory factors are exploited among different learning tasks. Such exploitation allows the possibility of knowledge transferring across different learning tasks. Moreover, the number of the model parameters is decreased by using the low-rank matrix factorization method to reduce the training time. Lastly, the recognition results from different acoustic features are combined by using recognizer output voting error reduction (ROVER) algorithm to further improve the performance. Experimental results of continuous speech recognition on TIMIT database show that the joint acoustic modeling method performs better than modeling independently with different features. In terms of phone error rates (PER), the result combined by ROVER based on the joint acoustic models yields a relative gain of 4.6% over the result based on the independent acoustic models.

Key words: speech recognition, deep neural network (DNN), acoustic models, low-rank matrix factorization, fusion