Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, Chinese Academy of Sciences, Beijing, China 100190

---

Abstract—Recently, deep neural network (DNN) based acoustic modeling has been successfully applied to large vocabulary continuous speech recognition (LVCSR) tasks. A relative word error reduction around 20% can be achieved compared to a state-of-the-art discriminatively trained Gaussian Mixture Model (GMM). However, due to the huge number of parameters in the DNN, real-time decoding is a bottleneck for the DNN based speech recognition systems. In this paper, we adopt several techniques for the speed optimization of the DNN-based system. Specifically, we use singular value decomposition (SVD) to reduce the model parameters, use the SSE instruction sets for the parallel calculation in the data space, and quantize the model parameters reasonably to convert the floating-point arithmetic into fixed-point arithmetic. Besides, taking the characteristics of speech signal into account, we use a frameskipping method when evaluating the posterior probabilities. Finally, compared to the un-optimized baseline system, with negligible recognition performance loss, the decoding realtime factor of the optimized one is significantly reduced, from 6.1 to 0.31. And this response speed can basically meet the requirement of our real applications.

Index Terms—Large Vocabulary Continuous Speech Recognition, Acoustic Modeling, Deep Neural Network, SSE Instructions.

[PDF]

Cite: Yeming Xiao, Yujing Si, Ji Xu, Jielin Pan, Yonghong Yan, "Speeding up deep neural network based speech recognition systems," Journal of Software vol. 9, no. 10, pp. 2706-2712, 2014.