Benchmarking CPU vs. GPU Performance in Building Predictive LSTM Deep Learning Models
Michael Siek

Business Information Systems Program, Information Systems Department, Faculty of Computing and Media, Bina Nusantara University, Jakarta, Indonesia 11480

Abstract

Constructing deep recurrent neural network (DRNN) models using back-propagation through time learning algorithm often requires intensive computation. The long short-term memory (LSTM) network as a variation of DRNN has solved the vanishing problem by enforcing constant in the gradient-based algorithm to transfer the error to the internal states of neurons, but not for exploding issue. Such instability in error back-propagated along deep neural network in time and its regularization problem could entail more computational time. Despite these issues, the LSTM network with deep learning has shown much promising results for classification and regression tasks in various applications, which many industries introduce novel AI-based processor products allowing for parallel computation, like Nvidia GPU with CUDA programming. This research aimed at benchmarking CPU vs GPU performance in building LSTM deep learning model for prediction. Several experiments were conducted to optimize hyper parameters, number of epochs, network size, learning rate and others for providing accurate predictive models as a decision support system. The modelling results indicate that the utilization of GPU in LSTM deep learning considerably outperforms the one using CPU.

Keywords: computational intelligence, dynamic neural network, predictive model, parallel computing

Topic: Computer Science