Complex neural networks will take much time for training; we can achieve better accuracy with simpler models by tuning hyper-parameters of the model. Hyper parameter tuning is required for neural networks to improve the accuracy and to reduce the training time of neural networks. In this paper we used simple CNN model with four convolution layers, two pooling layers and two fully connected layers with hyper parameter tuning, batch normalization, learning rate decay, and normalization techniques to recognize hand written digit recognition. This model is giving 99.54% on test set.

Convolution Neural Networks, CNN, Deep Learning, Parameter Tuning, Batch Normalization

[1] James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13 (February 2012), 281-305.
[2] Y. Hou and H. Zhao, "Handwritten digit recognition based on depth neural network," 2017 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), Okinawa, 2017, pp. 35-38
[3] U. Meier, D. C. Ciresan, L. M. Gambardella and J. Schmidhuber, "Better Digit Recognition with a Committee of Simple Neural Nets," 2011 International Conference on Document Analysis and Recognition, Beijing, 2011, pp. 1250-1254.
[4] L. Deng, "The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]," in IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141-142, Nov. 2012.
[5] C. Laurent, G. Pereyra, P. Brakel, Y. Zhang and Y. Bengio, "Batch normalized recurrent neural networks," 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, 2016, pp. 2657-2661.
[6] Y. Xie, H. Jin and E. C. C. Tsang, "Improving the lenet with batch normalization and online hard example mining for digits recognition," 2017 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), Ningbo, 2017, pp. 149-153.
[7] D. Ito, T. Okamoto and S. Koakutsu, "A learning algorithm with a gradient normalization and a learning rate adaptation for the mini-batch type learning," 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), Kanazawa, 2017, pp. 811-816.
[8] L. Lv, X. Cai, Y. Zeng and C. Chen, "Facial feature extraction method based on fast and effective convolutional neural network," 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, 2017, pp. 2011-2015.
[9] L. Chen, S. Wang, W. Fan, J. Sun and S. Naoi, "Cascading Training for Relaxation CNN on Handwritten Character Recognition," 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), Shenzhen, 2016, pp. 162-167.
[10] Tai-cong Chen, Da-jian Han, F. T. K. Au and L. G. Tham, "Acceleration of Levenberg-Marquardt training of neural networks with variable decay rate," Proceedings of the International Joint Conference on Neural Networks, 2003., Portland, OR, 2003, pp. 1873-1878 vol.3.
[11] T. Tanprasert and T. Kripruksawan, "An approach to control aging rate of neural networks under adaptation to gradually changing context," Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP `02., Singapore, 2002, pp. 174-178 vol.1.
[12] K. T. Islam, G. Mujtaba, R. G. Raj and H. F. Nweke, "Handwritten digits recognition with artificial neural network," 2017 International Conference on Engineering Technology and Technopreneurship (ICE2T), Kuala Lumpur, 2017, pp. 1-4.