In the field of Artificial Intelligence (AI), Deep Learning is a method falls in the wider family of Machine Learning algorithms that works on the principle of learning. Deep learning models basically works without human intervention and they are equivalent, and sometimes even, superior than humans. With the rise of emerging technology, deep learning draws an attention by many researchers and it is widely used in several areas including image, sound and text analysis. The paper discussed deep learning background, types of deep learning architectures and applications from different domains where researchers used deep learning models successfully.

Deep Learning, Convolutional Neural Network, Deep Belief Network, Recurrent Neural Network

[1] Deng, Li. Three classes of deep learning architectures and their applications: a tutorial survey, APSIPA transactions on signal and information processing, 2012
[2] Bengio, Y., Learning deep architectures for AI. Foundations and trends in Machine Learning 2, 1-127, 2009
[3] What Is Deep Learning?, retrieved from https://in.mathworks.com/discovery/deep-learning.html on May 20, 2018
[4] Convolutional Neural Networks Tutorial in TensorFlow, retrieved from http://adventuresinmachinelearning.com/convolutional-neural-networks-tutorial-tensorflow/ on May 20, 2018
[5] Min, Seonwoo & Lee, Byunghan & Yoon, Sungroh, Deep Learning in Bioinformatics. Briefings in Bioinformatics, 2016, 18. 10.1093/bib/bbw068
[6] Zhang W, et al., Deep convolutional neural networks for multi-modality isointense infant brain image segmentation, Neuroimage, 108, 214–224, 2015, doi: 10.1016/j.neuroimage.2014.12.061
[7] Moeskops P, et al., Automatic segmentation of MR brain images with a convolutional neural network, IEEE Trans. Med. Imaging, 35(5), 1252–1261, 2016, doi: 10.1109/TMI.2016.2548501
[8] Nie D, Dong N, Li W, Yaozong G, Dinggang S, Fully convolutional networks for multi-modality isointense infant brain image segmentation, in 2016 I.E. 13th International Symposium on Biomedical Imaging (ISBI), 2016
[9] Chen, Lele & Wu, Yue & Dsouza, Adora & Z. Abidin, Anas & Xu, Chenliang & Wismüller, Axel, MRI tumor segmentation with densely connected 3D CNN, 2018, 10.1117/12.2293394
[10] Raunaq Rewari, Automatic Tumor Segmentation from MRI scans, http://cs231n.stanford.edu/reports/2016/pdfs/328_Report.pdf
[11] Y. Dong, Y. Pan, X. Zhao, R. Li, C. Yuan and W. Xu, Identifying Carotid Plaque Composition in MRI with Convolutional Neural Networks, 2017 IEEE International Conference on Smart Computing (SMARTCOMP), Hong Kong, pp. 1-8, 2017
[12] Margeta, J., Criminisi, A., Cabrera Lozoya, R., Lee, D.C., Ayache, N., Fine-tuned convolutional neural nets for cardiac MRI acquisition plane recognition, Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, Taylor & Francis, pp. 339 – 349, 2017
[13] Vázquez Romaguera, Liset, Costa, Marly Guimarães Fernandes, Romero, Francisco Perdigón, Costa Filho, Cicero Ferreira Fernandes, Left ventricle segmentation in cardiac MRI images using fully convolutional neural networks, Proceedings of the SPIE, Volume 10134, id. 101342Z 11 pp. 2017
[14] Wang, Xinggang & Yang, Wei & Weinreb, Jeffrey & Han, Juan & Li, Qiubai & Kong, Xiangchuang & Yan, Yongluan & Ke, Zan & Luo, Bo & Liu, Tao & Wang, Liang, Searching for prostate cancer by fully automated magnetic resonance imaging classification: Deep learning versus non-deep learning. Scientific Reports, 7, 2017, 10.1038/s41598-017-15720-y.
[15] M. Srinivas, D. Roy and C. K. Mohan, Discriminative feature extraction from X-ray images using deep convolutional neural networks, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, 2016, pp. 917-921
[16] Y. Dong, Y. Pan, J. Zhang and W. Xu, Learning to Read Chest X-Ray Images from 16000+ Examples Using CNN, 2017 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), Philadelphia, PA, 2017, pp. 51-57.
[17] C. Liu et al., TX-CNN: Detecting tuberculosis in chest X-ray images using convolutional neural network, 2017 IEEE International Conference on Image Processing (ICIP), Beijing, 2017, pp. 2314-2318. doi: 10.1109/ICIP.2017.8296695
[18] Cernazanu-Glavan, C & Stefan, Holban, Segmentation of Bone Structure in X-ray Images using Convolutional Neural Network. Advances in Electrical and Computer Engineering. 13. 87-94, 2013, 10.4316/aece.2013.01015
[19] P. Rao, N. A. Pereira and R. Srinivasan, Convolutional neural networks for lung cancer screening in computed tomography (CT) scans, 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I), Noida, 2016, pp. 489-493
[20] Xiangrong Zhou, Ryosuke Takayama, Song Wang; Xinxin Zhou, Takeshi Hara, Hiroshi Fujita, Automated segmentation of 3D anatomical structures on CT images by using a deep convolutional network based on end-to-end learning approach, Proc. SPIE 10133, Medical Imaging 2017: Image Processing, 1013324 (24 February 2017); doi: 10.1117/12.2254201
[21] Lisowska, Aneta & Beveridge, Erin & Muir, Keith & Poole, Ian, Thrombus Detection in CT Brain Scans using a Convolutional Neural Network. 24-33, 2017, 10.5220/0006114600240033.
[22] K. J. Piczak, Environmental sound classification with convolutional neural networks, 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), Boston, MA, 2015, pp. 1-6. doi: 10.1109/MLSP.2015.7324337
[23] S. Dieleman, P. Brakel, B. Schrauwen, Audio-based music classification with a pretrained convolutional network, Proceedings of the 12th International Society for Music Information Retrieval (ISMIR) conference, pp. 669-674, 2011.
[24] Aykanat, M., Kılıç, Ö., Kurt, B. et al. J Image Video Proc. (2017) 2017: 65. https://doi.org/10.1186/s13640-017-0213-2
[25] Q. Chen, W. Zhang, X. Tian, X. Zhang, S. Chen and W. Lei, "Automatic heart and lung sounds classification using convolutional neural networks," 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Jeju, 2016, pp. 1-4.
[26] O. Abdel-Hamid, A. r. Mohamed, H. Jiang, L. Deng, G. Penn and D. Yu, "Convolutional Neural Networks for Speech Recognition," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 22, no. 10, pp. 1533-1545, Oct. 2014..
[27] X. Ouyang, P. Zhou, C. H. Li and L. Liu, "Sentiment Analysis Using Convolutional Neural Network," 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, Liverpool, 2015, pp. 2359-2364.
[28] Stojanovski, Dario & Strezoski, Gjorgji & Madjarov, Gjorgji & Dimitrovski, Ivica. (2015). Twitter Sentiment Analysis Using Deep Convolutional Neural Network. Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science). 9121. 10.1007/978-3-319-19644-2_60.
[29] A. Hassan and A. Mahmood, "Deep Learning approach for sentiment analysis of short texts," 2017 3rd International Conference on Control, Automation and Robotics (ICCAR), Nagoya, 2017, pp. 705-710. doi: 10.1109/ICCAR.2017.7942788
[30] Cai G., Xia B. (2015) Convolutional Neural Networks for Multimedia Sentiment Analysis. In: Li J., Ji H., Zhao D., Feng Y. (eds) Natural Language Processing and Chinese Computing. Lecture Notes in Computer Science, vol 9362. Springer, Cham
[31] Geoffrey E. Hinton (2009), Deep belief networks, Scholarpedia, 4(5):5947.
[32] HussamHebbo, Jae Won Kim, Classification with Deep Belief Networks, https://www.ki.tu-berlin.de/fileadmin/fg135/publikationen/Hebbo_2013_CDB.pdf
[33] M. Tim Jones, Deep learning architectures, https://www.ibm.com/developerworks/library/cc-machine-learning-deep-learning-architectures/index.html
[34] Wang, Hai & Cai, Yingfeng & Chen, Long. (2014). A Vehicle Detection Algorithm Based on Deep Belief Network. The Scientific World Journal. 2014. 647380. 10.1155/2014/647380
[35] R. Sarikaya, G. E. Hinton and A. Deoras, "Application of Deep Belief Networks for Natural Language Understanding," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 22, no. 4, pp. 778-784, April 2014. doi: 10.1109/TASLP.2014.2303296
[36] Bei Zhong, Jin Liu, Yuanda Du, Yunlu Liaozheng and Jiachen Pu, Extracting Attributes of Named Entity from Unstructured Text with Deep Belief Network, International Journal of Database Theory and Application Vol.9, No.5 (2016), pp.187-196
[37] Jin Y., Zhang H., Du D. (2017) Incorporating Positional Information into Deep Belief Networks for Sentiment Classification. In: Perner P. (eds) Advances in Data Mining. Applications and Theoretical Aspects. ICDM 2017. Lecture Notes in Computer Science, vol 10357. Springer, Cham
[38] Liu T. (2010) A Novel Text Classification Approach Based on Deep Belief Network. In: Wong K.W., Mendis B.S.U., Bouzerdoum A. (eds) Neural Information Processing. Theory and Algorithms. ICONIP 2010. Lecture Notes in Computer Science, vol 6443. Springer, Berlin, Heidelberg
[39] A. J. Yepes, A. MacKinlay, J. Bedo, R. Garnavi, and Q. Chen. Deep belief networks and biomedical text categorisation. In Proceedings of the Twelfth Annual Workshop of the Australasia Language Technology Association, page 123, 2014.
[40] G. Liu, L. Xiao and C. Xiong, "Image Classification with Deep Belief Networks and Improved Gradient Descent," 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), Guangzhou, 2017, pp. 375-380.
[41] S. Zhou, Q. Chen and X. Wang, "Discriminative Deep Belief Networks for image classification," 2010 IEEE International Conference on Image Processing, Hong Kong, 2010, pp. 1561-1564. doi: 10.1109/ICIP.2010.5649922
[42] P. Zhong, Z. Gong, S. Li and C. B. Schönlieb, "Learning to Diversify Deep Belief Networks for Hyperspectral Image Classification," in IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 6, pp. 3516-3530, June 2017. doi: 10.1109/TGRS.2017.2675902
[43] Kim J, Kang U, Lee Y. Statistics and Deep Belief Network-Based Cardiovascular Risk Prediction. Healthc Inform Res. 2017 Jul;23(3):169-175.
[44] ALTAN, Gökhan & Allahverdi, Novruz & Kutlu, Yakup. (2017). Diagnosis of Coronary Artery Disease Using Deep Belief Networks. European Journal of Engineering and Natural Sciences. 2. 29-36.
[45] Turner, J.T. & Page, Adam & Mohsenin, Tinoosh & Oates, Tim. (2014). Deep Belief Networks used on High Resolution Multichannel Electroencephalography Data for Seizure Detection. 75-81.
[46] Mohamed abd el Zaher, Ahmed allah & Eldeib, Ayman. (2015). Breast cancer classification using deep belief networks. Expert Systems with Applications. 46. 139-144. 10.1016/j.eswa.2015.10.015.
[47] Guo-Ping Liu, Jian-Jun Yan, Yi-Qin Wang, et al., “Deep Learning Based Syndrome Diagnosis of Chronic Gastritis,” Computational and Mathematical Methods in Medicine, vol. 2014, Article ID 938350, 8 pages, 2014. https://doi.org/10.1155/2014/938350.
[48] M. D. Prasetio, T. Hayashida, I. Nishizaki and S. Sekizaki, "Deep belief network optimization in speech recognition," 2017 International Conference on Sustainable Information Engineering and Technology (SIET), Malang, 2017, pp. 138-143.
[49] Zulkarneev M., Grigoryan R., Shamraev N. (2013) Acoustic Modeling with Deep Belief Networks for Russian Speech Recognition. In: Železný M., Habernal I., Ronzhin A. (eds) Speech and Computer. SPECOM 2013. Lecture Notes in Computer Science, vol 8113. Springer, Cham
[50] Farahat, Mahboubeh & Halavati, Ramin. (2016). Noise Robust Speech Recognition Using Deep Belief Networks. International Journal of Computational Intelligence and Applications. 15. 1650005. 10.1142/S146902681650005X.
[51] Lee JG, Jun S, Cho YW, Lee H, Kim GB, Seo JB, Kim N., Deep Learning in Medical Imaging: General Overview. Korean J Radiol. 2017 Jul-Aug;18(4):570-584
[52] Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs, http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/
[53] Min, Seonwoo & Lee, Byunghan & Yoon, Sungroh. (2016). Deep Learning in Bioinformatics. Briefings in Bioinformatics. 18. 10.1093/bib/bbw068.
[54] Wim De Mulder, Steven Bethard, Marie-Francine Moens, A survey on the application of recurrent neural networks to statistical language modeling, Computer Speech & Language, Volume 30, Issue 1, 2015, Pages 61-98
[55] T. Ishitaki, R. Obukata, T. Oda and L. Barolli, "Application of Deep Recurrent Neural Networks for Prediction of User Behavior in Tor Networks," 2017 31st International Conference on Advanced Information Networking and Applications Workshops (WAINA), Taipei, 2017, pp. 238-243.
[56] Malek, Alaeddin. (2008). Applications of Recurrent Neural Networks to Optimization Problems. 10.5772/5556.
[57] B. Q. Huang, Tarik Rashid and M-T. Kechadi, Multi-Context Recurrent Neural Network for Time Series Applications, World Academy of Science, Engineering and Technology International Journal of Computer and Information Engineering Vol:1, No:10, 2007
[58] Serban, I. V., Klinger, T., Tesauro, G., Talamadupula, K., Zhou, B., Bengio, Y., & Courville, A. C. (2017, February). Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation. In AAAI (pp. 3288-3294).
[59] V. Pham, T. Bluche, C. Kermorvant and J. Louradour, "Dropout Improves Recurrent Neural Networks for Handwriting Recognition," 2014 14th International Conference on Frontiers in Handwriting Recognition, Heraklion, 2014, pp. 285-290.
[60] Graves A. (2012) Offline Arabic Handwriting Recognition with Multidimensional Recurrent Neural Networks. In: Märgner V., El Abed H. (eds) Guide to OCR for Arabic Scripts. Springer, London
[61] Chakraborty, Bappaditya & Sarathi Mukherjee, Partha & Bhattacharya, Ujjwal. (2016). Bangla online handwriting recognition using recurrent neural network architecture. 1-8. 10.1145/3009977.3010072.
[62] P. Voigtlaender, P. Doetsch and H. Ney, "Handwriting Recognition with Large Multidimensional Long Short-Term Memory Recurrent Neural Networks," 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), Shenzhen, 2016, pp. 228-233