CFP last date
20 December 2024
Call for Paper
January Edition
IJCA solicits high quality original research papers for the upcoming January edition of the journal. The last date of research paper submission is 20 December 2024

Submit your paper
Know more
The week's pick
Responsive image
Improved Shuffled Frog Leaping Algorithm with Self-Adaptive Shuffling for Fuzzy Logic PD+G Controller Optimization in Robotic Manipulators

Duc Hoang Nguyen
Random Articles
Reseach Article

Review of Deep Learning: Architectures, Applications and Challenges

by Ankit Sirmorya, Milind Chaudhari, Suhail Balasinor
journal cover thumbnail

International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 184 - Number 18
Year of Publication: 2022
Authors: Ankit Sirmorya, Milind Chaudhari, Suhail Balasinor
10.5120/ijca2022922164

Ankit Sirmorya, Milind Chaudhari, Suhail Balasinor . Review of Deep Learning: Architectures, Applications and Challenges. International Journal of Computer Applications. 184, 18 ( Jun 2022), 1-13. DOI=10.5120/ijca2022922164

@article{ 10.5120/ijca2022922164,
author = { Ankit Sirmorya, Milind Chaudhari, Suhail Balasinor },
title = { Review of Deep Learning: Architectures, Applications and Challenges },
journal = { International Journal of Computer Applications },
issue_date = { Jun 2022 },
volume = { 184 },
number = { 18 },
month = { Jun },
year = { 2022 },
issn = { 0975-8887 },
pages = { 1-13 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume184/number18/32415-2022922164/ },
doi = { 10.5120/ijca2022922164 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2024-02-07T01:21:44.923579+05:30
%A Ankit Sirmorya
%A Milind Chaudhari
%A Suhail Balasinor
%T Review of Deep Learning: Architectures, Applications and Challenges
%J International Journal of Computer Applications
%@ 0975-8887
%V 184
%N 18
%P 1-13
%D 2022
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Deep Learning is a continuously evolving subset of machine learning techniques. New technology has provided solutions to a wide range of complex problems that were once unsolvable due to limitations in human intelligence. Since its conception, several DL architectures have been developed, including recursive neural networks, recurrent neural networks, artificial neural networks, and convolution neural networks. Many of their contributions have been in the area of computer vision, natural language processing, sequence generation, etc. Despite their increasing popularity, many individuals cannot see the bigger picture or comprehend these techniques. In this paper, the various deep learning models are described, as well as how they work. In addition, the article explains a few prominent DL models and their relevance in contemporary technology. As with every rapidly changing technology, DL has some limitations. These limitations are mitigated to some extent in this paper. Further, it emphasizes their continued development, the challenges they face, and the possibilities for future research in their fields.

References
  1. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44
  2. Zhang Z, Cui P, Zhu W. Deep learning on graphs: a survey. IEEE Trans Knowl Data Eng. 2020. https://doi.org/10. 1109/TKDE.2020.2981333.
  3. Shrestha A, Mahmood A. Review of deep learning algorithms and architectures. IEEE Access. 2019;7:53040–65
  4. Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E. Deep learning applications and challenges in big data analytics. J Big Data. 2015;2(1):1.
  5. Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning, vol. 1. Cambridge: MIT press; 2016
  6. Hasan RI, Yusuf SM, Alzubaidi L. Review of the state of the art of deep learning for plant diseases: a broad analysis and discussion. Plants. 2020;9(10):1302.
  7. Shorten C, Khoshgoftaar TM, Furht B. Deep learning applications for COVID-19. J Big Data. 2021;8(1):1–54
  8. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
  9. Goh GB, Hodas NO, Vishnu A. Deep learning for computational chemistry. J Comput Chem. 2017;38(16):1291–307.
  10. Li Y, Zhang T, Sun S, Gao X. Accelerating flash calculation through deep learning methods. J Comput Phys. 2019;394:153–65.
  11. Yang W, Zhang X, Tian Y, Wang W, Xue JH, Liao Q. Deep learning for single image super-resolution: a brief review. IEEE Trans Multimed. 2019;21(12):3106–21
  12. Tang J, Li S, Liu P. A review of lane detection methods based on deep learning. Pattern Recogn. 2020;111:107623.
  13. Zhao ZQ, Zheng P, Xu ST, Wu X. Object detection with deep learning: a review. IEEE Trans Neural Netw Learn Syst. 2019;30(11):3212–32
  14. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classifcation with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
  15. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 770–8.
  16. Metz C. Turing award won by 3 pioneers in artifcial intelligence. The New York Times. 2019;27
  17. Nevo S, Anisimov V, Elidan G, El-Yaniv R, Giencke P, Gigi Y, Hassidim A, Moshe Z, Schlesinger M, Shalev G, et al. Ml for food forecasting at scale; 2019. arXiv preprint arXiv:1901.09583.
  18. Chen H, Engkvist O,Wang Y, Olivecrona M, Blaschke T. The rise of deep learning in drug discovery. Drug Discov Today. 2018;23(6):1241–50.
  19. Benhammou Y, Achchab B, Herrera F, Tabik S. Breakhis based breast cancer automatic diagnosis using deep learning: taxonomy, survey and insights. Neurocomputing. 2020;375:9–24.
  20. Wulczyn E, Steiner DF, Xu Z, Sadhwani A,Wang H, Flament- Auvigne I, Mermel CH, Chen PHC, Liu Y, Stumpe MC. Deep learning-based survival prediction for multiple cancer types using histopathology images. PLoS ONE. 2020;15(6):e0233678.
  21. Brunese L, Mercaldo F, Reginelli A, Santone A. Explainable deep learning for pulmonary disease and coronavirus COVID- 19 detection from X-rays. Comput Methods Programs Biomed. 2020;196(105):608.
  22. Jamshidi M, Lalbakhsh A, Talla J, Peroutka Z, Hadjilooei F, Lalbakhsh P, Jamshidi M, La Spada L, Mirmozafari M, Dehghani M, et al. Artifcial intelligence and COVID-19: deep learning approaches for diagnosis and treatment. IEEE Access. 2020;8:109581–95.
  23. Shorfuzzaman M, Hossain MS. Metacovid: a siamese neural network framework with contrastive loss for n-shot diagnosis of COVID-19 patients. Pattern Recogn. 2020;113:107700.
  24. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 770–8.
  25. Saeed MM, Al Aghbari Z, Alsharidah M. Big data clustering techniques based on spark: a literature review. PeerJ Comput Sci. 2020;6:321.
  26. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al. Human-level control through deep reinforcement learning. Nature. 2015;518(7540):529–33.
  27. Socher R, Perelygin A,Wu J, Chuang J, Manning CD, Ng AY, Potts C. Recursive deep models for semantic compo- sitionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing; 2013. p. 1631–42.
  28. Goller C, Kuchler A. Learning task-dependent distributed representations by backpropagation through structure. In: Proceedings of international conference on neural networks (ICNN’96), vol 1. IEEE; 1996. p. 347–52.
  29. Socher R, Lin CCY, Ng AY, Manning CD. Parsing natural scenes and natural language with recursive neural net- works. In: ICML; 2011.
  30. Louppe G, Cho K, Becot C, Cranmer K. QCD-aware recursive neural networks for jet physics. J High Energy Phys. 2019;2019(1):57.
  31. Sadr H, Pedram MM, Teshnehlab M. A robust sentiment analysis method based on sequential combination of convolutional and recursive neural networks. Neural Process Lett. 2019;50(3):2745–61.
  32. Urban G, Subrahmanya N, Baldi P. Inner and outer recursive neural networks for chemoinformatics applications. J Chem Inf Model. 2018;58(2):207–11.
  33. Batur Dinler O¨ , Aydin N. An optimal feature parameter set based on gated recurrent unit recurrent neural net- works for speech segment detection. Appl Sci. 2020;10(4):1273.
  34. Jagannatha AN, Yu H. Structured prediction models for RNN based sequence labeling in clinical text. In: Proceed- ings of the conference on empirical methods in natural language processing. conference on empirical methods in natural language processing, vol. 2016, NIH Public Access; 2016. p. 856.
  35. Pascanu R, Gulcehre C, Cho K, Bengio Y. How to construct deep recurrent neural networks. In: Proceedings of the second international conference on learning representations (ICLR 2014); 2014.
  36. Glorot X, Bengio Y. Understanding the difculty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artifcial intelligence and statistics; 2010. p. 249–56.
  37. Gao C, Yan J, Zhou S, Varshney PK, Liu H. Long short-term memory-based deep recurrent neural networks for target tracking. Inf Sci. 2019;502:279–96.
  38. Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J, et al. Recent advances in convolutional neural networks. Pattern Recogn. 2018;77:354–77.
  39. Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning, vol. 1. Cambridge: MIT press; 2016
  40. O. M. Parkhi, A. Vedaldi, C. V. Jawahar and A. Zisserman, ”The truth about cats and dogs,” 2011 International Conference on Computer Vision, 2011, pp. 1427-1434, doi: 10.1109/ICCV.2011.6126398.
  41. HUBEL DH, WIESEL TN. Receptive fields of single neurones in the cat’s striate cortex. J Physiol. 1959;148(3):574- 591. doi:10.1113/jphysiol.1959.sp006308
  42. Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Santamar ´ıa J, Duan Y, Oleiwi SR. Towards a better understanding of transfer learning for medical imaging: a case study. Appl Sci. 2020;10(13):4523.
  43. Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C. A survey on deep transfer learning. In: International conference on artifcial neural networks. Springer; 2018. p. 270–9.
  44. Weiss K, Khoshgoftaar TM, Wang D. A survey of transfer learning. J Big Data. 2016;3(1):9
  45. Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data. 2019;6(1):60
  46. Wang S, Sun S, Xu J. Auc-maximized deep convolutional neural felds for sequence labeling 2015. arXiv preprint arXiv:1511.05265.
  47. Johnson JM, Khoshgoftaar TM. Survey on deep learning with class imbalance. J Big Data. 2019;6(1):27
  48. Yang P, Zhang Z, Zhou BB, Zomaya AY. Sample subset optimization for classifying imbalanced biological data. In: Pacifc- Asia conference on knowledge discovery and data mining. Springer; 2011. p. 333–44
  49. Li Y, Huang C, Ding L, Li Z, Pan Y, Gao X. Deep learning in bioinformatics: introduction, application, and perspective in the big data era. Methods. 2019;166:4–21
  50. Choi E, Bahadori MT, Sun J, Kulas J, Schuetz A, Stewart W. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In: Advances in neural information processing systems. San Mateo: Morgan Kaufmann Publishers; 2016. p. 3504–12
  51. Ching T, Himmelstein DS, Beaulieu-Jones BK, Kalinin AA, Do BT, Way GP, Ferrero E, Agapow PM, Zietz M, Hof- man MM, et al. Opportunities and obstacles for deep learning in biology and medicine. J R Soc Interface. 2018;15(141):20170,387
  52. Pokuri BSS, Ghosal S, Kokate A, Sarkar S, Ganapathysubramanian B. Interpretable deep learning for guided microstructure-property explorations in photovoltaics. NPJ Comput Mater. 2019;5(1):1–11.
  53. Ribeiro MT, Singh S, Guestrin C. “Why should I trust you?” explaining the predictions of any classifer. In: Proceed- ings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining; 2016. p. 1135–44.
  54. Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks; 2017. arXiv preprint arXiv:1703.01365
  55. Platt J, et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood meth- ods. Adv Large Margin Classif. 1999;10(3):61–74
  56. Nair T, Precup D, Arnold DL, Arbel T. Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation. Med Image Anal. 2020;59:101557
  57. Herzog L, Murina E, D¨urr O, Wegener S, Sick B. Integrating uncertainty in deep neural networks for MRI based stroke analysis. Med Image Anal. 2020;65:101790
  58. Pereyra G, Tucker G, Chorowski J, Kaiser Ł, Hinton G. Regularizing neural networks by penalizing confdent output distributions; 2017. arXiv preprint arXiv:1701.06548
  59. Naeini MP, Cooper GF, Hauskrecht M. Obtaining well calibrated probabilities using bayesian binning. In: Proceed- ings of the... AAAI conference on artifcial intelligence. AAAI conference on artifcial intelligence, vol. 2015. NIH Public Access; 2015. p. 2901
  60. Li M, Sethi IK. Confdence-based classifer design. Pattern Recogn. 2006;39(7):1230–40.
  61. Zadrozny B, Elkan C. Obtaining calibrated probability estimates from decision trees and Naive Bayesian classifers. In: ICML, vol. 1, Citeseer; 2001. p. 609–16
  62. Steinwart I. Consistency of support vector machines and other regularized kernel classifers. IEEE Trans Inf Theory. 2005;51(1):128–42
  63. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural net- works from overftting. J Mach Learn Res. 2014;15(1):1929–58
  64. Xu Q, Zhang M, Gu Z, Pan G. Overftting remedy by sparsifying regularization on fully-connected layers of CNNs. Neurocomputing. 2019;328:69–74
  65. Zhang C, Bengio S, Hardt M, Recht B, Vinyals O. Understanding deep learning requires rethinking generalization. Commun ACM. 2018;64(3):107–15
  66. Xu X, Jiang X, Ma C, Du P, Li X, Lv S, Yu L, Ni Q, Chen Y, Su J, et al. A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering. 2020;6(10):1122–9.
  67. Zhang G, Wang C, Xu B, Grosse R. Three mechanisms of weight decay regularization; 2018. arXiv preprint arXiv: 1810.12281
  68. Laurent C, Pereyra G, Brakel P, Zhang Y, Bengio Y. Batch normalized recurrent neural networks. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE; 2016. p. 2657–61
  69. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural net- works from overftting. J Mach Learn Res. 2014;15(1):1929–58
  70. Pereyra G, Tucker G, Chorowski J, Kaiser Ł, Hinton G. Regularizing neural networks by penalizing confdent output distributions; 2017. arXiv preprint arXiv:1701.06548
  71. D’Amour A, Heller K, Moldovan D, Adlam B, Alipanahi B, Beutel A, Chen C, Deaton J, Eisenstein J, Hofman MD, et al. Underspecifcation presents challenges for credibility in modern machine learning; 2020. arXiv preprint arXiv: 2011.03395
  72. Bharati S, Podder P, Mondal MRH. Hybrid deep learning for detecting lung diseases from X-ray images. Inform Med Unlocked. 2020;20:100391
  73. Heidari M, Mirniaharikandehei S, Khuzani AZ, Danala G, Qiu Y, Zheng B. Improving the performance of CNN to predict the likelihood of COVID-19 using chest X-ray images with preprocessing algorithms. Int J Med Inform. 2020;144:104284
  74. Al-Timemy AH, Khushaba RN, Mosa ZM, Escudero J. An efcient mixture of deep and machine learning models for COVID-19 and tuberculosis detection using X-ray images in resource limited settings 2020. arXiv preprint arXiv: 2007.08223.
  75. Abraham B, Nair MS. Computer-aided detection of COVID- 19 from X-ray images using multi-CNN and Bayesnet classifer. Biocybern Biomed Eng. 2020;40(4):1436–45.
  76. Shin HC, Orton MR, Collins DJ, Doran SJ, Leach MO. Stacked autoencoders for unsupervised feature learn- ing and multiple organ detection in a pilot study using 4D patient data. IEEE Trans Pattern Anal Mach Intell. 2012;35(8):1930–43.
  77. Sirazitdinov I, Kholiavchenko M, Mustafaev T, Yixuan Y, Kuleev R, Ibragimov B. Deep neural network ensemble for pneumonia localization from a large-scale chest X-ray database. Comput Electr Eng. 2019;78:388–99
  78. Hossin M, Sulaiman M. A review on evaluation metrics for data classifcation evaluations. Int J Data Min Knowl Manag Process. 2015;5(2):1
  79. Dr.R.Parthasarathi1: ANN Based Modeling for Performance and Exhaust Emission of DI Diesel Engine using Emulsified Diesel Fuel
  80. Sushma Tamta: Estimation of Evaporation in Hilly Area by Using Ann and Canfis System Based Models
  81. Mruga Gurjar: STOCK MARKET PREDICTION USING ANN
  82. Laura E. Suarez : Learning function from structure in neuromorphic networks
  83. Arlene Casey1*, Emma Davidson2 , Michael Poon2 , Hang Dong3,4, Daniel Duma1 , Andreas Grivas5 , Claire Grover5 , V´ıctor Su´arez-Paniagua3,4, Richard Tobin5 , William Whiteley2,6, Honghan Wu4,7 and Beatrice Alex1: A systematic review of natural language processing applied to radiology reports
  84. Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  85. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. Bengio,Y. (2014)Generative Adversarial Nets. NIPS 2014
  86. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein gan. arXiv preprint
  87. Chen Y, Lai YK, Liu YJ (2018) Cartoongan: Generative adversarial networks for photo cartoonization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9465–9474
  88. Alec Radford Luke Metz, Soumit chintala: UNSUPERVISED REPRESENTATION LEARNING WITH DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORKS
  89. Mehdi Mirza,Simon Osindero: Conditional Generative Adversarial Nets
  90. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1. chap. Learning Internal Representations by Error Propagation, pp. 318–362. MIT Press, Cambridge, MA, USA (1986). URL http://dl.acm. org/citation.cfm?id=104279.104293
  91. Jinbo Xua: Distance-based protein folding powered by deep learning
  92. Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique
  93. Alan L. Yuille Chenxi Liu, ”Limitations of Deep Learning for Vision, and How We Might Fix Them”, The Gradient, 2019.
  94. Raghavendra Chalapathy, Sanjay Chawla: DEEP LEARNING FOR ANOMALY DETECTION: A SURVEY,2019.
  95. Wen, Andrew,Fu, Sunyang: Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLPas- a-service implementation.
  96. Spyns, Peter. (1997). Natural Language Processing in Medicine: An Overview. Methods of information in medicine. 35. 285-301. 10.1055/s-0038-1634681.
  97. Pereira S, Pinto A, Alves V, Silva CA. Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images. IEEE Trans Med Imaging. 2016 May;35(5):1240-1251. doi: 10.1109/TMI.2016.2538465. Epub 2016 Mar 4. PMID: 26960222.
  98. Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE conference on computer vision and pattern recognition; 2014 Jun 23-28; Washington, DC, USA. Piscataway (NJ): Institute of Electrical and Electronics Engineers; 2014. pp. 580–7.
  99. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 5, No. 12, 2014
  100. Chen, Chung-Chi Huang, Hen-Hsen Chen, Hsin-Hsi. (2020). NLP in FinTech Applications: Past, Present and Future.
  101. Hirschman, L., Gaizauskas, R. (2001). Natural language question answering:the view from here. Natural Language Engineering, 7, 275–300.
  102. Michael Tschannen, Olivier Bachem, Mario Lucic: “Recent Advances in Autoencoder-Based Representation Learning”, 2018
  103. Autoencoders, Lopez Pinaya W,Viera S et.al 2019.
  104. Chandola et al.
  105. Pang, Guansong Cao, Longbing Aggarwal, Charu. (2021). Deep Learning for Anomaly Detection: Challenges, Methods, and Opportunities. 1127-1130. 10.1145/3437963.3441659.
  106. Sublime, Jeremie Kalinicheva, Ekaterina. (2019). Automatic Post-Disaster Damage Mapping Using Deep-Learning Techniques for Change Detection: Case Study of the Tohoku Tsunami. Remote Sensing. 11. 1123. 10.3390/rs11091123.
  107. arXiv:1312.5663 [cs.LG]
  108. Dong, Ganggang Liao, Guisheng Liu, Hongwei Kuang, Gangyao. (2018). A Review of the Autoencoder and Its Variants: A Comparative Perspective from Target Recognition in Synthetic-Aperture Radar Images. IEEE Geoscience and Remote Sensing Magazine. 6. 44-68. 10.1109/MGRS.2018.2853555.
  109. Changfan Zhang, Xiang Cheng, Jianhua Liu, Jing He, Guangwei Liu, ”Deep Sparse Autoencoder for Feature Extraction and Diagnosis of Locomotive Adhesion Status”, Journal of Control Science and Engineering, vol. 2018, Article ID 8676387, 9 pages, 2018. https://doi.org/10.1155/2018/8676387
  110. Huajie Shao , Zhisheng Xiao :ControlVAE: Tuning, Analytical Properties, and Performance Analysis
  111. X. Yan, J. Yang, K. Sohn, and H. Lee, “Attribute2image: Conditional image generation from visual attributes,” in European Conference on Computer Vision. Springer, 2016, pp. 776–791
  112. M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised image-toimage translation networks,” in Advances in neural information processing systems, 2017, pp. 700–708.
  113. A. Zhang, Z. C. Lipton, M. Li, and A. J. Smola, Dive into Deep Learning, 2020, https://d2l.ai.
  114. W. Wang, Z. Gan, H. Xu, R. Zhang, G. Wang, D. Shen, C. Chen, and L. Carin, “Topic-guided variational autoencoders for text generation,” arXiv preprint arXiv:1903.07137, 2019.
  115. Z. Hu, Z. Yang, X. Liang, R. Salakhutdinov, and E. P. Xing, “Toward controlled generation of text,” in Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017, pp. 1587–1596
  116. I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework.” ICLR, vol. 2, no. 5, p. 6, 2017.
  117. H. Kim and A. Mnih, “Disentangling by factorising,” in International Conference on Machine Learning, 2018, pp. 2654–2663
  118. Feng, Jie Feng, Xueliang Chen, Jiantong Cao, Xianghai Zhang, Xiangrong Jiao, Licheng Yu, Tao. (2020). Generative Adversarial Networks Based on Collaborative Learning and Attention Mechanism for Hyperspectral Image Classification. Remote Sensing. 12. 1149. 10.3390/rs12071149.
  119. Feng, Jie Feng, Xueliang Chen, Jiantong Cao, Xianghai Zhang, Xiangrong Jiao, Licheng Yu, Tao. (2020). Generative Adversarial Networks Based on Collaborative Learning and Attention Mechanism for Hyperspectral Image Classification. Remote Sensing. 12. 1149. 10.3390/rs12071149.
  120. Phung, Rhee,. (2019). A High-Accuracy Model Average Ensemble of Convolutional Neural Networks for Classification of Cloud Image Patches on Small Datasets. Applied Sciences. 9. 4500. 10.3390/app9214500.
  121. Mishra, Vidushi AGARWAL, SMT PURI, NEHA. (2018). COMPREHENSIVE AND COMPARATIVE ANALYSIS OF NEURAL NETWORK. INTERNATIONAL JOURNAL OF COMPUTER APPLICATION. 2. 10.26808/rs.ca.i8v2.15.
  122. M. S. B. M. M. P. W. “Research Paper on Basic of Artificial Neural Network”. International Journal on Recent and Innovation Trends in Computing and Communication, vol. 2, no. 1, Jan. 2014, pp. 96-100, doi:10.17762/ijritcc.v2i1.2920.
  123. Grossi, Enzo Buscema, Massimo. (2008). Introduction to artificial neural networks. European journal of gastroenterology hepatology. 19. 1046-54. 10.1097/MEG.0b013e3282f198a0.
  124. Christos Stergiou and Dimitrios Siganos,“Neural Networks”
  125. Gopal, Sucharita. ”Unit 188-Artificial Neural Networks for Spatial Data Analysis.” (2000).
  126. Wang, Su. (2017). Generative Adversarial Networks (GAN): A Gentle Introduction [UPDATED].
  127. Lotter, W., Kreiman, G. Cox, D. (2015) Unsupervised Learning of Visual Structure Using Predictive Generative Networks CoRR, arXiv:1151.06380.
  128. Ledig, C., Theis, L., Huszar, F., Caballero, J., Aitken, A. P., Tejani, A., Totz, J., Wang, Z., Shi, W. (2016) Photo-realistic single Image Super-resolution Using a Generative Adversarial Network CoRR,arXiv:1609.04802.
  129. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A. A. (2016). Imageto- image
  130. Translation with ConditionalAdversarial Networks. CoRR, arXiv:1611.07004.
  131. Donahue, J., Kr¨ahenb¨uhl, P. Darrell, T. (2017) Adversarial Feature Learning CoRR, arXiv:1605.09782.
  132. Glover, J. (2016)Modeling Documents with Generative Adversarial Networks. CoRR, arXiv:1612.09122.
  133. Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A. Jurafsky, D. (2017)Adversarial Learning for Neural Dialogue Generation CoRR, arXiv:1701.06547.
  134. Chen X., Athiwaratkun, B., Sun, Y.,Weinberger, K. Cardie, C. (2016) Adversarial Deep Averaging Networks for Crosslingual Sentiment Classification,. CoRR, arXiv:1606.01614.
  135. Zhang, Y., Barzilay, R. Jaakkola, T. (2017)Aspectaugmented Adversarial Networks for Domain Adaptation. CoRR, arXiv:1701.00188
  136. arXiv:1312.6114v10 [stat.ML]
  137. Khurana, Diksha, et al. ”Natural language processing: State of the art, current trends and challenges.” arXiv preprint arXiv:1708.05148 (2017).
  138. Alshawi, H. (1992). The core language engine. MIT press. Kamp, H., Reyle, U. (1993). Tense and Aspect. In From Discourse to Logic (pp. 483- 689). Springer Netherlands.
  139. Tillmann, C., Vogel, S., Ney, H., Zubiaga, A., Sawaf, H. (1997, September). Accelerated DP based search for statistical translation. In Eurospeech.
  140. Bangalore, S., Rambow, O., Whittaker, S. (2000, June). Evaluation metrics for generation. In Proceedings of the first international conference on Natural language generation- Volume 14 (pp. 1-8). Association for Computational Linguistics
  141. Nießen, S., Och, F. J., Leusch, G., Ney, H. (2000, May). An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In LREC
  142. Papineni, K., Roukos, S., Ward, T., Zhu, W. J. (2002, July). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 311-318). Association for Computational Linguistics
  143. Doddington, G. (2002, March). Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second international conference
  144. McCray, A. T., Nelson, S. J. (1995). The representation of meaning in the UMLS. Methods of information in medicine, 34(1-2), 193-201.
  145. McGray, A. T., Sponsler, J. L., Brylawski, B., Browne, A. C. (1987, November). The role of lexical knowledge in biomedical text understanding. In Proceedings of the Annual Symposium on Computer Application in Medical Care (p. 103). American Medical Informatics Association.
  146. McCray, A. T. (1991). Natural language processing for intelligent information retrieval. In Engineering in Medicine and Biology Society, 1991. Vol. 13: 1991., Proceedings of the Annual International Conference of the IEEE (pp. 1160-1161). IEEE.
  147. McCray, A. T. (1991). Extending a natural language parser with UMLS knowledge. In Proceedings of the Annual Symposium on Computer Application in Medical Care (p. 194). American Medical Informatics Association.
  148. McCray, A. T., Srinivasan, S., Browne, A. C. (1994). Lexical methods for managing variation in biomedical terminologies. In Proceedings of the Annual Symposium on Computer Application in Medical Care (p. 235). American Medical Informatics Association.
  149. Liu S, Liu S, Cai W, et al. Early diagnosis of Alzheimer’s disease with deep learning. In: International Symposium on Biomedical Imaging, Beijing, China 2014, 1015–18.
  150. Brosch T, Tam R. Manifold learning of brain MRIs by deep learning. Med Image Comput Comput Assist Interv 2013;16:633–40.
  151. Prasoon A, Petersen K, Igel C, et al. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. Med Image Comput Comput Assist Interv 2013;16:246–53
  152. Deenan, Surya Prabha SatheeshKumar, J.. (2014). Image processing methods and its Role in agricultural sector – A study. International Journal of Business Intelligents. 3. 366- 373.
  153. Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. ”Reducing the dimensionality of data with neural networks.” science 313.5786 (2006): 504-507.
  154. Ashish Vaswani, Noam Shazeer: “Attention i sall you need”, arXiv:1706.03762v5
  155. Maxime: “What is a Transformer?”, https://medium.com/inside-machine-learning/what-is-atransformer- d07dd1fbec04
  156. Vaswani, Ashish Bengio, Samy Brevdo, Eugene Chollet, Francois Gomez, Aidan Gouws, Stephan Jones, Llion Kaiser, Łukasz Kalchbrenner, Nal Parmar, Niki Sepassi, Ryan Shazeer, Noam Uszkoreit, Jakob. (2018). Tensor2Tensor for Neural Machine Translation.
  157. Wolf, Thomas, et al. ”Transformers: State-of-the-art natural language processing.” Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations. 2020.
  158. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473, 2014
  159. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. CoRR, abs/1406.1078, 2014.
  160. Alex Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850, 2013.
  161. Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112, 2014.
  162. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  163. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016
  164. Stefania Cristina : “The Transformer Model”, https://machinelearningmastery.com/the-transformer-model/
  165. Nurtas, Marat Baishemirov, Zharasbek Tsay, V. Tastanov, M. Zhanabekov, Zh. (2020). APPLYING NEURAL NETWORK FOR PREDICTING CARDIOVASCULAR DISEASE RISK. PHYSICO-MATHEMATICAL SERIES. 4. 28- 34. 10.32014/2020.2518-1726.62.
  166. Ensar Seker: ”Recursive Neural Networks( RvNN) and Recurrent Neural Network(RNN)” ”https://ai.plainenglish.io/recursive-neural-networks-rvnnsand- recurrent-neural-networks-rnns-2ff6a067ad01
  167. Andrew Ng: ”Sparse Autoencoder”, https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf
  168. rhum Shafkat:”Intuitively Understanding Variational Autoencoders”,https://towardsdatascience.com/intuitivelyunderstanding- variational-autoencoders-1bfe67eb5daf
Index Terms

Computer Science
Information Sciences

Keywords

Deep Learning Machine Learning

Powered by PhDFocusTM