CFP last date
20 January 2025
Call for Paper
February Edition
IJCA solicits high quality original research papers for the upcoming February edition of the journal. The last date of research paper submission is 20 January 2025

Submit your paper
Know more
The week's pick
Responsive image
SFLA-Based Line Balancing and Task Optimization in Master-Slave Controlled Hanger Transportation Systems for Garment Production

Duc Hoang Nguyen
Random Articles
Reseach Article

Knowledge Engineering on Internet of Things through Reinforcement Learning

by Wasswa Shafik, Seyed Akabr Mostafavi
journal cover thumbnail

International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 177 - Number 44
Year of Publication: 2020
Authors: Wasswa Shafik, Seyed Akabr Mostafavi
10.5120/ijca2020919952

Wasswa Shafik, Seyed Akabr Mostafavi . Knowledge Engineering on Internet of Things through Reinforcement Learning. International Journal of Computer Applications. 177, 44 ( Mar 2020), 1-7. DOI=10.5120/ijca2020919952

@article{ 10.5120/ijca2020919952,
author = { Wasswa Shafik, Seyed Akabr Mostafavi },
title = { Knowledge Engineering on Internet of Things through Reinforcement Learning },
journal = { International Journal of Computer Applications },
issue_date = { Mar 2020 },
volume = { 177 },
number = { 44 },
month = { Mar },
year = { 2020 },
issn = { 0975-8887 },
pages = { 1-7 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume177/number44/31198-2020919952/ },
doi = { 10.5120/ijca2020919952 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2024-02-07T00:48:35.109287+05:30
%A Wasswa Shafik
%A Seyed Akabr Mostafavi
%T Knowledge Engineering on Internet of Things through Reinforcement Learning
%J International Journal of Computer Applications
%@ 0975-8887
%V 177
%N 44
%P 1-7
%D 2020
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Reinforcement learning (RL) is a new research area practical in the internet of things (IoT) where it addresses a broad and relevant task through about making decisions. RL enables interaction of devices and with the environment through a probabilistic approach using the response from its own actions and experiences. RL permits the machine and software agent to attain its behavior constructed on feedback from the environment. The IoTs extends to devices to the internet like smart electronic devices that can network and interconnect with others over through connectivity of remote resources being supervised and meticulous. In this paper, we examine the main four RL techniques including Markov Decision Process (MDP), Learning Automata (LA), artificial neural network (ANN), Q-learning in relation to its applicability in IoT, challenges and link them to state of art solutions. This review provides a summarized analysis of RL techniques that researchers can use to identify current bottlenecks in IoT and suggest models that are in line with the move.

References
  1. C. H. Liu, Q. Lin, and S. Wen, “Blockchain-enabled Data Collection and Sharing for Industrial IoT with Deep Reinforcement Learning,” IEEE Transactions on Industrial Informatics, 2018.
  2. J. Chen, S. Chen, Q. Wang, B. Cao, G. Feng, and J. Hu, “iRAF: a Deep Reinforcement Learning Approach for Collaborative Mobile Edge Computing IoT Networks,” IEEE Internet of Things Journal, 2019.
  3. Y. Wei, F. R. Yu, M. Song, and Z. Han, “Joint Optimization of Caching, Computing, and Radio Resources for Fog-Enabled IoT Using Natural Actor-Critic Deep Reinforcement Learning,” IEEE Internet of Things Journal, 2018.
  4. M. Mohammadi, A. Al-Fuqaha, M. Guizani, and J.-S. Oh, “Semisupervised deep reinforcement learning in support of IoT and smart city services,” IEEE Internet of Things Journal, vol. 5, no. 2, pp. 624–635, 2018.
  5. A. Singla and A. Sharma, “Physical Access System Security of IoT Devices using Machine Learning Techniques,” Available at SSRN 3356785, 2019.
  6. C. H. Liu, Q. Lin, and S. Wen, “Blockchain-enabled Data Collection and Sharing for Industrial IoT with Deep Reinforcement Learning,” IEEE Transactions on Industrial Informatics, 2018.
  7. S. Yousefi, F. Derakhshan, and A. Bokani, "Mobile Agents for Route Planning in the Internet of Things Using Markov Decision Process," in 2018 IEEE International Conference on Smart Energy Grid Engineering (SEGE), 2018, pp. 303–307.
  8. P. Sun, J. Li, M. Z. A. Bhuiyan, L. Wang, and B. Li, “Modeling and clustering attacker activities in IoT through machine learning techniques,” Information Sciences, vol. 479, pp. 456–471, 2019.
  9. A. Singla and A. Sharma, “Physical Access System Security of IoT Devices using Machine Learning Techniques,” Available at SSRN 3356785, 2019.
  10. P. Punithavathi, S. Geetha, M. Karuppiah, S. H. Islam, M. M. Hassan, and K.-K. R. Choo, “A lightweight machine learning-based authentication framework for smart IoT devices,” Information Sciences, vol. 484, pp. 255–268, 2019.
  11. F. Hussain, R. Hussain, S. A. Hassan, and E. Hossain, “Machine Learning in IoT Security: Current Solutions and Future Challenges,” arXiv preprint arXiv:1904.05735, 2019.
  12. I. Grondman, L. Busoniu, G. A. Lopes, and R. Babuska, “A survey of actor-critic reinforcement learning: Standard and natural policy gradients,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 1291–1307, 2012.
  13. J. Qiu, Q. Wu, G. Ding, Y. Xu, and S. Feng, “A survey of machine learning for big data processing,” EURASIP Journal on Advances in Signal Processing, vol. 2016, no. 1, p. 67, 2016.E. K. Wang, T.-Y. Wu, C.-M. Chen, Y. Ye, Z. Zhang, and F. Zou, "Malpas: Markov decision process based adaptive security for sensors in the internet of things," in Genetic and Evolutionary Computing, Springer, 2015, pp. 389–397.
  14. Mostafavi, M. Dehghan, "Game-theoretic Bandwidth Procurement Mechanisms in Live P2P Streaming Systems", Multimedia Tools and Applications, vol. 75, no. 14, pp. 8545-8568, 2016.
  15. S. Mostafavi, M. Dehghan, "Game-theoretic Auction Design for Bandwidth Sharing in Helper-assisted P2P Streaming", International Journal of Communication Systems, vol. 29, no. 6, pp. 1057-1072, 2016.
  16. S. M. Matinkhah and W. Shafik, “A Study on Financial Pricing and Applications models on 5G,” in 4th international Mathematical Conference and Modelling, pp. 52.
  17. W. Shafik and S. M. Matinkhah, “Admitting New Requests in Fog Networks According to Erlang B Distribution,” in 2019 27th Iranian Conference on Electrical Engineering (ICEE), 2019, pp. 2016–2021.
  18. S. M. Matinkhah, W. Shafik, and M. Ghasemzadeh, “Emerging Artificial Intelligence Application: Reinforcement Learning Issues on Current Internet of Things.” in 2019 16th international Conference in information knowledge and Technology(ikt2019), pp. 2019 ICIKT10_062.
  19. S. Mostafavi and W. Shafik, “Fog Computing Architectures, Privacy and Security Solutions,” J. Commun. Technol. Electron. Comput. Sci., vol. 24, pp. 1–14, 2019.
  20. W. Shafik, S. M. Matinkhah, and M. Ghasemazade, “Fog-Mobile Edge Performance Evaluation and Analysis on Internet of Things,” J. Adv. Res. Mob. Comput., vol. 1, no. 3.
  21. W. Shafik and S. M. Matinkhah, “How to use Erlang B to determine the blocking probability of packet loss in a wireless communication,” in presented at the 13th Symposium on Advances in Science & Technology, 2018.
  22. W. Shafik and S. M. Matinkhah, “Privacy Issues in Social Web of Things,” in 2019 5th International Conference on Web Research (ICWR), 2019, pp. 208–214.
  23. S.-H. Zahiri, “Learning automata-based classifier,” Pattern Recognition Letters, vol. 29, no. 1, pp. 40–48, 2008.
  24. B. Braune, S. Diehl, A. Kerren, and R. Wilhelm, “Animation of the generation and computation of finite automata for learning software,” in International Workshop on Implementing Automata, 1999, pp. 39–47.
  25. K. S. Narendra and M. A. Thathachar, “Learning automata-a survey,” IEEE Transactions on systems, man, and cybernetics, no. 4, pp. 323–334, 1974.
  26. A. K. Ghosh, C. Michael, and M. Schatz, “A real-time intrusion detection system based on learning program behavior,” in International Workshop on Recent Advances in Intrusion Detection, 2000, pp. 93–109.
  27. C. L. Giles, C. B. Miller, D. Chen, H.-H. Chen, G.-Z. Sun, and Y.-C. Lee, “Learning and extracting finite state automata with second-order recurrent neural networks,” Neural Computation, vol. 4, no. 3, pp. 393–405, 1992.
  28. B. B. Zarpelão, R. S. Miani, C. T. Kawakani, and S. C. de Alvarenga, “A survey of intrusion detection in Internet of Things,” Journal of Network and Computer Applications, vol. 84, pp. 25–37, 2017.
  29. A. Abduvaliyev, A.-S. K. Pathan, J. Zhou, R. Roman, and W.-C. Wong, “On the vital areas of intrusion detection systems in wireless sensor networks,” IEEE Communications Surveys & Tutorials, vol. 15, no. 3, pp. 1223–1237, 2013.
  30. R. Zhao, R. Yan, Z. Chen, K. Mao, P. Wang, and R. X. Gao, “Deep learning and its applications to machine health monitoring: A survey,” arXiv preprint arXiv:1612.07640, 2016.
  31. S. Misra, P. V. Krishna, H. Agarwal, A. Saxena, and M. S. Obaidat, “A learning automata-based solution for preventing distributed denial of service in Internet of things,” in 2011 International Conference on Internet of Things and 4th International Conference on Cyber, Physical and Social Computing, 2011, pp. 114–122.
  32. M. Weisman et al., “Machine Learning and Data Mining for IPv6 Network Defence,” in International Conference on Cyber Warfare and Security, 2018, pp. 681–XVI.
  33. W. Jiang, C.-L. Zhao, S.-H. Li, and L. Chen, “A new learning automata-based approach for online tracking of event patterns,” Neurocomputing, vol. 137, pp. 205–211, 2014.
  34. S. Raza, L. Wallgren, and T. Voigt, “SVELTE: Real-time intrusion detection in the Internet of Things,” Ad hoc networks, vol. 11, no. 8, pp. 2661–2674, 2013.
  35. Z. Yan, P. Zhang, and A. V. Vasilakos, “A survey on trust management for Internet of Things,” Journal of network and computer applications, vol. 42, pp. 120–134, 2014.
  36. M. A. Al-Garadi, A. Mohamed, A. Al-Ali, X. Du, and M. Guizani, “A survey of machine and deep learning methods for internet of things (IoT) security,” arXiv preprint arXiv:1807.11023, 2018.
  37. K. Zaheer, M. Othman, M. H. Rehmani, and T. Perumal, “A Survey of Decision-Theoretic Models for Cognitive Internet of Things (CIoT),” IEEE Access, vol. 6, pp. 22489–22512, 2018.
  38. L. Cao, G. Weiss, and S. Y. Philip, “A brief introduction to agent mining,” Autonomous Agents and Multi-Agent Systems, vol. 25, no. 3, pp. 419–424, 2012.
  39. O. B. Sezer, E. Dogdu, and A. M. Ozbayoglu, “Context-aware computing, learning, and big data in internet of things: a survey,” IEEE Internet of Things Journal, vol. 5, no. 1, pp. 1–27, 2018.
  40. C. Gomez, A. Shami, and X. Wang, “Machine Learning Aided Scheme for Load Balancing in Dense IoT Networks,” Sensors, vol. 18, no. 11, p. 3779, 2018.
  41. F. M. Al-Turjman, “Information-centric sensor networks for cognitive IoT: an overview,” Annals of Telecommunications, vol. 72, no. 1–2, pp. 3–18, 2017.
  42. M. A. Alsheikh, S. Lin, D. Niyato, and H.-P. Tan, “Machine learning in wireless sensor networks: Algorithms, strategies, and applications,” IEEE Communications Surveys & Tutorials, vol. 16, no. 4, pp. 1996–2018, 2014.
  43. K. Ye, “Key Feature Recognition Algorithm of Network Intrusion Signal Based on Neural Network and Support Vector Machine,” Symmetry, vol. 11, no. 3, p. 380, 2019.
  44. J. Abreu, L. Fred, D. Macêdo, and C. Zanchettin, “Hierarchical Attentional Hybrid Neural Networks for Document Classification,” arXiv preprint arXiv:1901.06610, 2019.
  45. K. Wang, “Network data management model based on Naïve Bayes classifier and deep neural networks in heterogeneous wireless networks,” Computers & Electrical Engineering, vol. 75, pp. 135–145, 2019.
  46. P. F. Fantoni, “A neuro-fuzzy model applied to full range signal validation of PWR nuclear power plant data,” INTERNATIONAL JOURNAL OF GENERAL SYSTEM, vol. 29, no. 2, pp. 305–320, 2000.
  47. M. Kahng, N. Thorat, D. H. P. Chau, F. B. Viégas, and M. Wattenberg, “GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation,” IEEE transactions on visualization and computer graphics, vol. 25, no. 1, pp. 310–320, 2019.
  48. D. Popa, F. Pop, C. Serbanescu, and A. Castiglione, “Deep learning model for home automation and energy reduction in a smart home environment platform,” Neural Computing and Applications, pp. 1–21, 2019.
  49. S. Baruah, “Botnet Detection: Analysis of Various Techniques,” International Journal of Computational Intelligence & IoT, vol. 2, no. 2, 2019, pp 7-14.
  50. A. Mollalo, L. Mao, P. Rashidi, and G. E. Glass, “A GIS-Based Artificial Neural Network Model for Spatial Distribution of Tuberculosis across the Continental United States,” International journal of environmental research and public health, vol. 16, no. 1, p. 157, 2019.
  51. R. V. McCarthy, M. M. McCarthy, W. Ceccucci, and L. Halawi, “Predictive Models Using Neural Networks,” in Applying Predictive Analytics, Springer, 2019, pp. 145–173.
  52. A. Rezvanian, B. Moradabadi, M. Ghavipour, M. M. D. Khomami, and M. R. Meybodi, “Introduction to Learning Automata Models,” in Learning Automata Approach for Social Networks, Springer, 2019, pp. 1–49.
  53. A. Rezvanian, B. Moradabadi, M. Ghavipour, M. M. D. Khomami, and M. R. Meybodi, “Wavefront Cellular Learning Automata: A New Learning Paradigm,” in Learning Automata Approach for Social Networks, Springer, 2019, pp. 51–74.
  54. S. Matwin, L. Tesei, and R. Trasarti, “Computational modelling and data-driven techniques for systems analysis,” Journal of Intelligent Information Systems, pp. 1–3, 2019.
  55. S. Misra, P. V. Krishna, V. Saritha, H. Agarwal, and A. Ahuja, “Learning automata-based multi-constrained fault-tolerance approach for effective energy management in smart grid communication network,” Journal of Network and Computer Applications, vol. 44, pp. 212–219, 2014.
  56. I. Erev and G. Barron, “On adaptation, maximization, and reinforcement learning among cognitive strategies.,” Psychological review, vol. 112, no. 4, p. 912, 2005.
Index Terms

Computer Science
Information Sciences

Keywords

Internet of Things Markov Decision Process Learning Automata Artificial neural networks Q-learning

Powered by PhDFocusTM