QoS Optimization Technique with Deep Reinforcement Learning in SDN-Based IoT

Majlesi Journal of Electrical Engineering

مقاله 10، دوره 15، شماره 3، آذر 2021، صفحه 105-113   

نوع مقاله: Review Article

شناسه دیجیتال (DOI): 10.52547/mjee.15.3.105

نویسندگان

Mohammadreza Moslehi^{* 1}؛ Hossein Ebrahimpor-Komleh¹؛ Salman Goli¹؛ Reza Taji²

¹Department of Computer Engineering, University of Kashan, Kashan, Iran.

²Independent Researcher in the Field of AI and Neural Networking.

چکیده

In recent years, exponential growth of communication devices in Internet of Things (IoT) has become an emerging technology which facilitates heterogeneous devices to connect with each other in heterogeneous networks. This communication requires different level of Quality-of-Service (QoS) and policies depending on the device type and location. To provide a specific level of QoS, we can utilize emerging new technological concepts in IoT infrastructure, software-defined network (SDN) and, machine learning algorithms. We use deep reinforcement learning in the process of resource management and allocation in control plane. We present an algorithm that aims to optimize resource allocation. Simulation results show that the proposed algorithm improved network performances in terms of QoS parameters, including delay and throughput compared to Random and Round Robin methods. Compared to similar methods the performance of the proposed method is also as good as the fuzzy and predictive methods.

کلیدواژه‌ها

Internet of Things؛ Software-Defined Networking (SDN)؛ Deep Reinforcement Learning؛ Qos

مراجع

[1] “Cisco Annual Internet Report - Cisco Annual Internet Report (2018–2023) White Paper - Cisco.” [Online]. Available: https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html. [Accessed: 23-Feb-2020].

[2] “Software-Defined Networking (SDN) Definition - Open Networking Foundation.” [Online]. Available: https://opennetworking.org/sdn-definition/. [Accessed: 07-Dec-2020].

[3] “Software-Defined Networking (SDN) Definition - Open Networking Foundation.” [Online]. Available: https://www.opennetworking.org/sdn-definition/?nab=1&utm_referrer=https%3A%2F%2Fsdn.itrc.ac.ir%2F%3Fq%3Dfa%2Fcontent%2Fsdn. [Accessed: 08-Jan-2020].

[4] “Examples of Network Programmability and SDN > Introduction to Controller-Based Networking | Cisco Press.” [Online]. Available: https://www.ciscopress.com/articles/article.asp?p=2995354&seqNum=3. [Accessed: 07-Dec-2020].

[5] A. Montazerolghaem and M. H. Yaghmaee, “Load-Balanced and QoS-Aware Software-Defined Internet of Things,” IEEE Internet Things J., vol. 7, no. 4, pp. 3323–3337, Apr. 2020.

[6] S. Din, M. M. Rathore, A. Ahmad, A. Paul, and M. Khan, “SDIoT: Software Defined Internet of Thing to Analyze Big Data in Smart Cities,” in Proceedings - 2017 IEEE 42nd Conference on Local Computer Networks Workshops, LCN Workshops 2017, 2017, pp. 175–182.

[7] H. E.-K. S. G.-B. Mohammadreza Moslehi, “Improving QoS using Software-Defined Networking for Smart City (SDSC),” Int. J. Futur. Gener. Commun. Netw., vol. 13, no. 3, pp. 2757–2767–2757–2767, Aug. 2020.

[8] S. K. Tayyaba, M. A. Shah, O. A. Khan, and A. W. Ahmed, “Software Defined Network (SDN) Based Internet of Things (IoT),” in Proceedings of the International Conference on Future Networks and Distributed Systems - ICFNDS ’17, 2017, pp. 1–8.

[9] D. Sinh, L. V. Le, B. S. P. Lin, and L. P. Tung, “SDN/NFV - A new approach of deploying network infrastructure for IoT,” in 2018 27th Wireless and Optical Communication Conference, WOCC 2018, 2018, pp. 1–5.

[10] Y. Njah, C. Pham, and M. Cheriet, “Service and Resource Aware Flow Management Scheme for an SDN-Based Smart Digital Campus Environment,” IEEE Access, vol. 8, pp. 119635–119653, 2020.

[11] X. Guo, H. Lin, Z. Li, and M. Peng, “Deep-Reinforcement-Learning-Based QoS-Aware Secure Routing for SDN-IoT,” IEEE Internet Things J., vol. 7, no. 7, pp. 6242–6251, Dec. 2019.

[12] K. S. Bhandari, I. H. Ra, and G. Cho, “Multi-Topology Based QoS-Differentiation in RPL for Internet of Things Applications,” IEEE Access, vol. 8, pp. 96686–96705, 2020.

[13] G. C. Deng and K. Wang, “An Application-aware QoS Routing Algorithm for SDN-based IoT Networking,” in Proceedings - IEEE Symposium on Computers and Communications, 2018, vol. 2018-June, pp. 186–191.

[14] K. Streit, C. Schmitt, and C. Giannelli, “SDN-Based Regulated Flow Routing in MANETs,” 2020, pp. 73–80.

[15] N. Hu, F. Luan, X. Tian, and C. Wu, “A Novel SDN-Based Application-Awareness Mechanism by Using Deep Learning,” IEEE Access, vol. 8, pp. 160921–160930, 2020.

[16] M. Naeem, S. T. H. Rizvi, and A. Coronato, “A Gentle Introduction to Reinforcement Learning and its Application in Different Fields,” IEEE Access, vol. 8, pp. 209320–209344, Nov. 2020.

[17] Yichen Qian, Jun Wu, Rui Wang, Fusheng Zhu, and Wei Zhang, “Survey on Reinforcement Learning Applications in Communication Networks,” J. Commun. Inf. Networks, vol. 4, no. 2, pp. 30–39, 2019.

[18] A. Alharin, T.-N. Doan, and M. Sartipi, “Reinforcement Learning Interpretation Methods: A Survey,” IEEE Access, vol. 8, pp. 171058–171077, Sep. 2020.

[19] V. Mnih et al., “Playing Atari with Deep Reinforcement Learning,” Dec. 2013.

[20] V. Mnih et al., “Human-level control through deep reinforcement learning.,” Nature, vol. 518, no. 7540, pp. 529–33, Feb. 2015.

[21] Y. He, N. Zhao, and H. Yin, “Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach,” IEEE Trans. Veh. Technol., vol. 67, no. 1, pp. 44–55, Jan. 2018.

[22] R. S. Sutton and A. G. Barto, “Reinforcement Learning: An Introduction Second edition, in progress.”

[23] “Mininet: An Instant Virtual Network on your Laptop (or other PC) - Mininet.” [Online]. Available: http://mininet.org/. [Accessed: 22-Mar-2020].

[24] M. Runsungnoen and T. Anusas-amornkul, “Round Robin Scheduling Based on Remaining Time and Median (RR_RT&M) for Cloud Computing,” in Smart Innovation, Systems and Technologies, 2020, vol. 165, pp. 21–29.

[25] “RFC 3549 - Linux Netlink as an IP Services Protocol.” [Online]. Available: https://datatracker.ietf.org/doc/rfc3549/. [Accessed: 11-Dec-2020].

[26] A. Kuznetsov, J. Salim, A. Kleen, and H. Khosravi, “Linux Netlink as an IP Services Protocol.”

[27] “sFlow.org - Making the Network Visible.” [Online]. Available: https://sflow.org/index.php. [Accessed: 26-Nov-2020].

[28] “sFlow-RT.” [Online]. Available: https://sflow-rt.com/. [Accessed: 17-Dec-2020].

[29] “Gym.” [Online]. Available: https://gym.openai.com/. [Accessed: 11-Nov-2020].

[30] “MQTTset | Kaggle.” [Online]. Available: https://www.kaggle.com/cnrieiit/mqttset. [Accessed: 26-Dec-2020].

[31] “esnet/iperf: iperf3: A TCP, UDP, and SCTP network bandwidth measurement tool.” [Online]. Available: https://github.com/esnet/iperf. [Accessed: 05-Dec-2020].