The week's pick
Random Articles
Reseach Article
Adaptive Congestion Control Protocol based on Meta-Reinforcement Learning for Data Communication Networks
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 186 - Number 65 |
| Year of Publication: 2025 |
| Authors: Mugoh Mwaura, Stephen Kiambi, Hezekiah Nganga |
10.5120/ijca2025924423
|
Mugoh Mwaura, Stephen Kiambi, Hezekiah Nganga . Adaptive Congestion Control Protocol based on Meta-Reinforcement Learning for Data Communication Networks. International Journal of Computer Applications. 186, 65 ( Feb 2025), 1-7. DOI=10.5120/ijca2025924423
Abstract
Congestion control protocols aim to optimize network capacity utilization and maximize throughput by selecting appropriate transmission rates for the sender. When approached as a Reinforcement Learning (RL) problem, congestion control policies derived from this framework exhibit superior performance compared to manually designed protocols. However, the practical implementation of RL algorithms encounters a significant challenge due to the dynamic nature of network conditions, which hampers their ability to generalize to new scenarios. This study presents a solution to this issue by considering the vast range of network conditions as an unknown task, treating it as a concealed variable that can be inferred from observed network history. By acquiring the capability to estimate this task as an underlying state and conditioning the protocol to respond accordingly, the proposed approach achieves continuous adaptation to evolving network conditions. The results demonstrate that this method not only enhances the utilization of network capacity in congestion control algorithms but also ensures protocol consistency across diverse network characteristics.
References
- Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, and Shimon Whiteson. A survey of meta-reinforcement learning, 2023.
- David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518):859–877, Apr 2017.
- Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Soheil Hassas Yeganeh, and Van Jacobson. Bbr: Congestionbased congestion control. Queue, 14(5):20–53, 2016.
- Hao-Tien Lewis Chiang, Aleksandra Faust, Marek Fiser, and Anthony Francis. Learning navigation behaviors endto- end with autorl. IEEE Robotics and Automation Letters, 4(2):2007–2014, 2019.
- Dah-Ming Chiu and Raj Jain. Analysis of the increase and decrease algorithms for congestion avoidance in computer networks. Computer Networks and ISDN Systems, 17(1):1–14, 1989.
- Justin Domke and Daniel Sheldon. Importance weighting and variational inference. CoRR, abs/1808.09034, 2018.
- Mo Dong, Tong Meng, Doron Zarchy, Engin Arslan, Yossi Gilad, Brighten Godfrey, and Michael Schapira. PCC vivace: Online-learning congestion control. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18), pages 343–356, Renton, WA, April 2018. USENIX Association.
- B. Everett. An Introduction to Latent Variable Models. Springer Science & Business Media, Mar 2013.
- Benjamin Eysenbach, Tianjun Zhang, Ruslan Salakhutdinov, and Sergey Levine. Contrastive learning as goal-conditioned reinforcement learning, 2023.
- Chelsea Finn, Pieter Abbeel, and Sergey Levine. Modelagnostic meta-learning for fast adaptation of deep networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1126–1135. PMLR, 06–11 Aug 2017.
- S. Floyd and K. Fall. Promoting the use of end-to-end congestion control in the internet. IEEE/ACM Transactions on Networking, 7(4):458–472, 1999.
- Sangtae Ha, Injong Rhee, and Lisong Xu. Cubic: A new tcpfriendly high-speed tcp variant. ACM SIGOPS Operating Systems Review, 42(5):64–74, 2008.
- T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey. Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(09):5149–5169, sep 2022.
- Chia-Chun Hung, Timothy Lillicrap, Josh Abramson, Yan Wu, Mehdi Mirza, Federico Carnevale, Arun Ahuja, and Greg Wayne. Optimizing agent behavior over long time scales by transporting value. Nature Communications, 10(1), 2019.
- Nathan Jay, Noga Rotman, Brighten Godfrey, Michael Schapira, and Aviv Tamar. A deep reinforcement learning perspective on internet congestion control. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 3050–3059. PMLR, 09–15 Jun 2019.
- Huiling Jiang, Qing Li, Yong Jiang, GengBiao Shen, Richard Sinnott, Chen Tian, and Mingwei Xu. When machine learning meets congestion control: A survey and comparison. Computer Networks, 192:108033, 2021.
- Wei Li, Fan Zhou, Kaushik Roy Chowdhury, and Waleed Meleis. Qtcp: Adaptive congestion control with reinforcement learning. IEEE Transactions on Network Science and Engineering, 6(3):445–458, 2019.
- Daniel J. Mankowitz, Nir Levine, Rae Jeong, Yuanyuan Shi, Jackie Kay, Abbas Abdolmaleki, Jost Tobias Springenberg, Timothy Mann, Todd Hester, and Martin Riedmiller. Robust reinforcement learning for continuous control with model misspecification, 2020.
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning, 2013.
- Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. 2018 IEEE International Conference on Robotics and Automation (ICRA), May 2018.
- R. Prosad, C. Davrolis, M. Murray, and K.c. Claffy. Bandwidth estimation: metrics, measurement techniques, and tools. IEEE Network, 17(6):27–35, 2003.
- Kate Rakelly, Aurick Zhou, Deirdre Quillen, Chelsea Finn, and Sergey Levine. Efficient off-policy meta-reinforcement learning via probabilistic context variables, 2019.
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017.
- Avi Singh, Huihan Liu, Gaoyue Zhou, Albert Yu, Nicholas Rhinehart, and Sergey Levine. Parrot: Data-driven behavioral priors for reinforcement learning, 2020.
- Aravind Srinivas, Michael Laskin, and Pieter Abbeel. Curl: Contrastive unsupervised representations for reinforcement learning, 2020.
- Richard S. Sutton, Francis Bach, and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press Ltd, 2018.
- Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning (reinforce). Reinforcement Learning, page 5–32, 1992.
- Zhenchang Xia, Libing Wu, Fei Wang, Xudong Liao, Haiyan Hu, Jia Wu, and Dan Wu. Glider: rethinking congestion control with deep reinforcement learning. World Wide Web, 26(1):115–137, Jun 2022.
- Francis Y. Yan, Jestin Ma, Greg D. Hill, Deepti Raghavan, Riad S. Wahby, Philip Levis, and Keith Winstein. Pantheon: the training ground for internet congestion-control research. In 2018 USENIX Annual Technical Conference (USENIX ATC 18), pages 731–743, Boston, MA, July 2018. USENIX Association.
- Amy Zhang, Rowan McAllister, Roberto Calandra, Yarin Gal, and Sergey Levine. Learning invariant representations for reinforcement learning without reconstruction, 2021.
- Tony Z. Zhao, Anusha Nagabandi, Kate Rakelly, Chelsea Finn, and Sergey Levine. Meld: Meta-reinforcement learning from images via latent state models, 2021.
Index Terms
Keywords