CFP last date
21 April 2025
Call for Paper
May Edition
IJCA solicits high quality original research papers for the upcoming May edition of the journal. The last date of research paper submission is 21 April 2025

Submit your paper
Know more
The week's pick
Responsive image
Serverless Architecture for Real-Time Stock Market Data Analytics in Cloud Environments

Gautam Solaimalai
Random Articles
Reseach Article

Distributed Deep Reinforcement Learning for Decentralized Autonomous Vehicle Coordination in Urban Environment

by Isha Das, Md. Jisan Ahmed
journal cover thumbnail

International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 186 - Number 75
Year of Publication: 2025
Authors: Isha Das, Md. Jisan Ahmed
10.5120/ijca2025924643

Isha Das, Md. Jisan Ahmed . Distributed Deep Reinforcement Learning for Decentralized Autonomous Vehicle Coordination in Urban Environment. International Journal of Computer Applications. 186, 75 ( Mar 2025), 43-51. DOI=10.5120/ijca2025924643

@article{ 10.5120/ijca2025924643,
author = { Isha Das, Md. Jisan Ahmed },
title = { Distributed Deep Reinforcement Learning for Decentralized Autonomous Vehicle Coordination in Urban Environment },
journal = { International Journal of Computer Applications },
issue_date = { Mar 2025 },
volume = { 186 },
number = { 75 },
month = { Mar },
year = { 2025 },
issn = { 0975-8887 },
pages = { 43-51 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume186/number75/distributed-deep-reinforcement-learning-for-decentralized-autonomous-vehicle-coordination-in-urban-environment/ },
doi = { 10.5120/ijca2025924643 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2025-03-25T22:41:46.506087+05:30
%A Isha Das
%A Md. Jisan Ahmed
%T Distributed Deep Reinforcement Learning for Decentralized Autonomous Vehicle Coordination in Urban Environment
%J International Journal of Computer Applications
%@ 0975-8887
%V 186
%N 75
%P 43-51
%D 2025
%I Foundation of Computer Science (FCS), NY, USA
Abstract

This research tackles the problem of coordinating self-driving vehicles in crowded cities using a decentralized strategy based on deep reinforcement learning. This research seeks to design a smart and sturdy framework that can assist numerous agents in driving differently in real time, contributing to reduced crowding and higher safety overall. Each automobile may smartly work together with surrounding cars without a centralized controller by making local judgments and with selective information sharing. The results demonstrate that crashes and average traveling time considerably diminish in different traffic circumstances. This would enhance traffic flow and possibly enable self-organizing traffic systems. City planners and car manufacturers can employ this decentralized strategy for major traffic control schemes, which can help in smooth commuting and better load on infrastructure. Unlike what’s been done before, this work provides a unique aspect by emphasizing on-the-fly flexibility and strong reward shaping in a truly distributed architecture. The study’s distinctive contribution is in proving that coordination of multi-agents can be performed and sustained despite communication latencies as well as large vehicle densities. The suggested technology permits on-the-fly collaboration among autonomous cars, a critical step towards safer, greener, and more vibrant urban travel.

References
  1. Li, D., Zhu, F., Chen, T., Wong, Y. D., Zhu, C., & Wu, J. (2023). COOR-PLT: A hierarchical control model for coordinating adaptive platoons of connected and autonomous vehicles at signal-free intersections based on deep reinforcement learning. Transportation Research Part C: Emerging Technologies, 146, 103933.
  2. Qian, X. (2016). Model predictive control for autonomous and cooperative driving (Doctoral dissertation, Université Paris sciences et lettres).
  3. M. Wiering, F. van Veenen, J. van de Walle, and A. Koopman, “Intelligent traffic light control,” Mach. Learn., vol. 105, no. 1, pp. 41–62, 2016.
  4. X. Li, G. Zhao, Z. Kong, and C. Wu, “Toward robust multi-agent cooperation for connected autonomous vehicles: A novel single-point-of-failure detection approach,” IEEE Trans. Veh. Technol., vol. 70, no. 6, pp. 5501–5512, Jun. 2021.
  5. T. Chu, J. Wang, L. Codecà, and Z. Li, “Multi-agent deep reinforcement learning for large-scale traffic signal control,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 3, pp. 1086–1095, Mar. 2020.
  6. Z. Zhu, X. Yu, and Y. Wang, “Deep reinforcement learning for intelligent traffic light control in IoT-based smart transportation,” IEEE Internet Things J., vol. 7, no. 6, pp. 5557–5566, Jun. 2020.
  7. Y. Feng, S. Ruan, Q. Yang, and B. Ran, “Adaptive cruise control strategy based on deep reinforcement learning under varying traffic flow,” in Proc. IEEE Int. Conf. Intell. Transp. Syst. (ITSC), Auckland, New Zealand, 2019, pp. 2369–2374.
  8. Y. Ye, W. He, and G. Xiong, “Hierarchical multi-agent reinforcement learning approach for connected automated driving in large-scale partial centralized environment,” in Proc. 23rd IEEE Int. Conf. Intell. Transp. Syst. (ITSC), Rhodes, Greece, 2020, pp. 1236–1243.
  9. J. Zhang, A. Kumar, and R. Gupta, “Fully decentralized reinforcement learning for connected autonomous vehicle coordination,” in Proc. IEEE Global Conf. Signal Inf. Process., Atlanta, GA, USA, 2021, pp. 435–439.
  10. H. Liu, T. Gao, M. Li, and C. Wu, “Scalable multi-agent deep reinforcement learning in resource-constrained environments for urban traffic management,” Transp. Res. Part C Emerg. Technol., vol. 132, p. 103420, Nov. 2021.
  11. T. Chu, J. Wang, L. Codeca, and Z. Li, “Multi-agent deep reinforcement learning for large-scale traffic signal control,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 3, pp. 1086–1095, Mar. 2020.
  12. Z. Zhu, X. Yu, and Y. Wang, “Deep reinforcement learning for intelligent traffic light control in IoT-based smart transportation,” IEEE Internet Things J., vol. 7, no. 6, pp. 5557–5566, Jun. 2020.
  13. Y. Feng, S. Ruan, Q. Yang, and B. Ran, “Adaptive cruise control strategy based on deep reinforcement learning under varying traffic flow,” in Proc. IEEE Int. Conf. Intell. Transp. Syst. (ITSC), Auckland, New Zealand, 2019, pp. 2369–2374.
  14. K. Kamal, J. Imura, and K. Aihara, “Multi-vehicle cooperative driving using hierarchical model predictive control with dynamic equilibrium approach,” IET Intell. Transp. Syst., vol. 14, no. 1, pp. 40–48, Jan. 2020.
  15. A. A. Malikopoulos, “Optimal coordination of connected and automated vehicles at intersections with mixed traffic: An emergent behavior approach,” IEEE Trans. Intell. Veh., vol. 6, no. 4, pp. 592–602, Dec. 2021.
  16. X. Li, G. Zhao, Z. Kong, and C. Wu, “Toward robust multi-agent cooperation for connected autonomous vehicles: A novel single-point-of-failure detection approach,” IEEE Trans. Veh. Technol., vol. 70, no. 6, pp. 5501–5512, Jun. 2021.
  17. A. A. Ghazanfari and M. R. Khalili, “A distributed reinforcement learning-based method for multi-intersection traffic signal control,” in Proc. 23rd IEEE Int. Conf. Intell. Transp. Syst. (ITSC), Rhodes, Greece, 2020, pp. 1236–1243.
  18. Y. Ye, W. He, and G. Xiong, “Hierarchical multi-agent reinforcement learning approach for connected automated driving in large-scale partial centralized environment,” in Proc. 23rd IEEE Int. Conf. Intell. Transp. Syst. (ITSC), Rhodes, Greece, 2020, pp. 1256–1263.
  19. H. van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double Q-learning,” in Proc. AAAI Conf. Artif. Intell., Phoenix, AZ, USA, 2016, pp. 2094–2100.
  20. D. Silver et al., “Mastering the game of Go without human knowledge,” Nature, vol. 550, pp. 354–359, 2017.
  21. Y. Zhou, S. Feng, Y. Wen, and D. O. Wu, “Adaptive and efficient deep reinforcement learning for large-scale multi-agent systems,” IEEE Trans. Parallel Distrib. Syst., vol. 31, no. 10, pp. 2394–2407, Oct. 2020.
  22. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
  23. R. Lowe et al., “Multi-agent actor-critic for mixed cooperative-competitive environments,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS), Long Beach, CA, USA, 2017, pp. 6379–6390.
  24. J. Zhang, A. Kumar, and R. Gupta, “Fully decentralized reinforcement learning for connected autonomous vehicle coordination,” in Proc. IEEE Global Conf. Signal Inf. Process., Atlanta, GA, USA, 2021, pp. 435–439.
  25. P. Foerster, N. Nardelli, G. Farquhar, P. H. de Witt, T. P. Kohli, and S. Whiteson, “Stabilising experience replay for deep multi-agent reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), Stockholm, Sweden, 2018, pp. 1146–1155.
  26. T. Rashid, M. Samvelyan, C. S. de Witt, G. Farquhar, J. N. Foerster, and S. Whiteson, “QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), Stockholm, Sweden, 2018, pp. 4292–4301.
  27. A. A. Bukhari, H. C. Kim, and S. Kim, “Parameter sharing-based deep reinforcement learning for multi-agent cooperative collision avoidance,” IEEE Access, vol. 8, pp. 107850–107861, Jun. 2020.
  28. N. V. Long, S. T. Quoc, and M. Kojima, “Incorporating partial observability in multi-agent reinforcement learning for intelligent transportation systems,” in Proc. 16th Int. Conf. Control, Autom. Syst. (ICCAS), Gyeongju, Korea, 2016, pp. 724–729.
  29. M. Wiering, F. van Veenen, J. van de Walle, and A. Koopman, “Intelligent traffic light control,” Mach. Learn., vol. 105, no. 1, pp. 41–62, Apr. 2016.
  30. C. Bouton, D. G. Lawrence, S. K. Rathinam, and P. R. Pagilla, “Analysis of safe and scalable vehicle platooning,” IEEE Trans. Intell. Transp. Syst., vol. 19, no. 9, pp. 2844–2855, Sep. 2018.
  31. Q. Wang, K. Liu, J. Ma, J. Cao, and H. Jin, “Federated reinforcement learning for cooperative traffic signal control,” in Proc. IEEE 40th Int. Conf. Distrib. Comput. Syst. (ICDCS), Singapore, 2020, pp. 688–697.
  32. B. Yan, L. Lin, Y. Chen, and Y. Wang, “Edge intelligence-empowered multi-intersection signal control: A federated reinforcement learning approach,” IEEE Internet Things J., vol. 8, no. 8, pp. 6481–6491, Apr. 2021.
  33. E. Tolstaya et al., “Learning decentralized controllers for robot swarms with graph neural networks,” in Proc. Conf. Robot Learn. (CoRL), Osaka, Japan, 2019, pp. 671–680.
  34. C. Wu, Z. Zhang, H. Wang, and L. Chen, “Adaptive distributed multi-agent reinforcement learning for real-time traffic signal control,” Transp. Res. Part C Emerg. Technol., vol. 125, p. 103058, May 2021.
  35. G. Christopoulos, S. Jin, H. Abou-zeid, and M. Verhelst, “Rethinking resource allocation for low-latency multi-agent deep reinforcement learning in V2X networks,” in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC), Austin, TX, USA, 2022, pp. 492–497.
  36. S. E. Li, X. Qin, K. Li, and J. Wang, “Coordinated platoon control of connected vehicles in mixed traffic: A deep reinforcement learning approach,” Transp. Res. Part C Emerg. Technol., vol. 102, pp. 1–12, Mar. 2019.
  37. A. Bhattacharya, T. Basar, D. Bauso, and C. Langbort, “Distributed reinforcement learning for multi-agent networked systems with unknown dynamics,” IEEE Trans. Autom. Control, vol. 65, no. 12, pp. 4965–4980, Dec. 2020.
  38. H. Liu, T. Gao, M. Li, and C. Wu, “Scalable multi-agent deep reinforcement learning in resource-constrained environments for urban traffic management,” Transp. Res. Part C Emerg. Technol., vol. 132, p. 103420, Nov. 2021.
  39. Y. J. Kim and B. Kim, “GAN-based traffic data augmentation strategy for robust multi-agent reinforcement learning in varying traffic conditions,” IEEE Access, vol. 8, pp. 111967–111978, Jun. 2020.
  40. A. O. Al-Ani, S. L. Smith, and J. Recker, “Multi-agent deep reinforcement learning for robust cooperative driving in congested urban scenarios,” in Proc. IEEE Intell. Veh. Symp. (IV), Las Vegas, NV, USA, 2020, pp. 479–486.
Index Terms

Computer Science
Information Sciences

Keywords

Autonomous vehicle coordination multi-agent systems deep reinforcement learning traffic management decentralized autonomy connected cars

Powered by PhDFocusTM