CFP last date
21 April 2025
Call for Paper
May Edition
IJCA solicits high quality original research papers for the upcoming May edition of the journal. The last date of research paper submission is 21 April 2025

Submit your paper
Know more
The week's pick
Responsive image
Serverless Architecture for Real-Time Stock Market Data Analytics in Cloud Environments

Gautam Solaimalai
Random Articles
Reseach Article

Critical Challenges of Continuous Integration and Testing (CI/CT) in DevOps: A Systematic Literature Review Protocol with preliminary Results

by Anum Bahar, Muhammad Yaseen, Muhammad Asif Nauman, Amara Parveen
journal cover thumbnail

International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 186 - Number 68
Year of Publication: 2025
Authors: Anum Bahar, Muhammad Yaseen, Muhammad Asif Nauman, Amara Parveen
10.5120/ijca2025924521

Anum Bahar, Muhammad Yaseen, Muhammad Asif Nauman, Amara Parveen . Critical Challenges of Continuous Integration and Testing (CI/CT) in DevOps: A Systematic Literature Review Protocol with preliminary Results. International Journal of Computer Applications. 186, 68 ( Feb 2025), 29-41. DOI=10.5120/ijca2025924521

@article{ 10.5120/ijca2025924521,
author = { Anum Bahar, Muhammad Yaseen, Muhammad Asif Nauman, Amara Parveen },
title = { Critical Challenges of Continuous Integration and Testing (CI/CT) in DevOps: A Systematic Literature Review Protocol with preliminary Results },
journal = { International Journal of Computer Applications },
issue_date = { Feb 2025 },
volume = { 186 },
number = { 68 },
month = { Feb },
year = { 2025 },
issn = { 0975-8887 },
pages = { 29-41 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume186/number68/critical-challenges-of-continuous-integration-and-testing-cict-in-devops-a-systematic-literature-review-protocol-with-preliminary-results/ },
doi = { 10.5120/ijca2025924521 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2025-02-25T22:58:09.079216+05:30
%A Anum Bahar
%A Muhammad Yaseen
%A Muhammad Asif Nauman
%A Amara Parveen
%T Critical Challenges of Continuous Integration and Testing (CI/CT) in DevOps: A Systematic Literature Review Protocol with preliminary Results
%J International Journal of Computer Applications
%@ 0975-8887
%V 186
%N 68
%P 29-41
%D 2025
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Continuous Integration (CI) and Continuous Testing (CT) are essential components in DevOps for improving software delivery efficiency and quality. However, the implementation of CI and CT in DevOps settings faces considerable challenges that affect scalability, efficiency and dependability. This Systematic Literature Review (SLR) protocol aims to precise and examine the key challenges in adopting CI/CT. The investigation revolves around one primary research questions: recognizing challenges, evaluating their effects, creating an assessment framework, and confirming findings through case studies. The study employs a methodical approach, encompassing review planning, execution, and reporting. An initial screening process evaluated 2,200 research papers, reducing them to 100 pertinent studies that offer valuable insights into the most pressing issues impacting CI/CT. Preliminary findings reveal recurring challenges such as automation issues, integration challenges, test instability, scalability limitations, security weaknesses, and the absence of standardized procedures. These challenges hinder smooth software development and deployment, underscoring the necessity for additional research and enhancements in CI/CT practices. Through a systematic analysis of these challenges, this study contributes to a more comprehensive understanding of CI/CT constraints in DevOps. The results provide a foundation for future research, assisting organizations and researchers in developing more effective strategies to enhance software development processes and optimize CI/CT workflows.

References
  1. None Suprit Pattanayak, N. P. Murthy, and N. A. Mehra, “Integrating AI into DevOps pipelines: Continuous integration, continuous delivery, and automation in infrastructural management: Projections for future,” International Journal of Science and Research Archive, vol. 13, no. 1, pp. 2244–2256, Oct. 2024, doi: https://doi.org/10.30574/ijsra.2024.13.1.1838.
  2. D. Pandey and N. Parmar, “Unleashing the Power of DevOps: Revolutionizing Software Development Practices,” International Journal of Scientific Research in Computer Science Engineering and Information Technology, vol. 10, no. 2, pp. 767–772, Apr. 2024, doi: https://doi.org/10.32628/cseit24102121.
  3. J. M. Ali, “DevOps and continuous integration/continuous deployment (CI/CD) automation,” Adv. Eng. Innov., vol. 4, no. 1, pp. 38–42, Nov. 2023, doi: 10.54254/2977-3903/4/2023031.
  4. A. Nalamwar and P. Sirsat, “An Empirical Study on DevOps,” vol. 3, no. 11, 2019.
  5. B. Erdenebat, B. Bud, T. Batsuren, and T. Kozsik, “Multi-Project Multi-Environment Approach—An Enhancement to Existing DevOps and Continuous Integration and Continuous Deployment Tools,” Computers, vol. 12, no. 12, p. 254, Dec. 2023, doi: https://doi.org/10.3390/computers12120254.
  6. A. Qumer Gill, A. Loumish, I. Riyat, and S. Han, “DevOps for information management systems,” VINE Journal of Information and Knowledge Management Systems, vol. 48, no. 1, pp. 122–139, Feb. 2018, doi: https://doi.org/10.1108/vjikms-02-2017-0007.
  7. X. Zhou et al., “Revisit security in the era of DevOps: An evidence‐based inquiry into DevSecOps industry,” IET Software, vol. 17, no. 4, pp. 435–454, Aug. 2023, doi: https://doi.org/10.1049/sfw2.12132.
  8. Harshali Rohit Kadaskar, “UNLEASHING THE POWER OF DEVOPS IN SOFTWARE DEVELOPMENT,” Int. J. Sci. Res. Mod. Sci. Technol., vol. 3, no. 3, pp. 01–07, Mar. 2024, doi: 10.59828/ijsrmst.v3i3.185.
  9. L. Prates, J. Faustino, M. Silva, and R. Pereira, “DevSecOps Metrics,” Information Systems: Research, Development, Applications, Education, pp. 77–90, 2019, doi: https://doi.org/10.1007/978-3-030-29608-7_7.
  10. F. Almeida, J. Simões, and S. Lopes, “Exploring the Benefits of Combining DevOps and Agile,” Future Internet, vol. 14, no. 2, p. 63, Feb. 2022, doi: https://doi.org/10.3390/fi14020063.
  11. A. Alnafessah, A. U. Gias, R. Wang, L. Zhu, G. Casale, and A. Filieri, “Quality-Aware DevOps Research: Where Do We Stand?,” IEEE Access, vol. 9, pp. 44476–44489, 2021, doi: https://doi.org/10.1109/access.2021.3064867.
  12. D. Srivastava, M. Verma, Shashank Sheshar, and M. Gupta, “DevOps Tools: Silver Bullet for Software Industry,” Studies in computational intelligence, pp. 105–118, Dec. 2022, doi: https://doi.org/10.1007/978-981-19-6290-5_6.
  13. M. Gokarna, “DevOps phases across Software Development Lifecycle.” institute of Electrical Electronics Engineers, Jan. 06, 2021. doi: 10.36227/techrxiv.13207796.v2.
  14. Chellamalla Mamatha and S C V S L S Ravi Kiran, “Implementation of DevOps Architecture in the project development and deployment with help of tools,” International Journal of Scientific Research in Computer Science and Engineering, vol. 6, no. 2, pp. 87–95, Apr. 2018, doi: https://doi.org/10.26438/ijsrcse/v6i2.8795.
  15. A. Sen, “DevOps, DevSecOps, AIOPS- Paradigms to IT Operations,” springer singapore, 2020, pp. 211–221. doi: 10.1007/978-981-15-7804-5_16.
  16. E. Naresh, S. V. N. Murthy, N. Sreenivasa, S. Merikapudi, and C. R. Rakhi Krishna, “Continuous Integration, Testing Deployment and Delivery in Devops,” 2024 International Conference on Knowledge Engineering and Communication Systems (ICKECS), pp. 1–4, Apr. 2024, doi: https://doi.org/10.1109/ickecs61492.2024.10616918.
  17. P. G. Gowda, Stanley, and J. E. Joyce, “DevOps Dynamics: Tools Driving Continuous Integration and Deployment,” pp. 1–7, Jun. 2024, doi: https://doi.org/10.1109/iciteics61368.2024.10624986.
  18. K. Nayak, S. Route, M. Sundararajan, A. Jain, and Shashidhar R, “Sustainable Continuous Testing in DevOps Pipeline,” May 2024, doi: https://doi.org/10.1109/incccs60947.2024.10593566.
  19. C. Feio, N. Santos, N. Escravana, and B. Pacheco, “An Empirical Study of DevSecOps Focused on Continuous Security Testing,” doi: https://doi.org/10.1109/EuroSPW61312.2024.00074.
  20. “Compliance and Audit Challenges in DevOps: A Security Perspective,” International Research Journal of Modernization in Engineering Technology and Science, Oct. 2023, doi: https://doi.org/10.56726/irjmets45309.
  21. A. Hemon, B. Lyonnet, F. Rowe, and B. Fitzgerald, “Conceptualizing the Transition from Agile to DevOps: A Maturity Model for a Smarter IS Function,” inria.hal.science, Jun. 25, 2018.
  22. B. Erdenebat, B. Bud, T. Batsuren, and T. Kozsik, “Multi-Project Multi-Environment Approach—An Enhancement to Existing DevOps and Continuous Integration and Continuous Deployment Tools,” Computers, vol. 12, no. 12, p. 254, Dec. 2023, doi: https://doi.org/10.3390/computers12120254.
  23. J. Zulkarnain, R. F. Mulya, T. Pratiwi, W. Pangestuti, and F. I. Ilmawati, “DevOps Main Area and Core Capabilities Adopting DevOps in the Last Decade: A Systematic Literature Review,” International Journal of Research and Applied Technology (INJURATECH), vol. 2, no. 2, pp. 184–197, Nov. 2022, doi: https://doi.org/10.34010/injuratech.v2i2.8364.
  24. D. A. Meedeniya, I. D., and I. Perera, “Traceability Establishment and Visualization of Software Artefacts in DevOps Practice: A Survey,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 7, 2019, doi: https://doi.org/10.14569/ijacsa.2019.0100711.
  25. “Compliance and Audit Challenges in DevOps: A Security Perspective,” International Research Journal of Modernization in Engineering Technology and Science, Oct. 2023, doi: https://doi.org/10.56726/irjmets45309.
  26. Dinesh Reddy Chittibala, “Web Application Firewalls (WAF) Integration in DevOps Practices: A Scholarly Exploration of Security, Automation and Continuous Protection,” pp. 1–5, Dec. 2022, doi: https://doi.org/10.47363/jaicc/2022(1)210.
  27. F. M. A. Erich, C. Amrit, and M. Daneva, “A qualitative study of DevOps usage in practice,” Journal of Software: Evolution and Process, vol. 29, no. 6, p. e1885, Jun. 2017, doi: https://doi.org/10.1002/smr.1885.
  28. C. Vassallo, S. Proksch, H. C. Gall, and Massimiliano Di Penta, “Automated Reporting of Anti-Patterns and Decay in Continuous Integration,” May 2019, doi: https://doi.org/10.1109/icse.2019.00028.
  29. Y. Li, J. Wang, Q. Wang, and J. Hu, “A Class-level Test Selection Approach Toward Full Coverage For Continuous Integration,” Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering, vol. 2019, pp. 49–54, Jul. 2019, doi: https://doi.org/10.18293/seke2019-011.
  30. F. Moyón, R. Soares, M. Pinto-Albuquerque, D. Mendez, and K. Beckers, “Integration of Security Standards in DevOps Pipelines: An Industry Case Study,” Product-Focused Software Process Improvement, pp. 434–452, 2020, doi: https://doi.org/10.1007/978-3-030-64148-1_27.
  31. Amit, Bhanushali. (2023). 2. Challenges and Solutions in Implementing Continuous Integration and Continuous Testing for Agile Quality Assurance. International journal of science and research, doi: 10.21275/sr231021114758
  32. G. Rothermel, “Improving regression testing in continuous integration development environments (keynote),” Oct. 2018, doi: https://doi.org/10.1145/3278186.3281454.
  33. J. Sosnowski, J. Baker, O. Arnold, M. Silva, and M. West, “Reflections on 10 years of Operating a Computer-based Testing Facility: Lessons Learned, Best Practices,” 2024 ASEE Annual Conference & Exposition, Jun. 2024, doi: https://doi.org/10.18260/1-2--47930.
  34. M. Cheok, N. Michael, and R. Wojnar, “Compatibility of medication admixtures in continuous subcutaneous infusions: prioritizing laboratory testing for common combinations,” International Journal of Pharmacy Practice, vol. 32, no. 5, Aug. 2024, doi: https://doi.org/10.1093/ijpp/riae039.
  35. M. Yaseen, Samad Baseer, and S. Sherin, “Critical challenges for requirement implementation in context of global software development: A systematic literature review,” Dec. 2015, doi: https://doi.org/10.1109/icosst.2015.7396413.
  36. M. Yaseen, S. Ali, Abullah, and N. Ullah, “An Improved Framework for Requirement Implementation in the context of Global Software Development: A Systematic Literature Review Protocol,” International Journal of Database Theory and Application, vol. 9, no. 6, pp. 161–170, Jun. 2016, doi: https://doi.org/10.14257/ijdta.2016.9.6.16.
  37. M. Yaseen, “Exploratory study of existing research on software requirements prioritization: A systematic literature review,” Journal of Software Evolution and Process, vol. 36, no. 6, Sep. 2023, doi: https://doi.org/10.1002/smr.2613.
  38. M. A. Akbar, A. Alsanad, S. Mahmood, A. A. Alsanad, and A. Gumaei, “A Systematic Study to Improve the Requirements Engineering Process in the Domain of Global Software Development,” IEEE Access, vol. 8, pp. 53374–53393, 2020, doi: https://doi.org/10.1109/access.2020.2979468.
Index Terms

Computer Science
Information Sciences

Keywords

DevOps Continuous Integration Continuous Testing Challenges Best Practices Systematic literature Review

Powered by PhDFocusTM