The week's pick
Random Articles
Reseach Article
Event driven Fraud Detection Pipeline: Real-Time Processing with Kafka, ksqlDB & Apache Flink
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 187 - Number 60 |
| Year of Publication: 2025 |
| Authors: Ronak S. Dev, Usha J. |
10.5120/ijca2025925872
|
Ronak S. Dev, Usha J. . Event driven Fraud Detection Pipeline: Real-Time Processing with Kafka, ksqlDB & Apache Flink. International Journal of Computer Applications. 187, 60 ( Nov 2025), 13-18. DOI=10.5120/ijca2025925872
Abstract
In an era dominated by digital transactions and real-time decision-making, traditional fraud detection systems have become inadequate due to their reliance on delayed, batch-based processing. This research presents an event-driven architecture for real-time fraud detection, leveraging Apache Kafka for high-throughput data ingestion, ksqlDB for rule-based stream querying, and Apache Flink for complex event processing and machine learning inference. The system ingests transaction, login, and geolocation data streams, applies immediate filters, and performs stateful anomaly detection to identify suspicious behaviors such as velocity violations and improbable access patterns. A fully containerized implementation validates the architecture’s performance under simulated load conditions, achieving a true positive rate of 94.2% and sub-second latency. The hybrid approach unifies rule-based and ML-enhanced detection, offering low false positives and high adaptability. This work demonstrates how modern stream processing technologies can transform fraud detection from a reactive, offline task into a proactive, real-time analytics pipeline embedded within the data infrastructure. The architecture is modular, scalable, and production-ready, making it suitable for deployment in financial and e-commerce ecosystems.
References
- Venkata Karunakar Uppalapati, “AI In Financial Services: Real-Time Fraud Detection on Cloud Native GPU Clusters,” Journal of Computer Science and Technology Studies, Vol. 7, No. 7, July 2025, pp. 183–190 https://www.jcsts.org/articles/ai-financial-gpu-detection-2025
- Akash Vijayrao Chaudhari, “A Cloud Native Unified Platform for Real Time Fraud Detection,” (unpublished), April 2025 https://www.researchgate.net/publication/378621221
- Chen Liu, Hengyu Tang, Zhixiao Yang, Ke Zhou, Sangwhan Cha, “Big Data Driven Fraud Detection Using Machine Learning and Real Time Stream Processing,” arXiv preprint, May 2025 https://arxiv.org/abs/2506.02008
- Dyapa S., “Real Time Fraud Detection: Leveraging Apache Kafka and Flink,” International Journal on Science and Technology (IJSAT), Vol. 16, No. 1, 2025 https://www.ijsat.org/papers/2025/1/2654.pdf
- Srijan Saket, Vivek Chandela, Md. Danish Kalim, “Real Time Event Joining in Practice with Kafka and Flink,” arXiv preprint, October 2024 https://arxiv.org/abs/2410.15533
- Md. Kamrul Hasan Chy, “Proactive Fraud Defense: Machine Learning’s Evolving Role in Protecting Against Online Fraud,” arXiv preprint, October 2024 https://arxiv.org/abs/2410.06812
- Adeyinka Orelaja, Adenike F. Adeyemi, “Developing Real Time Fraud Detection and Response Mechanisms for Financial Transactions,” IRE Journals, Vol. 8, No. 1, August 2024 https://irejournals.com/formatedpaper/1705034.pdf
- Parin Patel, “Real Time Fraud Detection Using Apache Flink and Machine Learning,” Medium, September 2024 https://medium.com/@parinpatel22/real-time-fraud-detection-using-apache-flink-and-machine-learning-70b6490a01b6
- Adriano Vogel, Sören Henning, Esteban Perez Wohlfeil, Otmar Ertl, Rick Rabiser, “A Comprehensive Benchmarking Analysis of Fault Recovery in Stream Processing Frameworks,” arXiv preprint, April 2024 https://arxiv.org/abs/2404.11949
- Kai Waehner, “Real Time Model Inference with Apache Kafka and Flink for Predictive AI And Genai,” Blog Post, December 2024 https://www.kai-waehner.de/blog/2024/10/01/real-time-model-inference-with-apache-kafka-and-flink-for-predictive-ai-and-genai/
- Kai Waehner, “Fraud Detection with Apache Kafka, Ksqldb and Apache Flink,” Kai Waehner Blog, October 2022 https://kai-waehner.medium.com/fraud-prevention-in-under-60-seconds-with-apache-kafka-9542224f9ec8
- Kai Waehner, “Fraud Detection in Mobility Services (Ride Hailing, Food Delivery) With Kafka & Flink,” Kai Waehner Blog, April 2025 https://www.kai-waehner.de/blog/2025/04/28/fraud-detection-in-mobility-services-ride-hailing-food-delivery-with-data-streaming-using-apache-kafka-and-flink/
- Confluent Inc., “Real-Time Fraud Detection – Use Case Implementation,” White Paper, 2025 https://www.confluent.io/resources/white-paper/real-time-fraud-detection-use-case-implementation/
- International Journal on Multidisciplinary Engineering (IJMIE), “From Batch Processing to Real Time Streaming in Financial Fraud Detection,” Vol. 13, No. 3, March 2025 https://www.ijmra.us/project%20doc/2025/IJME_MARCH2025/IJMIE7_March2025.pdf
- IRJMETS, “Streaming Analytics and Real Time Decision Making,” IRJMETS Journal, March 2025 https://www.irjmets.com/uploadedfiles/paper//issue_3_march_2025/70449/final/fin_irjmets1743171816.pdf
- Vashisht, B. S. Rekha, “Microservices and Real-Time Processing in Retail IT,” arXiv preprint, June 2025 https://arxiv.org/abs/2506.09938
- P. Singh, “Advanced Techniques in Real-Time Monitoring for Financial Transactions,” MDRG Journal, Vol. 3, No. 3, June 2025 https://www.allmultidisciplinaryjournal.com/uploads/archives/20250621125159_MGE-2025-3-305.1.pdf
- Sugumar P., “A Poc Approach: Real-Time Fraud Detection with Kafka, Flink & ML,” Medium Blog, February 2025 https://medium.com/@sugumarp/real-time-fraud-detection-using-kafka-flink-machine-learning-dbd6c1dc80e6
- S. Fedulov, “Streaming Machine Learning Pipelines with Flink SQL,” Ververica Blog, January 2025 https://www.ververica.com/blog/streaming-machine-learning-pipelines-with-flink-sql
- TimePlus, “Proton: An Open Source Alternative to ksqlDB for Real-Time Analytics,” TimePlus Blog, 2024 https://www.timeplus.com/post/proton-ksqldb-alternative
- ACM Digital Library, “Design and Implementation of a Real-Time Stream Processing Engine for Financial Risk,” ACM Conference Proceedings, 2024 https://dl.acm.org/doi/10.1145/3729706.3729765
- S. Malviya, “Limitations of Batch Fraud Detection Techniques in Dynamic Financial Networks,” IJFMR, Vol. 11, No. 1, January 2025 https://www.ijfmr.com/papers/2025/January/IJFMR0112345.pdf
- Yasir, V. J., et al., “Trends in Payment Fraud in Indian Financial Systems (2019–2022),” Indian Journal of FinTech Studies, 2025 https://indianfintechjournal.org/articles/2025/trends-in-payment-fraud
- Singh A., Banerjee R., “CEP Strategies in Fraud Detection Using Apache Flink,” Journal of Streaming Analytics, 2025 https://www.streaminganalyticsjournal.org/cep-strategies-2025
- Fabrizio Carcillo, Andrea Dal Pozzolo, Yann Aël Le Borgne, Olivier Caelen, Yannis Mazzer, Gianluca Bontempi, “SCARFF: A Scalable Framework for Streaming Credit Card Fraud Detection with Spark,” arXiv preprint, September 2017 https://arxiv.org/abs/1708.08905.
Index Terms
Keywords