The week's pick
Reseach Article
Anomaly Detection using Hadoop and MapReduce Technique in Cloud with Sensor Data
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 125 - Number 1 |
| Year of Publication: 2015 |
| Authors: Ihab I.M. Alghussein, Walid Mohamed Aly, Mohamad Abou El-Nasr |
10.5120/ijca2015904922
|
Ihab I.M. Alghussein, Walid Mohamed Aly, Mohamad Abou El-Nasr . Anomaly Detection using Hadoop and MapReduce Technique in Cloud with Sensor Data. International Journal of Computer Applications. 125, 1 ( September 2015), 22-26. DOI=10.5120/ijca2015904922
Abstract
This paper presents a model to observation the Cloud computing for any anomalous activity. Hadoop it is a largely used open source Cloud Computing framework to huge data. It uses the model Machine Learning technique to detect classify anomalies of sensory observation and help to in ensuring the stabilization of virtual sensor networks. The framework it’s built on top of the Hadoop and MapReduce implementation which is use one of the Machines Learning techniques to detect these anomalies. Preliminary results show that our classification mechanism is promising and able to detect anomalous events that may cause a threat to the Cloud Computing.
References
- Mell and T. Grance, ―The NIST definition of cloud computing ,‖ NIST special publication, 800(145), 7, 2011.
- An architecture for overlaying private clouds on public providers Shtern, M. ; Simmons, B. ; Smit, M. ; Litoiu, M. Publication Year: 2012 , Page(s): 371 – 377.
- Hadoop website. http://hadoop.apache.org/. Last vist 14 augest 2014.
- "MapReduce: Simplified Data Processing on Large Clusters", by JeffreyDean and Sanjay Ghemawat; from http://research.google.com/archive/mapreduce.html Last vist 12 augest 2014.
- T. White, Hadoop: The Definitive Guide, original ed.O’Reilly Media, Jun. 2009.
- Kai Wang, Ying Wang, Bo Yin, "A Density-Based Anomaly Detection Method for MapReduce," nca, pp.159-162, 2012 IEEE 11th International Symposium on Network Computing and Applications, 2012.
- https://wiki.apache.org/hadoop/PoweredBy last vist 12 augest 2014. (Ebay).
- Magorzata Steinder, Adarshpal S. Sethi, “A survey of fault localization techniques in computer networks”, in Sci Comput Program, Vol. 53, pp.165-194.
- Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey”. In ACM Computing Surveys, 2009.
- Kher, S. ; Arkansas State Univ., AR, USA ; Nutt, V. Dasgupta, D. ; Ali, H.more authors”A detection model for anomalies in smart grid with sensor network” Future of Instrumentation International Workshop (FIIW), 2012.
- K. Wang, Y. Wang, and B. Yin, "A Density-Based Anomaly Detection Method for MapReduce", ;in Proc. NCA, 2012, pp.159-162.
- A. Shilton, S. Rajasegarar, and M. Palaniswami, “Combined multiclass classification and anomaly detection for large-scale wireless sensor networks,” in Intelligent Sensors, Sensor Networks and Information Processing, 2013 IEEE Eighth International Conference on, 2013, pp.491–496.
- J. R. Lee, S.-K. Ye, and H.-D. J. Jeong, “Detecting anomaly teletraffic using stochastic self-similarity based on Hadoop,” in Network-Based Information Systems (NBiS), 2013 16th International Conference on, 2013, pp. 282–287.
- M. Xie, J. Hu, and B. Tian, “Histogram-based online anomaly detection in hierarchical wireless sensor networks,” in Trust, Security and Pri- vacy in Computing and Communications (TrustCom), 2012 IEEE 11th International Conference on, 2012, pp. 751–759.
- Gaber, M. Data Stream Processing in Sensor Networks. In Learning from Data Streams Processing Techniques in Sensor Networks; Springer: Berlin/Heidelberg, Germany, 2007; pp. 41–48.
- Tan, P.-N.; Steinbach, M.; Kumar, V. Introduction to Data Mining; Addison Wesley: Boston, MA, USA, 2005.
- Aggarwal, C.; Yu, P. Outlier Detection for High Dimensional Data. In Proceedings of the ACM SIGMOD International Conference on Management of Data, Santa Barbara, CA, USA, 21–24 May 2001.
- Janakiram, D.; Adi Mallikarjuna Reddy, V.; Phani Kumar, A.V.U. Outlier Detection in Wireless Sensor Networks Using Bayesian Belief Networks, In Proceedings of the First International Conference on Communication System Software and Middleware (COMSWARE 2006), Delhi, India, 8–12 January 2006; pp. 1–6.
- Li, Y. Anomaly Detection in Unknown Environments Using Wireless Sensor Networks; The University of Tennessee: Knoxville, TN, USA, 2010.
- Jeffery, S.; Alonso, G.; Franklin, M.; Hong, W.; Widom, J. Declarative Support for Sensor Data Cleaning. In Pervasive Computing, Springer: Berlin/Heidelberg, Germany, 2006; pp. 83–100.
- Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler. Yahoo! Sunnyvale, California USA “The Hadoop Distributed File System”, IEEE, 2010.
- ] Module 1: Tutorial Introduction Mapreduce Yahoo https://developer.yahoo.com/hadoop/tutorial/module1.html. 12 augest 2014.
- L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone. Classification and Regression Trees. Wadswort, Belmont, 1984.
- OneR one Rule . http://www.saedsayad.com/oner.htm. Last vist 12 augest 2014.
- Shinde, Amit, Anshuman Sahu, Daniel Apley, and George Runger. "Preimages for Variation Patterns from Kernel PCA and Bagging." IIE Transactions, Vol. 46, Iss. 5, 2014.
- C4.5algorithm. Http://en.wikipedia.org/wiki/C4.5_algorithm. Last vist 12 augest 2014.
- WEKA: A machine learning tool set. Software downloadable from http://www.cs.waikato.ac.nz/ml/weka/index_downloading.html. last vist 12 augest 2014.
Index Terms
Keywords