We apologize for a recent technical issue with our email system, which temporarily affected account activations. Accounts have now been activated. Authors may proceed with paper submissions. PhDFocusTM
CFP last date
20 December 2024
Call for Paper
January Edition
IJCA solicits high quality original research papers for the upcoming January edition of the journal. The last date of research paper submission is 20 December 2024

Submit your paper
Know more
The week's pick
Responsive image
Improved Shuffled Frog Leaping Algorithm with Self-Adaptive Shuffling for Fuzzy Logic PD+G Controller Optimization in Robotic Manipulators

Duc Hoang Nguyen
Random Articles
Reseach Article

Enhancing NameNode Fault Tolerance in Hadoop Distributed File System

by Ohnmar Aung, Thandar Thein
journal cover thumbnail

International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 87 - Number 12
Year of Publication: 2014
Authors: Ohnmar Aung, Thandar Thein
10.5120/15264-4020

Ohnmar Aung, Thandar Thein . Enhancing NameNode Fault Tolerance in Hadoop Distributed File System. International Journal of Computer Applications. 87, 12 ( February 2014), 41-47. DOI=10.5120/15264-4020

@article{ 10.5120/15264-4020,
author = { Ohnmar Aung, Thandar Thein },
title = { Enhancing NameNode Fault Tolerance in Hadoop Distributed File System },
journal = { International Journal of Computer Applications },
issue_date = { February 2014 },
volume = { 87 },
number = { 12 },
month = { February },
year = { 2014 },
issn = { 0975-8887 },
pages = { 41-47 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume87/number12/15264-4020/ },
doi = { 10.5120/15264-4020 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2024-02-06T22:05:46.699542+05:30
%A Ohnmar Aung
%A Thandar Thein
%T Enhancing NameNode Fault Tolerance in Hadoop Distributed File System
%J International Journal of Computer Applications
%@ 0975-8887
%V 87
%N 12
%P 41-47
%D 2014
%I Foundation of Computer Science (FCS), NY, USA
Abstract

In today's cloud computing environment, Hadoop is applied for handling huge data, tens of terabytes to petabytes, with commodity hardware (HDFS) for storage and software (MapReduce) for parallel data processing. In Hadoop version 1. 0. 3, there is a single metadata server called NameNode which stores the entire file system metadata in main memory and most of I/O operations are associated with those credential metadata. Hadoop is out of commission if NameNode is crashed because it works on memory which becomes exhausted due to multiple concurrent accesses [3]. Therefore, NameNode is a single point of failure (SPOF) in Hadoop and it has to tolerate faults. To solve this issue, a proactive predictive solution is proposed for enhancing NameNode fault tolerance. The solution is designed to proactively calculate the predicted time to crash of NameNode due to resource exhaustion by evaluating the use of powerful Back Propagation Algorithm Neural Network. The proposed approach can give prediction accuracy with minimal error compared to the actual result. Therefore, NameNode's single point of failure can overcome through proposed proactively predicting the time to crash of NameNode caused by memory resource exhaustion.

References
  1. Anil K. Jain, "Artificial Neural Networks: A Tutorial", in Proceedings of Neural Computing: Companion issue to Engineering, Vol. 29 Issue 3, March 1996, pp. 31-44
  2. Cristina L. Abad, Huong Luu, Nathan Roberts, Kihwal Lee, Yi Lu and Roy H. Campbell, "Metadata Traces and Workload Models for Evaluating Big Storage Systems", in Proceedings of IEEE 5th International Conference on Utility and Cloud Computing (UCC), Chicago, IL, November 5-8, 2012, pp. 125-132.
  3. Chuck Lam, "Hadoop in Action", Manning Publications Co. 180 Broad St. Suite 1323, Stamfor, CT 06901, December 22, 2010.
  4. Diane Hatcher, "Considerations for Implementing a Highly Available or Disaster Recovery Environment," SAS Institute Inc, Cary, NC, USA, 2011.
  5. Dhruba Borthakur, "Apache Hadoop and Its Usage in Facebook", UC Berkeley, April 4, 2011. Online Available : http://www. facebook. com/hadoopfs
  6. Eric Sammer, "Hadoop Operations", O'Reilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472, United States of America, September 9, 2012.
  7. Feng Wang, Jie Qiu, Jie Yang, "Hadoop High Availability through Metadata Replication", IBM Research, China, 2009.
  8. Javier Alonso and Jordi Torres, "Predicting Web Server Crashes: A Case Study in Comparing Prediction Algorithms", in Proceedings of 5th IEEE International Conference on Autonomic and Autonomous Systems (ICAS'09), Valencia, April 20-25, 2009, pp. 264-269.
  9. Javier Alonso Lopez, "Proactive Software Rejuvenation Solution for Web Environment on Virtualized Platforms," Doctoral thesis, Barcelona, Spain 2011.
  10. Jimmy Lin and Chris Dyer, "Data-Intensive Text Processing with MapReduce", University of Maryland, College Park, April 11, 2010.
  11. Roman Dudko, Abhishek Sharma, Jon Tedsco, "Effective Failure Prediction in Hadoop Clusters", March, 2012. Online Available: http://www. techrepublic. com/resource-library/whitepapers/effective-failure-prediction-in-hadoop-clusters/
  12. Simon Haykin, "Neural Network: A Comprehensive Foundation," Prentice Hall, Delhi, India, 1999.
  13. Tom White, "Hadoop: The Definitive Guide", O'Reilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472, United States of America, May 2012.
  14. Xiaojuan Ren, Seyong Lee, Rudolf Eigenmann, Saurabh Bagchi, "Prediction of Resource Availability in Fine-Grained Cycle Sharing Systems Empirical Evaluation", J Grid Computing (2007), Vol 5, pp 173-195.
Index Terms

Computer Science
Information Sciences

Keywords

HDFS NameNode Memory Resource Exhaustion Prediction Back Propagation Neural Network

Powered by PhDFocusTM