CFP last date
20 December 2024
Call for Paper
January Edition
IJCA solicits high quality original research papers for the upcoming January edition of the journal. The last date of research paper submission is 20 December 2024

Submit your paper
Know more
The week's pick
Responsive image
Improved Shuffled Frog Leaping Algorithm with Self-Adaptive Shuffling for Fuzzy Logic PD+G Controller Optimization in Robotic Manipulators

Duc Hoang Nguyen
Random Articles
Reseach Article

PRISM: Fine-Grained Phase and Resource Information-aware Scheduler for Map-Reduce

by Swati R. Mahendrakar, B. M. Patil
journal cover thumbnail

International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 172 - Number 4
Year of Publication: 2017
Authors: Swati R. Mahendrakar, B. M. Patil
10.5120/ijca2017915130

Swati R. Mahendrakar, B. M. Patil . PRISM: Fine-Grained Phase and Resource Information-aware Scheduler for Map-Reduce. International Journal of Computer Applications. 172, 4 ( Aug 2017), 32-39. DOI=10.5120/ijca2017915130

@article{ 10.5120/ijca2017915130,
author = { Swati R. Mahendrakar, B. M. Patil },
title = { PRISM: Fine-Grained Phase and Resource Information-aware Scheduler for Map-Reduce },
journal = { International Journal of Computer Applications },
issue_date = { Aug 2017 },
volume = { 172 },
number = { 4 },
month = { Aug },
year = { 2017 },
issn = { 0975-8887 },
pages = { 32-39 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume172/number4/28241-2017915130/ },
doi = { 10.5120/ijca2017915130 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2024-02-07T00:19:28.055114+05:30
%A Swati R. Mahendrakar
%A B. M. Patil
%T PRISM: Fine-Grained Phase and Resource Information-aware Scheduler for Map-Reduce
%J International Journal of Computer Applications
%@ 0975-8887
%V 172
%N 4
%P 32-39
%D 2017
%I Foundation of Computer Science (FCS), NY, USA
Abstract

In recent years, Map Reduce has become a popular model with regard to data-intensive computation. Map Reduce can significantly reduce the execution time of data-intensive jobs. In order to achieve this objective, Map Reduce breaks down each job into small map and reduce tasks and executes them in parallel across a large number of machines. However, existing solutions mainly focus on scheduling at the task-level, which offer sub-optimal job performance, because tasks may have resource requirements which may vary during their lifetime. This makes it difficult for existing system’s task-level schedulers to effectively utilize available resources in order to reduce job execution time. To avoid this limitation, PRISM is introduced. PRISM stands for Phase and Resource Information-aware Scheduler for Map-Reduce. PRISM consists of various clusters that perform resource-aware scheduling at the level of phases. PRISM can be defined as a fine-grained resource-aware Map Reduce scheduler that divides tasks into phases. Here, each phase has a constant resource usage profile, so that not a single phase suffers from starvation. PRISM also offers high resource utilization and provides 1:3x improvements in job running time as compared to the current Hadoop schedulers.

References
  1. Hadoop MapReduce distribution [Online]. Available: http://hadoop.apache.org, 2015.
  2. Hadoop Capacity Scheduler [Online]. Available: http://hadoop.apache.org/docs/stable/capacity_scheduler html/, 2015.
  3. Hadoop Fair Scheduler [Online]. Available: http://hadoop.apache.org/docs/r0.20.2/fair_scheduler.html, 2015.
  4. Hadoop Distributed File System [Online]. Available: hadoop.apache.org/docs/hdfs/current/, 2015.
  5. GridMix benchmark for Hadoop clusters [Online]. Available:http://hadoop.apache.org/docs/mapreduce/curt/gridmix.html, 2015.
  6. PUMA benchmarks [Online]. Available: http://web.ics.purdue.edu/fahmad/benchmarks/datasets.htm, 2015.
  7. The Next Generation of Apache Hadoop MapReduce [Online].Available:http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html, 2015.
  8. T. Condie, N. Conway, P. Alvaro, J. Hellerstein, K. Elmeleegy, and R. Sears, “MapReduce online,” in Proc. USENIX Symp. Netw. Syst. Des. Implementation, 2010, p. 21.
  9. J. Dean and S. Ghemawat, “Mapreduce: Simplified data processing on large clusters,” Commun. ACM, vol. 51, no. 1, pp. 107–113, 2008.
  10. A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker, and I. Stoica, “Dominant resource fairness: Fair allocation of multiple resource types,” in Proc. USENIX Symp. Netw. Syst. Des. Implementation, 2011, pp. 323–336.
  11. H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. Cetin, and S. Babu, ”Starfish: A self-tuning system for big data analytics,” in Proc. Conf. Innovative Data Syst. Res., 2011, pp. 261–272.
  12. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, and K. Talwar, “Quincy: Fair scheduling for distributed computing clusters,” in Proc. ACMSIGOPS Symp. Oper. Syst. Principles, 2009, pp. 261–276.
  13. C. Joe-Wong, S. Sen, T. Lan, and M. Chiang. “Multi-resource allocation: Flexible tradeoffs in a unifying framework,” in Proc. IEEE Int. Conf. Comput. Commun., 2012, pp. 1206–1214.
  14. J. Polo, C. Castillo, D. Carrera, Y. Becerra, I. Whalley, M. Steinder, J. Torres, and E. Ayguad_e, “Resource-aware adaptive scheduling for MapReduce clusters,” in Proc. ACM/IFIP/USENIX Int. Conf. Middleware, 2011, pp. 187–207.
  15. Verma, L. Cherkasova, and R. Campbell, “Resource provisioning framework for MapReduce jobs with performance goals,” in Proc. ACM/IFIP/USENIX Int. Conf. Middleware, 2011, pp. 165–186.
  16. Qi Zhang, Student Member, IEEE, Mohamed Faten Zhani, Member, IEEE, Yuke Yang, Raouf Boutaba, Fellow, IEEE, and Bernard Wong, “PRISM: Fine-Grained Resource-Aware Scheduling for Map-Reduce,” in ieee transactions on cloud computing, vol. 3, no. 2, april/june 2015.
Index Terms

Computer Science
Information Sciences

Keywords

Map Reduce scheduling resource allocation.

Powered by PhDFocusTM