원문정보
초록
영어
MapReduce, as a popular programming model for processing large data sets, has been widely applied. MapReduce 2.0 (MRV2) is a newly adopted one, which has a better performance. Those machines which have a lower performance in a cluster usually play a role who pull down the pace of job execution time. Speculative execution known as an approach for dealing with the above problems works by backing up those tasks running on a low performance machine to a higher one. Although multiple speculative execution strategies have been proposed, there are still a lot of pitfalls existing in the strategies. In this paper, Some pitfalls in proposed strategy have been modified and computer hardware has been taken into consideration (HWC-Speculation). In Hadoop-2.6, we have implemented it, called Hadoop-HWC. Experiment results show that our method can find a slow task correctly, also, the performance of MRV2 is improved.
목차
1. Introduction
2. Related Work
3. The HWC-Speculation Strategy
3.1 Getting the Remaining Time of Current Task
3.2 Method of Selecting a Backup Node and Predicting Time
3.3 Selection of Backup Task
4. Experiment and Analysis
5. Conclusion
Acknowledgments
References[1]