earticle

영어

Job scheduling is an important component of Hadoop. Hadoop default FIFO scheduling algorithm is simple and easy to achieve and widely used; but it lacks consideration in the characteristic of data locality, which will lead to heavy traffic during network transmission and task execution, then computing resources cannot be fully utilized and a series of other drawbacks. Meanwhile the static function of resource slots in Map and Reduce stages make such defects worse. This paper proposes a job scheduling algorithm (Interactive Scheduler Algorithm) IS based on interacting the master node and slave nodes from the data locality and tasks allocation. The algorithm improves FIFO and the usage of computing resource, realizes the dynamic conversion of map slots and reduces slots. In the end through the comparison of experiment it is proved that the IS has a great improvement in job scheduling for Hadoop.