Xu, Xiaolong, Cao, Lingling and Wang, Xinheng ORCID: https://orcid.org/0000-0001-8771-8901 (2014) Adaptive task scheduling strategy based on dynamic workload adjustment for heterogeneous Hadoop clusters. IEEE Systems Journal, 10 (2). pp. 471-482. ISSN 1932-8184

Full text not available from this repository.

Abstract

The original task scheduling algorithm of Hadoop cannot meet the performance requirements of heterogeneous clusters. According to the dynamic change of load of each task node and the difference of node performance of different tasks in the heterogeneous Hadoop cluster, a novel adaptive task scheduling strategy based on dynamic workload adjustment (ATSDWA) is presented. With ATSDWA, tasktrackers can adapt to the change of load at runtime, obtain tasks in accordance with the computing ability of their own, and realize the self-regulation, while avoiding the complexity of algorithm, which is the prime reason to make jobtracker the system performance bottleneck. Experimental results show that ATSDWA is a highly efficient and reliable algorithm, which can make heterogeneous Hadoop clusters stable, scalable, efficient, and load balancing. Furthermore, its performance is superior to the original and improved task scheduling strategy of Hadoop, from the aspects of the execution time of tasks, the resource utilization, and the speed-up ratio.

Item Type:	Article
Identifier:	10.1109/JSYST.2014.2323112
Keywords:	Adaptive scheduling, clustering methods, computational efficiency, distributed computing, dynamic scheduling
Subjects:	Computing > Systems > Computer networking
Depositing User:	Henry Wang
Date Deposited:	30 Jan 2018 17:01
Last Modified:	06 Feb 2024 15:55
URI:	https://repository.uwl.ac.uk/id/eprint/4346

Actions (login required)

View Item