An optimized model for mapreduce based on Hadoop

doi:10.12928/TELKOMNIKA.v14i4.3606

IR > 计算机与通信学院

	An optimized model for mapreduce based on Hadoop
	Zhang, Hong1,2 ; Wang, Xiao-Ming 1; Cao, Jie2 ; Ma, Yan-Hong 3; Guo, Yi-Rong 1; Wang, Min 1
	2016-12-01
发表期刊	Telkomnika (Telecommunication Computing Electronics and Control)
ISSN	16936930
卷号	14 期号:4 页码:1552-1558
摘要	Aiming at the waste of computing resources resulting from sequential control of running mechanism of MapReduce model on Hadoop platform, Fork/Join framework has been introduced into this model to make full use of CPU resource of each node. From the perspective of fine-grained parallel data processing, combined with Fork/Join framework, a parallel and multi-thread model, this paper optimizes MapReduce model and puts forward a MapReduce+Fork/Join programming model which is a distributed and parallel architecture combined with coarse-grained and fine-grained on Hadoop platform to Support two-tier levels of parallelism architecture both in shared and distributed memory machines. A test is made under the environment of Hadoop cluster composed of four nodes. And the experimental results prove that this model really can improve performance and efficiency of the whole system and it is not only suitable for handling tasks with data intensive but also tasks with computing intensive. it is an effective optimization and improvement to the MapReduce model of big data processing. © 2016 Universitas Ahmad Dahlan.
关键词	Data handling Dynamical systems Memory architecture Parallel architectures Distributed Fork/Join Hadoop Map-reduce Parallel
DOI	10.12928/TELKOMNIKA.v14i4.3606
收录类别	EI
语种	英语
出版者	Universitas Ahmad Dahlan
EI入藏号	20170803381807
EI主题词	Big data
EI分类号	722 Computer Systems and Equipment - 723.2 Data Processing and Image Processing
来源库	Compendex
分类代码	722 Computer Systems and Equipment - 723.2 Data Processing and Image Processing
引用统计	无
文献类型	期刊论文
条目标识符	https://ir.lut.edu.cn/handle/2XXMBERH/112994
专题	计算机与通信学院兰州理工大学
作者单位	1.College of Electrical and Information Engineering\, Lanzhou University of Technology, Lanzhou; 730050, China; 2.College of Computer and Communication, Lanzhou University of Technology, Lanzhou; 730050, China; 3.State Grid Gansu Electric Company, Lanzhou; 730030, China
第一作者单位	兰州理工大学
第一作者的第一单位	兰州理工大学
推荐引用方式 GB/T 7714	Zhang, Hong,Wang, Xiao-Ming,Cao, Jie,et al. An optimized model for mapreduce based on Hadoop[J]. Telkomnika (Telecommunication Computing Electronics and Control),2016,14(4):1552-1558.
APA	Zhang, Hong,Wang, Xiao-Ming,Cao, Jie,Ma, Yan-Hong,Guo, Yi-Rong,&Wang, Min.(2016).An optimized model for mapreduce based on Hadoop.Telkomnika (Telecommunication Computing Electronics and Control),14(4),1552-1558.
MLA	Zhang, Hong,et al."An optimized model for mapreduce based on Hadoop".Telkomnika (Telecommunication Computing Electronics and Control) 14.4(2016):1552-1558.