The Hadoop Dispersed File system: Balancing Movability And Performance
Keywords:
Component, Formatting, Style, StylingAbstract
Hadoop is a general open-source application of map reduce for the examination of big datasets. To achieve storing capitals crosswise the cluster, Hadoop uses a dispersed user-level file system. This file system — HDFS — is written in java and envisioned for movability crosswise varied hardware and software platforms. This newspaper inspects the presentation of HDFS and uncovers numerous presentation issues. First, architectural blockages be in the Hadoop application that consequence in incompetent HDFS usage owing to postponements in preparation new map reduce tasks. Second, movability limits stop the java application after abusing topographies of the innate platform. Third, HDFS implicitly brands movability molds about how the innate phase achieves storing resources, smooth however innate file systems and I/O schedulers vary widely in arrangement and behavior. This newspaper examines the root details of these presentation blockages in instruction to assess tradeoffs among movability and presentation in the Hadoop dispersed file system.
References
Chandrasekar, S. ; Dakshinamurthy, R. ; Seshakumar, P.G. ;Prabavathy, B. ; Babu, C. “A novel indexing scheme for efficient handling of small filesin Hadoop Distributed File System”Computer Communication and Informatics (ICCCI), 2013 International Conference on Publication Year: 2013, Page(s): 1 - 8
Jing Zhang ; Gongqing Wu ; Xuegang Hu ; Xindong Wu “A Distributed Cache for Hadoop Distributed File System in Real-Time Cloud Services” Grid Computing (GRID), 2012 ACM/IEEE 13th International Conference on Publication Year: 2012 , Page(s): 12 - 21
Kaushik, R.T. ; Bhandarkar, M. ; Nahrstedt, K. “Evaluation and Analysis of GreenHDFS: A Self-Adaptive, Energy-Conserving Variant of the Hadoop Distributed File System” Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on Publication Year: 2010 , Page(s): 274 - 287
Hieu Hanh Le ; Hikida, S. ; Yokota, H. Dependable, “An Evaluation of Power-Proportional Data Placement forHadoop Distributed File Systems Autonomic and Secure Computing (DASC), 2011 IEEE Ninth International Conference on Publication Year: 2011 , Page(s): 752 - 759
Attebury, G. ; Baranovski, A. ; Bloom, K. ; Bockelman, B. ; Kcira, D. ;Letts, J. ; Levshina, T. ; Lundestedt, C. ; Martin, T. ; Maier, W. ;Haifeng Pi ; Rana, A. ; Sfiligoi, I. ; Sim, A. ; Thomas, M. ;Wuerthwein, “Hadoop distributed file system for the Grid” F. Nuclear Science Symposium Conference Record (NSS/MIC), 2009 IEEE Publication Year: 2009 , Page(s): 1056 - 1061
Youwei Wang ; WeiPing Wang ; Can Ma ; Dan Meng “Zput: A speedy data uploading approach for the Hadoop Distributed File System” Cluster Computing (CLUSTER), 2013 IEEE International Conference on Publication Year: 2013 , Page(s): 1 – 5
Shahabinejad, M. ; Khabbazian, M. ; Ardakani, M. “An Efficient Binary Locally Repairable Code for Hadoop Distributed File System” Communications Letters, IEEE Volume: 18 , Issue: 8 Publication Year: 2014 , Page(s): 1287 – 1290
Tomasic, I. ; Ugovsek, J. ; Rashkovska, A. ; Trobec, R. “Multicluster Hadoop Distributed File System”MIPRO, 2012 Proceedings of the 35th International Convention Publication Year: 2012 , Page(s): 301 - 305
Hsiao-Ying Lin ; Shiuan-Tzuo Shen ; Wen-Guey Tzeng ; Lin, B.-S.P. “Toward Data Confidentiality via Integrating Hybrid Encryption Schemes and Hadoop Distributed File System” Advanced Information Networking and Applications (AINA), 2012 IEEE 26th International Conference on Publication Year: 2012, Page(s): 740 - 747
Moraveji, R. ; Taheri, J. ; Farahabady, M.R.H. ; Rizvandi, N.B. ;Zomaya, A.Y. “Data-Intensive Workload Consolidation for the Hadoop Distributed File System” Grid Computing (GRID), 2012 ACM/IEEE 13th International Conference on Publication Year: 2012 , Page(s): 95 – 103 “The Hadoop Distributed File System”
Shvachko, K. ; Hairong Kuang ; Radia, S. ; Chansler, R. Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on Publication Year: 2010 , Page(s): 1 - 10
Xiayu Hua ; Hao Wu ; Shangping Ren “Enhancing Throughput of Hadoop Distributed File System for Interaction-Intensive Task Parallel, Distributed and Network-Based Processing (PDP), 2014 22nd Euromicro International Conference on Publication Year: 2014 , Page(s): 508 – 511
Wei Zhou ; Jizhong Han ; Zhang Zhang ; Jiao Dai ,“Dynamic Random Access for Hadoop Distributed File System” Distributed Computing Systems Workshops (ICDCSW), 2012 32nd International Conference on Publication Year: 2012 , Page(s): 17 – 22
Madaan, S. ; Agrawal, R.K. “Implementation of identity based distributed cloud storage encryption scheme using PHP and C for Hadoop File System” Tier 2 Federation Grid, Cloud & High Performance Computing Science (RO-LCG), 2012 5th Romania Publication Year: 2012 , Page(s): 74 - 77
Yang Jin ; Tang Deyu ; Zhou Yi “A Distributed Storage Model for EHR Based on HBase” Information Management, Innovation Management and Industrial Engineering (ICIII), 2011 International Conference on Volume: 2 Publication Year: 2011 , Page(s): 369 - 372
Yang Jin ; Tang Deyu ; Zheng Xianrong “Research on the distributed electronic medical records storage model” IT in Medicine and Education (ITME), 2011 International Symposium on Volume: 2 Publication Year: 2011 , Page(s): 288 - 292
Krish, K.R. ; Anwar, A. ; Butt, A.R. “hatS: A Heterogeneity-Aware Tiered Storage for Hadoop” Cluster, Cloud and Grid Computing (CCGrid), 2014 14th IEEE/ACM International Symposium on Publication Year: 2014 , Page(s): 502 – 511
Changjian Fu ; Zhihua Leng “A Framework for Recommender Systems in E-Commerce Based on Distributed Storage and Data-Mining” E-Business and E-Government (ICEE), 2010 International Conference on Publication Year: 2010 , Page(s): 3502 - 3505
Bo Dong ; Xiao Zhong ; Qinghua Zheng ; Lirong Jian ; Jian Liu ; Jie Qiu ; Ying Li “Correlation Based File Prefetching Approach for Hadoop” Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on Publication Year: 2010 , Page(s): 41 - 48
Bo Dong ; Jie Qiu ; Qinghua Zheng ; Xiao Zhong ; Jingwei Li ; Ying Li “A Novel Approach to Improving the Efficiency of Storing and Accessing Small Files on Hadoop: A Case Study by PowerPoint Files” Services Computing (SCC), 2010 IEEE International Conference on Publication Year: 2010 , Page(s): 65 - 72
Martha, V.S. ; Weizhong Zhao ; Xiaowei Xu “h-MapReduce: A Framework for Workload Balancing in MapReduce” Advanced Information Networking and Applications (AINA), 2013 IEEE 27th International Conference on Publication Year: 2013 , Page(s): 637 - 644
Li Wang ; Zhiwei Ni ; Yiwen Zhang ; Zhang Jun Wu ; “Liyang Tang Notice of Violation of IEEE Publication Principles “Pipelined-MapReduce: An Improved MapReduce Parallel Programing Model “ Intelligent Computation Technology and Automation (ICICTA), 2011 International Conference on Volume: 1 Publication Year: 2011 , Page(s): 871 - 874
Kaiqi Xiong ; Yuxiong He “Power-effiicent resource allocation in MapReduce clusters”Integrated Network Management (IM 2013), 2013 IFIP/IEEE International Symposium on Publication Year: 2013 , Page(s): 603 – 608
Yanpei Chen ; Ganapathi, A. ; Griffith, R. ; Katz, R. “The Case for Evaluating MapReduce Performance Using Workload Suites” Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), 2011 IEEE 19th International Symposium on Publication Year: 2011 , Page(s): 390 - 399 “Getting more for less in optimized MapReduce workflows”
Zhuoyao Zhang ; Cherkasova, L. ; Boon Thau Loo Integrated Network Management (IM 2013), 2013 IFIP/IEEE International Symposium on Publication Year: 2013 , Page(s): 93 – 100
Guigang Zhang ; Jian Wang ; Weixing Huang ; Chao Li ; Yong Zhang; Chunxiao Xing “A Semantic++ MapReduce: A Preliminary Report”Semantic Computing (ICSC), 2014 IEEE International Conference on Publication Year: 2014 , Page(s): 330 – 336
Kewen Wang ; Wenzhong Predator An experience guided configuration optimizer for Hadoop MapReduce Tang Cloud Computing Technology and Science (CloudCom), 2012 IEEE 4th International Conference on Publication Year: 2012 , Page(s): 419 - 426
Chu Huang ; Sencun Zhu ; Dinghao Wu Towards Trusted Services: Result Verification Schemes for MapReduce Cluster, Cloud and Grid Computing (CCGrid), 2012 12th IEEE/ACM International Symposium on Publication Year: 2012 , Page(s): 41 - 48
Cura: A Cost-Optimized Model for MapReduce in a Cloud Palanisamy, B. ; Singh, A. ; Langston, B. Parallel & Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on Publication Year: 2013 , Page(s): 1275 – 1286
M2M: A simple Matlab-to-MapReduce translator for cloud computing Zhang, Junbo ; Xiang, Dong ; Li, Tianrui ; Pan, Yi Tsinghua Science and Technology Volume: 18 , Issue: 1 Publication Year: 2013 , Page(s): 1-9
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors contributing to this journal agree to publish their articles under the Creative Commons Attribution 4.0 International License, allowing third parties to share their work (copy, distribute, transmit) and to adapt it, under the condition that the authors are given credit and that in the event of reuse or distribution, the terms of this license are made clear.
