Uses of HDFS in Metadata Management System

Authors

  • N Durga Department of Computer Science, AVVM Sri Pushpam College, Bharathidasan University, India
  • D Ragupathi Department of Computer Science, AVVM Sri Pushpam College, Bharathidasan University, India
  • V Raj Kumar R and D of Computer Science, STEPINFOTECH, India

Keywords:

Hadoop, HDFS, Distributed File System, File Management System, Metadata Management

Abstract

A (HDFS) Hadoop Distributed File System is intended to store very large data sets unfailingly and to stream those data sets at elevated bandwidth to consumer applications. Metadata managing is serious to distributed file system. In HDFS design, a solo master server administers all metadata, while numerals of data servers pile up file data. This design can’t gather the exponentially improved storage claim in cloud computing, as the solo master server may happen to a performance traffic jam. Proportional study of a metadata managing proposal is done. Its consist of three techniques tree separation, hashing and reliable hashing of metadata managing proposal. Out of these three proposals reliable hashing is the best techniques which utilize multiple Name Nodes, and segregate the metadata into “Buckets” which can be energetically move around among Name Nodes according to system job weight. To uphold consistency, metadata is simulated in dissimilar Name Nodes with log duplication knowledge, and Paxos algorithm is adopted to keep duplication uniformity.

References

Zhanye Wang ; Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China ; Dongsheng Wang “NCluster: Using Multiple Active Name Nodes to Achieve High Availability for HDFS” Published in: High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC_EUC), 2013 IEEE 10th International Conference on Date of Conference: 13-15 Nov. 2013 Page(s): 2291 – 2297.

Azzedin, F. ; Inf. & Comput. Sci. Dept., King Fahd Univ. of Pet. & Miner., Dhahran, Saudi Arabia “Towards a scalable HDFS architecture” Published in: Collaboration Technologies and Systems (CTS), 2013 International Conference on Date of Conference: 20-24 May 2013 Page(s): 155 – 161.

Dutta, R. ; Dept. of Comput. Sci. & Eng., Nat. Inst. of Technol. Karnataka, Surathkal, India ; Annappa, B. “A Scalable Cloud Platform using Matlab Distributed Computing Server Integrated with HDFS” Published in: Cloud and Services Computing (ISCOS), 2012 International Symposium on Date of Conference: 17-18 Dec. 2012 Page(s): 141– 145.

Cheng Li-xin ; Suzhou Inst. of Nano-Tech & Nano-Bionics (SINANO), Suzhou, China ; Yue Yang ; Liu Yun-yun ; Ka Lok Man “A research on the storage structure and reconstruction algorithm of Meta data during the startup process of solid state storage systems” Published in: SoC Design Conference (ISOCC), 2013 International Date of Conference: 17-19 Nov. 2013 Page(s): 263 – 266.

Zhen-Chun Huang ; Dept. of Comput. Sci. & Eng., Tsinghua Univ., Beijing ; Guo-qing Li “Meta-Data Adapter: A Way to Enable Spatial Information Data Grid” Published in: Future Generation Communication and Networking Symposia, 2008. FGCNS '08. Second International Conference on (Volume:5 ) Date of Conference: 13-15 Dec. 2008 Page(s): 87 – 92.

Narayan, S. ; InfoBlox Inc., Santa Clara, CA, USA ; Bailey, S. ; Daga, A. “Hadoop Acceleration in an OpenFlow-Based Cluster” Published in: High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion: Date of Conference: 10-16 Nov. 2012 Page(s): 535 – 538.

Mandal, A. ; Renaissance Comput. Inst., Univ. of North Carolina at Chapel Hill, Chapel Hill, NC, USA ; Yufeng Xin ; Baldine, I. ; Ruth, P. “Provisioning and Evaluating Multi-domain Networked Clouds for Hadoop-based Applications” Published in: Cloud Computing Technology and Science (CloudCom), 2011 IEEE Third International Conference on Date of Conference: Nov. 29 2011-Dec. 1 2011 Page(s): 690 – 697.

Xiaoyi Lu ; Islam, N.S. ; Wasi-Ur-Rahman, M. ; Jose, J. “High-Performance Design of Hadoop RPC with RDMA over InfiniBand” Published in: Parallel Processing (ICPP), 2013 42nd International Conference on Date of Conference: 1-4 Oct. 2013 Page(s): 641 – 650.

Downloads

Published

2014-09-30

How to Cite

[1]
N. Durga, D. Ragupathi, and V. Raj Kumar, “Uses of HDFS in Metadata Management System”, Int. J. Comp. Sci. Eng., vol. 2, no. 9, pp. 145–151, Sep. 2014.

Issue

Vol. 2 No. 9 (2014): IJCSE September Edition

Section

Research Article

License

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors contributing to this journal agree to publish their articles under the Creative Commons Attribution 4.0 International License, allowing third parties to share their work (copy, distribute, transmit) and to adapt it, under the condition that the authors are given credit and that in the event of reuse or distribution, the terms of this license are made clear.