An Optimizational Scheme of Dealing with Massive Small Files based on HDFS
Available Online July 2015.
- 10.2991/csic-15.2015.23How to use a DOI?
- Massive small files, File merge, HDFS
This paper delves the hadoop distributed file system-HDFS, and presents a new file combining, indexing and extracting scheme to alleviate the question that HDFS cannot perform well when facing with massive small files instead of the perfect efficiency of dealing with big files in order, which could reduce the usage of memory of Namenode and improve the I/O performance in HDFS.
- © 2015, the Authors. Published by Atlantis Press.
- Open Access
- This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Cite this article
TY - CONF AU - Yongsheng Du PY - 2015/07 DA - 2015/07 TI - An Optimizational Scheme of Dealing with Massive Small Files based on HDFS BT - Proceedings of the 2015 International Conference on Computer Science and Intelligent Communication PB - Atlantis Press SP - 99 EP - 102 SN - 2352-538X UR - https://doi.org/10.2991/csic-15.2015.23 DO - 10.2991/csic-15.2015.23 ID - Du2015/07 ER -