Storage Mechanism of Processing Magnanimity Small Files Applying HDFS Architecture

DOI

10.2991/icsem.2013.167How to use a DOI?

Keywords

HDFS, I/O Efficiency, Disk Buffer.

Abstract

Pointing at the low efficiency problem of I/O operations of small files using HDFS application architecture, this article proposes a new method, that is, add disk buffer on DataNode to reduce addressing time of reading small files, and thus reduce reading time, and reduce the stress bringed by frequent NameNode access, so as to optimize the reading efficiency of small files. Experiment results show that the design programme is feasible, and adding disk buffer benefits for increasing efficiency of processing small files applying HDFS.

Copyright

© 2013, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Xin Wang
AU  - Jun Ma
PY  - 2013/04
DA  - 2013/04
TI  - Storage Mechanism of Processing Magnanimity Small Files Applying HDFS Architecture
BT  - Proceedings of the 2nd International Conference On Systems Engineering and Modeling (ICSEM 2013)
PB  - Atlantis Press
SP  - 812
EP  - 815
SN  - 1951-6851
UR  - https://doi.org/10.2991/icsem.2013.167
DO  - 10.2991/icsem.2013.167
ID  - Wang2013/04
ER  -