Building quotient cube with MapReduce in hadoop
- DOI
- 10.2991/isccca.2013.161How to use a DOI?
- Keywords
- Data warehousing, Quotient cube, Hadoop plateform, MapReduce model, cloud computing Datas
- Abstract
In order to solve the problem that how to improve the efficiency of query and calculation in massive data, a method of building quotient cubes in Hadoop plateform which combined the advantage of the quotient cube and MapReduce model is proposed in this paper. At first, all cubes will be established and their aggregate value will be calculated in the Mapping stage. All the key/value pair formed in Mapping stage will be passed to Reducing stage. Equivalence partitioning will be carried out In this stage, and the minimum aggregation cube of each equivalence partitioning will be the key with its aggregate value. According to the minimum aggregation cubes, we can get the quotient cubes. In order to improve the speed of parallel computing and reduce network traffic, equivalence class division will be executed locally after the Map stage, it is named as combiner stage. In this paper, MapReduce model is used to improve the efficiency of building quotient cube because of its ability of parallel computing in a large amount of data. In addition, the experiment proved that, under certain circumstances, increasing the number of Mapper/Reducer task can reduce the building time effectively, and improve the construction efficiency.
- Copyright
- © 2013, the Authors. Published by Atlantis Press.
- Open Access
- This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Cite this article
TY - CONF AU - Juan Zhang PY - 2013/02 DA - 2013/02 TI - Building quotient cube with MapReduce in hadoop BT - Proceedings of the 2nd International Symposium on Computer, Communication, Control and Automation (ISCCCA 2013) PB - Atlantis Press SP - 643 EP - 646 SN - 1951-6851 UR - https://doi.org/10.2991/isccca.2013.161 DO - 10.2991/isccca.2013.161 ID - Zhang2013/02 ER -