Materialized View Selection Based on Adaptive Genetic Algorithm and Its Implementation with Apache Hive
- DOI
- 10.1080/18756891.2015.1113744How to use a DOI?
- Keywords
- materialized view, multi-dimensional lattice, genetic algorithm, cost model, adaptive, Apache Hive
- Abstract
Frequently accessed views in data warehouses are usually materialized in order to accelerate the speed of querying big data. However, the view materialization itself incurs huge costs. Moreover, some latest products of non-traditional data warehouse software, such as Apache Hive, still lack the support of ma- terialized views. In order to select the appropriate views to be materialized with the possible minimized cost, we propose a novel approach to the materialized view selection problem based on an adaptive ge- netic algorithm. We establish a cost model that integrates the query, maintenance and storage costs to evaluate the performance of approaches and measure the ï¬tness of an individual in the genetic algorithm. In addition, we introduce the adjustable factors for crossover probability and mutation probability, allow- ing the genetic algorithm to run quickly and avoid premature convergence. We also conduct extensive experiments for its implementation with Apache Hive, which query and manage large datasets residing in distributed storage. Both the simulation results and experiments on Apache Hive show that the approx- imately optimal solution for selecting materialized views can be obtained effectively using the approach presented.
- Copyright
- © 2017, the Authors. Published by Atlantis Press.
- Open Access
- This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Cite this article
TY - JOUR AU - Dongjin Yu AU - Wensheng Dou AU - Zhixiang Zhu AU - Jiaojiao Wang PY - 2015 DA - 2015/12/01 TI - Materialized View Selection Based on Adaptive Genetic Algorithm and Its Implementation with Apache Hive JO - International Journal of Computational Intelligence Systems SP - 1091 EP - 1102 VL - 8 IS - 6 SN - 1875-6883 UR - https://doi.org/10.1080/18756891.2015.1113744 DO - 10.1080/18756891.2015.1113744 ID - Yu2015 ER -