Towards Query Performance Improvement in Big Data Environment

DOI

10.2991/978-94-6463-034-3_80How to use a DOI?

Keywords

Query efficiency; Data organization; Spark SQL; HBase + Phoenix

Abstract

In recent years, big data has exploded in the communications industry, especially in the 4G industry card data. However, the large amount of data and processing dimensions of 4G industry card data incur a large overhead of query performance. In order to improve its query efficiency, this paper studies the impact of data organization on query performance, and proposes a data organization framework suitable for 4G industry card data. The proposed framework integrates data organization patterns of Spark SQL and HBase + Phoenix, and can automatically select the appropriate data organization for different business types. The experimental results show that the proposed data organization framework improves the query efficiency by 35.35% on average.

Copyright

© 2023 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Jianbo Li
AU  - Jianghua Luo
AU  - Zhiwen Song
AU  - Zhao Luo
PY  - 2022
DA  - 2022/12/23
TI  - Towards Query Performance Improvement in Big Data Environment
BT  - Proceedings of the 2022 3rd International Conference on Big Data and Informatization Education (ICBDIE 2022)
PB  - Atlantis Press
SP  - 772
EP  - 788
SN  - 2589-4900
UR  - https://doi.org/10.2991/978-94-6463-034-3_80
DO  - 10.2991/978-94-6463-034-3_80
ID  - Li2022
ER  -