Proceedings of the 2019 International Conference on Mathematics, Big Data Analysis and Simulation and Modelling (MBDASM 2019)

Data Cleaning Algorithms for Power Information Communication Assets Data Based on Self-coding

Authors
Qiang Wang, Jianliang Zhang, Jian Wu, Feng Gao, Xuewu Ren
Corresponding Author
Feng Gao
Available Online October 2019.
DOI
10.2991/mbdasm-19.2019.39How to use a DOI?
Keywords
big data;data clean; self-coding network; residual analysis
Abstract

With the continuous application of new information and communication technology in the asset management of power information and communication equipment, a large amount of data will be generated in all aspects of asset management. In data acquisition, due to sensor faults and other reasons, data anomalies will inevitably occur in the data set, which causes great trouble to the subsequent data analysis. In this paper, a data cleaning algorithm based on stack self-encoder (DCbS) is proposed, which can improve the ability of distinguishing and recovering outliers of data by saving the short-term correlation between data in sliding windows and residual analysis between noisy data and lossless data. Finally, the advantages of the algorithm are highlighted from two aspects: data recovery and outlier identification.

Copyright
© 2019, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the 2019 International Conference on Mathematics, Big Data Analysis and Simulation and Modelling (MBDASM 2019)
Series
Advances in Computer Science Research
Publication Date
October 2019
ISBN
10.2991/mbdasm-19.2019.39
ISSN
2352-538X
DOI
10.2991/mbdasm-19.2019.39How to use a DOI?
Copyright
© 2019, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Qiang Wang
AU  - Jianliang Zhang
AU  - Jian Wu
AU  - Feng Gao
AU  - Xuewu Ren
PY  - 2019/10
DA  - 2019/10
TI  - Data Cleaning Algorithms for Power Information Communication Assets Data Based on Self-coding
BT  - Proceedings of the 2019 International Conference on Mathematics, Big Data Analysis and Simulation and Modelling (MBDASM 2019)
PB  - Atlantis Press
SP  - 170
EP  - 173
SN  - 2352-538X
UR  - https://doi.org/10.2991/mbdasm-19.2019.39
DO  - 10.2991/mbdasm-19.2019.39
ID  - Wang2019/10
ER  -