Proceedings of the 2017 2nd International Conference on Control, Automation and Artificial Intelligence (CAAI 2017)

A Highly-Efficient Fault Tolerance Method for a Scalable Stream Processing System

Authors
Guanghui Chang, Peizhen Li, Guangxia Xu
Corresponding Author
Guanghui Chang
Available Online June 2017.
DOI
10.2991/caai-17.2017.50How to use a DOI?
Keywords
SPS; dynamic scale out; fault tolerance
Abstract

An effective fault-tolerant mechanism becomes essential to the credible stream processing system (SPS). However, the traditional introduction of fault-tolerant mechanisms has a relatively negative influence on the systemic calculation efficiency. And enhancing the system scale out is an effective way to solve this problem. This paper proposes a highly-efficient fault recovery method that is for an extensible SPS. On the one hand, based on a reasonable partition and upstream backup of its internal calculation status. The SPS can monitor the bottleneck state of the operators and then migrate the upstream backup state of the operators that is in need of scale out to the new nodes to achieve dynamic scale out of the SPS; On the other hand, when a node fault in the SPS, the system can dynamically expand a secondary node for the erroneous one, and then use the upstream backup algorithm to achieve fault recovery. In this paper, a mathematical model is built for the SPS. And based on this, the fault recovery time and computational efficiency of the rapid fault tolerant method are analyzed. Experimental results have shown that the one proposed in this paper is more effective in fault tolerance which gives the system a higher calculation efficiency by reducing the frequency of upstream backup of node state.

Copyright
© 2017, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the 2017 2nd International Conference on Control, Automation and Artificial Intelligence (CAAI 2017)
Series
Advances in Intelligent Systems Research
Publication Date
June 2017
ISBN
978-94-6252-360-9
ISSN
1951-6851
DOI
10.2991/caai-17.2017.50How to use a DOI?
Copyright
© 2017, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Guanghui Chang
AU  - Peizhen Li
AU  - Guangxia Xu
PY  - 2017/06
DA  - 2017/06
TI  - A Highly-Efficient Fault Tolerance Method for a Scalable Stream Processing System
BT  - Proceedings of the 2017 2nd International Conference on Control, Automation and Artificial Intelligence (CAAI 2017)
PB  - Atlantis Press
SP  - 226
EP  - 230
SN  - 1951-6851
UR  - https://doi.org/10.2991/caai-17.2017.50
DO  - 10.2991/caai-17.2017.50
ID  - Chang2017/06
ER  -