Physical Computing Resource Scheduling on Distributed Stream Processing System

DOI

10.2991/ncce-18.2018.62How to use a DOI?

Keywords

real-time data; workload; system-tier; stream processing.; elastic scheduling.

Abstract

Recently, distributed stream computing has emerged as a popular model for real-time data stream analytics. This paper focuses on computing resources scheduling of distributed stream processing. To adapt to the fluctuation of data stream, distributed stream processing system must elastically provision the computing resources based on the observed workload. However, there is no system that satisfies all these requirements. Motivated by this, this paper presents a physical resource scheduling algorithm for elastic scheduling computing resources of stream processing task. Base on existing task tier elastic resource scheduling and system-tier resource scheduling, this paper proposes a collaborative algorithm. According to the fluctuant workload, task-tier dynamic resource scheduler config computing resource of stream processing task. Meanwhile, based on the scheduling decision of task-tier scheduler, the system-tier resource scheduler is assign physical computing resources to stream processing tasks.

Copyright

© 2018, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Kailin Tang
PY  - 2018/05
DA  - 2018/05
TI  - Physical Computing Resource Scheduling on Distributed Stream Processing System
BT  - Proceedings of the 2018 International Conference on Network, Communication, Computer Engineering (NCCE 2018)
PB  - Atlantis Press
SP  - 385
EP  - 388
SN  - 1951-6851
UR  - https://doi.org/10.2991/ncce-18.2018.62
DO  - 10.2991/ncce-18.2018.62
ID  - Tang2018/05
ER  -