Proceedings of the Advances in Materials, Machinery, Electrical Engineering (AMMEE 2017)

A Fine-Grained Task Monitoring Mechanism in Spark Platform

Authors
Cheng Chen, Fei Liu, Guangrui Li, Xiang Chen, Ying Hou
Corresponding Author
Cheng Chen
Available Online June 2017.
DOI
10.2991/ammee-17.2017.75How to use a DOI?
Keywords
Spark, In-memory Computing, Fine-grained, Task monitoring.
Abstract

The existing coarse-grained monitoring mechanism in Spark has much difficulty in analyzing and locating the bottleneck during the execution of tasks. Aiming to solve this problem, we divide the execution process of Spark tasks into several subphases, and propose a fine-grained subphases-based monitoring mechanism. The fine-grained monitoring mechanism can help users to understand the detail execution status of Spark tasks and be beneficial to analyze and locate the performance bottleneck in Spark.

Copyright
© 2017, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the Advances in Materials, Machinery, Electrical Engineering (AMMEE 2017)
Series
Advances in Engineering Research
Publication Date
June 2017
ISBN
978-94-6252-350-0
ISSN
2352-5401
DOI
10.2991/ammee-17.2017.75How to use a DOI?
Copyright
© 2017, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Cheng Chen
AU  - Fei Liu
AU  - Guangrui Li
AU  - Xiang Chen
AU  - Ying Hou
PY  - 2017/06
DA  - 2017/06
TI  - A Fine-Grained Task Monitoring Mechanism in Spark Platform
BT  - Proceedings of the Advances in Materials, Machinery, Electrical Engineering (AMMEE 2017)
PB  - Atlantis Press
SP  - 395
EP  - 399
SN  - 2352-5401
UR  - https://doi.org/10.2991/ammee-17.2017.75
DO  - 10.2991/ammee-17.2017.75
ID  - Chen2017/06
ER  -