Proceedings of the 2018 3rd International Conference on Electrical, Automation and Mechanical Engineering (EAME 2018)

Test Spark Installation on Linux Clusters

Authors
Lee-Hur Shing, Lee-Pin Shing, Marn-Ling Shing, Chen-Chi Shing
Corresponding Author
Lee-Hur Shing
Available Online June 2018.
DOI
10.2991/eame-18.2018.37How to use a DOI?
Keywords
spark cluster; big data; cloud computing
Abstract

Apache Spark is an open source big data analytic framework on clusters. It runs on top of Hadoop distributed clusters but faster using in-memory processing. Very often it seems run normally even Hadoop runs abnormally. This paper describes that after install Spark 1.4.0 over Hadoop 2.7.0, how to test both Hadoop and Spark on CentOS 7.0 Linux clusters.

Copyright
© 2018, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the 2018 3rd International Conference on Electrical, Automation and Mechanical Engineering (EAME 2018)
Series
Advances in Engineering Research
Publication Date
June 2018
ISBN
10.2991/eame-18.2018.37
ISSN
2352-5401
DOI
10.2991/eame-18.2018.37How to use a DOI?
Copyright
© 2018, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Lee-Hur Shing
AU  - Lee-Pin Shing
AU  - Marn-Ling Shing
AU  - Chen-Chi Shing
PY  - 2018/06
DA  - 2018/06
TI  - Test Spark Installation on Linux Clusters
BT  - Proceedings of the 2018 3rd International Conference on Electrical, Automation and Mechanical Engineering (EAME 2018)
PB  - Atlantis Press
SP  - 177
EP  - 181
SN  - 2352-5401
UR  - https://doi.org/10.2991/eame-18.2018.37
DO  - 10.2991/eame-18.2018.37
ID  - Shing2018/06
ER  -