Test Spark Installation on Linux Clusters
Lee-Hur Shing, Lee-Pin Shing, Marn-Ling Shing, Chen-Chi Shing
Available Online June 2018.
- 10.2991/eame-18.2018.37How to use a DOI?
- spark cluster; big data; cloud computing
Apache Spark is an open source big data analytic framework on clusters. It runs on top of Hadoop distributed clusters but faster using in-memory processing. Very often it seems run normally even Hadoop runs abnormally. This paper describes that after install Spark 1.4.0 over Hadoop 2.7.0, how to test both Hadoop and Spark on CentOS 7.0 Linux clusters.
- © 2018, the Authors. Published by Atlantis Press.
- Open Access
- This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Cite this article
TY - CONF AU - Lee-Hur Shing AU - Lee-Pin Shing AU - Marn-Ling Shing AU - Chen-Chi Shing PY - 2018/06 DA - 2018/06 TI - Test Spark Installation on Linux Clusters BT - Proceedings of the 2018 3rd International Conference on Electrical, Automation and Mechanical Engineering (EAME 2018) PB - Atlantis Press SP - 177 EP - 181 SN - 2352-5401 UR - https://doi.org/10.2991/eame-18.2018.37 DO - 10.2991/eame-18.2018.37 ID - Shing2018/06 ER -