Proceedings of the 2nd International Symposium on Computer, Communication, Control and Automation (ISCCCA 2013)

High Performance Data Processing Pipeline Of Chinese Solar Radio Heliograph

Authors
Fan Zhang, Feng Wang, Wei Wang, Wei Dai, Hui Deng, Kaifan Ji, Yihua Yan
Corresponding Author
Fan Zhang
Available Online February 2013.
DOI
https://doi.org/10.2991/isccca.2013.134How to use a DOI?
Keywords
CSRH, data processing, HPC
Abstract
The Chinese Solar Radio Heliograph (CSRH) is a new generation radio heliograph would produce more than 4 terabytes data every day. As a aperture synthesis telescope, CSRH is facing the challenge of processing and storing such a vast data. Pipeline system is the key issue of data automatical processing for CSRH. In this study, to push the development of CSRH, we present a framework of high performance data processing pipeline system for saving and processing real-time observation data. The related techniques of pipeline software are presented in detail including raw data acquisition, UVFITS file, data calibration, parallel computing and data publication. The pipeline has been deployed and has played an important role for the development of CSRH.
Open Access
This is an open access article distributed under the CC BY-NC license.

Download article (PDF)

Volume Title
Proceedings of the 2nd International Symposium on Computer, Communication, Control and Automation (ISCCCA 2013)
Series
Advances in Intelligent Systems Research
Publication Date
February 2013
ISBN
978-90-78677-63-5
ISSN
1951-6851
DOI
https://doi.org/10.2991/isccca.2013.134How to use a DOI?
Open Access
This is an open access article distributed under the CC BY-NC license.

Cite this article

TY  - CONF
AU  - Fan Zhang
AU  - Feng Wang
AU  - Wei Wang
AU  - Wei Dai
AU  - Hui Deng
AU  - Kaifan Ji
AU  - Yihua Yan
PY  - 2013/02
DA  - 2013/02
TI  - High Performance Data Processing Pipeline Of Chinese Solar Radio Heliograph
BT  - Proceedings of the 2nd International Symposium on Computer, Communication, Control and Automation (ISCCCA 2013)
PB  - Atlantis Press
SP  - 535
EP  - 538
SN  - 1951-6851
UR  - https://doi.org/10.2991/isccca.2013.134
DO  - https://doi.org/10.2991/isccca.2013.134
ID  - Zhang2013/02
ER  -