Normalized to: Liang, B.
[1]
oai:arXiv.org:2002.09816 [pdf] - 2053428
Estimating red noise in quasi-periodic signals with MCMC-based Bayesian
Submitted: 2020-02-22
Multi-parameter Bayesian inferences based on Markov chain Monte Carlo (MCMC)
samples have been widely used to estimate red noise in solar period-periodic
signals. To MCMC, proper priors and sufficient iterations are prerequisites
ensuring the accuracy of red noise estimation. We used MCMC-based Bayesian
inferences to estimate 100 groups of red noise synthesized randomly for
evaluating its accuracy. At the same time, the Brooks-Gelman algorithm was
employed to precisely diagnose the convergence of the Markov chains generated
by MCMC. The root-mean-square error of parameter inferences to the synthetic
data is only 1.14. Furthermore, we applied the algorithm to analyze the
oscillation modes in a sunspot and a flare. A 70 s period is detected in the
sunspot umbra in addition to 3- and 5-minute periods, and a 40 s period is
detected in the flare. The results prove that estimating red noise with
MCMC-based Bayesian has more high accuracy in the case of proper priors and
convergence. We also find that the number of iterations increases dramatically
to achieve convergence as the number of parameters grows. Therefore, we
strongly recommend that when estimating red noise with MCMC-based Bayesian,
different initial values must be selected to ensure that the entire posterior
distribution is covered.
[2]
oai:arXiv.org:1701.04907 [pdf] - 1534695
OpenCluster: A Flexible Distributed Computing Framework for Astronomical
Data Processing
Wei, Shoulin;
Wang, Feng;
Deng, Hui;
Liu, Cuiyin;
Dai, Wei;
Liang, Bo;
Mei, Ying;
Shi, Congming;
Liu, Yingbo;
Wu, Jingping
Submitted: 2017-01-17
The volume of data generated by modern astronomical telescopes is extremely
large and rapidly growing. However, current high-performance data processing
architectures/frameworks are not well suited for astronomers because of their
limitations and programming difficulties. In this paper, we therefore present
OpenCluster, an open-source distributed computing framework to support rapidly
developing high-performance processing pipelines of astronomical big data. We
first detail the OpenCluster design principles and implementations and present
the APIs facilitated by the framework. We then demonstrate a case in which
OpenCluster is used to resolve complex data processing problems for developing
a pipeline for the Mingantu Ultrawide Spectral Radioheliograph. Finally, we
present our OpenCluster performance evaluation. Overall, OpenCluster provides
not only high fault tolerance and simple programming interfaces, but also a
flexible means of scaling up the number of interacting entities. OpenCluster
thereby provides an easily integrated distributed computing framework for
quickly developing a high-performance data processing system of astronomical
telescopes and for significantly reducing software development expenses.
[3]
oai:arXiv.org:1612.07572 [pdf] - 1533783
Low-cost high performance distributed data storage for multi-channel
observations
Submitted: 2016-12-22
The New Vacuum Solar Telescope (NVST) is a 1-m solar telescope that aims to
observe the fine structures in both the photosphere and the chromosphere of the
Sun. The observational data acquired simultaneously from one channel for the
chromosphere and two channels for the photosphere bring great challenges to the
data storage of NVST. The multi-channel instruments of NVST, including
scientific cameras and multi-band spectrometers, generate at least 3 terabytes
data per day and require high access performance while storing massive
short-exposure images. It is worth studying and implementing a storage system
for NVST which would balance the data availability, access performance and the
cost of development. In this paper, we build a distributed data storage system
(DDSS) for NVST and then deeply evaluate the availability of real-time data
storage on a distributed computing environment. The experimental results show
that two factors, i.e., the number of concurrent read/write and the file size,
are critically important for improving the performance of data access on a
distributed environment. Referring to these two factors, three strategies for
storing FITS files are presented and implemented to ensure the access
performance of the DDSS under conditions of multi-host write and read
simultaneously. The real applications of the DDSS proves that the system is
capable of meeting the requirements of NVST real-time high performance
observational data storage. Our study on the DDSS is the first attempt for
modern astronomical telescope systems to store real-time observational data on
a low-cost distributed system. The research results and corresponding
techniques of the DDSS provide a new option for designing real-time massive
astronomical data storage system and will be a reference for future
astronomical data storage.
[4]
oai:arXiv.org:1612.07587 [pdf] - 1533784
NVST data archiving system based on fastbit nosql database
Submitted: 2016-12-22
The New Vacuum Solar Telescope (NVST) is a 1-meter vacuum solar telescope
that aims to observe the fine structures of active regions on the Sun. The main
tasks of the NVST are high resolution imaging and spectral observations,
including the measurements of the solar magnetic field. The NVST has been
collecting more than 20 million FITS files since it began routine observations
in 2012 and produces a maximum observational records of 120 thousand files in a
day. Given the large amount of files, the effective archiving and retrieval of
files becomes a critical and urgent problem. In this study, we implement a new
data archiving system for the NVST based on the Fastbit Not Only Structured
Query Language (NoSQL) database. Comparing to the relational database (i.e.,
MySQL; My Structured Query Language), the Fastbit database manifests
distinctive advantages on indexing and querying performance. In a large scale
database of 40 million records, the multi-field combined query response time of
Fastbit database is about 15 times faster and fully meets the requirements of
the NVST. Our study brings a new idea for massive astronomical data archiving
and would contribute to the design of data management systems for other
astronomical telescopes.
[5]
oai:arXiv.org:1612.06656 [pdf] - 1533663
Distributed Data-Processing Pipeline for Mingantu Ultrawide Spectral
Radioheliograph
Wang, F.;
Mei, Y.;
Deng, H.;
Liu, C. Y.;
Liu, D. H.;
Wei, S. L.;
Dai, W.;
Liang, B.;
Liu, Y. B.;
Zhang, X. L.;
Ji, K. F.
Submitted: 2016-12-20
The Chinese Spectral RadioHeliograph (CSRH) is a synthetic aperture radio
interferometer built in Inner Mongolia, China. As a solar-dedicated
interferometric array, CSRH is capable of producing high quality radio images
at frequency range from 400 MHz to 15 GHz with high temporal, spatial, and
spectral resolution.To implement high cadence imaging at wide-band and obtain
more than 2 order higher multiple frequencies, the implementation of the data
processing system for CSRH is a great challenge. It is urgent to build a
pipeline for processing massive data of CSRH generated every day. In this
paper, we develop a high performance distributed data processing pipeline
(DDPP) built on the OpenCluster infrastructure for processing CSRH
observational data including data storage, archiving, preprocessing, image
reconstruction, deconvolution, and real-time monitoring. We comprehensively
elaborate the system architecture of the pipeline and the implementation of
each subsystem. The DDPP is automatic, robust, scalable and manageable. The
processing performance under multi computers parallel and GPU hybrid system
meets the requirements of CSRH data processing. The study presents an valuable
reference for other radio telescopes especially aperture synthesis telescopes,
and also gives an valuable contribution to the current and/or future data
intensive astronomical observations.