Satoshi Eguchi, Wataru Kawasaki, Yuji Shirasaki, Yutaka Komiya, George Kosugi, Masatoshi Ohishi, Yoshihiko Mizumoto
ALMA is estimated to generate TB scale data during only one observation; astronomers manage to identify which part of the data they are really interested in. Now we have been developing new GUI software for this purpose utilizing the VO interface: ALMA Web Quick Look System (ALMAWebQL) and ALMA Desktop Application (Vissage). The former is written in JavaScript and HTML5 generated from Java codes by Google Web Toolkit, and the latter is in pure Java. An essential point of our approach is how to reduce network traffic: we prepare, in advance, "compressed" FITS files of 2x2x1 (horizontal, vertical, and spectral directions, respectively) binning, 2x2x2 binning, 4x4x2 binning data, and so on. These files are hidden from users, and Web QL automatically choose proper one by each user operation. Through this work, we find that network traffic in our system is still a bottleneck towards TB scale data distribution. Hence we have to develop alternative data containers for much faster data processing. In this paper, I introduce our data analysis systems, and describe what we learned through the development.
View original:
http://arxiv.org/abs/1211.3790
No comments:
Post a Comment