Goals

The main motivation for processing ALMA data in quasi-real time is to optimize the scientific efficiency of the array. The instrument will be dynamically scheduled (Section 4), so an evaluation of the data quality must be available very soon after data taking, using visibilities in quasi real time, in order to allow switching projects if the current one is not matched to the actual observing conditions. Though readily available information on the array behavior can be obtained by monitoring atmospheric data (such as water content, and it fluctuations), more valuable information can be obtained by monitoring the atmospheric phase itself (using phase calibrators). Finally it is important to be able to determine quite soon if a project's goals are being attained, and for this a first step is naturally to calibrate the data as completely as feasible, and evaluate the quality of that calibration. The instantaneous u,v coverage being reasonably good, pipeline data processing can include not only calibration but also imaging using, if possible, the best solution to the inversion problem: diagnostic tools could eventually select between competitive methods. Another motivation is naturally to make the instrument more accessible to first-time users by producing images in a quasi-automated mode.

The required data quality level will be specified by the astronomers in their proposals, so that the output of the data pipeline can be used to decide when a project is completed. This can be e.g. on the basis of a certain rms noise level at a certain spatial resolution, a dynamic range, the achieved angular resolution or eventually the translation of these into more technical specifications such as rms phase uncertainties on calibrators, bandpass calibration accuracy, tolerance on side lobe levels in the synthesized beam, etc...

The pipeline must be able to process systematically the quasi totality of the measurements obtained with the array in a fully automated procedure. Its output will constitute a data archive with rather homogeneous properties. However their quality will not necessarily be optimal: human intervention will often be required to enhance the quality of the output. These final results should also be archived too, but in an other base of reduced data. Comparison between these two databases can be very useful to optimize the observing and data reduction procedures. These two databases, being often of much more modest size than the raw database, will be much more easily manageable and accessible through the Internet. This should maximize the use of ALMA observations, e.g. for the preparation of new projects by the proposing astronomers or, when they become publicly accessible, for direct scientific use in a different astronomical context than that of the original proposal.

Pipeline data processing will also enhance the efficiency of interactive observing, either by the astronomer if so requested in the proposal, or by the staff during technical time. The data pipeline will not only provide to the astronomer the possibility of adjusting the observing strategy following the results in quasi-real time, but also of running projects more efficiently in focus with their scientific objectives. High level specifications could actually be given during Phase 2 of the proposal submission procedure. To illustrate this, let us consider a proposal with the following requirements: some wide region of the sky must be imaged in the continuum in the 1.2 mm window; this wide field imaging must reach a certain specified rms sensitivity; all compact sources found above 5 $\sigma$ in that image have to be imaged in pointed mode, one field per source, at higher frequency down to again a certain sensitivity limit such that their spatial morphology can be investigated. Such a high level of specifications implies the need of high level measurement tools as part of the data pipeline such as a source extractor to blindly find sources and their positions; in this case the observing procedure will include several observing modes (mosaicing, pointed observations, multi-frequency) and it will be set dynamically, on the basis of results obtained by the data pipeline during the sequence of observations.