Really Huge Problems

IMAGER basically assumes everything fits into memory. This is in general quite fine for NOEMA data. However, for ALMA data, if you are working with a too small computer (such as my laptop, which is otherwise fine), you may be lacking physical memory (RAM, Random Access Memory), and IMAGER may become really inefficient by using Virtual Memory instead.

To avoid time losses, IMAGER prevents reading UV data whose size exceeds the available RAM, and warns the user if it exceeds half of the RAM. To treat these cases, IMAGER provides instead a number of tools that can work sequentially on the data set, instead of loading it in memory all at once.

1. Working by subsets:
   the READ /RANGE command allows to select an ensemble of channels from a UV data. If this ensemble is small enough, IMAGER can work. At the end the WRITE /APPEND and WRITE /REPLACE command will allow to put these channels at their proper places in a deconvolved data cube.
2. Working on UV data files:
   Some operations on UV data, like time averaging, separation of line from continuum emission, or self-calibration, and of course, spectral resampling, are best done using all valid channels to avoid loosing sensitivity. To allow IMAGER to do them even for large data files, most UV-related commands have a /FILE option which instructs the command to work from the corresponding data file, instead of the UV buffers. This includes UV_PREVIEW /FILE, UV_BASELINE /FILE, UV_FILTER /FILE and especially the UV_SPLIT /FILE commands. Time averaging can be performed by UV_TIME /FILE, and prior sorting by time order can be done by UV_SORT /FILE. Spectral range extraction is possible by UV_EXTRACT /FILE, spectral resampling by UV_COMPRESS /FILE, UV_RESAMPLE /FILE, and UV_HANNING /FILE.

By using the above commands, all operations can be done in a quasi sequential way, avoiding to load in memory whole data sets.

Two commands have no equivalent using the IMAGER buffers, and work only on files. Their /FILE option is used to provide an homogeneous syntax, but must be present for the command line to be valid.

1. the UV_MERGE /FILE
   that allows to merge together an arbitrary number of UV tables, in spectral line (with spectral resampling) or continuum (with flux scaling according to a spectral index) modes.
2. the UV_SPLIT /FILE
   that combines the capabilities of UV_BASELINE and UV_FILTER in a single command, since both operations require the same parameters and provide complementary informations.

The UV_SORT /FILE command also has a different behaviour than its memory only version UV_SORT. While the latter creates a transposed version of the UV table for internal use (it cannot be saved), the former keeps the normal organisation with the visibility axis first.

An example of use of such facilities is the @  image-mosaic script that splits the uv_map step into chunks that fit in the computer memory available to the user.