MAPL I/O
From Maplcode.org
Contents |
[edit] Introduction
MAPL_CFIO provides Climate and Forecast (CF)
compliant I/O methods for high level ESMF data types by using the
CFIO Library. It currently includes read-write support for ESMF
Bundles and States, and read-only support for ESMF Fields and
Fortran arrays. The API consists of 4 basic methods:
This page presents an introduction to each of these, emphasizing their capabilities and limitations and showing examples of their usage. For a detailed presentation of the API, please consult the MAPL_CFIO Programmer's Reference Manual.
[edit] Reading a file
When reading data from a CFIO compliant file, in the very least the user needs to specify the file name, the time, and an ESMF object to receive the data. There is no need to explicitly involve the MAPL_CFIO object in this case. For example, assuming that one has already defined an ESMF grid and clock, here is how to read a single time instance of all variables from a file into an ESMF Bundle:
bundle = ESMF_BundleCreate ( name='Mary', grid=grid )
call MAPL_CFIORead ( 'forecast_data.nc', clock, bundle )
This method will read all variables on file, doing any necessary (horizontal) regriding to the ESMF grid used to create the bundle, and allocating memory for each variable, as necessary. Currently, the file is open, read from and subsequently closed. MAPL_CFIO also provides methods to read single variables into a simple fortran arrays:
real :: ps(im,jm)
call MAPL_CFIORead ( 'surfp', 'forecast_data.nc', clock, grid, ps )
This method will read the variable named 'surfp' into the 2D Fortran array ps, performing any necessary (horizontal) interpolation to the destination grid. Consult the MAPL_CFIO Programmer's Reference Manual for several optional parameters to the MAPL_CFIORead() method, including the ability to select the variables to read and transparently perform time interpolation.
[edit] Writing to a file
For writing, a new file is created, written to, and explicitly closed. Assuming one has already defined an ESMF bundle and clock here is how to save a bundle to a new file:
type(MAPL_CFIO) :: mcfio
call MAPL_CFIOcreate ( mcfio, 'climate_data', clock, bundle )
call MAPL_CFIOwrite ( mcfio, clock )
call MAPL_CFIOdestroy ( mcfio )
Consult the MAPL_CFIO Programmer's Reference Manual for many optional parameters controling the behavior of these methods. As of this writing, a MAPL_CFIOopen() function to write to an already existing file has not been implemented.
[edit] File formats
[edit] Supported Formats
MAPL_CFIO is designed to work with a variety of file formats, provided these files can be annotaded with the necessary CF metadata. The particularities of the specific file format is handled by the backend CFIO library. As of this writing the backend CFIO library supports self-describing formats such as NetCDF and HDF, and support for GrADS compatible binary files is in alpha testing. There are also plans to add support for GRIB versions 1, 2 or both.
[edit] Self-desbring (SDF) formats
The support for SDF formats is implemented in the backend CFIO library using the NetCDF Version 2 API. This API is currently supported by NetCDF versions 2 through 4 and HDF version 4. By selecting one (and only one) of these libraries at build time it is possible to read/write several versions of NetCDF/HDF, as summarized in the following table:
Library Reads Writes
----------- ------------- ----------------
HDF-4 NetCDF, HDF-4 HDF-4
NetCDF-2 NetCDF NetCDF
NetCDF-3 NetCDF NetCDF
NetCDF-4 NetCDF, HDF-5 NetCDF, HDF-5
NetCDF versions 2 and 3 can only read/write their own native NetCDF format. HDF version 4 offers some form of interoperability with NetCDF, but it can only write HDF-4 files. The new NetCDF version 4 is written on top of the HDF-5 library. This version of NetCDF still retains the ability of reading and writing its legacy NetCDF format, but advanced features such as compression is only available when writing in HDF-5 format. Beware that NetCDF-4 can only read the particular kind of HDF-5 files it writes; it cannot read generic HDF-5 files such has HDF-5 EOS. Because the standard HDF-5 library (without NetCDF-4) no longer supports the NetCDF 2 API (or the native HDF-4 API for that matter) it cannot be used with the SDF backend of the CFIO library.
It is important to note that, because of conflicts in the API, one cannot load more than one NetCDF or HDF library in one application.
[edit] Future Work
The MAPL_CFIO package is still under active development. The current state of the API was dictated by the features needed to build the GEOS-5 model at NASA/GSFC, and some asymmetry still remains in the API. In particular, the read methods utilize file names to specify the file object, while the write methods uses the MAPL_CFIO object much like a file handle. Both methods of access are valid and useful under different circunstances, and ought to be supported for both read and write operations. When using file name access mode the following should be possible:
call MAPL_CFIORead ( 'forecast_data.nc', clock, bundle )
call MAPL_CFIOWrite ( 'new_file.nc', clock, bundle )
call MAPL_CFIOWrite ( 'existing_file.nc', clock, bundle, append=.true. )
In this case, the file is opened, read from/written to and closed on exit. For users desiring to keep files open in between operations a file handle mode should be provided for both read and write. Here is a typical use case for reading:
mcfio = MAPL_CFIOopen ( 'forecast_data.nc' )
call MAPL_CFIORead ( mcfio, clock_now, bundle )
...
call MAPL_CFIORead ( mcfio, clock_later, bundle )
call MAPL_CFIOdestroy ( mcfio )
Future versions of MAPL_CFIO may support these features.
