MAPL interface to POSEIDON
From Maplcode.org
Contents |
The MAPL interface to Poseidon is implemented through a Plug component that wraps the Poseidon object and methods.
Poseidon [1] is a hybrid coordinate ocean model developed in the Department of Climate Dyamics[2], College of Science, George Mason University[3]. Paul Schopf is the lead architect and developer.
Poseidon is an object-oriented code that uses the GEMS parallelization libraries for communications, input and output, and the Zeus clock/calendar system. This plug allows the GEMS decomposition, layout and clocks to function in sync with the ESMF versions, then uses the MAPL framework to present Poseidon as an ESMF gridded component that can work with any ESMF based superstructure.
[edit] Docs and Code
The protex generated documentation for the the MAPL based Poseidon plug can be is here (POS_GEOS5PlugMod2.pdf).
Fortran90 code is here.
The actual model codes are contained in two sub-directories ./neptune and ./eq_state.
[edit] Implementation Notes
The Poseidon MAPL Plug component is a MAPL component without children.
[edit] Poseidon Structure
Poseidon was originally built with dynamically defined objects for the ocean, the grid associated with it, a forcing object and a clock. A supervisor created these and could make many instances.
In the above example, a supervisor runing one clock could have several instances of the ocean running on two separate grids. One set could run off a single forcing, the others group might have one forcing for each ocean. The custom supervisor program has to instantiate and initialize all the objects, set some alarms, tick the clock, and run all the instances.
In order to make a MAPL wrapper, the basic structure of the component was built to include the 4 elements:
the ocean, forcing, grid and clock.
! These are the Poseidon objects
type POS_MAPL_Type
type (T_poseidon_ocean ) :: ocean
type (T_poseidon_grid ) :: grid
type (T_poseidon_forcing) :: oforce
type (T_poseidon_history) :: ohist
type (Z_clock) :: zclock
Type (Z_Alarm ) :: EOR_Alarm ! Alarm for stopping Poseidon
Type (Z_Alarm ) :: history_Alarm ! Alarm for writing history
Type (Z_Alarm ) :: history_Acc_Alarm ! Alarm for accumulating history averages
integer :: history_unit ! Fortran unit number for writing history data
integer :: grads_descriptor_unit ! Fortran unit number for writing GrADS control
logical :: write_control_file = .true. ! Should we write GrADS control?
type (Z_DateTimeInterval) :: historyInterval
character(ESMF_MAXSTR) :: ohist_template
end type POS_MAPL_Type
type POS_MAPLWrap_Type
type(POS_MAPL_Type), pointer :: Ptr
end type POS_MAPLWrap_Type
At the module level we declare a type to hold these objects. This version works OK for simple wrapping of one ocean with one grid. A drawback is that instantiation of multiple oceans would require multiple copies of grids, and the oceans could not share a single grid via pointers. Since there is no means to call the Initialize method with a pointer to a non-ESMF structure, I could not figure out how to use pointers to grids to advantage.
[edit] SetServices
As with all MAPL components, the SetServices identifies the Import and Export quantities that will be required or available. In designing the Guest_Ocean component that will run Poseidon, MOM, and MITgcm plugs, the team decided upon a specific set of forcing quantities that the ocean models will share. It is the Plug component's responsibility to transform quantities or units, etc. But the list of imports presented in SetServices includes the basic set.
Poseidon also exports the basic set of quantities required by the Guest_Ocean component. We are working to export all the Poseidon diagnostics as well, so they can be surfaced to the MAPL History. There is no reason additional exports can not be added to the list used here.
After the Ex/Im quantities are declared, the entry points for Init, Run and Finalize are set using the MAPL (not ESMF) GridCompSetEntryPoint. This communicates the information up to the MAPL layer, so that the final call to MAPL_GenericSetServices can resolve them.
Finally, a number of basic timers are added and the MAPL_GenericSetServices routine is called.
[edit] Initialize
(Line numbers reference those in the Fortran source available above or here.)
! Begin...
! Get the target components name and set-up traceback handle.
! -----------------------------------------------------------
Iam = "Poseidon_Initialize"
call ESMF_GridCompGet( gc, NAME=comp_name, RC=status )
VERIFY_(STATUS)
Iam = trim(comp_name) // trim(Iam)
! Get my internal MAPL_Generic state (Only used for profilers in this routine)
!-----------------------------------
call MAPL_GetObjectFromGC ( GC, MAPL, RC=STATUS)
VERIFY_(STATUS)
! Profilers
!----------
call MAPL_TimerOn(MAPL,"TOTAL" )
call MAPL_TimerOn(MAPL,"INITIALIZE")
! Generic initialize
! ------------------
call MAPL_GenericInitialize( GC, IMPORT, EXPORT, CLOCK, RC=status )
VERIFY_(STATUS)
Line 520: Initialize, the MAPL_GenericInitialize routine is called at the start (not at the end, as in SetServices). In SetServices, you are passing information up the hierarchy, and you do this by loading information to the MAPL layer, then letting MAPL handle the transfer up. In initialize, you are getting instructions going down the hierarchy, so you process the MAPL_GenericInitialize first, then you aree free to do what you need to do.
! Get the grid, configuration
!----------------------------
call ESMF_GridCompGet( GC, grid=ESMFGrid, CONFIG = CF, RC=status )
VERIFY_(STATUS)
! Get the layout from the grid
!-----------------------------
call ESMF_GridGet(ESMFGrid, deLayout=Layout,rc=STATUS)
VERIFY_(STATUS)
! Get the dimensions of the DElayout
!-----------------------------------
call ESMF_DELayoutGet(Layout, dimCount=dimcount, RC=STATUS )
VERIFY_(STATUS)
ASSERT_(dimcount == 2)
call ESMF_DELayoutGetVm(Layout, VM=VM, RC=STATUS )
VERIFY_(STATUS)
! GEMS initialization
!------------------------------------------------------
call ESMF_VMGet(VM, mpiCommunicator=Comm, rc=STATUS)
VERIFY_(STATUS)
call GEMS_cp_Initarch(Comm)
Lines 529 - 543 and 580 - 614: Because Poseidon will not use ESMF for communications, halo filling, etc. it needs to set up its own communications environment. It uses the GEMS libraries, which permit decompositions and communications on an MPI communicator. These can be set up in flexible ways, so we pattern GEMS to work with the ESMF Grid object passed to the POS_Plug component. We extract the deLayout and VM (then MPI Communicator) from the ESMF Grid, then initialize GEMS to use this same communicator. In the internal routine Initialize_grids_and_layouts handles the details of this setup.
! These are the Poseidon objects
type POS_MAPL_Type
type (T_poseidon_ocean ) :: ocean
type (T_poseidon_grid ) :: grid
type (T_poseidon_forcing) :: oforce
type (T_poseidon_history) :: ohist
type (Z_clock) :: zclock
Type (Z_Alarm ) :: EOR_Alarm ! Alarm for stopping Poseidon
Type (Z_Alarm ) :: history_Alarm ! Alarm for writing history
Type (Z_Alarm ) :: history_Acc_Alarm ! Alarm for accumulating history averages
integer :: history_unit ! Fortran unit number for writing history data
integer :: grads_descriptor_unit ! Fortran unit number for writing GrADS control
logical :: write_control_file = .true. ! Should we write GrADS control?
type (Z_DateTimeInterval) :: historyInterval
character(ESMF_MAXSTR) :: ohist_template
#ifdef ASSIM_MODE
type (t_odas_iau) :: iau ! Incremental analysis updating object
#endif
end type POS_MAPL_Type
type POS_MAPLWrap_Type
type(POS_MAPL_Type), pointer :: Ptr
end type POS_MAPLWrap_Type
! Allocate this instance of the internal state and put it in wrapper.
! -------------------------------------------------------------------
allocate( POS_MAPL_internal_state, stat=status )
VERIFY_(STATUS)
wrap%ptr => POS_MAPL_internal_state
! Save pointer to the wrapped internal state in the GC
! ----------------------------------------------------
call ESMF_UserCompSetInternalState ( GC, 'POS_MAPL_state',wrap,status )
VERIFY_(STATUS)
Lines 560 - 573: The Poseidon model runs with a set of independent objects: the ocean object, a grid, an object holding the forcing, a history object, and a clock. When running Poseidon stand-alone, three Zeus alarms are typically used: an end-of-run alarm (EOR_Alarm), an alarm to control the writing of history (history_Alarm), and an alarm to control the accumulation of samples for history (history_Acc_Alarm). All of these objects are wrapped into a single POS_MAPL_Type During Initialize, this composite type must be instantiated. The Plug code declares POS_MAPL_internal_state as a pointer to a POS_MAPL_Type, and early on allocates this pointer. This instance is then added to the gridded component's internal state, so it is carried with the component in all uses. Separate calls to Initialize for a multi-model ensemble, for instance, will instantiate separate oceans, forcings, clocks, etc, by instantiating separate POS_MAPL_Type pointers.
ocean => POS_MAPL_internal_state%ocean
oforce => POS_MAPL_internal_state%oforce
ohist => POS_MAPL_internal_state%ohist
grid => POS_MAPL_internal_state%grid
zclock => POS_MAPL_internal_state%zclock
Lines 575 - 579:
Because POS_Plug is designed to be intimately connected to Poseidon, use separate pointers for objects of the T_poseidon_ocean, T_poseidon_grid, etc. type for convenience.
call Initialize_ocean( rc=status ) VERIFY_(status)
Line 620: Here an internal routine is called to retreive the Poseidon state from the private (ZDF format) restart file. The Poseidon restart file contains binary representations of all quantities required for transparent restart, including parameter values, clock alarm status, state variables and derived quantities that may be held in accumulators. The state of the T_Poseidon_ocean object is restored to its condition at the moment of checkpoint.
call Setup_clocks( rc=status ) VERIFY_(status)
Lines 547 - 557 and 623: Here the ESMF clock information is extracted, and the Zeus clock is set to agree with the ESMF clock.
call Choose_diagnostics( rc=status ) VERIFY_(status) call Setup_history( rc=status ) VERIFY_(status)
Line 626 - 629: Poseidon has many diagnostics, and a method for choosing which ones to compute and save to a history file. The internal routines Choose_diagnostics and Setup_history are used to get the user's input via the ESMF Config file, then initialize the ocean history object.
call Validate_ocean( ocean, QUIET=.false.)
Line 624: The state of the ocean and its run parameters are next checked to be sure they are within bounds and reasonable. This is the stage at which a model gone wrong is usually caught.
[edit] Run
! Begin
!------
! Get the component's name and set-up traceback handle.
! -----------------------------------------------------
Iam = "Run"
call ESMF_GridCompGet( gc, NAME=comp_name, RC=status )
VERIFY_(status)
Iam = trim(comp_name) // Iam
! Get my internal MAPL_Generic state
!-----------------------------------
call MAPL_GetObjectFromGC ( GC, MAPL, RC=status)
VERIFY_(STATUS)
! Profilers
!----------
call MAPL_TimerOn (STATE,"TOTAL")
call MAPL_TimerOn (STATE,"RUN" )
! Get GuestModel's private internal state
!---------------------------------
CALL ESMF_UserCompGetInternalState( GC, 'POS_MAPL_state', WRAP, STATUS )
VERIFY_(STATUS)
POS_MAPL_internal_state => WRAP%PTR
! shortcut pointers
!---------------------------------
ocean => POS_MAPL_internal_state%ocean
grid => POS_MAPL_internal_state%grid
oforce => POS_MAPL_internal_state%oforce
ohist => POS_MAPL_internal_state%ohist
zclock => POS_MAPL_internal_state%zclock
stop_Alarm => POS_MAPL_internal_state%EOR_Alarm
history_Alarm => POS_MAPL_internal_state%history_Alarm
history_Acc_Alarm => POS_MAPL_internal_state%history_Acc_Alarm
Lines 2042-2084: The first things that Run needs to do are to get the MAPL_MetaComp from the gridded component and to extract the POS_MAPL_state. Once the internal state pointer is obtained, we set some other convenience pointers the the T_Poseidon_ocean, T_Poseidon_grid, and other objects in the internal state.
! Get IMPORT pointers
!--------------------
call MAPL_GetPointer(IMPORT, TAUX, 'TAUX', RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(IMPORT, TAUY, 'TAUY', RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(IMPORT, U3, 'USTAR3', RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(IMPORT, PS, 'PS' , RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(IMPORT, HeatFlux, 'HFLX', RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(IMPORT, WaterFlux, 'QFLX', RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(IMPORT, SaltFlux, 'SFLX', RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(IMPORT, RadHeating, 'SWHEAT', RC=STATUS); VERIFY_(STATUS)
! Get EXPORT pointers
!--------------------
call MAPL_GetPointer(EXPORT, US, 'US' , RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(EXPORT, VS, 'VS' , RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(EXPORT, TS, 'TS' , RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(EXPORT, SS, 'SS' , RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(EXPORT, DH, 'DH' , RC=STATUS); VERIFY_(STATUS)
call MAPL_GetPointer(EXPORT, MASK, 'MASK' , RC=STATUS); VERIFY_(STATUS)
Lines 2089-2106: Next, we get pointers to the imports and exports and will tart go through the process of converting and loading fluxes into the T_Poseidon_forcing object that the model will use to run the ocean.
call assign_forcing( grid, HeatFlux, oforce%HeatFlux, rc=STATUS)
VERIFY_(STATUS)
call assign_forcing( grid, WaterFlux, oforce%WaterFlux, rc=STATUS)
VERIFY_(STATUS)
! We get surface heat flux with the enthalpy of rain water
! computed as waterFlux * T * C_p, where T is in Kelvin
! but we use it as though T is in Centigrade
! remove the difference before Poseidon uses it:
oforce%HeatFlux = oforce%HeatFlux - oforce%WaterFlux * CTOKELV * CP_OCN
call assign_forcing( grid, SaltFlux, oforce%SaltFlux, rc=STATUS)
VERIFY_(STATUS)
call assign_vector_forcing( grid, TAUX, TAUY, oforce%taux, oforce%tauy, rc=STATUS)
VERIFY_(STATUS)
call assign_forcing( grid, U3, oforce%U3, rc=STATUS)
VERIFY_(STATUS)
Lines 2118-2138: Here we start to copy imports to the ocean forcing object. Some are straight copies, some are missing, and some need to be derived.
The assign_forcing internal procedure handles halo filling and unit conversion, a repetitive task for all the quantities.
By agreement, we have decided that the import state variable HeatFlux includes the enthalpy of water added to the ocean, where the heat content of this water is ρCpT where temperature in Kelvin. Poseidon expects a similar enthalpy flux, but computes it based on temperature in Centigrade. We do a conversion here.
call Compute_Buoyancy_Flux( grid, oforce%WaterFlux, &
oforce%HeatFlux,oforce%SaltFlux,&
ocean%state%T(:,:,1),&
ocean%state%S(:,:,1)+ocean%state%Sbias, &
ocean%state%H(:,:,1), &
oforce%BuoyancyFlux, rc=status)
Line 2140: We compute the buoyancy flux here, which is needed by the Poseidon mixed layer.
oforce%Qrad_net(:,:) = 0.0 ! We handle this directly, not in poseidon
Line 2148: The version of Poseidon used in this implementation recognizes that penetrating radiation and the resultant radiative heating are computed by a separate (sibling) component, and it takes those tendencies from the import state and applies them to the ocean, skipping the built-in computation of radiative heating. There are some potential problems with this method, especially when computing the mixed layer turbulence. At this point, we set the Qrad_net part of the ocean forcing to zero. This ensures that the only way the penetrating radiative heating is included is through the import state heating.
call ESMF_ClockGet( clock, currTime=eCurrTime, &
timeStep=eTimeStep, rc=STATUS)
VERIFY_(STATUS)
eEndTime = eCurrTime + eTimeStep
zEndTime = eEndTime
zCurrTime = Z_as_DateTime( zclock )
! Set the stop_alarm to start ringing when it reaches
! the end time. (Set its origin to zEndTime, and its
! interval to 1.0)
call Z_Set_Time( stop_alarm, zEndTime , status=STATUS)
VERIFY_(STATUS)
call Z_Set_Interval( stop_alarm, 1.0 , status=STATUS)
VERIFY_(STATUS)
call Z_Shut_Alarm( stop_alarm, status=STATUS)
VERIFY_(STATUS)
Lines 2159-2175: Here we set the Zeus clock so that the Poseidon model runs for the time interval requested by the ESMF clock. The convention used with MAPL is that the clock is ticked after the run methods are called, so at this point, the ESMF clock shows the time at the end of the previous time step. We add the time step to the current time to get an end time. We set an Zeus alarm to ring when it reaches this end time.
call Apply_radiative_heating( eTimeStep, rc=STATUS)
Line 2178: This is where the radiative heating tendencies get applied.
do while ( .not. Z_Ringing( stop_Alarm ) )
! Poseidon uses a tick-then-run paradigm
call Z_tick( zclock )
...
call run_poseidon( ocean,oforce )
...
enddo
Lines 2180-2256: Perform a loop over time. Run until the End Time is reached. During each step, tick the Zeus clock, step the ocean, then check for history accumulation and history writing.
if(associated(US)) US = ocean%state%U(i1:i2,j1:j2,1)*grid%cos_chi(i1:i2,j1:j2) &
+ ocean%state%V(i1:i2,j1:j2,1)*grid%sin_chi(i1:i2,j1:j2)
if(associated(VS)) VS = ocean%state%V(i1:i2,j1:j2,1)*grid%cos_chi(i1:i2,j1:j2) &
- ocean%state%U(i1:i2,j1:j2,1)*grid%sin_chi(i1:i2,j1:j2)
if(associated(TS )) then
TS = ocean%state%T(i1:i2,j1:j2,1) + CTOKELV
end if
if(associated(SS )) then
SS = ocean%state%S(i1:i2,j1:j2,1) + ocean%state%sbias
end if
if(associated(DH)) then
DH = ocean%state%H(i1:i2,j1:j2,:)
end if
if(associated(MASK)) then
MASK = grid%bh3(i1:i2,j1:j2,:)
end if
call MAPL_TimerOff(MAPL,"RUN" )
call MAPL_TimerOff(MAPL,"TOTAL" )
! All Done
!---------
RETURN_(ESMF_SUCCESS)
Lines 2264-2283: Now that the ocean has advanced its state to the reuired time, load the export quantities and return.
