|
| 1 | +.. _design-details: |
| 2 | + |
| 3 | +================ |
| 4 | + Design Details |
| 5 | +================ |
| 6 | + |
| 7 | +---------------------- |
| 8 | +Data Model Performance |
| 9 | +---------------------- |
| 10 | + |
| 11 | +There are two primary costs associated with CDEPS share code: reading data and spatially mapping data. |
| 12 | +Time interpolation is relatively cheap in the current implementation. |
| 13 | +As much as possible, redundant operations are minimized. |
| 14 | +The upper and lower bound mapped input data is saved between time steps to reduce mapping costs in cases where data is time interpolated more often than new data is read. |
| 15 | +If the input data timestep is relatively small (for example, hourly data as opposed to daily or monthly data) the cost of reading input data can be quite large. |
| 16 | +Also, there can be significant variation in cost of the data model over the coarse of the run, for instance, when new inputdata must be read and interpolated, although it's relatively predictable. |
| 17 | +The present implementation doesn't support changing the order of operations, for instance, time interpolating the data before spatial mapping. |
| 18 | +Because the present computations are always linear, changing the order of operations will not fundamentally change the results. |
| 19 | +The present order of operations generally minimizes the mapping cost for typical data model use cases. |
| 20 | + |
| 21 | +---------------------- |
| 22 | +IO Through Data Models |
| 23 | +---------------------- |
| 24 | + |
| 25 | +At the present time, data models can only read netcdf data, and IO is handled through the PIO library using either netcdf or pnetcdf. |
| 26 | +PIO can read the data either serially or in parallel in chunks that are approximately the global field size divided by the number of IO tasks. |
| 27 | +If pnetcdf is used through PIO, then the pnetcdf library must be included during the build of the model. |
| 28 | + |
| 29 | +---------------------------------- |
| 30 | +IO Through Data Models In CIME-CCS |
| 31 | +---------------------------------- |
| 32 | + |
| 33 | +If CDEPS is used in CIME, the pnetcdf path and option is hardwired |
| 34 | +into the ``Macros.make`` file for the specific machine. To turn on |
| 35 | +``pnetcdf`` in the build, make sure the ``Macros.make`` variables |
| 36 | +``PNETCDF_PATH``, ``INC_PNETCDF``, and ``LIB_PNETCDF`` are set and |
| 37 | +that the PIO ``CONFIG_ARGS`` sets the ``PNETCDF_PATH`` argument. |
| 38 | +Beyond just the option of selecting IO with PIO, several namelist variables are available to help optimize PIO IO performance. |
| 39 | +Those are **TODO** - list these. |
| 40 | +The total mpi tasks that can be used for IO is limited to the total number of tasks used by the data model. |
| 41 | +Often though, using fewer IO tasks results in improved performance. |
| 42 | +In general, [io_root + (num_iotasks-1)*io_stride + 1] has to be less than the total number of data model tasks. |
| 43 | +In practice, PIO seems to perform optimally somewhere between the extremes of 1 task and all tasks, and is highly machine and problem dependent. |
| 44 | + |
| 45 | +------------- |
| 46 | +Restart Files |
| 47 | +------------- |
| 48 | +Restart files are generated automatically by the data models based on an attribute flag received in the NUOPC cap. |
| 49 | +The restart files must meet the CIME-CCS naming convention and an ``rpointer`` file is generated at the same time. |
| 50 | +An ``rpointer`` file is a *restart pointer* file which contains the name of the most recently created restart file. |
| 51 | +Normally, if restart files are read, the restart filenames are specified in the ``rpointer`` file. |
| 52 | +Optionally though, there are namelist variables such as ``restfilm`` to specify the restart filenames via namelist. If those namelist variables are set, the ``rpointer`` file will be ignored. |
| 53 | + |
| 54 | +In most cases, no restart file is required for the data models to restart exactly. |
| 55 | +This is because there is no memory between timesteps in many of the data model science modes. |
| 56 | +If a restart file is required, it will be written automatically and then must be used to continue the previous run. |
| 57 | + |
| 58 | +There are separate stream restart files that only exist for |
| 59 | +performance reasons. A stream restart file contains information about |
| 60 | +the time axis of the input streams. This information helps reduce the |
| 61 | +startup costs associated with reading the input dataset time axis |
| 62 | +information. If a stream restart file is missing, the code will |
| 63 | +restart without it but may need to reread data from the input data |
| 64 | +files that would have been stored in the stream restart file. This |
| 65 | +will take extra time but will not impact the results. |
| 66 | + |
| 67 | +.. _data-structures: |
| 68 | + |
| 69 | +--------------- |
| 70 | +Stream Modules |
| 71 | +--------------- |
| 72 | + |
| 73 | +The CDEPS stream code contains four modules: |
| 74 | + |
| 75 | +**dshr_strdata_mod.F90** |
| 76 | + Carries out stream IO along with the spatial and |
| 77 | + temporal interpolation of the stream data to the model mesh and |
| 78 | + model time. Initializes the module data type ``shr_strdata_type``. |
| 79 | + |
| 80 | +**dshr_stream_mod.F90** |
| 81 | + Reads in the stream xml file and returns the upper and |
| 82 | + lower bounds of the stream data. Initializes the module data type |
| 83 | + ``shr_stream_streamType``. |
| 84 | + |
| 85 | +**dshr_tinterp_mod.F90** |
| 86 | + Determines the time interpolation factors. |
| 87 | + |
| 88 | +**dshr_methods_mod.F90** |
| 89 | + Wrappers to ESMF such as getting a pointer to a field in a field bundle, etc. |
| 90 | + |
| 91 | +---------------- |
| 92 | +Stream Datatypes |
| 93 | +---------------- |
| 94 | + |
| 95 | +The most basic type, ``shr_stream_fileType`` is contained in |
| 96 | +``shr_stream_mod.F90`` and specifies basic information related to a |
| 97 | +given stream file. |
| 98 | + |
| 99 | +.. code-block:: Fortran |
| 100 | +
|
| 101 | + type shr_stream_fileType |
| 102 | + character(SHR_KIND_CL) :: name = shr_stream_file_null ! the file name |
| 103 | + logical :: haveData = .false. ! has t-coord data been read in? |
| 104 | + integer (SHR_KIND_IN) :: nt = 0 ! size of time dimension |
| 105 | + integer (SHR_KIND_IN),allocatable :: date(:) ! t-coord date: yyyymmdd |
| 106 | + integer (SHR_KIND_IN),allocatable :: secs(:) ! t-coord secs: elapsed on date |
| 107 | + end type shr_stream_fileType |
| 108 | +
|
| 109 | +The following type, ``shr_stream_streamType`` contains information |
| 110 | +that encapsulates the information related to all files specific to a |
| 111 | +target stream. (see the overview of the :ref:`stream_description_file`). |
| 112 | + |
| 113 | +.. code-block:: Fortran |
| 114 | +
|
| 115 | + type shr_stream_streamType |
| 116 | + !private ! no public access to internal components |
| 117 | + integer :: logunit ! stdout log unit |
| 118 | + type(iosystem_desc_t), pointer :: pio_subsystem |
| 119 | + integer :: pio_iotype |
| 120 | + integer :: pio_ioformat |
| 121 | + logical :: init = .false. ! has stream been initialized |
| 122 | + integer :: nFiles = 0 ! number of data files |
| 123 | + integer :: yearFirst = -1 ! first year to use in t-axis (yyyymmdd) |
| 124 | + integer :: yearLast = -1 ! last year to use in t-axis (yyyymmdd) |
| 125 | + integer :: yearAlign = -1 ! align yearFirst with this model year |
| 126 | + character(CS) :: taxMode = shr_stream_taxis_cycle ! cycling option for time axis |
| 127 | + character(CS) :: tInterpAlgo = 'linear' ! algorithm to use for time interpolation |
| 128 | + character(CS) :: mapalgo = 'bilinear' ! type of mapping - default is 'bilinear' |
| 129 | + character(CS) :: readMode = 'single' ! stream read model - 'single' or 'full_file' |
| 130 | + real(r8) :: dtlimit = 1.5_r8 ! delta time ratio limits for time interpolation |
| 131 | + integer :: offset = 0 ! offset in seconds of stream data |
| 132 | + character(CS) :: calendar = shr_cal_noleap ! stream calendar (obtained from first stream data file) |
| 133 | + character(CL) :: meshFile = ' ' ! filename for mesh for all fields on stream (full pathname) |
| 134 | + integer :: k_lvd = -1 ! file/sample of least valid date |
| 135 | + integer :: n_lvd = -1 ! file/sample of least valid date |
| 136 | + logical :: found_lvd = .false. ! T <=> k_lvd,n_lvd have been set |
| 137 | + integer :: k_gvd = -1 ! file/sample of greatest valid date |
| 138 | + integer :: n_gvd = -1 ! file/sample of greatest valid date |
| 139 | + logical :: found_gvd = .false. ! T <=> k_gvd,n_gvd have been set |
| 140 | + logical :: fileopen = .false. ! is current file open |
| 141 | + character(CL) :: currfile = ' ' ! current filename |
| 142 | + integer :: nvars ! number of stream variables |
| 143 | + character(CL) :: stream_vectors ! stream vectors names |
| 144 | + type(file_desc_t) :: currpioid ! current pio file desc |
| 145 | + type(shr_stream_file_type) , allocatable :: file(:) ! filenames of stream data files (full pathname) |
| 146 | + type(shr_stream_data_variable), allocatable :: varlist(:) ! stream variable names (on file and in model) |
| 147 | + end type shr_stream_streamType |
| 148 | +
|
| 149 | +Finally, the datatypes ``shr_strdata_per_stream`` and |
| 150 | +``shr_strdata_type`` in ``dshr_strdata_mod.F90`` are at the heart |
| 151 | +of the CDEPS stream code and contains information for |
| 152 | +all the streams that are active for the target data model. |
| 153 | + |
| 154 | +.. code-block:: Fortran |
| 155 | +
|
| 156 | + type shr_strdata_perstream |
| 157 | + character(CL) :: stream_meshfile ! stream mesh file from stream txt file |
| 158 | + type(ESMF_Mesh) :: stream_mesh ! stream mesh created from stream mesh file |
| 159 | + type(io_desc_t) :: stream_pio_iodesc ! stream pio descriptor |
| 160 | + logical :: stream_pio_iodesc_set =.false. ! true=>pio iodesc has been set |
| 161 | + type(ESMF_RouteHandle) :: routehandle ! stream n -> model mesh mapping |
| 162 | + character(CL), allocatable :: fldlist_stream(:) ! names of stream file fields |
| 163 | + character(CL), allocatable :: fldlist_model(:) ! names of stream model fields |
| 164 | + integer :: stream_lb ! index of the Lowerbound (LB) in fldlist_stream |
| 165 | + integer :: stream_ub ! index of the Upperbound (UB) in fldlist_stream |
| 166 | + type(ESMF_Field) :: field_stream ! a field on the stream data domain |
| 167 | + type(ESMF_Field) :: stream_vector ! a vector field on the stream data domain |
| 168 | + type(ESMF_FieldBundle), allocatable :: fldbun_data(:) ! stream field bundle interpolated to model grid |
| 169 | + type(ESMF_FieldBundle) :: fldbun_model ! stream n field bundle interpolated to model grid and time |
| 170 | + integer :: ucomp = -1 ! index of vector u in stream |
| 171 | + integer :: vcomp = -1 ! index of vector v in stream |
| 172 | + integer :: ymdLB = -1 ! stream ymd lower bound |
| 173 | + integer :: todLB = -1 ! stream tod lower bound |
| 174 | + integer :: ymdUB = -1 ! stream ymd upper bound |
| 175 | + integer :: todUB = -1 ! stream tod upper bound |
| 176 | + real(r8) :: dtmin = 1.0e30_r8 |
| 177 | + real(r8) :: dtmax = 0.0_r8 |
| 178 | + type(ESMF_Field) :: field_coszen ! needed for coszen time interp |
| 179 | + end type shr_strdata_perstream |
| 180 | +
|
| 181 | +.. code-block:: Fortran |
| 182 | +
|
| 183 | + type shr_strdata_type |
| 184 | + type(shr_strdata_perstream), allocatable :: pstrm(:) ! stream info |
| 185 | + type(shr_stream_streamType), pointer :: stream(:)=> null() ! stream datatype |
| 186 | + integer :: nvectors ! number of vectors |
| 187 | + logical :: masterproc |
| 188 | + integer :: logunit ! stdout unit |
| 189 | + integer :: io_type ! pio info |
| 190 | + integer :: io_format ! pio info |
| 191 | + integer :: modeldt = 0 ! model dt in seconds |
| 192 | + type(ESMF_Mesh) :: model_mesh ! model mesh |
| 193 | + real(r8), pointer :: model_lon(:) => null() ! model longitudes |
| 194 | + real(r8), pointer :: model_lat(:) => null() ! model latitudes |
| 195 | + integer :: model_nxg ! model global domain lon size |
| 196 | + integer :: model_nyg ! model global domain lat size |
| 197 | + integer :: model_nzg ! model global domain vertical size |
| 198 | + integer :: model_lsize ! model local domain size |
| 199 | + integer, pointer :: model_gindex(:) ! model global index spzce |
| 200 | + integer :: model_gsize ! model global domain size |
| 201 | + type(ESMF_CLock) :: model_clock ! model clock |
| 202 | + character(CL) :: model_calendar = shr_cal_noleap ! model calendar for ymd,tod |
| 203 | + integer :: ymd, tod ! model time |
| 204 | + type(iosystem_desc_t), pointer :: pio_subsystem => null() ! pio info |
| 205 | + real(r8) :: eccen = SHR_ORB_UNDEF_REAL ! cosz t-interp info |
| 206 | + real(r8) :: mvelpp = SHR_ORB_UNDEF_REAL ! cosz t-interp info |
| 207 | + real(r8) :: lambm0 = SHR_ORB_UNDEF_REAL ! cosz t-interp info |
| 208 | + real(r8) :: obliqr = SHR_ORB_UNDEF_REAL ! cosz t-interp info |
| 209 | + real(r8), allocatable :: tavCoszen(:) ! cosz t-interp data |
| 210 | + end type shr_strdata_type |
0 commit comments