This repo contain infromation necessary for data access, and examples of data processing for 1st EERIE hackathon
https://events.hifis.net/event/913/timetable/?view=standard
The EERIE simulations are in progress, so initially examples will be based on data from other projects, but there are already some EERIE based examples as well. We will gradually update examples with EERIE simulations when they become available.
We have compiled a list of variables that we aim to make available during the hackathon. https://docs.google.com/spreadsheets/d/1HWtNO28EBd4O6PdOh5RCHIHsgQ_TByT5F4i2ugNVTfg/edit#gid=0
There are two sheets:
-
EERIE: comprises first-priority variables interpolated to a 0.25-degree regular grid. We will try to deliver these for all simulations.
-
WP7: lists second-priority variables. We will also strive to deliver these, but availability might differ among modeling groups.
Kindly plan your hackathon tasks with this in mind.
Data will be available through intake catalogs. It might be new experience for many, so please have a loot at:
- General description about data structure and how to access data through catalogs on DKRZ
- Navigating through catalogs can be tricky at first, here is an example of how to search inside some of the catalogs
Current overview of simulations:
More detailed information about the availability of data on different grids and time frequencies can be found in the readme of each individual model's subfolder.
- nextGEMS Cycle 3 simulations. Description of simulations: https://easy.gems.dkrz.de/DYAMOND/NextGEMS/index.html#id4
- eerie-control-1950 - tco1279-NG5 (atmosphere 10 km, ocean 5-13 km). Currently 2 years available.
- cat['dkrz']['disk']['model-output']['ifs-fesom2-sr']['eerie-control-1950']
- nextGEMS Cycle 2 simulations Description of simulations: https://easy.gems.dkrz.de/DYAMOND/NextGEMS/index.html#id1
- eerie-control-1950 - 5 km ocean and 10km atmosphere
- cat['dkrz']['disk']['model-output']['icon-esm-er']['eerie-control-1950']
- Please note that for the first 7 years (model years 2002-2008), the 3d daily atmosphere (not 2d) are instantaneous values. This has been rectified to daily mean values from model year 2009 onwards.
Initial data from the full eerie-piControl simulation is available:
| Model | Suite ID | Atmos Data | Ocean Data |
|---|---|---|---|
| HadGEM3-GC5-EERIE-N96-ORCA1 | cy163 | 1851 - 2081 | |
| HadGEM3-GC5-EERIE-N216-ORCA025 | cy021 | 1851 - 1981 | 1851 - 1900 |
| HadGEM3-GC5-EERIE-N640-ORCA12 | cx993 | 1851 - 1901 | 1851 - 1890 |
CMORised atmosphere data is available on JASMIN in the /gws/nopw/j04/eerie/public/data/EERIE/ directory. The folder structure follows the CMIP conventions. The ocean data has not been CMORised but the raw ocean and seaice netCDF files have been included in the same directory structure (regridded and one variable per file).
Data from AMIP (atmosphere-only forced with sea surface temperature (SST) and sea ice concentration (SIC)) runs is available. Given the purpose of the AMIP runs to study the impact of the presence of mesoscale features, the runs exist in pairs: One is forced with observed SST and SIC, taken from the OSTIA dataset (also available under #OBSERVATIONS), while in a twin experiment the observed SST anomalies are smoothed out with a filter the length scale of which is a multiple of the local Rossby radius of deformation. The multiple is indicated in the run ID, i.e. lr30 implies a factor of 30. For more details see #OBSERVATIONS.
| Run | Data | SST |
|---|---|---|
| amip-hist-obs | 2010 - 2020 | unfiltered OSTIA SST |
| amip-hist-obs-lr30 | 2010 - 2020 | OSTIA SST smoothed with 30 x Rossby radius |
| amip-ng-obs | 2020 - 2021 | unfiltered OSTIA SST |
| amip-ng-obs-lr30 | 2020 - 2021 | OSTIA SST smoothed with 30 x Rossby radius |
The "hist" runs are historical runs with time-dependent forcing based on IFS CY48R1.1, including some new EERIE source updates. The "ng" runs use the NextGEMS cycle 3 configuration, i.e. perpetual 2020 forcing (except for SST & SIC).
Here is README with description of observations
- DKRZ account for IFS/FESOM and ICON (Science Hour presentation from Fabian Wachsmann
- use python3/unstable as kernel
- JASMIN for UM/NEMO, IFS/NEMO and IFS (Science Hour presentation from Malcolm Roberts)
- How to use the earlier version of this repo
- Use of the ESGF portal to screen and download data of the Pre-EERIE simulations
The easier way to move forward is to login to the system you going to work with (DKRZ or JASMIN) and clone this repo:
git clone https://github.com/eerie-project/EERIE_hackathon_2023.gitThen go to jupyterhub of DKRZ or JASMIN and execute notebooks from there.
To login to JASMIN to clone the repository, you can either use SSH or you can use the JASMIN jupyterhub and go to Launcher (blue cross) and then choose a new Terminal.
dkrz jupyterhub https://jupyterhub.dkrz.de/
- account: bk1377 (make sure you are a member of it)
- profile: at least 50GB memory
- 20gb home. more data can be temporarily saved in /scratch/b/BNUMBER . if your account starts with m use m/MNUMBER
- server runs for specific amount of time, you can close the browser, the server continues. Unless you stop it via the hub control panel.
jasmin notebook service https://notebooks.jasmin.ac.uk
stops when there is no activity
- For unstructured meshes:
/COMMON/FESOM2_ICON_grids_easy_plot_and_interpolate.ipynb - For data interpolated to regular grid: FESOM example
- For interpolation from NEMO/FESOM/IFS/UM grid to regular grid: WORK IN PROGRESS
- For effective parallel aggregations with dask: FESOM example
Each coupled model example is located in individual folders, which are in turn split into components. Begin with START_HERE.ipynb for each component, and then explore the notebooks from the list in the README.
- Basics (access data on the original grid, get data for one time step, open grid, have a look at the data)
- Regridding to a regular grid
- Plotting global and regional maps
- Finding a point nearest to coordinates, extracting time series
- Finding a set of points closest to a transect, plotting the transect
- Effective, parallel aggregations
- Area integral, Volume integral
- Plot transect
- Curl
- Transport through a transect
- Compute AMOC
Here is the list of variables, that will be available during hackathon.
- easy.gems as a user-driven site for documenting high-resolution climate simulation output and its increasingly challenging analysis
- Cycle 2 nextGEMS Hackathon repository
- Cycle 3 nextGEMS Hackthon repository
We welcome contributions to the EERIE_hackathon_2023 project! Depending on the extent of your contributions, you can follow one of the two processes:
For small, one-time contributions, please follow the standard GitHub process:
- Fork the repository.
- Create a new branch in your fork.
- Make your changes and commit them to your branch.
- Create a Pull Request (PR) from your branch to the main repository.
If you plan to make many contributions, you may request write access to the repository. Even with write access, it's best practice to:
- Create a new branch for your changes.
- Commit your changes to that branch.
- Create a Pull Request (PR) from your branch to the main branch.
This ensures that changes are reviewed and integrated systematically. Please avoid committing directly to the main branch.
Thank you for your interest in contributing to the EERIE_hackathon_2023 project!
- IFS website
- FESOM2 GitHub repo
- FESOM2 documentation
- pyfesom2, python library to work with FESOM2 data
- fint FESOM2 interpolation command line utilities