LMDZ Setup

LMDZ_Setup is a set of scripts that allows a light automatic setup of LMDZ long chained climate simulations, including the installation of the model itself.

LMDZ_Setup can only be used on the Jean-Zay computer at Idris so far.

Coming VERY soon (summer 2024) : extension to other computing centers ("spirit", adastra) and Linux PCs

Documentation (in French): LMDZ_Setup_HowTo

To extract LMDZ_Setup :

a/ Recommended : extraction from svn repository :

   svn co https://svn.lmd.jussieu.fr/LMDZ/BOL/LMDZ_Setup

b/ Download of the "tar" archive :

   wget https://lmdz.lmd.jussieu.fr/pub/Training/LMDZ_Setup.tar

c/ For those used with the old name "tutorial_prod.tar" : a link with this name still exists, pointing to LMDZ_Setup.tar :

   wget https://lmdz.lmd.jussieu.fr/pub/Training/tutorial_prod.tar

(Text extracted from LMDZ_Setup/README0_HowTo):

LMDZ_Setup consists in running LMDZ coupled to Orchidee for land surface (optional), but with imposed sea surface temperature.

The basic default configuration makes use of the IPSL-CM6A grid configuration and tuning, and runs a multi annual simulation on "climatological" amip sea surface temperature (with a mean annual cycle) using a calendar with 360 days.

Optionally : the configuration includes the ability to use interannually varying SST and to activate "nudging" by a reanalysis. In both cases, the calendar is then a real one. When nudging is activated, the simulation must be run on a monthly basis, while otherwise, it can be either monthly or yearly.

Since LMDZ_Setup automatically generates its own initial files, it can be run with zoom configurations by only changing the number of grid points ("resol" in main.sh) and the DEF/gcm.def file (see bellow).

Aerosols can be read for the year 2000 (weighted average over 1999-2001 cf Lurton et al 2020), and instantaneous forcing with respect to 1850 can be computed as well.

LMDZ_Setup can also run LMDZ coupled with the SPLA model (SimPLe Aerosol, activated with option aerosols=spla in setup.sh). Emissions of dust and sea salt are then computed interactively. For the time being, SPLA-specific input files (anthropic aerosol emissions)are only available on the grid zoomed over N Africa used by J. Escribano in his PhD thesis, and by B. Diallo and H. Senghor in subsequent work on N-African dust (file DEF/gcm.def_zNAfrica_BiJe).

A configuration with isotopes is also available since 2023-04.

---

WIP - Migration de LMDZ_Setup_HowTo + inclusion nouveau LMDZ_Setup

TODO Amaury: reprendre à "SSTs et SIC" + vérifier réponse Adriana pour xios+omp>2

Old documentation (French): LMDZ_Setup_HowTo

User guide

 /!\ WARNING 06/2024 - (ignore if installing a new version)
The JeanZay calculator, as well as Adastra, do not allow (anymore) communication from/to $STORE for running jobs on the main computing partitions.
As a result, make sure to replace $STORE by $WORK in lmdz_env.sh wherever relevant.

READ comments at the top of scripts ! Often your questions are answered there.

Note: LMDZ_Setup has been tested on Adastra and Spirit, and to a lesser extent on JeanZay (legacy support only). No guarantee is made on other platforms, although it should be easy to adapt it. It can also be ran locally - for expert use only.

Downloading LMDZ_Setup

• (recommended) Using subversion: svn co https://svn.lmd.jussieu.fr/LMDZ/BOL/LMDZ_Setup
• (alternative) wget https://lmdz.lmd.jussieu.fr/pub/Training/LMDZ_Setup.tar
• (deprecated) The old tutorial_prod version: wget https://lmdz.lmd.jussieu.fr/pub/Training/tutorial_prod.tar

Structure

Once you have downloaded LMDZ_Setup, you will find the following main files:

• main.sh: the main script. It contains the basic settings you will modify.
• setup.sh: this script contains the internal workings of LMDZ_Setup. It's where you can edit expert-level settings.
• lmdz_env.sh: this file contains platform-specific configuration, such as where to install the model, where to run the simulations, etc.

How to run a simulation

1. Edit lmdz_env.sh. In lmdz_env.sh, in the function set_env, find the case corresponding to your supercomputer: jean-) for JeanZay, spiri) for Spirit, adast) for Adastra. Edit the variables required for your use case. All the variables are documented in the last *) case. In particular:
   • you must set root_dir to the path where you extracted LMDZ_Setup.
2. Edit main.sh. In main.sh, edit all relevant settings.
3. [optional] Edit setup.sh. In setup.sh, in the function define_expert_options, edit all relevant settings.
4. [optional] Edit DEF/*. Edit the DEF/*.def or DEF/XMLfiles* files, such as config.def, to your liking. The options you specify in main.sh/setup.sh don't require any adjustment here; such adjustments are automatically performed at runtime.
5. Launch main.sh.

Miscellanous notes

• Simulations run in a subdirectory of $SIMRUNBASEDIR with the same name as the LMDZ_Setup folder (if you have extracted LMDZ_Setup and renamed it to My_Setup then simulations will run in $SIMRUNBASEDIR/My_Setup). Each simulation is contained in its own folder, named after the simulation's name + a process number. As a result, you don't need to clean $SIMRUNBASEDIR before relaunching a simulation.
• If your simulation crashes, look directly in $SIMRUNBASEDIR for the relevant log.
• If you need a version of ORCHIDEE different from the one provided by default in the model (such as CMIP6), a password is required - ask orchidee-help at listes.ipsl.fr if you need help.

XIOS

/!\ Although XIOS is not required for ORCHIDEE >=2.2, not using it will disable some features.

Different sets of *.xml files are available in LMDZ_Setup/DEF/XMLFiles*. However, only some common files (histf, histday, histmth) have been adapted for LMDZ_Setup. If you need other outputs, be sure to edit/check the relevant *.xml:

• replace the _AUTO_ for output_level and enabled by actual values;
• use compression_level=0 instead of the default values (2 or 4) (since LMDZ_Setup uses XIOS in attached mode rather than client-server mode, using non-zero compression levels would require running XIOS on 1+ dedicated MPI).

Note: After installing the model, two files are automatically initialized in DEF/XMLfiles*: context_lmdz.xml and field_def_lmdz.xml. Those files don't require user intervention, and must match the exact LMDZ revision used.

Compilation

When launching main.sh, before launching the simulation, we first download and compile the required models. Although we call the compilation script every time, if the model has already been compiled before, the script should detect that and skip the compilation, thus execute very quickly.

/!\ As written in the comments of main.sh/setup.sh, if you change the veget version or the aerosols, it is highly recommended to do so in a NEW clean LMDZ_Setup folder rather than recompiling in the same folder.

Cluster-specific information

Generic

ORCH 4 (trunk)

ORCHIDEE 4 is currently not able to compile with gfortran, and as a result can't be used. This has been reported to the relevant team, and hopefully should be fixed soon (06/2024).

CDO & aerosols

The current version of cdo on Adastra (2.4.0) and Spirit (2.3.0) contain a bug which interferes with the script interpolating aerosols. This bug is supposedly solved in version 2.4.1+. This bug corrupts the aerosols files, and results in a hgardfou error when running the simulation.

JeanZay

JeanZay was the main supported cluster of the old LMDZ_Setup. Although the new version can be ran there, it is discouraged.

Slowness

Compilation of the various components of the model, in particular of LMDZ and XIOS, is extremely slow on JeanZay compared to other clusters (for unknown reasons) (~15 minutes for LMDZ+rrtm). Moreover, it seems that for uninvestigated reasons, the LMDZ model is systematically recompiled on each call to main.sh (expected behavior, as on other clusters: the fcm system detects that it's already compiled and avoids compiling again). As a result, the whole process of launching a simulation is much slower.