Planets - User contributions [en]

Tool Box Venus PCM

2024-08-14T10:00:02Z

Rcapron: /* LMDZ.VENUS/Tools */

== Post-processing tools provided with the Venus PCM ==
First and foremost there are a number of postprocessing utilities (self-standing tools) which can be found in the ''UTIL'' and ''LMDZ.VENUS/Tools'' directories. Secondly one has some extra main programs in the dynamics-physics interface.

=== UTIL ===
In the top-level ''UTIL'' directory can be found some utilities for post-processing Venus PCM outputs such as:
* zrecast : a utility to vertically interpolate PCM outputs (which are on the native hybrid sigma-pressure vertical coordinate) onto pressure or altitude levels. An example of a compiling script, <code>compile</code>, to compile ''zrecast'' is also provided; to be adapted to your local settings.
* ... To be completed ...

=== LMDZ.VENUS/Tools ===
This subdirectory contains the following utilities (check out the README file present in that directory for additional information):
* angmom : to compute angular momentum and torque components
* energy : to compute specific and integarted potential and kinetic energy
* fft : to compute the Fourier decomposition
* localtime_mean_and_std : to interpolate variables at the same local time everywhere
* psi : to compute the streamfunction
* stability: to compute stability, Richardson number and distance to cyclostrophic equilibrium
* tem : to compute TransEulerianMean variables
* tmc : to compute angular momentum transport from high-frequency outputs

The ''startarchive2icosa'' subdirectory contains some programs and instructions (see the README there) to generate a DYNAMICO-Venus set of start files from a lon-lat <code>start_archive.nc</code> file

==== Using angmom ====
First, one need to compile the subprograms using this script :
<pre>
#!/bin/bash

# Source
source /chemin/vers/votre/XIOS/arch.env
source /chemin/vers/votre/XIOS/arch.path

# Chemins NetCDF
NETCDF_INCLUDE=$NETCDF_INCDIR
NETCDF_LIB=$NETCDF_LIBDIR

# Compilation des sous-programmes
mpif90 -c -g -traceback cpdet.F90 moyzon.F moyzon2.F moytim.F dx_dp.F epflux.F90 io.F90 dmass.F90 reverse.F90 \
$NETCDF_INCLUDE $NETCDF_LIB
</pre>

Then, compile the main programs with this script :
<pre>
#!/bin/bash

# Source
source /chemin/vers/votre/XIOS/arch.env
source /chemin/vers/votre/XIOS/arch.path

# Chemins NetCDF
NETCDF_INCLUDE=$NETCDF_INCDIR
NETCDF_LIB=$NETCDF_LIBDIR

main_programs=("stability.F90" "energy.F90" "psi.F90" "tem.F90" "angmom.F90" "tmc.F90")

for program in "${main_programs[@]}"; do
program_name=$(basename "$program" .F90)
mpif90 -g -traceback "$program" *.o \
-I /chemin/vers/votre/XIOS/inc -L /chemin/vers/votre/XIOS/lib -lxios -lstdc++ \
$NETCDF_INCLUDE $NETCDF_LIB -o "${program_name}.e"
done

\rm *.o
</pre>

Be aware that, if you modify a one of the main programs and wish to re-compile it, don't forget to re-compile the subprograms first. Without, you'll end up with an issue, that the script cannot find the .o files needed for the compilation.

After doing this, you should obtain the executable <code>angmom.e</code>. Copy it in the repository that contains the NetCDF files that you want to analyse.
<pre>
cp ../../your/path/
</pre>

Then execute it
<pre>
./angmom.e
</pre>

When the program is run, you will first be asked for the name of the file you wish to use. Write it down and then type enter.
You will then be asked if you wish to use the file "dynzon.nc". If you don't want to, simply type "enter".

Normally, at the end of execution, you should obtain a file called "your_file_GAM.nc" (if the original name was "your_file.nc").

'''Nota Bene :'''
Here is a modified "alternative" script of angmom.F90, that don't take into account the <code>dyzon.nc</code> file, and, which is run only by this command :
<pre>
./angmom.e your_file.nc
</pre>

This is a way more practical use of it.

'''Nota Bene 2 :'''
If you use angmom on a file that is a bit heavy, you will probably get an execution error, specifying that the machine has not been able to allocate enough memory. To remedy this, simply run a job in a compute node. The advantage of '''Nota Bene 1''' is that it is easier to run the program in a compute node that is not necessarily interactive (using the "alternative" script).

Example of script job submission in this case (simply use <code>sbatch your_submission_script</code>) :
<pre>
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --partition=zen16
#SBATCH --mem=24G
#SBATCH -J job_angmom
#SBATCH --time=1:00:00
#SBATCH --output=job_angmom.%j.out

source /your/path/trunk/ARCH/arch-ifort_MESOIPSL.env

./angmom.e your_file.nc > angmom.out 2>&1
</pre>
You can follow the run of the program in angmom.out, using for example <code>tail -f angmom.out</code>.

== Main programs (other than GCM) but included in the Venus PCM package ==
There are a few other main programs that are included with the GCM.

Advanced stuff: these main programs are located under ''LMDZ.VENUS/libf/dynphy_lonlat/phyvenus/'' as they are at the interface between lon-lat dynamics and the Venus physics package

=== start2archive ===
This program collects multiple <code>start.nc</code> and <code>startfi.nc</code> files from a series simulations and store them in a <code>start_archive.nc</code> file. For this one simply needs to run the <code>startarchive</code> program in the directory. It will automatically fetch <code>start.nc</code> and <code>startfi.nc</code> files and generate <code>start_archive.nc</code>. If a <code>start_archive.nc</code> file is already present then the current <code>start.nc</code> and <code>startfi.nc</code> files are added to the <code>start_archive.nc</code> file (which can contain multiple initial states, as long as they are on the same grid and correspond to different dates.

The <code>startarchive</code> program should be compiled at the same resolution as the gcm which produced the start files, using the [[The_makelmdz_fcm_GCM_Compilation_Script | makelmdz_fcm]] compilation script

=== newstart ===
This program is to:
* extract (and interpolate) <code>restart.nc</code> and <code>restartfi.nc</code> files from a <code>start_archive.nc</code> file or from a pair of <code>start.nc</code> and <code>startfi.nc</code> files. The subtle difference between the two setup is that grid interpolation (horizontal and/or vertical) is only possible if using a <code>start_archive.nc</code> input file
* modify values and fields contained in the initial condition file
* Compiling <code>newstart</code> is done using the [[The makelmdz fcm GCM Compilation Script|makelmdz_fcm]] utility. The program is then meant to be run interactively with the user providing options and choices when prompted.
* Once the program has run and finished without error, it will generate <code>restart.nc</code> and <code>restartfi.nc</code>

=== rearrange_startphy ===
In a nutshell, this program allows us to convert a <code>start_archive.nc</code> coming from Venus LMDZ simulation startfiles, into a <code>start.nc</code> and <code>startphy.nc</code> files, usable to begin a '''Venus - DYNAMICO''' simulation.
You can find it here :
<code>
/your/path/LMDZ.VENUS/Tools/startarchive2icosa/
</code>

Every step is well explained in the REAMDE file contained in the same directory.

== The rcm1d 1D column program ==
The source code is located under ''LMDZ.VENUS/libf/phyvenus/dyn1d''

... Compilation...

[[Category:Venus-Model]]

Tool Box Venus PCM

2024-08-14T09:23:23Z

Rcapron: /* Main programs (other than GCM) but included in the Venus PCM package */

Managing the Venus PCM outputs

2024-08-14T08:56:05Z

Rcapron: /* Venus PCM - LMDZ outputs with IOIPSL */

This page gives some comments and instructions on managing the Venus PCM outputs, depending on the dynamical core (LMDZ or DYNAMICO) and input library (IOIPSL or XIOS) that is used.

== Generalities ==
* We here describe only the outputs from the Venus physics package, not from the dynamics
* depending on which input/output library ([[The IOIPSL Library|IOIPSL]] or [[The XIOS Library|XIOS]]) things are quite different. Outputs using IOIPSL is depreciated (but still possible); '''using XIOS should be favored and is strongly recommended'''.

== Venus PCM - LMDZ outputs with IOIPSL ==
Warning: using IOIPSL for outputs is depreciated. When outputs are with IOIPSL they consist in (at most) two files, ''histmth.nc'' (averages) and ''histins.nc'' (instantaneous fields) and
are controlled by some flags in the physiq.def file, namely:
<pre>
# Parameters for IOIPSL output files
##~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
## OLD. Now we use XIOS => see context_lmdz_physics.xml to taylor the output files
#
### OK_journe= y for daily output file histday.nc, =n no histday.nc output
### Meaningless for Venus
OK_journe=n
### OK_mensuel= y for monthly output file histmth.nc, =n no histmth.nc
### For Venus, only these averaged outputs
OK_mensuel=n
## rate (in days) at which the Venus histmth file is to be written
# sets the output rate in histmth and/or histins
ecritphy=0.1
### OK_instan=y, make some "instantaneous" outputs (same rate as histmth)
OK_instan=n
#
# Output levels for the various output files
#
# output level for "day" lev_histday
# - lev_hist*=1 => baseline 2D fields
# - lev_hist*=2 => baseline 3D fields (default)
# - lev_hist*=3 => radiative transfert
# - lev_hist*=4 => 3D tendencies
# - lev_hist*=5 => tracers and others
lev_histday=2
#output level for "mth" lev_histmth
lev_histmth=2
</pre>
Note that the setup (and flags) is inherited from the Earth model hence some weird unusable options (like '''OK_journe''') and flag names.
The user thus can trigger the generation of ''histins.nc'' outputs by setting '''OK_instan=y''' and the generation of ''histmth.nc'' by setting '''OK_mensuel=y'''. In addition the output frequency (i.e. timesteps in the output file) can be adapted using the flag '''ecritphy''' which should be set to the (Venus) day fraction to be used. As concerns the fields that will be included in the output files, these will depend on the values of flags '''lev_histday''' and '''lev_histmth''' as mentioned in the comments above. Note that if the user needs to add variables to the output files or change the default behavior then this will require modifying the source code.

If running in parallel (MPI) with outputs using IOIPSL, each process will generate each its own set of output files: histmth_0000.nc, histins_0000.nc for process number 0, histmth_0001.nc, histins_0001.nc for process number 1, and so on. Once the run is finished it is up to the user to recombine these hist*_*.nc files into single files gathering the data over the entire planet, using the IOIPSL rebuild script. It is located it the directory :
<syntaxhighlight lang="bash">
/your/path/IOIPSL/rebuild
</syntaxhighlight>

You need to copy both the executables '''rebuild''' and '''flio_rbld''' to the directory containing your '''histins/histmth.nc''' files.
Then execute it use it like that (example given on histmth, works the same with histins files), the first name is the one you want for your new file, the second (and others), are the histins/histmth.nc files you are using the rebuild on :

<syntaxhighlight lang="bash">
rebuild histmth.nc histmth_*.nc
</syntaxhighlight>

Nota Bene :
It is possible to use it like this too (guessing you have 4 histmth.nc files in this example), less practical, but good to know :
<syntaxhighlight lang="bash">
rebuild histmth.nc histmth_0001.nc histmth_0002.nc histmth_0003.nc histmth_0004.nc
</syntaxhighlight>

== Venus PCM - LMDZ outputs with XIOS ==
Note that this configuration requires running in parallel and with XIOS enabled, i.e. having compiled with makelmdz_fcm options
<pre>
-parallel mpi -io xios
</pre>
XIOS outputs are managed and defined via dedicated XML files which must be alongside the executable when it is run. Check out the [[The XIOS Library| XIOS library]] page for more details about what the XML files should contain. In a nutshell one needs the following files:
* '''iodef.xml''' : the XIOS "master" input file (which includes all the other XML files)
* '''context_lmdz_physics.xml''' : the XIOS "main" file (concerns the physics package and includes other XML files) which also contains the description of the various input and output grids to be used.
* '''field_def_physics.xml''' : file which describes all the variables that the code sends to XIOS and which might be outputted.
* '''file_def_physics.xml''' : file which describes all the desired output files and their contents
Some examples of these xml files are given in ''LMDZ.VENUS/deftank''

TODO: MORE DETAILS ON THE XML SYNTAX AND RELEVANT PARAMETERS NEEDED HERE

== Venus PCM - DYNAMICO outputs with XIOS ==
Note that using XIOS is the only possibility (no IOIPSL outputs) when using DYNAMICO

TODO: MORE DETAILS NEEDED HERE

[[Category:Venus-Model]]
[[Category:Venus-LMDZ]]

Venus - DYNAMICO

2024-07-15T12:19:19Z

Rcapron: /* Where to find .xml files */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run :

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

=== Where to find .xml files ===

See the README.md in /ICOSA_LMDZ to know what '''xml''' files to take where (note that this exemple written right after correspond to the '''current''' way of choosing '''xml files''', this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

When writing this documentation, the '''context_lmdz_physics.xml''' contained in LMDZ.VENUS/deftank is probably lacking some lines, and you should have a '''"dom_glo"''' issue when running DYNAMICO, here are the lines to change/add :

Line 7 to completely replace by : (replacing between <domain_definition> and </domain_definition>)
<pre>
<domain_group id="dom_glo" data_dim="1" >
<domain id="dom_glo" />
</domain_group>

<domain id="dom_regular" ni_glo="96" nj_glo="97" type="rectilinear" >
<generate_rectilinear_domain lat_start="-90" lat_end="90" lon_start="180" lon_end="-176.25" />
<interpolate_domain order="1"/>
</domain>

<domain id="dom_out" domain_ref="dom_regular"/>

</pre>

Then lines 44 to 46, add : (line 34 from the original file)
<pre>
<grid id="grid_2D">
<domain domain_ref="dom_glo" />
</grid>

</pre>

Then lines 50 to 57 add : (line 37 from the original file)
<pre>
</grid>

<grid id="grid_3D_out">
<domain domain_ref="dom_out" />
<axis axis_ref="altitude" />
</grid>
<grid id="grid_2D_out">
<domain domain_ref="dom_out" />

</pre>

=== Where to find .def files ===
More '''.def''' files are needed in order to run the complete '''Venus-DYNAMICO with LMDZ physics''' (compared to the Held&Suarez testCase) :

- '''run_icosa.def''' : everything linked to the DYNAMICO dynamical core will be driven in this file, see this page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_run_icosa.def_Input_File The run_icosa.def Input File]

- '''physics.def''' : everything linked to the LMDZ physics will be driven in this file.

- '''run.def''' : just a "bridge" for run_icosa.def and physics.def.

- '''z2sig.def''' : defines the vertical level of discretization, to find in LMDZ.VENUS. (there are many vertical discretization, 50, 78 etc.... 50 levels will be quicker to run, therefore the best way to test if everything works). See this page for more information : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_z2sig.def_Input_File The z2sig.def Input File]

=== Where to find others needed files ===

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T11:56:35Z

Rcapron: /* Where to find .xml files */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run :

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

=== Where to find .xml files ===

See the README.md in /ICOSA_LMDZ to know what '''xml''' files to take where (note that this exemple written right after correspond to the '''current''' way of choosing '''xml files''', this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

When writing this documentation, the '''context_lmdz_physics.xml''' contained in LMDZ.VENUS/deftank is probably lacking some lines, and you should have a '''"dom_glo"''' issue when running DYNAMICO, here are the lines to change/add :

Line 7 to completely replace by :
<pre>
<domain_group id="dom_glo" data_dim="1" >
<domain id="dom_glo" />
</domain_group>

<domain id="dom_regular" ni_glo="96" nj_glo="97" type="rectilinear" >
<generate_rectilinear_domain lat_start="-90" lat_end="90" lon_start="180" lon_end="-176.25" />
<interpolate_domain order="1"/>
</domain>

<domain id="dom_out" domain_ref="dom_regular"/>

</pre>

Then lines 44 to 46, add : (line 34 from the original file)
<pre>
<grid id="grid_2D">
<domain domain_ref="dom_glo" />
</grid>

</pre>

Then lines 50 to 57 add : (line 37 from the original file)
<pre>
</grid>

<grid id="grid_3D_out">
<domain domain_ref="dom_out" />
<axis axis_ref="altitude" />
</grid>
<grid id="grid_2D_out">
<domain domain_ref="dom_out" />

</pre>

=== Where to find .def files ===
More '''.def''' files are needed in order to run the complete '''Venus-DYNAMICO with LMDZ physics''' (compared to the Held&Suarez testCase) :

- '''run_icosa.def''' : everything linked to the DYNAMICO dynamical core will be driven in this file, see this page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_run_icosa.def_Input_File The run_icosa.def Input File]

- '''physics.def''' : everything linked to the LMDZ physics will be driven in this file.

- '''run.def''' : just a "bridge" for run_icosa.def and physics.def.

- '''z2sig.def''' : defines the vertical level of discretization, to find in LMDZ.VENUS. (there are many vertical discretization, 50, 78 etc.... 50 levels will be quicker to run, therefore the best way to test if everything works). See this page for more information : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_z2sig.def_Input_File The z2sig.def Input File]

=== Where to find others needed files ===

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T09:57:28Z

Rcapron: /* Where to find .def files */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run :

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

=== Where to find .xml files ===

See the README.md in /ICOSA_LMDZ to know what '''xml''' files to take where (note that this exemple written right after correspond to the '''current''' way of choosing '''xml files''', this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

When writing this documentation, the context_lmdz_physics.xml contained in LMDZ.VENUS/deftank is probably lacking some lines, and you should have a '''"dom_glo"''' issue when running DYNAMICO, here are the lines to change/add :

Line 7 to completely replace by :
<pre>
<domain_group id="dom_glo" data_dim="1" >
<domain id="dom_glo" />
</domain_group>

<domain id="dom_regular" ni_glo="96" nj_glo="97" type="rectilinear" >
<generate_rectilinear_domain lat_start="-90" lat_end="90" lon_start="180" lon_end="-176.25" />
<interpolate_domain order="1"/>
</domain>

<domain id="dom_out" domain_ref="dom_regular"/>

</pre>

Then lines 44 to 46, add : (line 34 from the original file)
<pre>
<grid id="grid_2D">
<domain domain_ref="dom_glo" />
</grid>

</pre>

Then lines 50 to 57 add : (line 37 from the original file)
<pre>
</grid>

<grid id="grid_3D_out">
<domain domain_ref="dom_out" />
<axis axis_ref="altitude" />
</grid>
<grid id="grid_2D_out">
<domain domain_ref="dom_out" />

</pre>

=== Where to find .def files ===
More '''.def''' files are needed in order to run the complete '''Venus-DYNAMICO with LMDZ physics''' (compared to the Held&Suarez testCase) :

- '''run_icosa.def''' : everything linked to the DYNAMICO dynamical core will be driven in this file, see this page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_run_icosa.def_Input_File The run_icosa.def Input File]

- '''physics.def''' : everything linked to the LMDZ physics will be driven in this file.

- '''run.def''' : just a "bridge" for run_icosa.def and physics.def.

- '''z2sig.def''' : defines the vertical level of discretization, to find in LMDZ.VENUS. (there are many vertical discretization, 50, 78 etc.... 50 levels will be quicker to run, therefore the best way to test if everything works). See this page for more information : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_z2sig.def_Input_File The z2sig.def Input File]

=== Where to find others needed files ===

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T09:55:43Z

Rcapron: /* Where to find .def files */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run :

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

=== Where to find .xml files ===

See the README.md in /ICOSA_LMDZ to know what '''xml''' files to take where (note that this exemple written right after correspond to the '''current''' way of choosing '''xml files''', this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

When writing this documentation, the context_lmdz_physics.xml contained in LMDZ.VENUS/deftank is probably lacking some lines, and you should have a '''"dom_glo"''' issue when running DYNAMICO, here are the lines to change/add :

Line 7 to completely replace by :
<pre>
<domain_group id="dom_glo" data_dim="1" >
<domain id="dom_glo" />
</domain_group>

<domain id="dom_regular" ni_glo="96" nj_glo="97" type="rectilinear" >
<generate_rectilinear_domain lat_start="-90" lat_end="90" lon_start="180" lon_end="-176.25" />
<interpolate_domain order="1"/>
</domain>

<domain id="dom_out" domain_ref="dom_regular"/>

</pre>

Then lines 44 to 46, add : (line 34 from the original file)
<pre>
<grid id="grid_2D">
<domain domain_ref="dom_glo" />
</grid>

</pre>

Then lines 50 to 57 add : (line 37 from the original file)
<pre>
</grid>

<grid id="grid_3D_out">
<domain domain_ref="dom_out" />
<axis axis_ref="altitude" />
</grid>
<grid id="grid_2D_out">
<domain domain_ref="dom_out" />

</pre>

=== Where to find .def files ===
More '''.def''' files are needed in order to run the complete '''Venus-DYNAMICO with LMDZ physics''' (compared to the Held&Suarez testCase) :

- '''run_icosa.def''' : everything linked to the DYNAMICO dynamical core will be driven in this file, see this page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_run_icosa.def_Input_File The run_icosa.def Input File]

- '''physics.def''' : everything linked to the LMDZ physics will be driven in this file.

- '''run.def''' : just a "bridge" for run_icosa.def and physics.def.

- '''z2sig.def''' : defines the vertical level of discretization, to find in LMDZ.VENUS. (there are many vertical discretization, 50, 78 etc.... 50 levels will be quicker to run, therefore the best way to test if everything works)

=== Where to find others needed files ===

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T09:50:20Z

Rcapron: /* Compilation - DYNAMICO */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run :

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

=== Where to find .xml files ===

See the README.md in /ICOSA_LMDZ to know what '''xml''' files to take where (note that this exemple written right after correspond to the '''current''' way of choosing '''xml files''', this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

When writing this documentation, the context_lmdz_physics.xml contained in LMDZ.VENUS/deftank is probably lacking some lines, and you should have a '''"dom_glo"''' issue when running DYNAMICO, here are the lines to change/add :

Line 7 to completely replace by :
<pre>
<domain_group id="dom_glo" data_dim="1" >
<domain id="dom_glo" />
</domain_group>

<domain id="dom_regular" ni_glo="96" nj_glo="97" type="rectilinear" >
<generate_rectilinear_domain lat_start="-90" lat_end="90" lon_start="180" lon_end="-176.25" />
<interpolate_domain order="1"/>
</domain>

<domain id="dom_out" domain_ref="dom_regular"/>

</pre>

Then lines 44 to 46, add : (line 34 from the original file)
<pre>
<grid id="grid_2D">
<domain domain_ref="dom_glo" />
</grid>

</pre>

Then lines 50 to 57 add : (line 37 from the original file)
<pre>
</grid>

<grid id="grid_3D_out">
<domain domain_ref="dom_out" />
<axis axis_ref="altitude" />
</grid>
<grid id="grid_2D_out">
<domain domain_ref="dom_out" />

</pre>

=== Where to find .def files ===
More '''.def''' files are needed in order to run the complete '''Venus-DYNAMICO with LMDZ physics''' (compared to the Held&Suarez testCase) :

- '''run_icosa.def''' : everything linked to the DYNAMICO dynamical core will be driven in this file, see this page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_run_icosa.def_Input_File The run_icosa.def Input File]

- '''physics.def''' : everything linked to the LMDZ physics will be driven in this file.

- '''run.def''' : just a "bridge" for run_icosa.def and physics.def.

=== Where to find others needed files ===

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T09:45:50Z

Rcapron: /* Where to find .def files */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

=== Where to find .xml files ===

See the README.md in /ICOSA_LMDZ to know what '''xml''' files to take where (note that this exemple written right after correspond to the '''current''' way of choosing '''xml files''', this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

When writing this documentation, the context_lmdz_physics.xml contained in LMDZ.VENUS/deftank is probably lacking some lines, and you should have a '''"dom_glo"''' issue when running DYNAMICO, here are the lines to change/add :

Line 7 to completely replace by :
<pre>
<domain_group id="dom_glo" data_dim="1" >
<domain id="dom_glo" />
</domain_group>

<domain id="dom_regular" ni_glo="96" nj_glo="97" type="rectilinear" >
<generate_rectilinear_domain lat_start="-90" lat_end="90" lon_start="180" lon_end="-176.25" />
<interpolate_domain order="1"/>
</domain>

<domain id="dom_out" domain_ref="dom_regular"/>

</pre>

Then lines 44 to 46, add : (line 34 from the original file)
<pre>
<grid id="grid_2D">
<domain domain_ref="dom_glo" />
</grid>

</pre>

Then lines 50 to 57 add : (line 37 from the original file)
<pre>
</grid>

<grid id="grid_3D_out">
<domain domain_ref="dom_out" />
<axis axis_ref="altitude" />
</grid>
<grid id="grid_2D_out">
<domain domain_ref="dom_out" />

</pre>

=== Where to find .def files ===
More '''.def''' files are needed in order to run the complete '''Venus-DYNAMICO with LMDZ physics''' (compared to the Held&Suarez testCase) :

- '''run_icosa.def''' : everything linked to the DYNAMICO dynamical core will be driven in this file, see this page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_run_icosa.def_Input_File The run_icosa.def Input File]

- '''physics.def''' : everything linked to the LMDZ physics will be driven in this file.

- '''run.def''' : just a "bridge" for run_icosa.def and physics.def.

=== Where to find others needed files ===

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T09:39:25Z

Rcapron: /* Where to find .def files */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

=== Where to find .xml files ===

See the README.md in /ICOSA_LMDZ to know what '''xml''' files to take where (note that this exemple written right after correspond to the '''current''' way of choosing '''xml files''', this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

When writing this documentation, the context_lmdz_physics.xml contained in LMDZ.VENUS/deftank is probably lacking some lines, and you should have a '''"dom_glo"''' issue when running DYNAMICO, here are the lines to change/add :

Line 7 to completely replace by :
<pre>
<domain_group id="dom_glo" data_dim="1" >
<domain id="dom_glo" />
</domain_group>

<domain id="dom_regular" ni_glo="96" nj_glo="97" type="rectilinear" >
<generate_rectilinear_domain lat_start="-90" lat_end="90" lon_start="180" lon_end="-176.25" />
<interpolate_domain order="1"/>
</domain>

<domain id="dom_out" domain_ref="dom_regular"/>

</pre>

Then lines 44 to 46, add : (line 34 from the original file)
<pre>
<grid id="grid_2D">
<domain domain_ref="dom_glo" />
</grid>

</pre>

Then lines 50 to 57 add : (line 37 from the original file)
<pre>
</grid>

<grid id="grid_3D_out">
<domain domain_ref="dom_out" />
<axis axis_ref="altitude" />
</grid>
<grid id="grid_2D_out">
<domain domain_ref="dom_out" />

</pre>

=== Where to find .def files ===
More '''.def''' files are needed in order to run the complete '''Venus-DYNAMICO with LMDZ physics''' (compared to the Held&Suarez testCase) :

- '''run_icosa.def''' : everything linked to the DYNAMICO dynamical core will be driven in this file.

- '''physics.def''' : everything linked to the LMDZ physics will be driven in this file.

- '''run.def''' : just a "bridge" for run_icosa.def and physics.def.

=== Where to find others needed files ===

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T09:30:07Z

Rcapron: /* Running Venus - DYNAMICO */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

=== Where to find .xml files ===

See the README.md in /ICOSA_LMDZ to know what '''xml''' files to take where (note that this exemple written right after correspond to the '''current''' way of choosing '''xml files''', this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

When writing this documentation, the context_lmdz_physics.xml contained in LMDZ.VENUS/deftank is probably lacking some lines, and you should have a '''"dom_glo"''' issue when running DYNAMICO, here are the lines to change/add :

Line 7 to completely replace by :
<pre>
<domain_group id="dom_glo" data_dim="1" >
<domain id="dom_glo" />
</domain_group>

<domain id="dom_regular" ni_glo="96" nj_glo="97" type="rectilinear" >
<generate_rectilinear_domain lat_start="-90" lat_end="90" lon_start="180" lon_end="-176.25" />
<interpolate_domain order="1"/>
</domain>

<domain id="dom_out" domain_ref="dom_regular"/>

</pre>

Then lines 44 to 46, add : (line 34 from the original file)
<pre>
<grid id="grid_2D">
<domain domain_ref="dom_glo" />
</grid>

</pre>

Then lines 50 to 57 add : (line 37 from the original file)
<pre>
</grid>

<grid id="grid_3D_out">
<domain domain_ref="dom_out" />
<axis axis_ref="altitude" />
</grid>
<grid id="grid_2D_out">
<domain domain_ref="dom_out" />

</pre>

=== Where to find .def files ===

=== Where to find others needed files ===

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T09:07:53Z

Rcapron: /* Running Venus - DYNAMICO */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

See the README.md in /ICOSA_LMDZ to know what files to take where (note that this is an exemple that what's written right after correspond to the '''current''' way of choosing xml.file when writing this documentation, this if therefore strongly adviced to open the README that should be always up to date) :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml
- nudging_dynamico.xml
- sponge_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T08:52:50Z

Rcapron: /* Connection Venus - DYNAMICO */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update ICOSA_LMDZ
</pre>

And the ARCH directory that contains informations about every architectures.

<pre>
cd /your/path/trunk/
svn update ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

See the README.md in /ICOSA_LMDZ to know what files to take where :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-10T08:51:03Z

Rcapron: /* Installation - DYNAMICO */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ Venus''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

One should also see the '''PCM directory layout''' page to understand and install everything : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PCM_directory_layout PCM directory layout].
That said, we will explain it once more in this page to be sure everything works.

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update -r 2655 -q ICOSA_LMDZ
</pre>

You can also make a file named '''ARCH''', and put your '''arch-YOUR_ARCH.env''' and '''arch-YOUR_ARCH.path''' files.

<pre>
cd /your/path/trunk/
mkdir ARCH
cd /your/path/trunk/ICOSAGCM/arch
cp YOUR_MACHINE.env /your/path/trunk/ARCH
cp YOUR_MACHINE.path /your/path/trunk/ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

See the README.md in /ICOSA_LMDZ to know what files to take where :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-07-04T10:00:26Z

Rcapron: /* Compilation Venus - DYNAMICO */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update -r 2655 -q ICOSA_LMDZ
</pre>

You can also make a file named '''ARCH''', and put your '''arch-YOUR_ARCH.env''' and '''arch-YOUR_ARCH.path''' files.

<pre>
cd /your/path/trunk/
mkdir ARCH
cd /your/path/trunk/ICOSAGCM/arch
cp YOUR_MACHINE.env /your/path/trunk/ARCH
cp YOUR_MACHINE.path /your/path/trunk/ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

All these compiling steps are summed up in make_isoca_lmdz :
<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -full
</pre>

Here, -full option assure the compilation of each part ('''IOIPSL''', '''XIOS''', '''LMDZ.COMMON''', '''ICOSAGCM''' and '''ICOSA_LMDZ''') of the model.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

See the README.md in /ICOSA_LMDZ to know what files to take where :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-06-25T14:27:29Z

Rcapron:

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update -r 2655 -q ICOSA_LMDZ
</pre>

You can also make a file named '''ARCH''', and put your '''arch-YOUR_ARCH.env''' and '''arch-YOUR_ARCH.path''' files.

<pre>
cd /your/path/trunk/
mkdir ARCH
cd /your/path/trunk/ICOSAGCM/arch
cp YOUR_MACHINE.env /your/path/trunk/ARCH
cp YOUR_MACHINE.path /your/path/trunk/ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

See the README.md in /ICOSA_LMDZ to know what files to take where :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

TO CONTINUE

== Using the restart.nc file to continue your simulation ==
If you want to continue you simulation using the "end-data" of your previous one, all is explained here in the DYNAMICO page : [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core#Using_the_restart.nc_file_to_continue_your_simulation using restart.nc]

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-06-25T14:11:09Z

Rcapron: /* Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, to be quick, you should change the '''"etat0"''' parameter into '''"venus"''', the '''"physics"''' parameter into '''"Lebonnois2012"''', the '''"day_step"''' (because of the long day one Venus), etc.... Rather than explaining all the different parameters that changes, here's an example of a complete script (that should work from scratch) :

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>
(one can compare with the previous run.def to see the differences)

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update -r 2655 -q ICOSA_LMDZ
</pre>

You can also make a file named '''ARCH''', and put your '''arch-YOUR_ARCH.env''' and '''arch-YOUR_ARCH.path''' files.

<pre>
cd /your/path/trunk/
mkdir ARCH
cd /your/path/trunk/ICOSAGCM/arch
cp YOUR_MACHINE.env /your/path/trunk/ARCH
cp YOUR_MACHINE.path /your/path/trunk/ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

See the README.md in /ICOSA_LMDZ to know what files to take where :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

TO CONTINUE

[[Category:Venus-Model]]

Venus - DYNAMICO

2024-06-25T14:07:11Z

Rcapron: /* Execution - Test_Case Held&Suarez - DYNAMICO */

= Venus - DYNAMICO =

Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_DYNAMICO_dynamical_core The DYNAMICO dynamical core]

== Installation - DYNAMICO ==

Before installing '''DYNAMICO''', you should have previously installed '''LMDZ''' (and everything that refers to it), see this page:
[https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

Then, you should have '''XIOS''' too, do this:

<pre>
cd /your/path/trunk/
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS
</pre>

Or see this page: [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

It is more practical for what comes next, to have each package ('''LMDZ.COMMON''', '''LMDZ.VENUS''', '''XIOS''', etc.) installed alongside each other.

Then you should compile '''XIOS''' (only once):

<pre>
cd your/path/trunk/XIOS
./make_xios --prod --arch YOUR_ARCH --job 8
</pre>

(You have to replace “YOUR_ARCH” by your architecture. Every architecture is listed in /XIOS/arch)

For each architecture, there will be 3 files: '''arch-YOUR_ARCH.env''', '''arch-YOUR_ARCH.path''', and '''arch-YOUR_ARCH.fcm'''.

No need to specify everything in the command line, just the name. For example, if my architecture is “'''ifort_MESOIPSL'''”, there will be the 3 files '''arch-ifort_MESOIPSL.env''', '''arch-ifort_MESOIPSL.path''', and '''arch-ifort_MESOIPSL.fcm''', but my command line will be:

<pre>
./make_xios --prod --arch ifort_MESOIPSL --job 8
</pre>

This will be the same thing every time you have to specify your architecture.

To install '''DYNAMICO''', you should clone the '''GitLab''' repository (once more, alongside '''XIOS''', etc.):

<pre>
cd /your/path/trunk/
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</pre>

A new folder named “'''ICOSAGCM'''” will now appear, it contains the model.

== Compilation - DYNAMICO ==

Go to the repository /ICOSAGCM, then compile the model:

<pre>
cd /your/path/trunk/ICOSAGCM/
./make_icosa -parallel mpi_omp -with_xios -arch YOUR_ARCH -job 8 -debug
</pre>

You can use the “-debug” option to compile in debug mode, but it will be slower to run.

The executable “'''icosa_gcm.exe'''” will be in ICOSAGCM/bin.

== Execution - Test_Case Venus (type Held&Suarez) - DYNAMICO ==

Now, we will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_VENUS'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_VENUS
</pre>

Then, we need to copy the specific .def files for this testCase (they are in /ICOSAGCM), we will use (in the most part) the same as in the Held&Suarez basic testCase.

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_VENUS
cp *def ../../../test_VENUS
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_VENUS

cd ..
cp iodef.xml ../../test_VENUS
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” (it is placed in ICOSAGCM/bin) in the test directory test_VENUS :

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_VENUS
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Then, there are some changes to be made to the run.def and earth_const.def files.
First, rename the '''earth_const.def''' file into '''venus_const.def''' file :
<pre>
mv earth_const.def venus_const.def
</pre>

Next, you should change the '''venus_const.def''' completely, to match the Venus atmosphere, here's an example of script :
<pre>
# Rayon planétaire (m)
radius = 6.0518e6

# Durée d'un jour (s))
daysec = 20995200

# Gravité : réel (par défaut = 8.87)
g = 8.87

# Taux de rotation planétaire
omega = 2.992e-7

# kappa=Rd/cpp
kappa = 0.2857143

# Capacité thermique à pression constante : réel (par défaut = 1004.70885)
cpp = 1004

# Pression de référence : réel (par défaut = 9200000)
preff = 9.2e6
</pre>

After this, it is time to change the '''run.def''' file, rather than explaining all the different variables that changes here's an example of a complete script (that should work from scratch):

<pre>

#---------------- Mesh ----------------

# Number of subdivisions on a main triangle : integer (default=40)
nbp = 40

# Number of vertical layers : integer (default=19)
llm = 19

# Vertical grid : [std|ncar|ncarl30;strato;strato_custom;ncar;dcmip31;dcmip200;read_apbp;plugin] (default=std)
disvert = std

# Mesh optimisation : number of iterations : integer (default=0)
optim_it = 1000

# Sub splitting of main rhombus : integer (default=1)
nsplit_i = 1
nsplit_j = 1

#number of openmp task on vertical level
omp_level_size=1

#---------------- Numerics ----------------

# Advection called every itau_adv time steps : integer (default=2)
itau_adv = 1

# Time step in s : real (default=480)
# dt = 720
# Alternative to specifying "dt", specify number of steps per day : day_step
day_step = 240000

# Number of tracers : integer (default=1)
nqtot = 1

#---------------- Time and output ----------------

# Time style : [none|dcmip] (default=dcmip)
time_style = none

# Run length in s : real (default=??)
# run_length = 1036800
# Alternative to specifying "run_length", specify number of days to run : ndays
ndays=1

# Interval in s between two outputs : integer (default=??)
write_period = 314928

#---------------- Planet ----------------

INCLUDEDEF=venus_const.def

#---------------- Physical parameters ----------------

# Initial state :
# [jablonowsky06|academic|dcmip[1-4]|heldsz|dcmip2_schaer_noshear] (default=jablonowsky06)
etat0 = venus

# Si on souhaite repartir de la "fin" d'une précédente simulation
#etat0=start_file

#start_file_name=start

# Physics package : [none|held_suarez|dcmip] (default=none)
physics = Lebonnois2012

# Dissipation time for grad(div) : real (default=5000)
tau_graddiv = 18000

# Exponent of grad(div) disspation : integer (default=1)
nitergdiv = 2

# Dissipation time for curl(curl) : real (default=5000)
tau_gradrot = 18000

# Exponent of curl(curl) disspation : integer (default=1)
nitergrot = 2

# Dissipation time for div(grad) : real (default=5000)
tau_divgrad = 18000

# Exponent of div(grad) disspation : integer (default=1)
niterdivgrad = 2

</pre>

Everything is now ready to run the model. Go to '''test_VENUS''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

== Connection Venus - DYNAMICO ==

Now that we verified that the testCase '''HELD_and_SUAREZ''' is working, we will be able to “plug” the '''DYNAMICO''' dynamical core to some real Physics. For this, you already need '''LMDZ''', alongside '''XIOS''' and '''DYNAMICO'''.

In addition, you should have '''ICOSA_LMDZ''':

<pre>
cd /your/path/trunk/
svn update -r 2655 -q ICOSA_LMDZ
</pre>

You can also make a file named '''ARCH''', and put your '''arch-YOUR_ARCH.env''' and '''arch-YOUR_ARCH.path''' files.

<pre>
cd /your/path/trunk/
mkdir ARCH
cd /your/path/trunk/ICOSAGCM/arch
cp YOUR_MACHINE.env /your/path/trunk/ARCH
cp YOUR_MACHINE.path /your/path/trunk/ARCH
</pre>

Once more, it is more practical to install every new package alongside the others. Here’s what you should have after all the previous steps:

<pre>
cd /your/path/trunk/
ls
ARCH ICOSAGCM ICOSA_LMDZ LMDZ.COMMON LMDZ.VENUS IOIPSL XIOS test_HELD_SUAREZ
</pre>

If you are missing one of these folders, repeat the previous sections for '''ICOSAGCM'''.

Or follow the documentation for installing the Venus GCM (which will give you '''LMDZ.COMMON''', '''LMDZ.VENUS''', '''IOIPSL'''): [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/Quick_Install_and_Run_Venus_PCM Quick Install and Run Venus PCM]

For '''XIOS''': [https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/The_XIOS_Library The XIOS Library]

For '''ICOSA_LMDZ''': we just installed it before.

== Compilation Venus - DYNAMICO ==

Everything needs to be properly compiled before you can run the model. Here’s how to do it:

(some compilation has already been done, but this recaps everything)

!! Everything has to be compiled in the right order !!

Compile '''IOIPSL''':

<pre>
cd /your/path/trunk/LMDZ.COMMON/ioipsl
./install_ioipsl_YOUR_ARCH.bash
</pre>

Compile '''XIOS''':

<pre>
cd ../../XIOS/
./make_xios --prod --arch YOUR_ARCH --arch_path ../ARCH --job 8
</pre>

Compile '''LMDZ.COMMON''' (the Physics packages):

<pre>
cd ../LMDZ.COMMON/
./makelmdz_fcm -arch YOUR_ARCH -io xios -p venus -d 48x32x50 -j 8 gcm -parallel mpi
</pre>

“-d 48x32x50” is the model dimension, it can be changed.

Compile '''ICOSAGCM''' (the Dynamical Core):

<pre>
./make_icosa -parallel mpi -with_xios -arch YOUR_ARCH -job 8
</pre>

Eventually, compile '''ICOSA_LMDZ''':

<pre>
./make_icosa_lmdz -p venus -parallel mpi -arch YOUR_ARCH -arch_path ../ARCH -job 8 -nodeps
</pre>

The executable “'''icosa_lmdz.exe'''” will be in /ICOSA_LMDZ/bin/.

== Running Venus - DYNAMICO ==

After compiling everything in the right order, we need to prepare the directory. Make a new one alongside the others:

<pre>
cd /your/path/trunk/
mkdir Venus_DYNAMICO
</pre>

See the README.md in /ICOSA_LMDZ to know what files to take where :

<syntaxhighlight lang="text">
organization of XML files and synchronization with code
-------------------------------------------------------

from ICOSAGCM/xml [DYNAMICO dynamical core]
- context_input_dynamico.xml
- field_def_dynamico.xml

from ICOSA_LMDZ/xml [INTERFACE]
- iodef.xml

from LMDZ.VENUS/deftank [LMDZ physics]
- field_def_physics.xml
- context_lmdz_physics.xml

-----

to be created and adapted from ICOSAGCM/xml
>> check compatibility when changing ICOSAGCM version
- context_dynamico.xml

to be created and adapted from ICOSAGCM test cases
>> check compatibility when changing ICOSAGCM version
- file_def_dynamico.xml
</syntaxhighlight>

From ICOSAGCM/xml/DYNAMICO/ :

<pre>
cd /your/path/ICOSAGCM/xml/DYNAMICO
cp context_input_dynamico.xml field_def_dynamico.xml dynamico.xml nudging_dynamico.xml sponge_dynamico.xml ../../Venus_DYNAMICO
</pre>

From ICOSA_LMDZ :

<pre>
cd /your/path/ICOSA_LMDZ/xml
cp iodef.xml ../../Venus/DYNAMICO
</pre>

From LMDZ.VENUS :

<pre>
cd /your/path/LMDZ.VENUS/deftank
cp field_def_physics.xml context_lmdz_physics.xml ../../Venus_DYNAMICO
</pre>

TO CONTINUE

[[Category:Venus-Model]]

The DYNAMICO dynamical core

2024-06-25T13:43:11Z

Rcapron:

== What is DYNAMICO ? ==

DYNAMICO is a recent dynamical core for atmospheric general circulation models (GCM). It is based on an icosahedral hexagonal grid projected on the sphere, a hybrid pressure-based terrain-following vertical coordinate, second-order enstrophy-conserving finite-difference discretization and positive-definite advection.

DYNAMICO is coded in Fortran and meant to be used in a massively parallel environment (using MPI and OpenMP) and has been coupled to a number of physics packages, such as the Earth LMDZ6 physics package (see https://lmdz-forge.lmd.jussieu.fr/mediawiki/LMDZPedia/index.php/Accueil ; search there for the keyword DYNAMICO to get some related documentation) but also the planetary version of the Mars, Venus or Generic Planetary Climate Models (PCM).

The DYNAMICO source code is freely available and downloadable using git
<syntaxhighlight lang="bash">
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git
</syntaxhighlight>
The DYNAMICO project page (be warned that information there is somewhat obsolete and related to earlier version, now obsolete, on svn) can be found at http://forge.ipsl.jussieu.fr/dynamico

== Installing and running DYNAMICO ==
Here we just describe how to compile and run DYNAMICO by itself, i.e. without being coupled to any physics.
This is essentially done as an exercise to check that it has been correctly installed, before moving on to the more complex (and complete!) case of compiling and running with a given physics package

=== Prerequisites ===
There are a couple of prerequisites to installing and using DYNAMICO:
# An MPI library must be available (i.e. installed and ready to use)
# BLAS and LAPACK libraries must be available.
# The XIOS library, compiled with that same MPI library, must also be available. Check out the [[The_XIOS_Library|XIOS library]] page for some information on installing it.

=== Downloading and compiling DYNAMICO ===
Using git
<syntaxhighlight lang="bash">
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git ICOSAGCM
</syntaxhighlight>
will create a '''ICOSAGCM''' directory containing all the necessary source code.
Note that it is advised that this directory be alongside the '''XIOS''' library directory (because some relative paths in the dynamico arch*.env files assume it is the case. If not you will need to modify these files accordingly).

In the '''ICOSAGCM''' directory is the ''make_icosa'' compilation script, which is based on FCM and thus requires that adequate architecture ASCII files be available. The '''arch''' subdirectory contains examples for a few machines. Assuming you want to compile using ''somemachine'' architecture files (i.e. files ''arch/arch-somemachine.fcm'' , ''arch/arch-somemachine.env'', ''arch/arch-somemachine.path'' are available and contain the adequate information) you would run:
<syntaxhighlight lang="bash">
./make_icosa -parallel mpi_omp -with_xios -arch somemachine -job 8
</syntaxhighlight>
If compilation went well than you will find the executable '''icosa_gcm.exe''' in the '''bin''' subdirectory

==== For the experts: more about the arch files and their content ====
TO BE WRITTEN

==== For the experts: more about the make_icosa script ====
To know more about possible options to the ''make_icosa'' script:
<syntaxhighlight lang="bash">
./make_icosa -h
</syntaxhighlight>

=== Running a simple Held and Suarez test case ===
DYNAMICO comes with a simple test case corresponding to a Held and Suarez simulation (...DESCRIBE H&S IN A NUTSHELL + REF TO THEIR PAPER...).

We will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_HELD_SUAREZ'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_HELD_SUAREZ
</pre>

Then, we need to copy the specific .def files for this testCase. (they are in /ICOSAGCM)

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_HELD_SUAREZ
cp *def ../../../test_HELD_SUAREZ
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_HELD_SUAREZ

cd ..
cp iodef.xml ../../test_HELD_SUAREZ
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” in the test directory test_HELD_SUAREZ (it is placed in ICOSAGCM/bin):

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_HELD_SUAREZ
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Everything is ready to run the model. Go to '''test_HELD_SUAREZ''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that the test-case design is such that it assumes you will be using 10 MPI processes. Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

And here another one that should work on Irene-Rome (assuming you are in project "gen10391"):
<syntaxhighlight lang="bash">
#!/bin/bash
# Partition to run on:
#MSUB -q rome
# project to run on
#MSUB -A gen10391
# disks to use
#MSUB -m scratch,work,store
# Job name
#MSUB -r job_mpi
# Job standard output:
#MSUB -o job_mpi.%I
# Job standard error:
#MSUB -e job_mpi.%I
# number of OpenMP threads c
#MSUB -c 1
# number of MPI tasks n
#MSUB -n 40
# number of nodes to use N
#MSUB -N 1
# max job run time T (in seconds)
#MSUB -T 7200

source ../dynamico/arch.env
ccc_mprun -l icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

Moreover the following NetCDF output files should have been produced:
<syntaxhighlight lang="bash">
Ai.nc daily_output_native.nc restart.nc time_counter.nc
apbp.nc daily_output.nc start0.nc dynamico.nc
</syntaxhighlight>
where:
* ''Ai.nc'', ''apbp.nc'' and ''time_counter.nc'' are unimportant
* ''start0.nc'' is a file containing the initial conditions of the simulation
* ''restart.nc'' is the output file containing the final state of the simulation
* ''daily_output_native.nc'' is the output file containing a time series of a selection of variables on the native (icosahedral) grid
* ''daily_output.nc'' is the output file containing a time series of a selection of variables re-interpolated on a regular longitude-latitude grid
* ''dynamico.nc'' is the output file containing all your simulation data, that can be easily plotted with ferret (for example)

=== Using the restart.nc file to continue your simulation ===
If you want to use '''restart.nc''' to avoid restarting the simulation from scratch, here is the procedure to follow:

In the run.def file, for the first launch, change the variable "nqtot" to "1" instead of 0.

Then, run the model as usual.

Once the execution is complete, a '''restart.nc''' file will be created (in addition to all other .nc files), rename it to "start.nc".

<syntaxhighlight lang="bash">
mv restart.nc start.nc
</syntaxhighlight>

Next, modify the run.def file a second time. Change the variable "etat0 = held_suarez" to "etat0 = start_file", and add an additional line. You would then have these lines:

<syntaxhighlight lang="bash">
# etat0 = held_suarez (old line commented out)
etat0 = start_file
start_file_name = start
</syntaxhighlight>

Note, even if your file is named "start.nc", it is "start" that needs to be specified in the run.def (the .nc is already taken into account).

Then, run the model as usual with the same script.slurm and sbatch.

<syntaxhighlight lang="bash">
sbatch script.slurm
</syntaxhighlight>

You should see that the iterations start where those of the previous simulation stopped. With this command:

<syntaxhighlight lang="bash">
tail -f icosa_gcm.out
</syntaxhighlight>

If you want to do this again, remember to always use the new '''restart.nc''' file as the starting point (renaming it to start.nc etc...).

==== Mixed bag of comments about the run's setup and outputs ====
For those interested in more details about the key aspects and main parameters:
* The run control parameters are set in the ''run.def'' ASCII file which is read at run-time. This is where for instance one specifies the model resolution (parameters nbp: number of subdivisions of main triangles, and llm: number of vertical levels), time step (parameter day_step: number of steps per day) and length of the run (parameter ndays: number of days to run)
* The sub-splitting of the main rhombus (parameters nsplit_i and nsplit_j) controls the overall number of tiles (sub-domains). As a rule of thumb when running in parallel (MPI) one want to have as many sub-domains as available MPI processes. Since the icosahedron is made up of 10 rhombus this implies that one should target using a total of 10*nsplit_i*nspilt_j processes.
* Controlling the outputs made by XIOS is via the XML files, namely ''file_def_dynamico.xml''
* and so much more...

[[Category:FAQ]]
[[Category:DYNAMICO]]
[[Category:Generic-DYNAMICO]]
[[Category:Mars-DYNAMICO]]

The DYNAMICO dynamical core

2024-06-25T13:40:26Z

Rcapron:

== What is DYNAMICO ? ==

DYNAMICO is a recent dynamical core for atmospheric general circulation models (GCM). It is based on an icosahedral hexagonal grid projected on the sphere, a hybrid pressure-based terrain-following vertical coordinate, second-order enstrophy-conserving finite-difference discretization and positive-definite advection.

DYNAMICO is coded in Fortran and meant to be used in a massively parallel environment (using MPI and OpenMP) and has been coupled to a number of physics packages, such as the Earth LMDZ6 physics package (see https://lmdz-forge.lmd.jussieu.fr/mediawiki/LMDZPedia/index.php/Accueil ; search there for the keyword DYNAMICO to get some related documentation) but also the planetary version of the Mars, Venus or Generic Planetary Climate Models (PCM).

The DYNAMICO source code is freely available and downloadable using git
<syntaxhighlight lang="bash">
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git
</syntaxhighlight>
The DYNAMICO project page (be warned that information there is somewhat obsolete and related to earlier version, now obsolete, on svn) can be found at http://forge.ipsl.jussieu.fr/dynamico

== Installing and running DYNAMICO ==
Here we just describe how to compile and run DYNAMICO by itself, i.e. without being coupled to any physics.
This is essentially done as an exercise to check that it has been correctly installed, before moving on to the more complex (and complete!) case of compiling and running with a given physics package

=== Prerequisites ===
There are a couple of prerequisites to installing and using DYNAMICO:
# An MPI library must be available (i.e. installed and ready to use)
# BLAS and LAPACK libraries must be available.
# The XIOS library, compiled with that same MPI library, must also be available. Check out the [[The_XIOS_Library|XIOS library]] page for some information on installing it.

=== Downloading and compiling DYNAMICO ===
Using git
<syntaxhighlight lang="bash">
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git
</syntaxhighlight>
will create a '''dynamico''' directory containing all the necessary source code.
Note that it is advised that this directory be alongside the '''XIOS''' library directory (because some relative paths in the dynamico arch*.env files assume it is the case. If not you will need to modify these files accordingly).

In the '''dynamico''' directory is the ''make_icosa'' compilation script, which is based on FCM and thus requires that adequate architecture ASCII files be available. The '''arch''' subdirectory contains examples for a few machines. Assuming you want to compile using ''somemachine'' architecture files (i.e. files ''arch/arch-somemachine.fcm'' , ''arch/arch-somemachine.env'', ''arch/arch-somemachine.path'' are available and contain the adequate information) you would run:
<syntaxhighlight lang="bash">
./make_icosa -parallel mpi_omp -with_xios -arch somemachine -job 8
</syntaxhighlight>
If compilation went well than you will find the executable '''icosa_gcm.exe''' in the '''bin''' subdirectory

==== For the experts: more about the arch files and their content ====
TO BE WRITTEN

==== For the experts: more about the make_icosa script ====
To know more about possible options to the ''make_icosa'' script:
<syntaxhighlight lang="bash">
./make_icosa -h
</syntaxhighlight>

=== Running a simple Held and Suarez test case ===
DYNAMICO comes with a simple test case corresponding to a Held and Suarez simulation (...DESCRIBE H&S IN A NUTSHELL + REF TO THEIR PAPER...).

We will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_HELD_SUAREZ'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_HELD_SUAREZ
</pre>

Then, we need to copy the specific .def files for this testCase. (they are in /ICOSAGCM)

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_HELD_SUAREZ
cp *def ../../../test_HELD_SUAREZ
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_HELD_SUAREZ

cd ..
cp iodef.xml ../../test_HELD_SUAREZ
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” in the test directory test_HELD_SUAREZ (it is placed in ICOSAGCM/bin):

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_HELD_SUAREZ
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Everything is ready to run the model. Go to '''test_HELD_SUAREZ''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that the test-case design is such that it assumes you will be using 10 MPI processes. Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

And here another one that should work on Irene-Rome (assuming you are in project "gen10391"):
<syntaxhighlight lang="bash">
#!/bin/bash
# Partition to run on:
#MSUB -q rome
# project to run on
#MSUB -A gen10391
# disks to use
#MSUB -m scratch,work,store
# Job name
#MSUB -r job_mpi
# Job standard output:
#MSUB -o job_mpi.%I
# Job standard error:
#MSUB -e job_mpi.%I
# number of OpenMP threads c
#MSUB -c 1
# number of MPI tasks n
#MSUB -n 40
# number of nodes to use N
#MSUB -N 1
# max job run time T (in seconds)
#MSUB -T 7200

source ../dynamico/arch.env
ccc_mprun -l icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

Moreover the following NetCDF output files should have been produced:
<syntaxhighlight lang="bash">
Ai.nc daily_output_native.nc restart.nc time_counter.nc
apbp.nc daily_output.nc start0.nc dynamico.nc
</syntaxhighlight>
where:
* ''Ai.nc'', ''apbp.nc'' and ''time_counter.nc'' are unimportant
* ''start0.nc'' is a file containing the initial conditions of the simulation
* ''restart.nc'' is the output file containing the final state of the simulation
* ''daily_output_native.nc'' is the output file containing a time series of a selection of variables on the native (icosahedral) grid
* ''daily_output.nc'' is the output file containing a time series of a selection of variables re-interpolated on a regular longitude-latitude grid
* ''dynamico.nc'' is the output file containing all your simulation data, that can be easily plotted with ferret (for example)

=== Using the restart.nc file to continue your simulation ===
If you want to use '''restart.nc''' to avoid restarting the simulation from scratch, here is the procedure to follow:

In the run.def file, for the first launch, change the variable "nqtot" to "1" instead of 0.

Then, run the model as usual.

Once the execution is complete, a '''restart.nc''' file will be created (in addition to all other .nc files), rename it to "start.nc".

<syntaxhighlight lang="bash">
mv restart.nc start.nc
</syntaxhighlight>

Next, modify the run.def file a second time. Change the variable "etat0 = held_suarez" to "etat0 = start_file", and add an additional line. You would then have these lines:

<syntaxhighlight lang="bash">
# etat0 = held_suarez (old line commented out)
etat0 = start_file
start_file_name = start
</syntaxhighlight>

Note, even if your file is named "start.nc", it is "start" that needs to be specified in the run.def (the .nc is already taken into account).

Then, run the model as usual with the same script.slurm and sbatch.

<syntaxhighlight lang="bash">
sbatch script.slurm
</syntaxhighlight>

You should see that the iterations start where those of the previous simulation stopped. With this command:

<syntaxhighlight lang="bash">
tail -f icosa_gcm.out
</syntaxhighlight>

If you want to do this again, remember to always use the new '''restart.nc''' file as the starting point (renaming it to start.nc etc...).

==== Mixed bag of comments about the run's setup and outputs ====
For those interested in more details about the key aspects and main parameters:
* The run control parameters are set in the ''run.def'' ASCII file which is read at run-time. This is where for instance one specifies the model resolution (parameters nbp: number of subdivisions of main triangles, and llm: number of vertical levels), time step (parameter day_step: number of steps per day) and length of the run (parameter ndays: number of days to run)
* The sub-splitting of the main rhombus (parameters nsplit_i and nsplit_j) controls the overall number of tiles (sub-domains). As a rule of thumb when running in parallel (MPI) one want to have as many sub-domains as available MPI processes. Since the icosahedron is made up of 10 rhombus this implies that one should target using a total of 10*nsplit_i*nspilt_j processes.
* Controlling the outputs made by XIOS is via the XML files, namely ''file_def_dynamico.xml''
* and so much more...

[[Category:FAQ]]
[[Category:DYNAMICO]]
[[Category:Generic-DYNAMICO]]
[[Category:Mars-DYNAMICO]]

The DYNAMICO dynamical core

2024-06-25T13:28:08Z

Rcapron: /* Running a simple Held and Suarez test case */

== What is DYNAMICO ? ==

DYNAMICO is a recent dynamical core for atmospheric general circulation models (GCM). It is based on an icosahedral hexagonal grid projected on the sphere, a hybrid pressure-based terrain-following vertical coordinate, second-order enstrophy-conserving finite-difference discretization and positive-definite advection.

DYNAMICO is coded in Fortran and meant to be used in a massively parallel environment (using MPI and OpenMP) and has been coupled to a number of physics packages, such as the Earth LMDZ6 physics package (see https://lmdz-forge.lmd.jussieu.fr/mediawiki/LMDZPedia/index.php/Accueil ; search there for the keyword DYNAMICO to get some related documentation) but also the planetary version of the Mars, Venus or Generic Planetary Climate Models (PCM).

The DYNAMICO source code is freely available and downloadable using git
<syntaxhighlight lang="bash">
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git
</syntaxhighlight>
The DYNAMICO project page (be warned that information there is somewhat obsolete and related to earlier version, now obsolete, on svn) can be found at http://forge.ipsl.jussieu.fr/dynamico

== Installing and running DYNAMICO ==
Here we just describe how to compile and run DYNAMICO by itself, i.e. without being coupled to any physics.
This is essentially done as an exercise to check that it has been correctly installed, before moving on to the more complex (and complete!) case of compiling and running with a given physics package

=== Prerequisites ===
There are a couple of prerequisites to installing and using DYNAMICO:
# An MPI library must be available (i.e. installed and ready to use)
# BLAS and LAPACK libraries must be available.
# The XIOS library, compiled with that same MPI library, must also be available. Check out the [[The_XIOS_Library|XIOS library]] page for some information on installing it.

=== Downloading and compiling DYNAMICO ===
Using git
<syntaxhighlight lang="bash">
git clone https://gitlab.in2p3.fr/ipsl/projets/dynamico/dynamico.git
</syntaxhighlight>
will create a '''dynamico''' directory containing all the necessary source code.
Note that it is advised that this directory be alongside the '''XIOS''' library directory (because some relative paths in the dynamico arch*.env files assume it is the case. If not you will need to modify these files accordingly).

In the '''dynamico''' directory is the ''make_icosa'' compilation script, which is based on FCM and thus requires that adequate architecture ASCII files be available. The '''arch''' subdirectory contains examples for a few machines. Assuming you want to compile using ''somemachine'' architecture files (i.e. files ''arch/arch-somemachine.fcm'' , ''arch/arch-somemachine.env'', ''arch/arch-somemachine.path'' are available and contain the adequate information) you would run:
<syntaxhighlight lang="bash">
./make_icosa -parallel mpi_omp -with_xios -arch somemachine -job 8
</syntaxhighlight>
If compilation went well than you will find the executable '''icosa_gcm.exe''' in the '''bin''' subdirectory

==== For the experts: more about the arch files and their content ====
TO BE WRITTEN

==== For the experts: more about the make_icosa script ====
To know more about possible options to the ''make_icosa'' script:
<syntaxhighlight lang="bash">
./make_icosa -h
</syntaxhighlight>

=== Running a simple Held and Suarez test case ===
DYNAMICO comes with a simple test case corresponding to a Held and Suarez simulation (...DESCRIBE H&S IN A NUTSHELL + REF TO THEIR PAPER...).

We will run a testCase “without the physics”, to verify that the Dynamical Core works alone.

To do this, make a new folder “'''test_HELD_SUAREZ'''”, alongside '''ICOSAGCM''' and '''XIOS'''.

<pre>
cd /your/path/trunk/
mkdir test_HELD_SUAREZ
</pre>

Then, we need to copy the specific .def files for this testCase. (they are in /ICOSAGCM)

<pre>
# Go where the .def files are
cd /your/path/trunk/ICOSAGCM/TEST_CASE/HELD_SUAREZ

# Copy the .def files in the repository test_HELD_SUAREZ
cp *def ../../../test_HELD_SUAREZ
</pre>

Do the same for the .xml files:

<pre>
cd /your/path/trunk/ICOSAGCM/xml/DYNAMICO_XML
cp *xml ../../../test_HELD_SUAREZ

cd ..
cp iodef.xml ../../test_HELD_SUAREZ
</pre>

Then, you should modify the stack size to avoid any segmentation fault when running. Change your ~/.bashrc:

<pre>
gedit ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

If gedit doesn’t work, use nano:

<pre>
nano ~/.bashrc

# This option will unlimit the stack size
ulimit -s unlimited

source ~/.bashrc
</pre>

Then, copy the executable “icosa_gcm.exe” in the test directory test_HELD_SUAREZ (it is placed in ICOSAGCM/bin):

<pre>
cd /your/path/trunk/ICOSAGCM/bin
cp icosa_gcm.exe ../../test_HELD_SUAREZ
</pre>

If, when running the model, you want a NetCDF file (.nc) with all the data, you should modify the .xml file “'''file_def_dynamico.xml'''”, line 70, changing the “false” to “true” for “enabled”. This will enable the “dynamico.nc” file to be created, it is already a re-interpolation of the dynamico-grid into long-lat, which makes it usable directly with Ferret/Panoply.

Everything is ready to run the model. Go to '''test_HELD_SUAREZ''', then use the slurm command “'''sbatch'''” to submit a job to the cluster.

<pre>
cd /your/path/trunk/test_HELD_SUAREZ
sbatch script_d_execution.slurm
</pre>

Slurm script (example for spirit1):

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:20:00
#SBATCH --output %x.%j.out

source /your/path/trunk/ICOSAGCM/arch/arch-YOUR_ARCH.env

export OMP_NUM_THREADS=1
export OMP_STACKSIZE=400M

mpirun icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

In this script, you should modify the path and “YOUR_ARCH” with your architecture (for the source command). Note that the test-case design is such that it assumes you will be using 10 MPI processes. Note that we are not using OpenMP here, it is not functional for now (TO UPDATE).

And here another one that should work on Irene-Rome (assuming you are in project "gen10391"):
<syntaxhighlight lang="bash">
#!/bin/bash
# Partition to run on:
#MSUB -q rome
# project to run on
#MSUB -A gen10391
# disks to use
#MSUB -m scratch,work,store
# Job name
#MSUB -r job_mpi
# Job standard output:
#MSUB -o job_mpi.%I
# Job standard error:
#MSUB -e job_mpi.%I
# number of OpenMP threads c
#MSUB -c 1
# number of MPI tasks n
#MSUB -n 40
# number of nodes to use N
#MSUB -N 1
# max job run time T (in seconds)
#MSUB -T 7200

source ../dynamico/arch.env
ccc_mprun -l icosa_gcm.exe > icosa_gcm.out 2>&1
</syntaxhighlight>

To verify that the code is properly running, you can show directly the “icosa_gcm.out” file.

<pre>
tail -f icosa_gcm.out
</pre>

Once the code is finished running, something like this should appear (at the end of the icosa_gcm.out):

<pre>
GETIN restart_file_name = restart
masse advec mass rmsdpdt energie enstrophie entropie rmsv mt.ang
GLOB -0.999E-15 0.000E+00 0.79047E+01 0.110E-02 0.261E+00 0.155E-02 0.743E+01 0.206E-01

Time elapsed : 601.628763000000
</pre>

Moreover the following NetCDF output files should have been produced:
<syntaxhighlight lang="bash">
Ai.nc daily_output_native.nc restart.nc time_counter.nc
apbp.nc daily_output.nc start0.nc dynamico.nc
</syntaxhighlight>
where:
* ''Ai.nc'', ''apbp.nc'' and ''time_counter.nc'' are unimportant
* ''start0.nc'' is a file containing the initial conditions of the simulation
* ''restart.nc'' is the output file containing the final state of the simulation
* ''daily_output_native.nc'' is the output file containing a time series of a selection of variables on the native (icosahedral) grid
* ''daily_output.nc'' is the output file containing a time series of a selection of variables re-interpolated on a regular longitude-latitude grid
* ''dynamico.nc'' is the output file containing all your simulation data, that can be easily plotted with ferret (for example)

==== Mixed bag of comments about the run's setup and outputs ====
For those interested in more details about the key aspects and main parameters:
* The run control parameters are set in the ''run.def'' ASCII file which is read at run-time. This is where for instance one specifies the model resolution (parameters nbp: number of subdivisions of main triangles, and llm: number of vertical levels), time step (parameter day_step: number of steps per day) and length of the run (parameter ndays: number of days to run)
* The sub-splitting of the main rhombus (parameters nsplit_i and nsplit_j) controls the overall number of tiles (sub-domains). As a rule of thumb when running in parallel (MPI) one want to have as many sub-domains as available MPI processes. Since the icosahedron is made up of 10 rhombus this implies that one should target using a total of 10*nsplit_i*nspilt_j processes.
* Controlling the outputs made by XIOS is via the XML files, namely ''file_def_dynamico.xml''
* and so much more...

[[Category:FAQ]]
[[Category:DYNAMICO]]
[[Category:Generic-DYNAMICO]]
[[Category:Mars-DYNAMICO]]

Venus - DYNAMICO

2024-06-25T13:10:03Z

Rcapron: /* Execution - Test_Case Held&Suarez - DYNAMICO */

Venus - DYNAMICO

2024-06-25T13:09:04Z

Rcapron: /* Execution - Test_Case Held&Suarez - DYNAMICO */

Venus - DYNAMICO

2024-06-25T13:06:45Z

Rcapron: /* Execution - Test_Case - DYNAMICO */

Venus - DYNAMICO

2024-06-21T13:53:28Z

Rcapron:

Venus - DYNAMICO

2024-06-20T09:47:49Z

Rcapron:

Venus - DYNAMICO

2024-06-20T09:44:10Z

Rcapron:

Venus - DYNAMICO

2024-06-20T09:43:40Z

Rcapron:

Venus - DYNAMICO

2024-06-20T09:42:46Z

Rcapron: Created page with "= Venus - DYNAMICO = Dynamico is the recently developed Dynamical core, enabling better performance and solving some issues of the LMDZ model. To know more about it, see this..."