WRF dynamical core for LES/mesoscale simulations

Introduction

WRF is a mesoscale numerical weather prediction system designed for both atmospheric research and operational forecasting applications over the Earth's atmosphere: see WRF presentation.

In the PCM, we take advantage of the dynamical core of the WRF model to run LES (large-eddy simulations) or mesoscale simulations.

To do so, we take the WRF model, remove all its physical packages related to the Earth's atmosphere, and plug in the physical package from the PCM (either Venus, Mars, Titan, or Generic).

A description of the various equations solved by WRF can be found here: Skamarock et al. (2019, 2021).

Studies with PCM+WRF

Descriptions of the application of WRF dynamical core with the PCM can be found in:

- Noé CLEMENT's PhD thesis (chapter 3) for the Generic PCM.

- Maxence Lefevre's PhD thesis for the Venus PCM.

- Aymeric Spiga's PhD thesis for the Mars PCM

Studies with the PCM coupled to the WRF dynamical core have been done using WRF V2, WRF V3, WRF V4.

Our aim is now to use WRF V4 as much as possible, which offers significant improvements in terms of numerical schemes in particular.

Here is a list of studies using the PCM coupled to the WRF dynamical core:

Planet	Reference	Physical package	Dynamical core
Mars	Spiga and Forget (2009)	Mars PCM	WRF V2
Mars	Spiga et al. (2010)	Mars PCM	WRF V3
Venus	Lefèvre et al. (2018)	Venus PCM	WRF V3
Venus	Lefèvre et al. (2020)	Venus PCM	WRF V2
Exoplanets	Lefèvre et al. (2021)	Generic PCM	WRF V3
Exoplanets	Leconte et al. (2024)	Generic PCM	WRF V4
Uranus, Neptune	Clément et al. (2024)	Generic PCM	WRF V4

Remarks

For advanced users:

When compiling the model, the flags:

#ifndef MESOSCALE
      use ...
#else
      use ...

select lines compiled depending on the configuration.

WRF equations

Below is a synthesis of equations solved by WRF.

Installing the Generic PCM+WRF

3 git repositories to handle

Review of the repositories

- LES_planet_workflow

- git-trunk (physics & GCM dynamical core), which takes the name "code" when being cloned

- WRFV4 (dynamics)

The first is hosted on Aymeric Spiga's git, and will download (git clone) the two others, which are part of La communauté des modèles atmosphériques planétaires repositories.

The 3 repositories are managed by git. You will need to do a little juggling between these repositories to be sure they are on the right branches for your project (see later).

Downloading/Cloning the parent repository

The first step is downloading (i.e. cloning) the parent repository 'LES_planet_workflow' from Aymeric Spiga's git (you will download the other two later).

Put it on the branch "wrfv4".

.env file

!!! For now, only the Intel compiler has been tested and works. !!!

On any cluster, one has to build a file "mesoscale.env" which should look more or less like:

# mesoscale.env file

source YOUR_ifort_ENV

# for WRFV4
declare -x WRFIO_NCD_LARGE_FILE_SUPPORT=1

declare -x NETCDF=`nf-config --prefix`
declare -x NCDFLIB=$NETCDF/lib
declare -x NCDFINC=$NETCDF/include

# PCM/WRFV4 interface
declare -x LMDZ_LIBO='code/LMDZ.COMMON/libo/YOUR_ARCH_NAME'

On ADASTRA, the following one works:

# mesoscale.env file

source code/LMDZ.COMMON/arch/arch-ADASTRA-ifort.env

# for WRFV4
declare -x WRFIO_NCD_LARGE_FILE_SUPPORT=1

declare -x NETCDF=$NETCDF_DIR
# alternatively: declare -x NETCDF=`nf-config --prefix`
declare -x NCDFLIB=$NETCDF/lib
declare -x NCDFINC=$NETCDF/include

# specific to ADASTRA
declare -x NETCDF_classic=1

# fix for "undefined symbol: __libm_feature_flag" errors at runtime:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$NETCDF_DIR/lib

# PCM/WRFV4 interface
declare -x LMDZ_LIBO=code/LMDZ.COMMON/libo/ADASTRA-ifort

This file should be sourced before performing compilation steps. Ideally, it will be integrated automatically (no need to source manually). It should also be sourced before executing the compiled files to run simulations.

If you work on Adastra, you can directly use the one below by inserting it in the LES_planet_workflow, for example, and calling it "mesoscale.env". It will work for both the WRF terrestrial and the Generic PCM+WRF

Compiling the Generic PCM +WRF

Download repositories

Execute the 1_downloading.sh to download (i.e. clone) the 'git-trunk' repository (it takes the name 'code').

Download (i.e. clone) the WRFV4 repository in the 'LES_planet_workflow/code/WRF.COMMON' directory (keep the name "WRFV4")

Now you have the 3 repositories!

Modify options to fit your compiling environment & your modelled planetary atmosphere

Modify the 0_defining.sh file to fit your Intel environment

   archname="ADASTRA-ifort"  # (or other cluster)

Modify the 0_defining.sh file to have the right physics compiling options, for example:

   add="-t 1 -b 20x34 -d 25" # Uranus & Neptune with CH4 visible bands (compilation options are maybe depreciated)

When compiling terrestrial WRF, if you indeed selected 15 and 1, you do not have to modify the following. Otherwise, in the lines:

   echo 15 > configure_inputs
   echo 1 >> configure_inputs

replace 15 & 1 with the options you selected.

Terrestrial WRF

Compiling terrestrial WRF is a good first step to check if the environment is working.

Somewhere else in your work directory, do:

"git clone --recurse-submodules https://github.com/wrf-model/WRF"

Before executing the following commands, you have to source the "mesoscale.env" file:

   source mesoscale.env

Commands:

./configure

During the configure step, you will be asked about the compiler, choose "INTEL (ifort/icc) dmpar" (probably number 15), and then about nesting, choose "basic" (=1).

then

./compile em_les

WRF compilation can take some time. Consider doing:

nohup ./compile em_les &

Final built executables are: "ideal.exe" and "wrf.exe" (in the WRF/main directory).

Put the 3 repositories on the right branch/commit

- On branch "wrfv4" for LES_planet_workflow repo (already done normally)

- On a branch that matches your physics developments on the git-trunk (=code) repo.

The branch "hackathon_wrfv4" works for Uranus/Neptune test cases.

This commit shows you what to modify if you want to work with another tracer than water (e.g., CH4).

!!! This has to be corrected !!! See this commit to fix a little bug with the slab ocean model

- On branch "generic" for WRFV4 workflow

Execute installing and compiling files

One by one:

- 2_preparing.sh

- 3_compiling_physics.sh

- 4_compiling_dynamics.sh

Good luck!

Executables

If you succeed in compiling the model, executables should be in the 'LES_planet_workflow/code/WRF.COMMON/WRFV4/main' directory. Executables of the Generic PCM+WRF have the same name as terrestrial WRF.

"ideal.exe" prepares the domain. "wrf.exe" runs the simulation.

The modifications we make in these files during compilation aim to incorporate the Physics of the PCM as a library, as well as the interface between WRF and the PCM.

Some tricks

In case of a failed compilation, the command

./clean -a

in the WRFV4 directory allows you to remove intermediate compiled files and properly restart the compilation.