Difference between revisions of "Running Mars mesoscale model"

Latest revision as of 16:43, 2 April 2025

This is page I start to introduce some notes to remind that could be useful for other users (Jorge).

The user manual can be found at https://gitlab.in2p3.fr/la-communaut-des-mod-les-atmosph-riques-plan-taires/git-trunk/-/blob/master/MESOSCALE/MANUAL/SRC/user_manual.pdf?ref_type=heads

Regarding NaNs and parallel running

The mesoscale model cannot run in parallel in spirit at the moment, it runs but it outputs NaNs everywhere.

When running in a single core, in my case at least it produces NaNs with low output frequency (37) but by increasing the output frequency (6 works for me) it is solved. This is weird but seem to be right.

Increasing the output frequency in parallel runs doesn't work. Even with a frequency of 1 doesn't work.

Notes on clear input and output files (TBD)

Notes on configuration files (TBD)

namelist.input

Changing vertical resolution

Before running real.exe, a new levels file needs to be generated. The script to do this is in SIMU/RUN/vert_level_python.

Smoothing topography in the borders

This might help sometimes when simulations are breaking, but it didn't work for me so far. This script can smooth the topography in the borders of a wrfinput file.

import numpy as np
import xarray as xr
import sys
import os

from scipy.ndimage import gaussian_filter

def smooth_borders(a,margin):
    out=np.zeros(a.shape)
    for i in range(margin):
        if i>0:
            b=a[i:-i,i:-i]
        else:
            b=a[:]
        c=gaussian_filter(b, sigma=margin-i)
        if i>0:
            out[i:-i,i:-i]=c
        else:
            out=c[:]

    return out

margin=int(sys.argv[1])
input_file=sys.argv[2]


#Open file
ds=xr.open_dataset(input_file)

a=ds.HGT.values[0]
a=smooth_borders(a,margin)
ds.HGT.values[0]=a

os.system('mv %s %s_presmooth'%(input_file,input_file))
ds.to_netcdf(input_file)

Restarting simulations

In the file namelist.input there is the option "restart". The comment for this option says "Output restart files?" but actually it refers to whether or not the simulation should try to start from a previous simulation. Restart files are periodically saved, this can be configured with restart_interval. The value is an integer number corresponding to the number of "minutes" (with a Martian hour having 37 minutes) between restart files. By default this is very long (8880, corresponding to 10 sols). If we write 37 it will be every hour. If we write 37*6 it will be every 6 hours, etc.

To restart from an existing restart file, we must change the "restart" option to .true., and the start_month,start_day, and start_hour need to be updated to coincide with the time of the restart file. Since we can tune the starting of the simulation with a resolution of hours (and not minutes), this means that the restart files need to be saved at integer hours, and therefore the restart_interval option should have values 37*N, with N the number of hours of interval between restart files.

Troubleshooting restart files in big simulations

In big simulations restart files fail to be saved. This does not cause any problem, and the simulation keeps running. When increasing verbosity to one, the rsl.error file contains this error messages (Note that for this debug execution we set restart_interval to 1 to obtain the error quickly):

 d01 2024-10-01_06:01:00 med_restart_out: opening wrfrst_d01_2024-10-01_06:01:00
  for writing
 d01 2024-10-01_06:01:00  NetCDF error: NetCDF: One or more variable sizes viola
 te format constraints
 d01 2024-10-01_06:01:00  NetCDF error in ext_ncd_open_for_write_commit wrf_io.F
 90, line        1279
 med_restart_out: opening wrfrst_d01_2024-10-01_06:01:00 for writing
 d01 2024-10-01_06:01:00  Warning 2 DRYRUNS 1 VARIABLE in wrf_io.F90, line
   2303

[This line repeats many times]

d01 2024-10-01_06:01:00  Warning 2 DRYRUNS 1 VARIABLE in wrf_io.F90, line
   2303
Timing for Writing restart for domain        1:    0.06920 elapsed seconds.
 d01 2024-10-01_06:01:00  Warning TRY TO CLOSE DRYRUN in ext_ncd_ioclose wrf_io.
 F90, line        1311
 d01 2024-10-01_06:01:00  NetCDF error: NetCDF: One or more variable sizes viola
 te format constraints
 d01 2024-10-01_06:01:00  NetCDF error in ext_ncd_ioclose wrf_io.F90, line
   1329

The problem is in the file wrf_io.F90. We have to update two files

code/MESOSCALE/LMD_MM_MARS/SRC/WRFV2/external/io_netcdf/wrf_io.F90: line 1188
code/MESOSCALE/LMD_MM_MARS/SRC/WRFV2/external/io_pnetcdf/wrf_io.F90: line 1195

The problem is caused by the big size of the restart files, it will happen when files are bigger than 2gb.

Apparently, these parameters only work when writing files smaller than 2gb. We can fix that be replacing:

NF_CLOBBER

by

IOR(NF_CLOBBER,NF_64BIT_OFFSET)

Then the resulting line looks like

code/MESOSCALE/LMD_MM_MARS/SRC/WRFV2/external/io_netcdf/wrf_io.F90: line 1188
stat = NF_CREATE(FileName, IOR(NF_CLOBBER,NF_64BIT_OFFSET), DH%NCID)

code/MESOSCALE/LMD_MM_MARS/SRC/WRFV2/external/io_pnetcdf/wrf_io.F90: line 1195
stat = NFMPI_CREATE(Comm, FileName, IOR(NF_CLOBBER,NF_64BIT_OFFSET), MPI_INFO_NULL, DH%NCID)

This is documented in https://docs.unidata.ucar.edu/netcdf-fortran/current/nc_f77_interface_guide.html

Outputs in superhigh frequency

The way I found involves changing the file SRC/WRFV2/share/set_timekeeping.F, by adding the two lines that are commented in the following code, the outputs can be made every 20 seconds (every dynamical timestep)

CALL nl_get_history_interval( grid%id, history_interval )   ! same as minutes
   CALL nl_get_history_interval_mo( grid%id, history_interval_mo )
   CALL nl_get_history_interval_d( grid%id, history_interval_d )
   CALL nl_get_history_interval_h( grid%id, history_interval_h )
   CALL nl_get_history_interval_m( grid%id, history_interval_m )
   CALL nl_get_history_interval_s( grid%id, history_interval_s )
   IF ( history_interval_m .EQ. 0 ) history_interval_m = history_interval

   !history_interval_m=0
   !history_interval_s=20