Using MeSU

This page provides some information for those who use the Sorbonne Université clusters MeSU : https://sacado.sorbonne-universite.fr/plateforme-mesu/

WARNING: There has been a recent change, MeSU is now MCMeSU ... some information on this page might not be up-to-date... (sections with (old MeSU) in the title)

How to access the cluster

You need first to open an account (this is of course reserved to people in labs connected to Sorbonne Université) then proceed to this page: https://sacado.sorbonne-universite.fr/account-creation/ where you can download the registration form to fill and upload.

Once you have an account you can ssh to the MCMeSU cluster :

ssh username@login.mesu.sorbonne-universite.fr

Link to Online MCMeSU documentation : https://sacado.sorbonne-universite.fr/documentation-mcmesu/

OS and disk space (old MeSU)

As the welcome message will remind you:

               __  __      ____  _   _ 
              |  \/  | ___/ ___|| | | |
              | |\/| |/ _ \___ \| | | |
              | |  | |  __/___) | |_| |
              |_|  |_|\___|____/ \___/ 

  ⢎⡑ ⢀⡀ ⡀⣀ ⣇⡀ ⢀⡀ ⣀⡀ ⣀⡀ ⢀⡀   ⡇⢸ ⣀⡀ ⠄ ⡀⢀ ⢀⡀ ⡀⣀ ⢀⣀ ⠄ ⣰⡀ ⢠⡂
  ⠢⠜ ⠣⠜ ⠏  ⠧⠜ ⠣⠜ ⠇⠸ ⠇⠸ ⠣⠭   ⠣⠜ ⠇⠸ ⠇ ⠱⠃ ⠣⠭ ⠏  ⠭⠕ ⠇ ⠘⠤ ⠣⠭
 
      https://sacado.sorbonne-universite.fr/mesu
             mesu@sorbonne-universite.fr

 beta: 144x 24 Intel Haswell CPUs, 128 GB RAM
 gamma:  2x 12 Intel Haswell CPUs, 2 NVidia K5200, 256 GB RAM

This is Ubuntu Linux and the "HOME" is quite limited in size. Most work should be done on the /scratchbeta disks.

It is up to you to tailor your environment. By default it is quite bare; it is up to you to load the modules you'll need to have access to specific software or compilers or libraries (and versions thereof). To know what modules are available:

module avail

To load a given module, here the Panoply software:

module load panoply

The list of available software is available at https://sacado.sorbonne-universite.fr/mesu/index.php/usage/available-software/

Compiling the PCMs on MeSU (old MeSU)

To do

Compiling WRF on MeSU (old MeSU)

To do

Environment:

module load intel/intel-compilers-18.2/18.2
module load intel/intel-mpi/2018.2
module load intel/intel-cmkl-18.0/18.0
export NETCDF="/home/lefevrema/NETCDF/netcdf-4.0.1"
export NETCDF_INCDIR="/home/lefevrema/NETCDF/netcdf-4.0.1/include/"
export NETCDF_LIBDIR="/home/lefevrema/NETCDF/netcdf-4.0.1/lib/"
export WHERE_MPI="/opt/dev/intel/2018_Update2/impi/2018.2.199/bin64"

Submit your job (old MeSU)

MeSU Supercomputer uses a queuing system to match users jobs with available computing resources.

Users submit their programs to the job scheduler (PBS), which maintains a queue of jobs and distributes them on the compute nodes according to the servers status, scheduling policies and jobs parameters (number of compute nodes / cores, estimated execution time, required memory, etc.).

The interface with PBS is done via a text file – the PBS script – created by the user, which will define your job requirements and execution steps. This file is mainly comprised of two sections :

• the header in which you specify the job requirements (execution time, number of CPU cores to use, memory requirements…) in the form of PBS directives
• the body in which you will write the commands to load specific software, define environment variables, and run your job.

More information https://sacado.sorbonne-universite.fr/mesu/index.php/usage/howtos/job-submission/

Here is an example of job script to run using 24 cpus:

#!/bin/bash
#PBS -q beta
#PBS -l select=1:ncpus=24
#PBS -l place=free:exclhost
#PBS -l walltime=23:59:00
#PBS -N RUN_GCM
#PBS -j oe

# Load your environment
source ../trunk/LMDZ.COMMON/arch/arch-X64_MESU.env

# Go to job directory
cd $PBS_O_WORKDIR
cd ./RUN_DIR

#Magic trick
ulimit -s unlimited

mpirun gcm.e > gcm.out 2>&1

Submitting your job to the PBS scheduler is easy once you have authored the corresponding PBS script file, using the qsub command:

qsub ./myScript