Using the MESOIPSL cluster - Revision history

Emillour at 16:22, 10 April 2026

2026-04-10T16:22:58Z

Emillour at 14:18, 9 April 2026

2026-04-09T14:18:51Z

Emillour at 14:11, 9 April 2026

2026-04-09T14:11:03Z

Mwolff: /* Compiling the PCMs on Spirit */

2025-02-26T13:54:21Z

‎Compiling the PCMs on Spirit

Mwolff: /* Compiling the PCMs on Spirit */

2025-02-26T13:53:45Z

‎Compiling the PCMs on Spirit

Aurelien.falco: example of rsync script

2024-05-27T16:15:01Z

example of rsync script

Emillour at 07:44, 9 March 2023

2023-03-09T07:44:10Z

Emillour at 12:03, 1 March 2023

2023-03-01T12:03:27Z

Emillour at 17:04, 25 November 2022

2022-11-25T17:04:51Z

Emillour: Created page with "This page provides some information for those who use the MESOIPSL cluster (also known as "spirit", replacing "ciclad"). == How to access the cluster == If you had an account..."

2022-11-25T17:00:19Z

Created page with "This page provides some information for those who use the MESOIPSL cluster (also known as "spirit", replacing "ciclad"). == How to access the cluster == If you had an account..."

New page

This page provides some information for those who use the MESOIPSL cluster (also known as "spirit", replacing "ciclad").

== How to access the cluster ==
If you had an account on Ciclad, then you have one on Spirit. If you need to open an account (this is of course reserved to IPSL users) then proceed to this page: https://documentations.ipsl.fr/spirit/getting_started/account.html

Once you have an account you can ssh to the cluster via either of the spirit1 or spirit2 login nodes:
<syntaxhighlight lang="bash">
ssh yourMESOIPSLlogin@spirit1.ipsl.fr
</syntaxhighlight>

Here is probably also the right place to point to the MEOIPSL cluster's main page: https://documentations.ipsl.fr/spirit/

== OS and disk space ==
As the welcome message will remind you:
<pre>
Welcome to Ubuntu 20.04.5 LTS (GNU/Linux 5.4.0-125-generic x86_64)
CPU AMD EPYC 7402P 24-Core Processor 2.8GHz
=========================================================================
* Mesocentre ESPRI IPSL (Cluster Spirit) JUSSIEU *
=========================================================================
** Disk Space :
- /home/login (32Go and 300000 files max per user) : Backup every day.
- /data/login ( 1To and 300000 files max per user) : NO BACKUP
- /scratchu/login ( 2To and 300000 files max per user) : NO BACKUP
- /bdd/ : Datasets
- /climserv-home/, /homedata ,/scratchx : SpiritX workspace ( READ-ONLY)
------------------------------------------------------------------------------
Migration Documentation ( Temporary URL )
https://documentations.ipsl.fr/spirit/spirit_clusters/migration_from_ciclad_climserv.html
** Support Contact mailto:meso-support@ipsl.fr
------------------------------------------------------------------------------
</pre>
This is Ubuntu Linux and the "HOME" is quite limited in size. Most work should be done on the '''data''' and/or '''scratchu''' disks.

It is up to you to tailor your environment. By default it is quite bare; it is up to you to load the modules you'll need to have access to specific software or compilers or libraries (and versions thereof).
To know what modules are available:
<syntaxhighlight lang="bash">
module avail
</syntaxhighlight>
To load a given module, here the Panoply software:
<syntaxhighlight lang="bash">
module load panoply
</syntaxhighlight>

== Example of a job to run a GCM simulation ==
Here to run using 24 MPI tasks with 2 OpenMP threads each:
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=24
#SBATCH --cpus-per-task=2
#SBATCH --partition=zen4 # zen4: 64cores/node and 240GB of memory
##SBATCH --partition=zen16 # zen16: 32 cores/node core and 496GB of memory
#SBATCH -J job_mpi_omp
#SBATCH --time=0:55:00
#SBATCH --output %x.%j.out

source ../trunk/LMDZ.COMMON/arch/arch-ifort_MESOIPSL.env

export OMP_NUM_THREADS=2
export OMP_STACKSIZE=400M

mpirun gcm_64x48x54_phymars_para.e > gcm.out 2>&1
</syntaxhighlight>

@@ Line 56: / Line 56: @@
 Many compilers and compiler versions, along with precompiled NetCDF libraries are available on Spirit. Which can simplify installing and using the model there.
-In practice one should first decide which compiler suite to use : Intel ifort (depreciated?) or GNU gfortran
+In practice one should first decide which compiler suite to use : Intel ifort/icc (depreciated?) or GNU gfortran/gcc
 === Intel compilers ===
-Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''ifort_MESOIPSL''' (XIOS and DYNAMICO also have similarly named arch files). To use them when compiling the PCM, one simply needs to specify ''-arch ifort_MESOIPSL'' argument to the ''makelmdz_fcm'' script.
+Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''ifort_MESOIPSL''' (XIOS and DYNAMICO also have similarly named arch files). To use them when compiling the PCM, one simply needs to specify ''-arch ifort_MESOIPSL'' argument to the [[The makelmdz fcm GCM Compilation Script|makelmdz_fcm]] script.
 Likewise there is a dedicated '''install_ioipsl_ifort_MESOIPSL.bash''' install script for IOIPSL available in '''LMDZ.COMMON/ioipsl''', you simply need to execute it:
@@ Line 65: / Line 65: @@
 ./install_ioipsl_ifort_MESOIPSL.bash
 </syntaxhighlight>
+to download and compile the IOIPSL library
 === GNU compilers ===
-Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''MESOIPSL-gnu''' (XIOS and DYNAMICO also have similarly named arch files). To use them when compiling the PCM, one simply needs to specify ''-arch MESOIPSL-gnu'' argument to the ''makelmdz_fcm'' script.
+Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''MESOIPSL-gnu''' (XIOS and DYNAMICO also have similarly named arch files). To use them when compiling the PCM, one simply needs to specify ''-arch MESOIPSL-gnu'' argument to the [[The makelmdz fcm GCM Compilation Script|makelmdz_fcm]] script.
 Likewise there is a dedicated '''install_ioipsl_MESOIPSL-gnu.bash''' install script for IOIPSL available in '''LMDZ.COMMON/ioipsl''', you simply need to execute it:
@@ Line 78: / Line 74: @@
 ./install_ioipsl_MESOIPSL-gnu.bash
 </syntaxhighlight>
+to download and compile the IOIPSL library
 == Example of a job to run a GCM simulation ==

@@ Line 59: / Line 59: @@
 === Intel compilers ===
-Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''ifort_MESOIPSL''' (XIOS and DYNAMICO also have similarly named arch files).
+Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''ifort_MESOIPSL''' (XIOS and DYNAMICO also have similarly named arch files). To use them when compiling the PCM, one simply needs to specify ''-arch ifort_MESOIPSL'' argument to the ''makelmdz_fcm'' script.
 Likewise there is a dedicated '''install_ioipsl_ifort_MESOIPSL.bash''' install script for IOIPSL available in '''LMDZ.COMMON/ioipsl''', you simply need to execute it:
@@ Line 72: / Line 72: @@
 === GNU compilers ===
-Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''MESOIPSL-gnu''' (XIOS and DYNAMICO also have similarly named arch files).
+Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''MESOIPSL-gnu''' (XIOS and DYNAMICO also have similarly named arch files). To use them when compiling the PCM, one simply needs to specify ''-arch MESOIPSL-gnu'' argument to the ''makelmdz_fcm'' script.
 Likewise there is a dedicated '''install_ioipsl_MESOIPSL-gnu.bash''' install script for IOIPSL available in '''LMDZ.COMMON/ioipsl''', you simply need to execute it:

@@ Line 54: / Line 54: @@
 == Compiling the PCMs on Spirit ==
 Dedicated arch files are available in the '''LMDZ.COMMON/arch''' subdirectory! They are labeled '''ifort_MESOIPSL''' (XIOS and DYNAMICO also have similarly named arch files).
-Likewise there is a dedicated '''install_ioipsl_ifort_MESOIPSL.bash''' install script for IOIPSL available in '''LMDZ.COMMON/ioipsl'''
+Likewise there is a dedicated '''install_ioipsl_ifort_MESOIPSL.bash''' install script for IOIPSL available in '''LMDZ.COMMON/ioipsl''', you simply need to execute it:
-To compile the utilities, you should define the three varibles below as indicated, where the first line may have caused problems if you already tried it:
+To compile the utilities, you should define the three variables below as indicated, where the first line may have caused problems if you already tried it:
 <br> NETCDF_HOME=$NETCDF_FORTRAN_ROOT
 <br>COMPILER="ifort"
 <br>COMPILER_OPTIONS="-O2 -ip"
 == Example of a job to run a GCM simulation ==
@@ Line 76: / Line 98: @@
 #SBATCH --output %x.%j.out
-source ../trunk/LMDZ.COMMON/arch/arch-ifort_MESOIPSL.env
+source ../trunk/LMDZ.COMMON/arch.env
 export OMP_NUM_THREADS=2

@@ Line 59: / Line 59: @@
 To compile the utilities, you should define the three varibles below as indicated, where the first line may have caused problems if you already tried it:
-<br> NETCDF_HOME=$NETCDF_FORTRAN_ROOT</br>
+<br> NETCDF_HOME=$NETCDF_FORTRAN_ROOT
-<br>COMPILER="ifort"</br>
+<br>COMPILER="ifort"
-<br>COMPILER_OPTIONS="-O2 -ip"</br>
+<br>COMPILER_OPTIONS="-O2 -ip"
 == Example of a job to run a GCM simulation ==

← Older revision		Revision as of 13:53, 26 February 2025
Line 57:		Line 57:

	Likewise there is a dedicated '''install_ioipsl_ifort_MESOIPSL.bash''' install script for IOIPSL available in '''LMDZ.COMMON/ioipsl'''		Likewise there is a dedicated '''install_ioipsl_ifort_MESOIPSL.bash''' install script for IOIPSL available in '''LMDZ.COMMON/ioipsl'''
		+
		+	To compile the utilities, you should define the three varibles below as indicated, where the first line may have caused problems if you already tried it:
		+	<br> NETCDF_HOME=$NETCDF_FORTRAN_ROOT</br>
		+	<br>COMPILER="ifort"</br>
		+	<br>COMPILER_OPTIONS="-O2 -ip"</br>

	== Example of a job to run a GCM simulation ==		== Example of a job to run a GCM simulation ==

← Older revision		Revision as of 16:15, 27 May 2024
Line 80:		Line 80:

	Note that there is a per-user limitation of (maximum) 96 cores for a given job.		Note that there is a per-user limitation of (maximum) 96 cores for a given job.
		+
		+	== Sending data on Spirit ==
		+	This function could be used to send data to your scratch directory on spirit.
		+	<syntaxhighlight lang="bash">
		+	function rsend {
		+	a=${1:-.}
		+	b=${2:-$a}
		+	c=${3:-spirit}
		+	rsync -avzl $a/ $c:/scratchx/$USER/$b/
		+	}
		+	</syntaxhighlight>
		+
		+	To be used as such:
		+	<syntaxhighlight lang="bash">
		+	rsend folder # simply send folder to your scratch dir
		+	rsend folder1 folder2 # send folder1 into folder2
		+	rsend folder1 folder2 machine #send folder1 into folder2 on machine (depends on your ssh config)
		+	</syntaxhighlight>
		+

	[[Category:FAQ]]		[[Category:FAQ]]

@@ Line 1: / Line 1: @@
-This page provides some information for those who use the MESOIPSL cluster (also known as "spirit", replacing "ciclad").
+This page provides some information for those who use the MESOIPSL clusters (also known as "spirit", replacing "ciclad").
 == How to access the cluster ==
@@ Line 8: / Line 12: @@
 ssh yourMESOIPSLlogin@spirit1.ipsl.fr
 </syntaxhighlight>
 Here is probably also the right place to point to the MEOIPSL cluster's main page: https://documentations.ipsl.fr/spirit/
@@ Line 69: / Line 79: @@
 </syntaxhighlight>
 [[Category:FAQ]]

← Older revision		Revision as of 17:04, 25 November 2022
Line 63:		Line 63:
	mpirun gcm_64x48x54_phymars_para.e > gcm.out 2>&1		mpirun gcm_64x48x54_phymars_para.e > gcm.out 2>&1
	</syntaxhighlight>		</syntaxhighlight>
		+
		+
		+	[[Category:FAQ]]