Farming - Revision history

Lebonnois at 14:33, 2 September 2025

2025-09-02T14:33:43Z

Lebonnois at 14:32, 2 September 2025

2025-09-02T14:32:29Z

Lebonnois at 14:31, 2 September 2025

2025-09-02T14:31:59Z

Lebonnois at 14:30, 2 September 2025

2025-09-02T14:30:41Z

Lebonnois at 14:30, 2 September 2025

2025-09-02T14:30:25Z

Lebonnois at 14:29, 2 September 2025

2025-09-02T14:29:41Z

Martin.turbet: Created page with "On this page, we present working examples on how to "farm" simulations on computing clusters."

2025-07-15T08:11:26Z

Created page with "On this page, we present working examples on how to "farm" simulations on computing clusters."

New page

On this page, we present working examples on how to "farm" simulations on computing clusters.

@@ Line 60: / Line 60: @@
 Be aware of the use of memory ! If your jobs need more memory than the one allocated for its corresponding cores, you will need to waste cores to get all the memory each job needs.
 ''Example:''
 each node has 768 GB of memory, so 4 GB per core. If your job needs 8 GB/core, you will be able to use only 96 cores.

@@ Line 45: / Line 45: @@
 cd $workdir2
 srun --ntasks-per-node=96 --cpus-per-task=1 --threads-per-core=1 -- your.exe > output2 2>&1 &
-PID1=$!
+PID2=$!
 wait

@@ Line 42: / Line 42: @@
 srun --ntasks-per-node=96 --cpus-per-task=1 --threads-per-core=1 -- your.exe > output1 2>&1 &
 PID1=$!
 cd $workdir2
 srun --ntasks-per-node=96 --cpus-per-task=1 --threads-per-core=1 -- your.exe > output2 2>&1 &

@@ Line 4: / Line 4: @@
 One node on adastra is 192 cores. When you favorite PCM configuration does not scale easily to run on a full node, you can use shared nodes, but there are not a lot of them and you end up queueing a lot...
 If you opt for the
 <syntaxhighlight lang="bash">

@@ Line 3: / Line 3: @@
 '''On adastra (CINES)'''
-One node on adastra is 192 cores. When you favorite PCM configuration does not scale easily to run on a full node, you can use shared nodes, but there are not a lot of them and you end up queueing a lot... If you opt for the
+One node on adastra is 192 cores. When you favorite PCM configuration does not scale easily to run on a full node, you can use shared nodes, but there are not a lot of them and you end up queueing a lot...
 <syntaxhighlight lang="bash">
 #SBATCH --exclusive

← Older revision		Revision as of 14:29, 2 September 2025
Line 1:		Line 1:
	On this page, we present working examples on how to "farm" simulations on computing clusters.		On this page, we present working examples on how to "farm" simulations on computing clusters.
		+
		+	'''On adastra (CINES)'''
		+
		+	One node on adastra is 192 cores. When you favorite PCM configuration does not scale easily to run on a full node, you can use shared nodes, but there are not a lot of them and you end up queueing a lot... If you opt for the
		+	<syntaxhighlight lang="bash">
		+	#SBATCH --exclusive
		+	</syntaxhighlight>
		+	option, you'll get all the node for you, launch your jobs faster, but all the 192 cores will be billed for hours...
		+
		+	To make your runs more efficient, you can group several simulations on the same node, so that they run at the same time on your exclusive node, under the same launching script. You share only with you, on an exclusive node ! This means you avoid wasting unused-core hours.
		+
		+	To do this, here is an example :
		+	<syntaxhighlight lang="bash">
		+	#!/bin/bash
		+	#SBATCH --account=cin0391
		+	#SBATCH --job-name=vcd2dot4
		+	#SBATCH --output=%x_%A.out
		+	#SBATCH --error=%x_%A.err
		+	#SBATCH --constraint=GENOA
		+	#SBATCH --nodes=1
		+	#SBATCH --exclusive
		+	#SBATCH --time=24:00:00
		+
		+	### source your environment here
		+
		+	ulimit -s unlimited
		+	set -eux
		+
		+	workdir1=<path-to-yor-first-job>
		+	workdir2=<path-to-yor-second-job>
		+
		+	cd $workdir1
		+	### Activities to prepare your first job
		+	cd $workdir2
		+	### Activities to prepare your second job
		+
		+	#### launch first job:
		+	cd $workdir1
		+	srun --ntasks-per-node=96 --cpus-per-task=1 --threads-per-core=1 -- your.exe > output1 2>&1 &
		+	PID1=$!
		+	cd $workdir2
		+	srun --ntasks-per-node=96 --cpus-per-task=1 --threads-per-core=1 -- your.exe > output2 2>&1 &
		+	PID1=$!
		+
		+	wait
		+	status=$?
		+	echo $PID1.$status
		+	echo $PID2.$status
		+	date
		+
		+	### activities after the end of both jobs
		+	</syntaxhighlight>
		+	Each job runs in its own directory, and the last part of the script will be executed when both jobs are done.
		+	Here, 96 cores are used for each job. You can share the cores as you wish, but keep jobs in balance so that they finish roughly at the same time.
		+	You can also do this with 4 jobs on 48 cores, or 8 on 24 cores, etc...
		+
		+	Be aware of the use of memory ! If your jobs need more memory than the one allocated for its corresponding cores, you will need to waste cores to get all the memory each job needs.
		+	''Example:''
		+	each node has 768 GB of memory, so 4 GB per core. If your job needs 8 GB/core, you will be able to use only 96 cores.