submitting_a_job_array.md

title = "Submitting a Job Array"
description =  "How to use job arrays with the SLURM scheduler."

A job array is a set of jobs that share the same submit file, but will run multiple copies with a environment variable incremented.  These are useful when you need to process a series of files, or if you need to run the same application multiple times.

Creating a Array Submit File

An array submit file is very similar to the example submit files in Submitting Jobs.  

{{% panel theme="info" header="example.slurm" %}} {{< highlight batch >}} #!/bin/sh #SBATCH --array=0-31 #SBATCH --time=03:15:00 # Run time in hh:mm:ss #SBATCH --mem-per-cpu=1024 # Minimum memory required per CPU (in megabytes) #SBATCH --job-name=hello-world #SBATCH --error=/work/[groupname]/[username]/job.%A_%a.err #SBATCH --output=/work/[groupname]/[username]/job.%A_%a.out

module load example/test

echo "I am task $SLURM_ARRAY_TASK_ID on node hostname" sleep 60 {{< /highlight >}} {{% /panel %}}

The submit file above will output the $SLURM_ARRAY_TASK_ID, which will be different for every one of the 32 (0-31) jobs, to the output files.

Job Arrays in SLURM have more features than this brief description, please visit the SLURM Documentation for more detailed information.