The PARAM Pravega facility uses SLURM (Simple Linux Utility for Resource Management) to schedule jobs. SLURM is a workload manager that provides a framework for job queues, allocation of compute nodes, and the start and execution of jobs.
Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters. Slurm requires no kernel modifications for its operation and is relatively self-contained. As a cluster workload manager, Slurm has three key functions. First, it allocates exclusive and/or non-exclusive access to resources (compute nodes) to users for some duration of time so they can perform work. Second, it provides a framework for starting, executing, and monitoring work (normally a parallel job) on the set of allocated nodes. Finally, it arbitrates contention for resources by managing a queue of pending work.
It is important to note:
- Compilations are done on the login node. Only the execution is scheduled via SLURM on the compute nodes. To execute a job using Slurm a job script needs to be created. The job script describes the requirements of the job in terms of the resources, and libraries it requires to execute the job. The command that is used to submit a job is ‘sbatch’ command.
- Upon Submission of a Job script, each job gets a unique Job Id. This can be obtained from the ‘squeue’ command.
- The Job Id is also appended to the output and error filenames.
Some important commands that are used in SLRUM:
sbatch: It is used to submit a job script for later execution. The script will typically contain one or more srun commands to launch parallel tasks.
scancel: It is used to cancel a pending or running job or job step. It can also be used to send an arbitrary signal to all processes associated with a running job or job step.
squeue: It reports the state of jobs or job steps. It has a wide variety of filtering, sorting, and formatting options. By default, it reports the running jobs in priority order and then the pending jobs in priority order.
Parameters used in SLRUM job script:
The job flags are used with SBATCH command. The syntax for the SLURM directive in a script is “#SBATCH <flag>”.
| Resource | Flag syntax | Description |
| partition time |
–partition=partition name –time=01:00:00 |
Partition is a queue for jobs. Time limit for the job. |
| nodes | –nodes=2 | Number of compute nodes for the job. |
| cpu/cores | –ntasks-per-node=8 | Corresponds to number of cores on the compute node. |
| resource feature | –gres=gpu:2 | Request use of GPUs on compute nodes |
| account | –account=group-slurm-account | Users may belong to groups or accounts. |
| job name |
–job-name=”lammps” | Name of job. |
| output file | –output=lammps.out | Name of file for stdout |
| -w, –nodelist –mail-type= |
Request a specific list of hosts. Notify user by email when certain event types occur. Valid type values are NONE, BEGIN, END, FAIL, REQUEUE, ALL TIME_LIMIT, TIME_LIMIT_90 (reached 90 percent of time limit), TIME_LIMIT_80 (reached 80 percent of time limit), and TIME_LIMIT_50 (reached 50 percent of time limit). Multiple type values may be specified in a comma separated list |
|
| email address | –mail- userusername@iisc.ac.in User to receive email notification of state changes as defined by –mail-type. | User’s email address |
| access | –exclusive | Exclusive access to compute nodes. The job allocation cannot share nodes with other running jobs. |
Using Slurm on Param Pravega
The above-mentioned SLURM commands should be used to submit jobs for execution in the Param Pravega queues.
For the queues used in Param Pravega, refer to Param Pravega Queues.
For sample job submission scripts, refer to Sample Job Submission scripts.
Version of SLURM in PARAM Pravega: Slurm-20.11.08
For more information please visit: http://slurm.schedmd.com