Introduction

PBS Professional (version 10.2) is installed on the Dell Cluster.

It consists of three compute nodes, each with eight CPUs. It is configured to allow users to run compute based jobs within nodes.

How to use PBS Professional

Users can log on to dell-cluster,use the required PBS commands to submit their jobs.

Environmental setup

For c shell users

Add the following lines in your .cshrc file

set path=($path /opt/pbs/default/bin)
set path=($path /opt/pbs/default/sbin)

run the command source .cshrc

For bash shell users

Add the following lines in your .bashrc file

export PATH=$PATH:/opt/pbs/default/bin
export PATH=$PATH:/opt/pbs/default/sbin

run the command source .bashrc

Queue Configuration

There are three queues configured for running compute intensive parallel jobs and one routeq queue configured on the Dell cluster.

qp8: User can request for eight CPUs through this queue. To submit a job to this queue give ncpus=8 in your script or on the qsub command. For this queue the cpu time is 512hrs.

#PBS -l ncpus=8 in your script.

routeq: This is the default queue in which all jobs are placed when submitted. The purpose of this queue is to route the jobs to specific queues based on the parameters specified in the job script.

Note: Users cannot directly submit jobs to a particular queue.All the jobs are routed through routeq

How to submit jobs

PBS is configured to allow users to submit compute intensive parallel jobs using MPICH2.
In PBS, jobs are submitted through queues. A PBS job is a shell script with PBS directives requesting for resources.

Note: Temporary space – /home/dell scratch/<loginid> is available for Job runtime use. Files older than 10 days in this area will be deleted. Please do not install any software in this area.

To submit MPICH2 based jobs :-

PBS is configured to run MPICH2 jobs within and across nodes. MPICH2 jobs can use a maximum of eight CPUs, enabled for a maximun of two nodes.

Procedure To submit MPICH2 based jobs :-

SSH Passwordless SetUp :-

Running a MPICH2 program on eight CPUs requires a password-less inter-node communication to launch MPI-daemons and establish an MPI ring, prior to program execution. To do so, the MPI launcher should be able to log on to the different nodes of the cluster where PBS Server schedules the job non-interactively, using ssh.

To setup for password-less, ssh login for your account across the nodes, perform the following steps on very first login into dell cluster :-

$cd $HOME
$ssh-keygen -t rsa -N “” -f $HOME/.ssh/id_rsa
$cd.ssh
$touch authorized_keys
$cat id_rsa.pub >> authorized_keys
$chmod 600 authorized_keys

PBS Script to submit the MPICH2 job :-

To run a MPICH2 job on eight CPUs, create a script script1 as follows:

#! /bin/sh

#PBS -l ncpus=8

#PBS -V

mpirun -n 8 /home/dell-scratch/loginid/a.out

The job submission command

qsub script1

To submit Gaussian job

Frequently Used PBS Commands –

1. To submit a job to the queue

qsub <script-name>

e.g. qsub script2

2. To check the status of the job

qstat -a

Gives the details of the job like the job number, the queue through which it was fired etc.

3. To defer the execution of a job

qsub -a

e.g qsub -a 0401102230 script1

This could submit the job on year 2004 month 01 date 10 and hr:mm as
22:30. the same with in the script would be like:-

#PBS -a 0401102230

4. To give the job a name

qsub -N

This can also be put in the script as

# PBS N <job-name>.

This will give the job a particular name.
e.g qsub -N job1 script1.

5. To remove a job from the queue

qdel <jobid>

6. To know about the available queues

qstat -Q and qstat -q

For any help on writing job scripts. Please refer to or e-mail to: helpdesk.serc@auto.iisc.ac.in or contact System Administrator, #103, SERC .