PBS Professional (version 10.2) is installed on the Dell Cluster.
It consists of three compute nodes, each with eight CPUs. It is configured to allow users to run compute based jobs within nodes.
How to use PBS Professional
Users can log on to dell-cluster,use the required PBS commands to submit their jobs.
For c shell users
Add the following lines in your .cshrc file
set path=($path /opt/pbs/default/bin)
run the command source .cshrc
For bash shell users
Add the following lines in your .bashrc file
run the command source .bashrc
There are three queues configured for running compute intensive parallel jobs and one routeq queue configured on the Dell cluster.
qp8: User can request for eight CPUs through this queue. To submit a job to this queue give ncpus=8 in your script or on the qsub command. For this queue the cpu time is 512hrs.
#PBS -l ncpus=8 in your script.
routeq: This is the default queue in which all jobs are placed when submitted. The purpose of this queue is to route the jobs to specific queues based on the parameters specified in the job script.
Note: Users cannot directly submit jobs to a particular queue.All the jobs are routed through routeq
How to submit jobs
PBS is configured to allow users to submit compute intensive parallel jobs using MPICH2.
Note: Temporary space – /home/dell scratch/<loginid> is available for Job runtime use. Files older than 10 days in this area will be deleted. Please do not install any software in this area.
PBS is configured to run MPICH2 jobs within and across nodes. MPICH2 jobs can use a maximum of eight CPUs, enabled for a maximun of two nodes.
Procedure To submit MPICH2 based jobs :-
SSH Passwordless SetUp :-
Running a MPICH2 program on eight CPUs requires a password-less inter-node communication to launch MPI-daemons and establish an MPI ring, prior to program execution. To do so, the MPI launcher should be able to log on to the different nodes of the cluster where PBS Server schedules the job non-interactively, using ssh.
To setup for password-less, ssh login for your account across the nodes, perform the following steps on very first login into dell cluster :-
$ssh-keygen -t rsa -N “” -f $HOME/.ssh/id_rsa
$cat id_rsa.pub >> authorized_keys
$chmod 600 authorized_keys
PBS Script to submit the MPICH2 job :-
To run a MPICH2 job on eight CPUs, create a script script1 as follows:
#PBS -l ncpus=8
mpirun -n 8 /home/dell-scratch/loginid/a.out
The job submission command
Frequently Used PBS Commands –
1. To submit a job to the queue
e.g. qsub script2
2. To check the status of the job
Gives the details of the job like the job number, the queue through which it was fired etc.
3. To defer the execution of a job
e.g qsub -a 0401102230 script1
This could submit the job on year 2004 month 01 date 10 and hr:mm as
#PBS -a 0401102230
4. To give the job a name
This can also be put in the script as
# PBS N <job-name>.
This will give the job a particular name.
5. To remove a job from the queue
6. To know about the available queues
qstat -Q and qstat -q
For any help on writing job scripts. Please refer to or e-mail to: firstname.lastname@example.org or contact System Administrator, #103, SERC .