PBS User Guide
Introduction

We are using Portable Batch System Pro (PBSPro) version 5.0.6. The home page for the PBSPro is at http://www.pbspro.com.

The main components are the Server (pbs_server), Scheduler (pbs_sched) and the job executor (pbs_mom, otherwise known as Machine Oriented Mini-Server - MOM).

Currently in DSD, slappy and diesel are the two machines running PBS. They run PBS independently, in the sense that each have all of the three components. This means the jobs submitted to each of them are queued and executed in the same machine.

At the moment, we only allow batch jobs in these machines; no interactive jobs are allowed. Jobs are submitted, using qsub command, through job scripts to queues. We have default, small, medium, long, verylong queues. Based on the CPU time needed for running the job, as specified at the time of submitting the job, the job is moved on to one of the four execution queues.

Setting up your environment

To access PBS commands and man pages you need the following : 

setenv PATH /usr/grid/pbs/bin:$PATH
setenv MANPATH /usr/grid/pbs/man:$MANPATH
If you don't find these entries already in your $PATH and $MANPATH, just do a module load pbs. Edit your .cshrc or .login file to run this command everytime you login.

There are several things you should know about setting up your shell prompt and environment for use under PBS:
When PBS runs a job, it runs the jobs with your user priviledges your shell automatically executes

In order for PBS to work correctly, you must ensure that Sometimes, PBS is not able to deliver your output files if any of your init files (.cshrc, .login, .profile) produce output to the terminal. The most common problem is the use of a terminal setting command such as stty or resize, which prints an error message when not connected to a real terminal. If you execute these commands in an init file, make sure they are executed only for interactive shells.

If you still want to customize your environment, you can use the PBS environment variable ($PBS_ENVIRONMENT) to check if you are running under PBS. This environment variable is only defined when running under PBS. The variable takes two values: PBS_INTERACTIVE (interactive jobs) or PBS_BATCH (batch jobs). For example you may want to enclose any terminal or prompt setup in your init files as follows ("csh" sample): 

#
#       Normal setup executed only when we are not under PBS
#
if ( ! "$?PBS_ENVIRONMENT" ) then
      do terminal stuff here
      run setup commands with output here

      Your local setup for the prompt
      setenv prompt  myprompt

      Your local Terminal setup such as (stty, resize, etc)
      stty delete mydelete
endif
#
Relating PBS Parameters to RSL Parameters

While there is not a 100% overlap, many of the parameters used to specify resource requirements in PBS have a direct corresponding resource parameter in Globus' RSL. The following table shows the parameters available at this time.

PBS OptionsGlobus RSL Options
-l site=globus:target-system slappy.lbl.gov/jobmanager-pbs for example.
-l ncpus=nnn count=nnn (the number of CPU's desired on the remote system.
-A variable-list project=variable-list
-l walltime=nnn
-l cput=nnn
-l pcput=nnn
nnn is in seconds 		maxtime=mmm
mmm is in minutes!
Order of Precedence for PBS is walltime, cput, pcput.
-o stdout-path stdout=local-stdout-path
-e stderr-path stderr=local-stderr-path
-v var-list environment=var-list</td>

Globus reports the status of jobs with various "states" a job goes through. This is similar to PBS processing states. The following table shows these correspondences.

GlobusPBS
PENDING
ACTIVE
FAILED
DONE 		TRANSIT (T)
RUNNING (R)
EXITING (E)
EXITING (E)

Frequently used PBS Commands

Getting Status on the System, Queues, and Jobs
Command Short Description Manual Page
qstat list information about queues and jobs qstat man page
qstat -q list all queues on system
qstat -Q list queue limits for all queues
qstat -a list all jobs on system
qstat -au userid list all jobs owned by user userid
qstat -s list all jobs with status comments
qstat -r list all running jobs
qstat -f jobid list all information known about specified job
qstat -Qf queue list all information known about specified queue
qstat -B list summary information about the PBS server
Submitting a Job
Sample jobscript submit jobscript to PBS qsub man page
Using the Graphical User Interface (GUI)
xpbs Graphical User Interface to PBS commands xpbs man page

Man pages of PBS Commands

Page last modified: Thursday, 22-Feb-2001 15:38:39 PST Contact: grid-admin <Grid Administrators> Credits:DOE Science Grid research and development is funded by the U.S. Dept. of Energy, Office of Science, Office of Advanced Scientific Computing Research, Mathematical, Information, and Computational Sciences Division. Privacy and site security notice to Users
Grid Homepage
DSD Homepage
LBNL Homepage