openpbs/INSTALL

--------------------------------------------------------------------

How to install PBS Pro using the configure script.

1. Install the prerequisite packages for building PBS Pro.

  For CentOS systems you should run the following command as root:

    yum install -y gcc make rpm-build libtool hwloc-devel \
      libX11-devel libXt-devel libedit-devel libical-devel \
      ncurses-devel perl postgresql-devel postgresql-contrib python-devel tcl-devel \
      tk-devel swig expat-devel openssl-devel libXext libXft \
      autoconf automake

  For openSUSE systems you should run the following command as root:

    zypper install gcc make rpm-build libtool hwloc-devel \
      libX11-devel libXt-devel libedit-devel libical-devel \
      ncurses-devel perl postgresql-devel postgresql-contrib python-devel tcl-devel \
      tk-devel swig libexpat-devel libopenssl-devel libXext-devel \
      libXft-devel fontconfig autoconf automake

  For Debian systems you should run the following command as root:

    sudo apt-get install gcc make libtool libhwloc-dev libx11-dev \
      libxt-dev libedit-dev libical-dev ncurses-dev perl \
      postgresql-server-dev-all postgresql-contrib python-dev tcl-dev tk-dev swig \
      libexpat-dev libssl-dev libxext-dev libxft-dev autoconf \
      automake

2. Install the prerequisite packages for running PBS Pro. In addition
  to the commands below, you should also install a text editor of
  your choosing (vim, emacs, gedit, etc.).

  For CentOS systems you should run the following command as root:

    yum install -y expat libedit postgresql-server postgresql-contrib python \
      sendmail sudo tcl tk libical

  For openSUSE systems you should run the following command as root:

    zypper install expat libedit postgresql-server postgresql-contrib python \
      sendmail sudo tcl tk libical1

  For Debian systems you should run the following command as root:

    apt-get install expat libedit2 postgresql python postgresql-contrib sendmail-bin \
      sudo tcl tk libical1a

3. Open a terminal as a normal (non-root) user, unpack the PBS Pro
  tarball, and cd to the package directory.

    tar -xpvf pbspro-19.1.3.tar.gz
    cd pbspro-19.1.3

4. Generate the configure script and Makefiles. (See note 1 below)

    ./autogen.sh

5. Display the available build parameters.

    ./configure --help

6. Configure the build for your environment. You may utilize the
  parameters displayed in the previous step. (See note 2 below)

  For CentOS and Debian systems you should run the following
  command:

    ./configure --prefix=/opt/pbs

  For openSUSE systems (see note 3 below) you should run the
  following command:

    ./configure --prefix=/opt/pbs --libexecdir=/opt/pbs/libexec

  If PTL needs to be installed along with pbspro use the option 
  "--enable-ptl" (see note 5 below)
    eg ./configure --prefix=/opt/pbs --enable-ptl

7. Build PBS Pro by running "make". (See note 4 below)

    make

8. Configure sudo to allow your user account to run commands as
  root. Refer to the online manual pages for sudo, sudoers, and
  visudo.

9. Install PBS Pro. Use sudo to run the command as root.

    sudo make install

10. Configure PBS Pro by executing the post-install script.

    sudo /opt/pbs/libexec/pbs_postinstall

11. Edit /etc/pbs.conf to configure the PBS Pro services that
  should be started. If you are installing PBS Pro on only
  one system, you should change the value of PBS_START_MOM
  from zero to one. If you use vi as your editor, you would
  run:

    sudo vi /etc/pbs.conf

12. Some file permissions must be modified to add SUID privilege.

    sudo chmod 4755 /opt/pbs/sbin/pbs_iff /opt/pbs/sbin/pbs_rcp

13. Start the PBS Pro services.

    sudo /etc/init.d/pbs start

14. All configured PBS services should now be running. Update
  your PATH and MANPATH variables by sourcing the appropriate
  PBS Pro profile or logging out and back in.

  For Bourne shell (or similar) run the following:
    . /etc/profile.d/pbs.sh

  For C shell (or similar) run the following:
    source /etc/profile.d/pbs.csh

15. You should now be able to run PBS Pro commands to submit
  and query jobs. Some examples follow.

bash$ qstat -B
Server             Max   Tot   Que   Run   Hld   Wat   Trn   Ext Status
---------------- ----- ----- ----- ----- ----- ----- ----- ----- -----------
host1                0     0     0     0     0     0     0     0 Active
bash$ pbsnodes -a
host1
     Mom = host1
     ntype = PBS
     state = free
     pcpus = 2
     resources_available.arch = linux
     resources_available.host = host1
     resources_available.mem = 2049248kb
     resources_available.ncpus = 2
     resources_available.vnode = host1
     resources_assigned.accelerator_memory = 0kb
     resources_assigned.mem = 0kb
     resources_assigned.naccelerators = 0
     resources_assigned.ncpus = 0     
     resources_assigned.vmem = 0kb
     resv_enable = True
     sharing = default_shared
     license = l

bash$ echo "sleep 60" | qsub
0.host1
bash$ qstat -a

host1:
                                                            Req'd  Req'd   Elap
Job ID          Username Queue    Jobname    SessID NDS TSK Memory Time  S Time
--------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
0.host1         mike     workq    STDIN        2122   1   1    --    --  R 00:00

bash$

--------------------------------------------------------------------

NOTES:

Note 1: If you modify configure.ac or adjust timestamps on any files
  that are automatically generated, you will need to regenerate them
  by re-running autogen.sh.

Note 2: It is advisable to create a simple shell script that calls
  configure with the appropriate options for your environment. This
  ensures configure will be called with the same arguments during
  subsequent invocations. If you have already run configure you can
  regenerate all of the Makefiles by running "./config.status".
  The first few lines of config.status will reveal the options that
  were specified when configure was run. If you set envirnment
  variables such as CFLAGS it is best to do so as an argument to
  configure (e.g. ./configure CFLAGS="-O0 -g" --prefix=/opt/pbs).
  This will ensure consistency when config.status regenerates the
  Makefiles.

Note 3: The openSUSE rpm package expands %_libexecdir to /opt/pbs/lib
  rather than /opt/pbs/libexec which causes problems for the post-
  install scripts. Providing the --libexecdir value to configure
  overrides this behavior.

Note 4: You need to use a POSIX (or nearly POSIX) make. GNU make
  works quite well in this regard; BSD make does not. If you are
  having any sort of build problems, your make should be a prime
  suspect. Tremendous effort has been expended to provide proper
  dependency generation and makefiles without relying on any
  non-POSIX features. The build should work fine with a simple call
  to make, however, complicating things by using various make flags
  is not guaranteed to work. Don't be surprised if the first thing
  that make does is call configure again.

Note 5: PTL gets installed in the parent directory of where PBS Professional
  is installed. For example if you have given install prefix=/opt/pbs, then 
  you can find PTL installation in the directory /opt/ptl . You may need to 
  log out and log in from the terminal for PATH and PYTHONPATH to update.



Using valgrind with PBS Professional.
-------------------------------------
Here is a set of steps to detect memory errors/leaks within PBS code.

1. Install the valgrind development package.

   yum install valgrind-devel (zypper for OpenSUSE).


2. Compile Python in a way that valgrind can work with it, as follows:

   ./configure --prefix=<installdir> --without-pymalloc --with-pydebug --with-valgrind
   make; make install


3. Compile PBS Professional with the special python and in debug mode as follows:

   ./configure --prefix=<installdir> --with-python=<python-dir>  CFLAGS="-g -DPy_DEBUG -DDEBUG -Wall -Werror"

   
4. Run pbs daemons under valgrind.
   
   a) To detect memory errors (not leaks) run pbs daemons as follows: 
	
   export LD_LIBRARY_PATH=/opt/pbs/pgsql/lib:/opt/pbs/lib:$LD_LIBRARY_PATH
   valgrind --tool=memcheck --log-file=/tmp/val.out /opt/pbs/sbin/pbs_server.bin


   b) To detect memory leaks use the supplied leaks suppression file valgrind.supp, as follows: 
   
   export LD_LIBRARY_PATH=/opt/pbs/pgsql/lib:/opt/pbs/lib:$LD_LIBRARY_PATH
   valgrind --tool=memcheck --log-file=/tmp/val.out --suppressions=./valgrind.supp --leak-check=full --track-origins=yes /opt/pbs/sbin/pbs_server.bin