.\" Copyright (C) 1994-2018 Altair Engineering, Inc.
.\" For more information, contact Altair at www.altair.com.
.\"
.\" This file is part of the PBS Professional ("PBS Pro") software.
.\"
.\" Open Source License Information:
.\"
.\" PBS Pro is free software. You can redistribute it and/or modify it under the
.\" terms of the GNU Affero General Public License as published by the Free
.\" Software Foundation, either version 3 of the License, or (at your option) any
.\" later version.
.\"
.\" PBS Pro is distributed in the hope that it will be useful, but WITHOUT ANY
.\" WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
.\" FOR A PARTICULAR PURPOSE.
.\" See the GNU Affero General Public License for more details.
.\"
.\" You should have received a copy of the GNU Affero General Public License
.\" along with this program.  If not, see <http://www.gnu.org/licenses/>.
.\"
.\" Commercial License Information:
.\"
.\" For a copy of the commercial license terms and conditions,
.\" go to: (http://www.pbspro.com/UserArea/agreement.html)
.\" or contact the Altair Legal Department.
.\"
.\" Altair’s dual-license business model allows companies, individuals, and
.\" organizations to create proprietary derivative works of PBS Pro and
.\" distribute them - whether embedded or bundled with other software -
.\" under a commercial license agreement.
.\"
.\" Use of Altair’s trademarks, including but not limited to "PBS™",
.\" "PBS Professional®", and "PBS Pro™" and Altair’s logos is subject to Altair's
.\" trademark licensing policies.
.\"
.TH pbsdsh 1B "13 October 2017" Local "PBS Professional"
.SH NAME
.B pbsdsh 
\- distribute tasks to vnodes under PBS

.SH SYNOPSIS
.B pbsdsh 
[-c <copies>] [-s] [-v] [-o] -- <program> [<program args>]
.br
.B pbsdsh 
[-n <vnode index>] [-s] [-v] [-o] -- <program> [<program args>]
.br
.B pbsdsh 
--version

.SH DESCRIPTION
The 
.B pbsdsh
command allows you to distribute and execute a task on each of the vnodes
assigned to your job by executing (spawning) the application on each
vnode.  The 
.B pbsdsh 
command uses the PBS Task Manager, or TM, to distribute the program on 
the allocated vnodes.

When run without the 
.I -c 
or the 
.I -n 
option, 
.B pbsdsh 
will spawn the program
on all vnodes allocated to the PBS job.  The spawns take place concurrently;
all execute at (about) the same time.

Note that the double dash must come after the options and before the 
program and arguments.  The double dash is only required for Linux.

The 
.B pbsdsh 
command runs one task for each line in the $PBS_NODEFILE.  Each MPI rank 
gets a single line in the $PBS_NODEFILE, so if you are running multiple 
MPI ranks on the same host, you still get multiple 
.B pbsdsh 
tasks on that host.

.B Example
.br
The following example shows the 
.B pbsdsh 
command inside of a PBS batch job. The options indicate that the user wants 
.B pbsdsh 
to run the myapp program with one argument (app-arg1) on all four vnodes 
allocated to the job (i.e. the default behavior).

.RS 5
#!/bin/sh
.br
#PBS -l select=4:ncpus=1
.br
#PBS -l walltime=1:00:00

pbsdsh ./myapp app-arg1
.RE

.SH OPTIONS
.IP "-c copies"
The program is spawned 
.I copies
times on the vnodes allocated, one per vnode, unless
.I copies 
is greater than the number of vnodes.
If 
.I copies
is greater than the number of vnodes, 
it wraps around, running multiple instances on some vnodes.
This option is mutually exclusive with 
.I -n.
.IP "-n <vnode index>"
The program is spawned only on a single vnode, which is the 
.I vnode index -th
vnode allocated.  This option is mutually exclusive with 
.I -c.
.IP -o
No obit request is made for spawned tasks.  The program does not wait for
the tasks to finish.
.IP -s
Te program is run in turn on each vnode, one after the other.
.IP -v
Produces verbose output about error conditions and task exit status.

.IP --version
The 
.B pbsdsh
command returns its PBS version information and exits.
This option can only be used alone

.SH OPERANDS
.IP program
The first operand,
.I program ,
is the program to execute.  The double dash must precede the 
.I program 
under Linux.
.LP
.IP "program args"
Additional operands,
.I program args ,
are passed as arguments to the 
.I program.

.SH STANDARD ERROR
The 
.B pbsdsh 
command writes a diagnostic message to standard error for
each error occurrence.

.SH SEE ALSO
The
.B PBS Professional User's Guide,
the
.B PBS Professional Administrator's Guide,
qsub(1B), tm(3).