.\" Copyright (C) 1994-2018 Altair Engineering, Inc.
.\" For more information, contact Altair at www.altair.com.
.\"
.\" This file is part of the PBS Professional ("PBS Pro") software.
.\"
.\" Open Source License Information:
.\"
.\" PBS Pro is free software. You can redistribute it and/or modify it under the
.\" terms of the GNU Affero General Public License as published by the Free
.\" Software Foundation, either version 3 of the License, or (at your option) any
.\" later version.
.\"
.\" PBS Pro is distributed in the hope that it will be useful, but WITHOUT ANY
.\" WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
.\" FOR A PARTICULAR PURPOSE.
.\" See the GNU Affero General Public License for more details.
.\"
.\" You should have received a copy of the GNU Affero General Public License
.\" along with this program.  If not, see <http://www.gnu.org/licenses/>.
.\"
.\" Commercial License Information:
.\"
.\" For a copy of the commercial license terms and conditions,
.\" go to: (http://www.pbspro.com/UserArea/agreement.html)
.\" or contact the Altair Legal Department.
.\"
.\" Altair’s dual-license business model allows companies, individuals, and
.\" organizations to create proprietary derivative works of PBS Pro and
.\" distribute them - whether embedded or bundled with other software -
.\" under a commercial license agreement.
.\"
.\" Use of Altair’s trademarks, including but not limited to "PBS™",
.\" "PBS Professional®", and "PBS Pro™" and Altair’s logos is subject to Altair's
.\" trademark licensing policies.
.\"
.TH mpiexec 8B "12 October 2017" Local "PBS Professional"
.SH NAME
.B mpiexec 
\- run MPI programs under PBS on Linux

.SH SYNOPSIS
.B mpiexec 

.B mpiexec 
--version

.SH DESCRIPTION
The PBS 
.B mpiexec
command provides the standard mpiexec interface on a system
running supported versions of Performance Suite.  
If executed on a different system, it will assume
it was invoked by mistake.  In this case it will use the value of
.B PBS_O_PATH
to search for the correct mpiexec.  If one is found, the PBS 
.B mpiexec 
will
exec it.

The PBS
.B mpiexec
calls the HPE SGI mpirun(1).  The name of the array to use when invoking
mpirun is user-specifiable via the
.B PBS_MPI_SGIARRAY 
environment variable.

It is transparent to the user; MPI jobs submitted outside of PBS will
run as they would normally.  MPI jobs can be launched across multiple
HPE SGI machines.  PBS will manage, track, and cleanly terminate multi-host MPI
jobs.  PBS users can run MPI jobs within specific partitions.

If CSA has been configured and enabled, PBS will collect accounting
information on all tasks launched by an MPI job.  CSA information will
be associated with the PBS job ID that invoked it, on each execution
host.

If the 
.B PBS_MPI_DEBUG 
environment variable's value has a nonzero
length, PBS will write debugging information to standard output.

.SH USAGE 
The PBS
.B mpiexec 
command presents the mpiexec interface described in section 
"4.1 Portable MPI Process Startup" of the "MPI-2: Extensions 
to the Message-Passing Interface" document in 
http://www.mpi-forum.org/docs/mpi-20-html/node42.htm

.SH OPTIONS
.IP "--version" 8
The 
.B mpiexec
command returns its PBS version information and exits.
This option can only be used alone.

.SH REQUIREMENTS
System running a supported version of Performance Suite.

PBS uses HPE SGI's mpirun(1) command to launch MPI jobs.  HPE SGI's mpirun
must be in the standard location.

The location of 
.B pbs_attach() 
on each node of a multinode MPI job
must be the same as it is on the mother superior node.

In order to run multihost jobs, the HPE SGI Array Services must be
correctly configured.  HPE SGI systems communicating via HPE SGI's Array
Services must all use the same version of the sgi-arraysvcs package.
HPE SGI systems communicating via HPE SGI's Array Services must have been
configured to interoperate with each other using the default array.
See HPE SGI's array_services(5) man page.

.SH ENVIRONMENT VARIABLES

The PBS 
.B mpiexec 
script sets the 
.B PBS_CPUSET_DEDICATED 
environment variable to assert exclusive use of the resources in the
assigned cpuset.

The PBS 
.B mpiexec 
checks the 
.B PBS_MPI_DEBUG 
environment variable.  If
this variable has a nonzero length, debugging information is written.

If the 
.B PBS_MPI_SGIARRAY 
environment variable is present, the PBS
.B mpiexec 
will use its value as the name of the array to use when
invoking mpirun.

The 
.B PBS_ENVIRONMENT 
environment variable is used to determine whether
.B mpiexec 
is being called from within a PBS job.

The PBS 
.B mpiexec 
uses the value of 
.B PBS_O_PATH 
to search for the correct
.B mpiexec 
if it was invoked by mistake.

.SH PATH
PBS' 
.B mpiexec 
is located in 
.I PBS_EXEC/bin/mpiexec.


.SH SEE ALSO
The 
.B PBS Professional Administrator's Guide
.LP
HPE SGI man pages: 
HPE SGI's mpirun(1), 
HPE SGI's mpiexec_mpt(1),
HPE SGI's array_services(5)
.LP
PBS man pages:
pbs_attach(8B)