Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MPI Sessions Use-case #349

Merged
merged 3 commits into from
Jul 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions App_Use_Cases.tex
Original file line number Diff line number Diff line change
Expand Up @@ -448,4 +448,61 @@ \subsubsection{Coordinating at Runtime with Multiple Event Handlers}
\refconst{PMIX_EVENT_PARTIAL_ACTION_TAKEN} \\
\refconst{PMIX_EVENT_ACTION_DEFERRED} \\

\section{MPI Sessions}
\label{app:uc-MPI-sessions}

\subsection{Use Case Summary}
MPI Sessions addresses a number of the limitations of the current MPI programming model. Among the immediate problems MPI Sessions is intended to address are the following:

\begin{itemize}
\item MPI cannot be initialized within an MPI process from different application components without a priori knowledge or coordination,
\item MPI cannot be initialized more than once, and MPI cannot be reinitialized after MPI finalize has been called.
\item With MPI Sessions, an application no longer needs to explicitly call \code{MPI_Init} to make use of MPI, but rather can use a Session to only initialize MPI resources for specific communication needs.
\item Unless the MPI process explicitly calls MPI_Init, there is also no explicit \code{MPI_COMM_WORLD} communicator. Sessions can be created and destroyed multiple times in an MPI process.
\end{itemize}

\subsection{Use Case Details}

\begingroup
\begin{figure*}
\begin{center}
\includegraphics[width=.5\textwidth,keepaspectratio]{figs/mpi-sessions1}
\end{center}
\caption{MPI Communicator from MPI Session Handle using PMIx}
\label{fig:mpi_s1}
\end{figure*}
\endgroup

A PMIx Process Set (PSET) is a user-provided or host environment assigned
label associated with a given set of application processes. Processes can
belong to multiple process sets at a time. Definition of a PMIx
process set typically occurs at time of application execution - e.g., on a
command line: \code{prun -n 4 --pset ocean myoceanapp : -n 3 --pset ice myiceapp}

PMIx PSETs are used for query functions (\code{MPI_SESSION_GET_NUM_PSETS}, \code{MPI_SESSION_GET_NTH_PSET)} and to create \code{MPI_GROUP} from a process set name.

In OpenMPI's MPI Sessions prototype, PMIx groups are used during creation of \code{MPI_COMM} from an \code{MPI_GROUP}. The PMIx group constructor returns a 64-bit PMIx Group Context Identifier (PGCID) that is guaranteed to be unique for the duration of an allocation (in the case of a batch managed environment). This PGCID could be used as a direct replacement for the existing unique identifiers for communicators in MPI (E.g. Communicator Identifiers (CIDs) in Open MPI), but may have performance implications.

There is an important distinction between process sets and process groups. The process set identifiers are set by the host environment and currently there are no PMIx APIs provided by which an application can change a process set membership. In contrast, PMIx process groups can only be defined dynamically by the application.

\littleheader{Related Interfaces}

{\large \refapi{PMIx_Get}}
\pasteSignature{PMIx_Get}

{\large \refapi{PMIx_Group_construct}}
\pasteSignature{PMIx_Group_construct}

\littleheader{Related Attributes}

\pasteAttributeItem{PMIX_PSET_NAMES}
\pasteAttributeItem{PMIX_QUERY_NUM_GROUPS}
\pasteAttributeItem{PMIX_QUERY_GROUP_NAMES}
\pasteAttributeItem{PMIX_QUERY_GROUP_MEMBERSHIP}

\littleheader{Related Constants}

\refconst{PMIX_SUCCESS}
\refconst{PMIX_ERR_NOT_SUPPORTED}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Binary file added figs/mpi-sessions1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.