Skip to content

Commit

Permalink
Add callback doc
Browse files Browse the repository at this point in the history
  • Loading branch information
yzhang-23 committed Jan 24, 2024
1 parent 08cc66b commit 522bc45
Show file tree
Hide file tree
Showing 2 changed files with 49 additions and 2 deletions.
Binary file modified docs/source/developer/design/assets/runtime_view.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
51 changes: 49 additions & 2 deletions docs/source/developer/design/runtime_view.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,34 @@ number one supercomputer in the world, or anything in between.
#. MPI compatability.
#. Flexibility of backend.

***************************************************************
How Does RuntimeView Manage the Parallel Computing Environment?
***************************************************************

.. _design_parallel_manage:

Currently when a RuntimeView object is created, the standard MPI function
``MPI_Init()`` is called to initialize the parallel computing
environment. One legacy of different NWChemEx repositories is that many
repositories have their own functions to initialize and finalize the parallel
computing environment. These functions may rely on some external packages, such
as MADNESS we used to use in ParallelZone, to manage the paralell environment.
Different repositories depending on common external packages makes the
dependency relation complicated and could create a diamond dependency problem.
Installing the correct versions of package may be very tricky and building all
NWChemEx repositories would be very chanllenging. To make our lives easier we
want every NWChemEx repository which is downstream to ParallelZone to use
ParallelZone for parallel environment management.

In order to provide a common API to other downstream repositories for parallel
environment management, callback functions are implemented in the
``RuntimeView`` class to finalize the parallel environment. A function stack
member is put into the ``RuntimeView`` class to store the callbacks. A
downstream repository can register its version of finalize functions as
finalize callbacks into this stack. When a calculation is done, the destructor
of the ``RuntimeView`` object is called, and in the destructor the finalize
registered callbacks in the stack are called in a last-in-first-out fashion.

************************
RuntimeView Architecture
************************
Expand All @@ -72,7 +100,11 @@ addresses the above consideration by (numbering is from above):
``GPU`` objects in a particular ``ResourceSet``.
- This facilitates selecting start/end points.

#. MADNESS is built on MPI. MPI is exposed through MADNESS.
#. MPI_Init() is called directly to initialize the parallel computing
environment. Callbacks are implemented to allow different finalize function
to clean the parallel computing environment. For the design considerations
on callbacks please see :ref:`design_parallel_manage`.

#. The use of the PIMPL design allows us to hide many of the backend types. It
also facilitates writing an implementation for a different backend down the
line (although the API would need to change too).
Expand All @@ -92,7 +124,7 @@ Some finer points:
Proposed APIs
*************

Examples of all-to-all communications
Examples of all-to-all communications:

.. code-block:: c++

Expand All @@ -106,6 +138,20 @@ Examples of all-to-all communications
// This is an all reduce
auto output2 = rt.reduce(data, op);


An example of using callbacks to finalize the parallel computing environment:

.. code-block:: c++

// Create a RuntimeView object
RuntimeView rt;

// Register the corresponding finalize() function into the callback stack
rt.stack_callback(your_finalize_function());

The designated function ``your_finalize_function()`` would be called when the
RuntimeView object is destroyed.

.. note::

As written the APIs assume the data is going to/from RAM. If we eventually
Expand All @@ -118,3 +164,4 @@ Examples of all-to-all communications

because that would result in deadlock (it calls a series of all-to-one calls
where each rank thinks it's the root).

0 comments on commit 522bc45

Please sign in to comment.