Skip to content

Commit

Permalink
add documentation on parallelism
Browse files Browse the repository at this point in the history
  • Loading branch information
cedricleroy committed Apr 3, 2018
1 parent fb9e563 commit dcc0798
Showing 1 changed file with 27 additions and 0 deletions.
27 changes: 27 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,33 @@ in the ``graph`` instance.

The result will be (a + b) / 10 - a = -1.5

parallelism
-----------

When resolving the dag, pyungo figure out nodes that can be run
in parallel. When creating a graph, we can specify the option
`parallel=True` for running calculations concurently when possible,
using `Python multiprocessing module <https://docs.python.org/3.6/library/multiprocessing.html>`_.
We can specify the pool size when instantiating the Graph. This will
set the maximum number of processes that will be launched. If 3 nodes
can run in parallel and just 2 processes are used, pyungo will run
calculation on the first 2 nodes first and will run the last one as soon
as a process will be free.

Instantiating a `Graph` with a pool of 5 processes for running calculations
in parralel:

.. code-block:: python
graph = Graph(parallel=True, pool_size=5)
Note: Running functions in parallel has a cost. Python will spend time
creating / deleting new processes. Parallelism is recommended when at
least 2 concurrent nodes have heavy calculations which takes a significant
amount of time.


sanity check
------------

Expand Down

0 comments on commit dcc0798

Please sign in to comment.