Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: added operators and zfp section #3161

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/user_guide/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Funded by the `Exascale Computing Project (ECP) <https://www.exascaleproject.org
components/components
engines/virtual_engines
engines/engines
operators/operators
api_full/api_full
api_high/api_high

Expand Down
56 changes: 56 additions & 0 deletions docs/user_guide/source/operators/ZFPCompressor.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
*************
ZFPCompressor
*************

The ``ZFPCompressor`` Operator is compressor that uses a lossy but optionally
error-bounded compression to achieve high compression ratios.

ZFP provides compressed-array classes that support high throughput read and
write random access to individual array elements. ZFP also supports serial and
parallel (OpenMP and CUDA) compression of whole arrays, e.g., for applications
that read and write large data sets to and from disk.

ADIOS2 provides a ``ZFPCompressor`` operator that lets you compress an
decompress variables. Below there is an example of how to invoke
``ZFPCompressor`` operator:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The operator is called CompressZFP, also the name of the class

.. code-block:: c++

adios2::IO io = adios.DeclareIO("Output");
auto ZFPOp = adios.DefineOperator("ZFPCompressor", adios2::ops::LossyZFP);

auto var_r32 = io.DefineVariable<float>("r32", shape, start, count);
var_r32.AddOperation(ZFPOp, {{adios2::ops::zfp::key::rate, rate}});

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ZFPCompressor Specific parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``ZFPCompressor`` operator accepts the following operator specific
parameters:

+-------------------+---------------------------------------------+
| ``ZFPCompressor`` available parameters |
+===================+=============================================+
| ``accuracy`` | Fixed absolute error tolerance |
+-------------------+---------------------------------------------+
| ``rate`` | Fixed number of bits in a compression unit |
+-------------------+---------------------------------------------+
| ``precision`` | Fixed number of uncompressed bits per value |
+-------------------+---------------------------------------------+
| ``backend`` | Backend device: ``cuda`` ``omp`` ``serial`` |
+-------------------+---------------------------------------------+

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ZFPCompressor Execution Policy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``ZFPCompressor`` can run in multiple backend devices: GPUs (CUDA), OpenMP, and
in the host CPU. By default ``ZFPCompressor`` will choose its backend following
the above order upon the availability of the device adapter.

Exceptionally, if its corresponding ADIOS2 variable contains a CUDA memory
address, this is a CUDA buffer, the CUDA backend will be called if available.

In any case, the user can manually set the backend using the ZFPOperator
specific parameter ``backend``.
27 changes: 27 additions & 0 deletions docs/user_guide/source/operators/operators.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
###################
Supported Operators
###################

The Operator abstraction allows ADIOS2 to act upon the user application data,
either from a ``adios2::Variable`` or a set of Variables in an ``adios2::IO``
object. Current supported operations are:

1. Data compression/decompression, lossy and lossless.
2. Callback functions (C++11 bindings only) supported by specific engines
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callback operators are currently not used in any engines.


This section provides a description of the :ref:`Available Operators` in ADIOS2
and their specific parameters to allow extra-control from the user. Parameters
are passed in key-value pairs for:

1. Operator general supported parameters.

2. Operator specific supported parameters.

Parameters are passed at:

1. Compile time: using the second parameter of the method
``ADIOS2::DefineOperator``

2. :ref:`Runtime Configuration Files` in the :ref:`ADIOS` component.

.. include:: ZFPCompressor.rst