-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: added operators and zfp section #3161
Merged
JasonRuonanWang
merged 1 commit into
ornladios:release_28
from
vicentebolea:add-operators-docs-section
Apr 16, 2022
+84
−0
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
************* | ||
ZFPCompressor | ||
************* | ||
|
||
The ``ZFPCompressor`` Operator is compressor that uses a lossy but optionally | ||
error-bounded compression to achieve high compression ratios. | ||
|
||
ZFP provides compressed-array classes that support high throughput read and | ||
write random access to individual array elements. ZFP also supports serial and | ||
parallel (OpenMP and CUDA) compression of whole arrays, e.g., for applications | ||
that read and write large data sets to and from disk. | ||
|
||
ADIOS2 provides a ``ZFPCompressor`` operator that lets you compress an | ||
decompress variables. Below there is an example of how to invoke | ||
``ZFPCompressor`` operator: | ||
|
||
.. code-block:: c++ | ||
|
||
adios2::IO io = adios.DeclareIO("Output"); | ||
auto ZFPOp = adios.DefineOperator("ZFPCompressor", adios2::ops::LossyZFP); | ||
|
||
auto var_r32 = io.DefineVariable<float>("r32", shape, start, count); | ||
var_r32.AddOperation(ZFPOp, {{adios2::ops::zfp::key::rate, rate}}); | ||
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
ZFPCompressor Specific parameters | ||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
The ``ZFPCompressor`` operator accepts the following operator specific | ||
parameters: | ||
|
||
+-------------------+---------------------------------------------+ | ||
| ``ZFPCompressor`` available parameters | | ||
+===================+=============================================+ | ||
| ``accuracy`` | Fixed absolute error tolerance | | ||
+-------------------+---------------------------------------------+ | ||
| ``rate`` | Fixed number of bits in a compression unit | | ||
+-------------------+---------------------------------------------+ | ||
| ``precision`` | Fixed number of uncompressed bits per value | | ||
+-------------------+---------------------------------------------+ | ||
| ``backend`` | Backend device: ``cuda`` ``omp`` ``serial`` | | ||
+-------------------+---------------------------------------------+ | ||
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
ZFPCompressor Execution Policy | ||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
``ZFPCompressor`` can run in multiple backend devices: GPUs (CUDA), OpenMP, and | ||
in the host CPU. By default ``ZFPCompressor`` will choose its backend following | ||
the above order upon the availability of the device adapter. | ||
|
||
Exceptionally, if its corresponding ADIOS2 variable contains a CUDA memory | ||
address, this is a CUDA buffer, the CUDA backend will be called if available. | ||
|
||
In any case, the user can manually set the backend using the ZFPOperator | ||
specific parameter ``backend``. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
################### | ||
Supported Operators | ||
################### | ||
|
||
The Operator abstraction allows ADIOS2 to act upon the user application data, | ||
either from a ``adios2::Variable`` or a set of Variables in an ``adios2::IO`` | ||
object. Current supported operations are: | ||
|
||
1. Data compression/decompression, lossy and lossless. | ||
2. Callback functions (C++11 bindings only) supported by specific engines | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Callback operators are currently not used in any engines. |
||
|
||
This section provides a description of the :ref:`Available Operators` in ADIOS2 | ||
and their specific parameters to allow extra-control from the user. Parameters | ||
are passed in key-value pairs for: | ||
|
||
1. Operator general supported parameters. | ||
|
||
2. Operator specific supported parameters. | ||
|
||
Parameters are passed at: | ||
|
||
1. Compile time: using the second parameter of the method | ||
``ADIOS2::DefineOperator`` | ||
|
||
2. :ref:`Runtime Configuration Files` in the :ref:`ADIOS` component. | ||
|
||
.. include:: ZFPCompressor.rst |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The operator is called CompressZFP, also the name of the class