Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* changes according to tvmc enhancements.
Browse files Browse the repository at this point in the history
srkreddy1238 committed Mar 6, 2023
1 parent b2bb23b commit 5b9b1f2
Showing 5 changed files with 91 additions and 126 deletions.
70 changes: 59 additions & 11 deletions docs/how_to/deploy/adreno.rst
Original file line number Diff line number Diff line change
@@ -559,12 +559,21 @@ To do the conversion you need to call adreno specific transformation API as soon

.. code:: python
from tvm.relay.op.contrib import adreno
adreno.convert_to_dtype(mod["main"], "float16")
from tvm.driver.tvmc.transform import apply_graph_transforms
mod = apply_graph_transforms(
mod,
{
"mixed_precision": True,
"mixed_precision_ops": ["nn.conv2d", "nn.dense"],
"mixed_precision_calculation_type": "float16",
"mixed_precision_acc_type": "float16",
},
)
``tvm.relay.op.contrib.adreno.convert_to_dtype`` is simplified API over ``ToMixedPrecision`` pass to get desired precision.
We then can compile our model in any convinient way
``tvm.driver.tvmc.transform.apply_graph_transforms`` is simplified API over ``ToMixedPrecision`` pass to get desired precision.

We can then compile our model in any convinient way

.. code:: python
@@ -573,8 +582,23 @@ We then can compile our model in any convinient way
mod, target_host=target_host, target=target, params=params
)
While using ``tvmc`` python interface, argument ``pre_build_hooks=[adreno.mixed_precision_fp16]`` enables precision conversion to float16.
Similarly, ``tvmc`` command line interface option ``--pre-build-hooks "adreno.mixed_precision_fp16]"`` does the same.
While using ``tvmc`` python interface, the below arguments enables precision conversion to float16.

.. code:: python
mixed_precision = True,
mixed_precision_ops = ["nn.conv2d", "nn.dense"],
mixed_precision_calculation_type = "float16",
mixed_precision_acc_type = "float16"
Similarly, ``tvmc`` command line interface option bas below listed options.

.. code:: bash
--mixed-precision
--mixed-precision-ops nn.conv2d nn.dense
--mixed-precision-calculation-type float16
--mixed-precision-acc-type float16
**float16_acc32 (Mixed Precision)**
@@ -595,12 +619,21 @@ as it halves the expected size of the weights (FP16_acc16 case).
``ToMixedPrecision`` pass usage is simplified into a simple call as shown below for usage.

.. code:: python
from tvm.driver.tvmc.transform import apply_graph_transforms
mod = apply_graph_transforms(
mod,
{
"mixed_precision": True,
"mixed_precision_ops": ["nn.conv2d", "nn.dense"],
"mixed_precision_calculation_type": "float16",
"mixed_precision_acc_type": "float32",
},
)
from tvm.relay.op.contrib import adreno
adreno.convert_to_dtype(mod["main"], "float16_acc32")
``tvm.driver.tvmc.transform.apply_graph_transforms`` is simplified API over ``ToMixedPrecision`` pass to get desired precision.

We then can compile our model in any convinient way
We can then compile our model in any convinient way

.. code:: python
@@ -609,8 +642,23 @@ We then can compile our model in any convinient way
mod, target_host=target_host, target=target, params=params
)
While using ``tvmc`` python interface, argument ``pre_build_hooks=[adreno.mixed_precision_fp16_acc32]`` enables precision conversion to float16.
Similarly, ``tvmc`` command line interface option ``--pre-build-hooks "adreno.mixed_precision_fp16_acc32]"`` does the same.
While using ``tvmc`` python interface, the below arguments enables precision conversion to float16.

.. code:: python
mixed_precision = True,
mixed_precision_ops = ["nn.conv2d", "nn.dense"],
mixed_precision_calculation_type = "float16",
mixed_precision_acc_type = "float32"
Similarly, ``tvmc`` command line interface option bas below listed options.

.. code:: bash
--mixed-precision
--mixed-precision-ops nn.conv2d nn.dense
--mixed-precision-calculation-type float16
--mixed-precision-acc-type float32
.. |High-level overview of the Adreno™ A5x architecture for OpenCL| image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/how-to/adreno_architecture.png
24 changes: 15 additions & 9 deletions gallery/how_to/deploy_models/deploy_model_on_adreno.py
Original file line number Diff line number Diff line change
@@ -145,9 +145,10 @@
# it's advisable to use lower precision.
# We have a helper API to make the precision conversion simple and
# it supports dtype with "float16" and "float16_acc32" modes.
# Let's choose "float16_acc32" for this example.
# Let's choose "float16" for calculation and "float32" for accumulation.

dtype = "float16_acc32"
calculation_dtype = "float16"
acc_dtype = "float32"

# Specify Adreno target before compiling to generate texture
# leveraging kernels and get all the benefits of textures
@@ -240,16 +241,21 @@
# We may need to register precision rules like precision type, accumultation
# datatype ...etc. for the required operators to override the default settings.
# The below helper api simplifies the precision conversions across the module.
# Now it supports dtypes "float16" and "float16_acc32".

# dtype is set to "float16_acc32" in configuration section above.
# Calculation dtype is set to "float16" and accumulation dtype is set to "float32"
# in configuration section above.

from tvm.relay.op.contrib import adreno
from tvm.driver.tvmc.transform import apply_graph_transforms

adreno.convert_to_dtype(mod["main"], dtype)

dtype = "float32" if dtype == "float32" else "float16"
print(mod)
mod = apply_graph_transforms(
mod,
{
"mixed_precision": True,
"mixed_precision_ops": ["nn.conv2d", "nn.dense"],
"mixed_precision_calculation_type": calculation_dtype,
"mixed_precision_acc_type": acc_dtype,
},
)

#################################################################
# As you can see in the IR, the architecture now contains cast operations, which are
1 change: 0 additions & 1 deletion python/tvm/relay/op/contrib/__init__.py
Original file line number Diff line number Diff line change
@@ -27,4 +27,3 @@
from .tensorrt import *
from .cutlass import *
from .clml import *
from .adreno import *
98 changes: 0 additions & 98 deletions python/tvm/relay/op/contrib/adreno.py

This file was deleted.

24 changes: 17 additions & 7 deletions tests/python/relay/opencl_texture/test_network.py
Original file line number Diff line number Diff line change
@@ -22,20 +22,30 @@
import tvm
from tvm import relay
from tvm.contrib import utils
from tvm.relay.op.contrib import adreno
from tvm.relay import testing
from tvm.relay.op import register_mixed_precision_conversion
from utils.adreno_utils import build_run_compare, get_model, gpu_preprocess


def _test_mobilenet_v1(remote, target, dtype):
def _test_mobilenet_v1(remote, target, calc_dtype, acc_dtype):
mod, params, inputs, dtypes = get_model(
"https://github.com/mlcommons/mobile_models/raw/main/v0_7/tflite/mobilenet_edgetpu_224_1.0_float.tflite",
"mobilenet_edgetpu_224_1.0_float.tflite",
"tflite",
)
if dtype == "float16" or dtype == "float16_acc32":
mod = adreno.convert_to_dtype(mod["main"], dtype)
if calc_dtype == "float16":
from tvm.driver.tvmc.transform import apply_graph_transforms

mod = apply_graph_transforms(
mod,
{
"mixed_precision": True,
"mixed_precision_ops": ["nn.conv2d", "nn.dense"],
"mixed_precision_calculation_type": calc_dtype,
"mixed_precision_acc_type": acc_dtype,
},
)

build_run_compare(remote, mod, params, inputs, dtypes, target, [])


@@ -44,21 +54,21 @@ def _test_mobilenet_v1(remote, target, dtype):
@tvm.testing.parametrize_targets("opencl -device=adreno")
@pytest.mark.skipif(tvm.testing.utils.IS_IN_CI, reason="CI doesn't support fp16(half datatypes)")
def test_mobilenet_v1_fp16(remote, target):
_test_mobilenet_v1(remote, target, "float16")
_test_mobilenet_v1(remote, target, "float16", "float16")


@pytest.mark.skip(reason="See https://github.com/apache/tvm/issues/13443")
@tvm.testing.requires_opencl
@tvm.testing.parametrize_targets("opencl -device=adreno")
def test_mobilenet_v1_fp32(remote, target):
_test_mobilenet_v1(remote, target, "float32")
_test_mobilenet_v1(remote, target, "float32", "float32")


@pytest.mark.skip(reason="See https://github.com/apache/tvm/issues/13443")
@tvm.testing.requires_opencl
@tvm.testing.parametrize_targets("opencl -device=adreno")
def test_mobilenet_v1_fp16_acc32(remote, target):
_test_mobilenet_v1(remote, target, "float16_acc32")
_test_mobilenet_v1(remote, target, "float16", "float32")


if __name__ == "__main__":

0 comments on commit 5b9b1f2

Please sign in to comment.