[TVM][RELAY] Incomplete quantize annotate

[ehsanmok]

I changed the compile function in deploy ssd gluoncv tutorial to have default quantization pass

def compile(target):
    net, params = relay.frontend.from_mxnet(block, {"data": dshape})
    with tvm.relay.quantize.qconfig():
        qgraph = relay.quantize.quantize(net, params)

    with relay.build_config(opt_level=3):
        graph, lib, params = relay.build(qgraph, target)

    return graph, lib, params

and then

for target, ctx in target_list:
    if target == "cuda":
        print("GPU not supported yet, skip.")
        continue
    graph, lib, params = compile(target)
    class_IDs, scores, bounding_boxs = run(graph, lib, params, ctx)

failed with

TVMError                                  Traceback (most recent call last)
<ipython-input-23-f6d8f7869cc8> in <module>
      3         print("GPU not supported yet, skip.")
      4         continue
----> 5     graph, lib, params = compile(target)
      6     class_IDs, scores, bounding_boxs = run(graph, lib, params, ctx)

<ipython-input-22-f3b5a820bcdd> in compile(target)
     17                      debug_enabled_ops=None):
     18 
---> 19         qgraph = relay.quantize.quantize(net, params)
     20 
     21     with relay.build_config(opt_level=3):

~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/relay/quantize/quantize.py in quantize(graph, params, dataset)
    284         graph = _build.optimize(graph, params=params)
    285 
--> 286     graph = annotate(graph)
    287     graph = calibrate(graph, dataset)
    288     graph = realize(graph)

~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/relay/quantize/quantize.py in annotate(graph)
    175     """
    176     _set_conv_counter(0)  # reset counter
--> 177     return _quantize.annotate(graph)
    178 
    179 

tvm/_ffi/_cython/./function.pxi in tvm._ffi._cy3.core.FunctionBase.__call__()

tvm/_ffi/_cython/./function.pxi in tvm._ffi._cy3.core.FuncCall()

tvm/_ffi/_cython/./function.pxi in tvm._ffi._cy3.core.FuncCall3()

tvm/_ffi/_cython/./base.pxi in tvm._ffi._cy3.core.CALL()

TVMError: TVMCall CFunc Error:
Traceback (most recent call last):
  File "tvm/_ffi/_cython/./function.pxi", line 38, in tvm._ffi._cy3.core.tvm_callback
  File "/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/relay/quantize/_annotate.py", line 96, in frewrite_with_guard
    return func(ref_call, new_args, ctx)
  File "/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/relay/quantize/_annotate.py", line 176, in multiply_rewrite
    raise ValueError
ValueError

1. How to fix the compilation for quantization?
2. Fix error handling of multiply_rewrite.

• TVM master
• mxnet-cu9.0==1.4
• gluoncv==0.4 pre release

cc @ZihengJiang @vinx13

[vinx13]

@ZihengJiang we missed the case where lhs_kind is QInput and rhs_kind is None in multiply_rewrite

[ehsanmok]

@ZihengJiang any good news on this?

[vinx13]

Fix: #2848

[ehsanmok]

@vinx13 thanks! now there's this issue quantizing "yolo3_mobilenet1.0_coco" loaded from gluoncv:

TVMError  Traceback (most recent call last)
<ipython-input-23-f6d8f7869cc8> in <module>
      3         print("GPU not supported yet, skip.")
      4         continue
----> 5     graph, lib, params = compile(target)
      6     class_IDs, scores, bounding_boxs = run(graph, lib, params, ctx)

<ipython-input-21-5901f3423aef> in compile(target, quantize)
      4     if quantize:
      5         with tvm.relay.quantize.qconfig(skip_k_conv=0):
----> 6             net = relay.quantize.quantize(net, params)
      7 
      8     with relay.build_config(opt_level=3):

~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/relay/quantize/quantize.py in quantize(graph, params, dataset)
    284         graph = _build.optimize(graph, params=params)
    285 
--> 286     graph = annotate(graph)
    287     graph = calibrate(graph, dataset)
    288     graph = realize(graph)

~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/relay/quantize/quantize.py in annotate(graph)
    175     """
    176     _set_conv_counter(0)  # reset counter
--> 177     return _quantize.annotate(graph)
    178 
    179 

tvm/_ffi/_cython/./function.pxi in tvm._ffi._cy3.core.FunctionBase.__call__()

tvm/_ffi/_cython/./function.pxi in tvm._ffi._cy3.core.FuncCall()

tvm/_ffi/_cython/./function.pxi in tvm._ffi._cy3.core.FuncCall3()

tvm/_ffi/_cython/./base.pxi in tvm._ffi._cy3.core.CALL()

TVMError: Traceback (most recent call last):
  [bt] (8) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xa9759b) [0x7f4a605ce59b]
  [bt] (7) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xc84471) [0x7f4a607bb471]
  [bt] (6) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xc85953) [0x7f4a607bc953]
  [bt] (5) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xa8d837) [0x7f4a605c4837]
  [bt] (4) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xa92d6d) [0x7f4a605c9d6d]
  [bt] (3) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xa94cf9) [0x7f4a605cbcf9]
  [bt] (2) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xa9759b) [0x7f4a605ce59b]
  [bt] (1) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xc847f4) [0x7f4a607bb7f4]
  [bt] (0) ~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/libtvm.so(+0xf262b4) [0x7f4a60a5d2b4]
  File "tvm/_ffi/_cython/./function.pxi", line 38, in tvm._ffi._cy3.core.tvm_callback
  File "~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/relay/quantize/_annotate.py", line 96, in frewrite_with_guard
    return func(ref_call, new_args, ctx)
  File "~/anaconda3/lib/python3.7/site-packages/tvm-0.6.dev0-py3.7-linux-x86_64.egg/tvm/relay/quantize/_annotate.py", line 258, in concatenate_rewrite
    assert k is None
TVMError: AssertionError

[vinx13]

@ehsanmok I also met this before. We need to quantize all fields in case that there are float fields in concat. I will make a pr soon.