resnet50_quantized: errors in load and initialization

[NathanSegerlind]

Symptom: When I try to load the resnet50_quantized network using the method from the "load pretrained models" tutorial (https://github.com/caffe2/tutorials/blob/master/Loading_Pretrained_Models.ipynb), I get a Cannot create operator of type 'Int8GivenTensorFill' on the device 'CPU'. error.

Reproduction:

INIT_NET= '/home/ubuntu/segmentation_notebooks/resnet_quantized/resnet50_quantized_init_net.pb'
PREDICT_NET ='/home/ubuntu/segmentation_notebooks/resnet_quantized/resnet50_quantized_predict_net.pb'

# the predictor approach
# Read the contents of the input protobufs into local variables
with open(INIT_NET, "rb") as f:
    init_net = f.read()
with open(PREDICT_NET, "rb") as f:
    predict_net = f.read()

# Initialize the predictor from the input protobufs
p = workspace.Predictor(init_net, predict_net)

spew:

RuntimeError                              Traceback (most recent call last)
<ipython-input-7-5bf6f491eeeb> in <module>()
     10 
     11 # Initialize the predictor from the input protobufs
---> 12 p = workspace.Predictor(init_net, predict_net)

RuntimeError: [enforce fail at operator.cc:114] op. Cannot create operator of type 'Int8GivenTensorFill' on the device 'CPU'. Verify that implementation for the corresponding device exist. It might also happen if the binary is not linked with the operator implementation code. If Python frontend is used it might happen if dyndep.InitOpsLibrary call is missing. Operator def: output: "gpu_0/conv1_w_0_int8" name: "" type: "Int8GivenTensorFill" arg { name: "shape" ints: 64 ints: 7 ints: 7 ints: 3 } arg { name: "values" s:

Here's what happens if I try to run it with the GPU:
code:

workspace.ResetWorkspace()
device_opts = core.DeviceOption(caffe2_pb2.CUDA,0)


init_def = caffe2_pb2.NetDef()
with open(INIT_NET, 'rb') as f:
    init_def.ParseFromString(f.read())
    init_def.device_option.CopyFrom(device_opts)
    workspace.RunNetOnce(init_def.SerializeToString())

net_def = caffe2_pb2.NetDef()
with open(PREDICT_NET, 'rb') as f:
    net_def.ParseFromString(f.read())
    net_def.device_option.CopyFrom(device_opts)
    workspace.CreateNet(net_def.SerializeToString())

name = net_def.name
out_name = net_def.external_output[-1];
in_name = net_def.external_input[0]

spew:

UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-8-29ddafde7509> in <module>()
      7     init_def.ParseFromString(f.read())
      8     init_def.device_option.CopyFrom(device_opts)
----> 9     workspace.RunNetOnce(init_def.SerializeToString())
     10 
     11 net_def = caffe2_pb2.NetDef()

/home/ubuntu/src/caffe2/build/caffe2/python/workspace.pyc in RunNetOnce(net)
    181         C.Workspace.current._last_failed_op_net_position,
    182         GetNetName(net),
--> 183         StringifyProto(net),
    184     )
    185 

/home/ubuntu/src/caffe2/build/caffe2/python/workspace.pyc in CallWithExceptionIntercept(func, op_id_fetcher, net_name, *args, **kwargs)
    168         op_id = op_id_fetcher()
    169         net_tracebacks = operator_tracebacks.get(net_name, None)
--> 170         print("Traceback for operator {} in network {}".format(op_id, net_name))
    171         if net_tracebacks and op_id in net_tracebacks:
    172             tb = net_tracebacks[op_id]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xaf in position 21: ordinal not in range(128)

System information:

AWS p3.2xlarge in the Python environment caffe2_p27

FWIW, I get a similar error using my own build from source of caffe2 running with python 3.6 on Ubuntu 16.04 and a GTX1080

[dzung-hoang]

You can find the fixed predict_net.pb file for resnet50_quantized model here.

[NathanSegerlind]

cool, thanks

[Roger8]

@ALL, May someone tell me the process of quantizing caffe model? I'll be very appreciated. Thanks.