Error to convert TensorFlow trained model using Inference Engine.

[Bahramudin]

I trying to convert the model which was trained by using a pre-trained of the TensorFlow Object Detection API (faster_rcnn_inception_v2_coco) using Inference Engine.
In the documentation of the OpenVINO™ Toolkit there shows how to convert, when I tried, for example, the command below:
python3 mo_tf.py --input_model <INPUT_MODEL>.pb
Or
mo_tf.py --saved_model_dir <SAVED_MODEL_DIRECTORY>
Or
mo_tf.py --input_model <INFERENCE_GRAPH>.pb --input_checkpoint <INPUT_CHECKPOINT>
These all throw the error below:

Model Optimizer arguments:
Common parameters:
- Path to the Input Model: None
- Path for generated IR: C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer.
- IR output name: saved_model
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: Not specified, inherited from the model
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: False
- Reverse input channels: False
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Offload unsupported operations: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: None
- Operations to offload: None
- Patterns to offload: None
- Use the config file: None
Model Optimizer version: 1.5.12.49d067a0
[ FRAMEWORK ERROR ] Cannot load input model: No op named GatherV2 in defined operations.

But when I run the following command:
<INSTALL_DIR>/deployment_tools/model_optimizer/mo_tf.py --input_model=C:\workspace\output/frozen_inference_graph.pb --tensorflow_use_custom_operations_config <INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf/ssd_v2_support.json --tensorflow_object_detection_api_pipeline_config C:\workspace\pipeline.config --reverse_input_channels

Then it throws this error:

[ WARNING ] Model Optimizer removes pre-processing block of the model which resizes image keeping aspect ratio. The Inference Engine does not support dynamic image size so the Intermediate Representation file is generated with the input image size of a fixed size.
Specify the "--input_shape" command line parameter to override the default shape which is equal to (600, 600).
The Preprocessor block has been removed. Only nodes performing mean value subtraction and scaling (if applicable) are kept.
The graph output nodes "num_detections", "detection_boxes", "detection_classes", "detection_scores" have been replaced with a single layer of type "Detection Output". Refer to IR catalogue in the documentation for information about this layer.
[ ERROR ] Cannot infer shapes or values for node "ToFloat_3".
[ ERROR ] NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: ToFloat_3 = CastDstT=DT_FLOAT, SrcT=DT_UINT8, Truncate=false, _device="/job:localhost/replica:0/task:0/cpu:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: ToFloat_3 = CastDstT=DT_FLOAT, SrcT=DT_UINT8, Truncate=false, _device="/job:localhost/replica:0/task:0/cpu:0"]]

Caused by op 'ToFloat_3', defined at:
File "C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo_tf.py", line 31, in
sys.exit(main(get_tf_cli_parser(), 'tf'))
File "C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo\main.py", line 325, in main
return driver(argv)
File "C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo\main.py", line 267, in driver
mean_scale_values=mean_scale)
File "C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo\pipeline\tf.py", line 256, in tf2nx
partial_infer(graph)
File "C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo\middle\passes\infer.py", line 153, in partial_infer
node.infer(node)
File "C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo\front\tf\partial_infer\tf.py", line 60, in tf_native_tf_node_infer
tf_subgraph_infer(tmp_node)
File "C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo\front\tf\partial_infer\tf.py", line 135, in tf_subgraph_infer
all_constants, output_tensors = get_subgraph_output_tensors(node)
File "C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo\front\tf\partial_infer\tf.py", line 115, in get_subgraph_output_tensors
tf.import_graph_def(graph_def, name='')
File "C:\Users\bahra\Anaconda3\lib\site-packages\tensorflow\python\framework\importer.py", line 311, in import_graph_def
op_def=op_def)
File "C:\Users\bahra\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "C:\Users\bahra\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1269, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: ToFloat_3 = CastDstT=DT_FLOAT, SrcT=DT_UINT8, Truncate=false, _device="/job:localhost/replica:0/task:0/cpu:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: ToFloat_3 = CastDstT=DT_FLOAT, SrcT=DT_UINT8, Truncate=false, _device="/job:localhost/replica:0/task:0/cpu:0"]]

[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <function tf_native_tf_node_infer at 0x00000222F0BB4378>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ] Stopped shape/value propagation at "ToFloat_3" node.
For more information please refer to Model Optimizer FAQ (<INSTALL_DIR>/deployment_tools/documentation/docs/MO_FAQ.html), question #38.

So I have installed OpenVINO™ Toolkit correctly, I have test everything goes well, and detected successfully using C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools

I am uding OpenVINO ltest version (R5)

Note: during running the commands above, it throws error that requires TensorFlow 1.2, but in my machine I have installed 1.12, so I uninstalled and re installed the 1.2 version. If it is possible, please also support TF version 1.12 (the latest version) so there wont be need for the users to use OpenVINO to uninstall their TF.

Thanks!

[lazarevevgeny]

@Bahramudin the Model Optimizer needs TensorFlow >= 1.2, not exactly 1.2. The error that is shown in the your second attempt is because of too old version of the TensorFlow, so, please, install 1.12 again and conversion should work.

[Bahramudin]

@lazarevevgeny I reinstalled TF 1.12, now when I run this command:
mo_tf.py --input_model <INFERENCE_GRAPH>.pb --input_checkpoint <INPUT_CHECKPOINT>
it throws this error:

[ FRAMEWORK ERROR ] Cannot load input model: Unable to open table file C:\tf\models\research\object_detection\inference_graph\checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

and when to run threse commands:
mo_tf.py --saved_model_dir <SAVED_MODEL_DIRECTORY>
python3 mo_tf.py --input_model <INPUT_MODEL>.p
it throws this error:

C:\Intel\computer_vision_sdk_2018.5.445\deployment_tools\model_optimizer\mo\ops\slice.py:111: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result.
value = value[slice_idx]
[ ERROR ] Shape [-1 -1 -1 3] is not fully defined for output 0 of "image_tensor". Use --input_shape with positive integers to override model input shapes.
[ ERROR ] Cannot infer shapes or values for node "image_tensor".
[ ERROR ] Not all output shapes were inferred or fully defined for node "image_tensor".
For more information please refer to Model Optimizer FAQ (<INSTALL_DIR>/deployment_tools/documentation/docs/MO_FAQ.html), question #40.
[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <function tf_placeholder_ext.. at 0x000001DBD7413EA0>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ] Stopped shape/value propagation at "image_tensor" node.
For more information please refer to Model Optimizer FAQ (<INSTALL_DIR>/deployment_tools/documentation/docs/MO_FAQ.html), question #38.

What can be the reason!?

[lazarevevgeny]

You should use command line parameters as specified in the documentation: https://docs.openvinotoolkit.org/2018_R5/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Object_Detection_API_Models.html

[Bahramudin]

@lazarevevgeny Thanks a lot! After a lot of trying, uninstalling and reinstalling, finally successfully created .bin and .xml files. BTW I want to ask, after converting the model, then the speed was the same as before converting, I am using faster_rcnn_inception_v2, so is there anything I did forget?
I have done these steps:

• Creating data-set

• Train the data-set by using one of the pre-trained models of Object Detection API (in my case faster_rcnn_inception_v2)

• Export inference_graph

• Then converting to .bin and .xml (IR)

Are these steps enough? Or there can be something other optimization operation to do?

Very appreciate!

[lazarevevgeny]

@Bahramudin what do you mean when say "the speed was the same as before converting"? Did you compare performance of TF on CPU and converted the model on CPU?

[Bahramudin]

@lazarevevgeny Yes, when the model train finished, then I run it using CPU with TF, and then I used this wiki to use the model in OpenCV application by using cv::dnn::Net net = cv::dnn::readNetFromTensorflow(pb, pbtxt);, the speed remains the same as TF, and then converted the model to IR, and again used in OpenCV this time bu using cv::dnn::Net::readFromModelOptimizer(xml, bin); again the speed was the same as it was running in TF application. There was no implicit difference between these three runs.

So I asked the steps how did I produce the model, I am not sure if I forgot some step to do?!

[lazarevevgeny]

@Bahramudin could you try to run the IR with "object_detection_sample_ssd" from the samples directory and measure the performance?

@dkurt could you take a look at the previous comment from @Bahramudin ?

[dkurt]

@Bahramudin, Please provide reproducer of how you measure efficiency

• With TensorFlow python script
• With OpenCV and origin model
• With OpenCV and IR model

You may use samples from wiki as a reference.

[agileselenium]

@lazarevevgeny Thanks a lot! After a lot of trying, uninstalling and reinstalling, finally successfully created .bin and .xml files. BTW I want to ask, after converting the model, then the speed was the same as before converting, I am using faster_rcnn_inception_v2, so is there anything I did forget?
I have done these steps:

• Creating data-set
• Train the data-set by using one of the pre-trained models of Object Detection API (in my case faster_rcnn_inception_v2)
• Export inference_graph
• Then converting to .bin and .xml (IR)

Are these steps enough? Or there can be something other optimization operation to do?

Very appreciate!

Can you please provide what is the command to generate the .bin and .xml using with the .pb file?

[s0r2637]

@agileselenium: Follow the command below. Hope this helps. I should warn you that using the frozen.pb and pbtxt format runs faster than the .xml and .bin format that you are trying to create.
sudo python3 mo_tf.py -m ~/INCISIVE/ssd_mobilenet_v1_coco_2018_01_28/frozen_inference_graph.pb -o ~/INCISIVE/ssd_mobilenet_v1_coco_2018_01_28/ --tensorflow_use_custom_operations_config ./extensions/front/tf/ssd_support.json --tensorflow_object_detection_api_pipeline_config ~/INCISIVE/ssd_mobilenet_v1_coco_2018_01_28/pipeline.config --data_type=FP32

[dlliu0422]

How to fix this problem ? I have the same problem.

@lazarevevgeny Thanks a lot! After a lot of trying, uninstalling and reinstalling, finally successfully created .bin and .xml files. BTW I want to ask, after converting the model, then the speed was the same as before converting, I am using faster_rcnn_inception_v2, so is there anything I did forget?
I have done these steps:

Creating data-set

Train the data-set by using one of the pre-trained models of Object Detection API (in my case faster_rcnn_inception_v2)

Export inference_graph

Then converting to .bin and .xml (IR)

Are these steps enough? Or there can be something other optimization operation to do?

Very appreciate!

[lazarevevgeny]

@dlliu0422 There are nothing to do. I close the ticket as it seems to be resolved.