Why the input of int64 automatically converted to int32 when the model is loaded?

[LinGeLin]

I work on an extension. The input must be int64. But when I debug it, I find that the input will be converted to int32. Is there any special mechanism to control it? I have observed that the validate_and_infer_types function is called many times. May I ask where in ov will this function of the extension plugin be called at runtime? And from the log I printed, the first few times the input is still int64, but later it becomes int32, which is very confusing.

version 2024.0.0

[ilya-lavrenov]

Please, create an issue instead of discussion.

[LinGeLin]

Is this a bug?

[LinGeLin]

created.
#24072

[LinGeLin]

int64 input is not actually supported? Will it convert to int32?

[LinGeLin]

Can I remove this line of code and recompile openvino to ensure that int64 is not converted to int32?

[ilya-lavrenov]

yes. You can either change CPU plugin code or assume that i32 is coming to your operation.

Currently, OpenVINO CPU plugin changes the precisions for better performance, but it does not assume that precisions for custom operations should be preserved. It's a bug and I've assigned people from CPU team to have a look at it

[LinGeLin]

openvino/src/common/transformations/include/transformations/convert_precision.hpp

Line 43 in 34825fe

* List of operations that are supported by this transformations for i64 -> i32 conversion:

 * List of operations that are supported by this transformations for i64 -> i32 conversion:
 *     opset4::Parameter
 *     opset4::Convert
 *     opset4::ShapeOf
 *     opset4::Range
 *     opset3::NonMaxSuppression
 *     opset4::NonMaxSuppression
 *     opset4::TopK
 *     opset4::NonZero
 *     opset4::Bucketize

In my usage scenario, all INT64s before the custom operator cannot be converted. But I've noticed that some op are supported by the i64 - > i32 conversion. Is it that even if I remove the code which setting conver precision, if I use these operators, the input to int64 will be converted to int32 ?

[ilya-lavrenov]

CC @nshchego

[LinGeLin]

My program is now running properly and I have commented out the i64->i32 code in transformation_pipeline.cpp. The next step is to verify the performance of the existing production model. thank u