openvino
openvino copied to clipboard
openvinotoolkit
[Bug]: Why the input of int64 automatically converted to int32 when the model is loaded?
OpenVINO Version
2024.0.0
Operating System
Ubuntu 20.04 (LTS)
Device used for inference
CPU
Framework
ONNX
Model used
TFRA
Issue description
I work on an extension. The input must be int64. But when I debug it, I find that the input will be converted to int32. Is there any special mechanism to control it? I have observed that the validate_and_infer_types function is called many times. May I ask where in ov will this function of the extension plugin be called at runtime? And from the log I printed, the first few times the input is still int64, but later it becomes int32, which is very confusing.
Step-by-step reproduction
No response
Relevant log output
No response
Issue submission checklist
- [X] I'm reporting an issue. It's not a question.
- [X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
- [X] There is reproducer code and related data files such as images, videos, models, etc.
Hello, Please check which data types the CPU plugin supports here. In particular, i64 is not supported. You can execute extension node in i64, but inputs/outputs will be converted anyway. There is no mechanism to force it. Thanks
Hello, Please check which data types the CPU plugin supports here. In particular, i64 is not supported. You can execute extension node in i64, but inputs/outputs will be converted anyway. There is no mechanism to force it. Thanks
What happens if I delete the conver precision for i64->i32?
Hello, Please check which data types the CPU plugin supports here. In particular, i64 is not supported. You can execute extension node in i64, but inputs/outputs will be converted anyway. There is no mechanism to force it. Thanks
What happens if I delete the conver precision for i64->i32?
Various issues may arise during the compilation and execution of the graph, depending on the model. Even if graph will be successfully compiled end inferred, the CPU's nodes will perform an internal conversion i64->i32.
Hello, Please check which data types the CPU plugin supports here. In particular, i64 is not supported. You can execute extension node in i64, but inputs/outputs will be converted anyway. There is no mechanism to force it. Thanks
What happens if I delete the conver precision for i64->i32?
Various issues may arise during the compilation and execution of the graph, depending on the model. Even if graph will be successfully compiled end inferred, the CPU's nodes will perform an internal conversion i64->i32.
I notice that you have some development branch as support i64, such as cpu/i64/proto, Do any of these branches meet my needs? How should I use it if supported? For example, how to configure parameters when starting tritonserver?
Yes, you can use this branch for i64. Just set enableNativeI64 to True here. Also take into account that it has quite old base (more than one year).
Closing this, I hope previous responses were sufficient to help you proceed. Feel free to reopen and ask additional questions related to this topic.
