onnx
onnx.shape_inference.infer_shapes creates a blank graph when passed a >2GB model
Bug Report
Is the issue related to model conversion?
No.
Describe the bug
The infer_shapes call creates a blank graph.
I export microsoft/Phi-3-mini-4k-instruct from the HF hub using torch.onnx.export with training mode on. Since the model is >2GB, it exports with external data. The Torch exported model loads in Netron fine and looks correct to me.
Upon loading the onnx model, I try the onnx.shape_inference.infer_shapes call, and attempt to save a model, which saves an empty ONNX file. When attempting to step through the nodes, no nodes are stepped through.
After exporting, I also try passing the Torch exported model to the ONNXRuntime generate_artifacts call, which uses infer_shapes here -- this call also seems to produce a blank graph since it causes issues later on.
System information
- OS Platform and Distribution (e.g. Linux Ubuntu 20.04): Windows
- ONNX version (e.g. 1.13): 1.15.0
- Python version: 3.9
- GCC/Compiler version (if compiling from source):
- CMake version:
- Protobuf version:
- Visual Studio version (if applicable):
Reproduction instructions
Github repo (readme is slightly out of date, but the pip requirements should still apply)
Run the torch onnx export script
Then run the generate artifacts script -- the infer shapes call is commented out and the onnx.save call is removed, but can be added back in.
Expected behavior
infer_shapes call should create a graph that is not blank.
Notes
I believe ONNXRuntime and ONNXRuntime Training are not compatible with ONNX >= 1.16.0.
It's most likely that the model proto provided is not serialized correctly because it's larger than 2GB. I talked with @carzh. We suggest loading the model without weights to run shape inference, or call infer_shapes_path. In the long run, we should implement shape inference in Python.
