Regression: eraseOutput: Assertion `outputs_[i]->uses().empty()` failed.

[BowenBao]

This is a regression from running onnxoptimizer. Calling optimize from onnx 1.7 with the same model and same passes did not throw any error.

Traceback (most recent call last):
  File "repro.py", line 21, in <module>
    onnxoptimizer.optimize(m, passes)
  File "onnxoptimizer/onnxoptimizer/__init__.py", line 50, in optimize
    optimized_model_str = C.optimize(model_str, passes)
RuntimeError: onnxoptimizer/third_party/onnx/onnx/common/ir.h:1291: eraseOutput: Assertion `outputs_[i]->uses().empty()` failed.

Script to repro:

import onnx

model_str = b'\x08\x06\x12\x07pytorch\x1a\x031.9:\xe5\x02\n\'\x12\x011"\x08Constant*\x18\n\x05value*\x0c\x10\x07J\x08\x05\x00\x00\x00\x00\x00\x00\x00\xa0\x01\x04\n \x12\x012"\x08Constant*\x11\n\x05value*\x05\x10\tJ\x01\x01\xa0\x01\x04\n\xd1\x01\n\x011\n\x012\n\x03x.1\x12\x013"\x04Loop*\xba\x01\n\x04body2\xae\x01\n\x1a\n\x04x.11\n\x03i.1\x12\x017\x1a\x05Add_0"\x03Add\n\x1c\n\x012\x12\x018\x1a\nIdentity_1"\x08Identity\x12\x11torch-jit-export1Z\r\n\x03i.1\x12\x06\n\x04\x08\x07\x12\x00Z\x0e\n\x04cond\x12\x06\n\x04\x08\t\x12\x00Z\x1a\n\x04x.11\x12\x12\n\x10\x08\x07\x12\x0c\n\x02\x08\x01\n\x02\x08\x02\n\x02\x08\x03b\x0b\n\x018\x12\x06\n\x04\x08\t\x12\x00b\x17\n\x017\x12\x12\n\x10\x08\x07\x12\x0c\n\x02\x08\x01\n\x02\x08\x02\n\x02\x08\x03\xa0\x01\x05\x12\x10torch-jit-exportZ\x19\n\x03x.1\x12\x12\n\x10\x08\x07\x12\x0c\n\x02\x08\x01\n\x02\x08\x02\n\x02\x08\x03b\x17\n\x013\x12\x12\n\x10\x08\x07\x12\x0c\n\x02\x08\x01\n\x02\x08\x02\n\x02\x08\x03B\x02\x10\t'
m = onnx.load_from_string(model_str)

import onnxoptimizer

passes =  ['fuse_consecutive_transposes',
            'eliminate_nop_transpose',
            'fuse_transpose_into_gemm',
            'lift_lexical_references',
            'split_predict']
from onnx import optimizer
onnx.optimizer.optimize(m, passes)  # pass
onnxoptimizer.optimize(m, passes)  # fail

[daquexian]

Thanks for your issue!

What are "split_predict" and "lift_lexical_references" used for? I thought they are legacy passes and only useful for the caffe2 backend. Please correct me if I am wrong.

[BowenBao]

Thanks for your issue!

What are "split_predict" and "lift_lexical_references" used for? I thought they are legacy passes and only useful for the caffe2 backend. Please correct me if I am wrong.

Hi @daquexian , yes this is a case from Caffe2 backend test. The background is that we are updating ONNX version under PyTorch, and was trying the onnxoptimizer module due to the dependency of Caffe2 on onnx.optimize. This error came up for these tests

TestCaffe2Backend_opset9::test_dynamic_loop
TestCaffe2Backend_opset9::test_loop
TestCaffe2Backend_opset9::test_nested_loops

[daquexian]

Is caffe2 backend still used? If needed I will take a look at it. Another choice is dropping the support for "lift_lexical_references" and "split_predict" passes if the only user -- caffe2 backend is deprecated.

[neginraoof]

Hi @daquexian Caffe2 backend is not updated recently, but the existing code and tests for older opsets are maintained as part of the repo. This issue looks like a bug in the split_init_and_predict logic where nodes that are not in the predic net get deleted.

This is the CI for updating ONNX (currently failing): https://github.com/pytorch/pytorch/pull/55889

Could you please take a look? Thanks

[neginraoof]

cc @gramalingam and @askhade FYI, this issue is currently blocking the use of onnx/optimizer for us in: https://github.com/pytorch/pytorch/pull/55889

[daquexian]

@BowenBao @neginraoof I'll take a look today tomorrow 😂

[neginraoof]

@daquexian Thanks for your help. This is currently needed for ONNX in PyTorch. Do you think we can help with anything here? I can prepare a PR for split pass.

[daquexian]

I can prepare a PR for split pass.

Could you please submit a pr if applicable? I'm quite busy these days. 🤣 I'll take a look at this problem but it will help a lot if you also submit a pr

[neginraoof]

@daquexian This is high priority for us as mentioned. Could you please take a look at the PR: https://github.com/onnx/optimizer/pull/41

Thanks.

[neginraoof]

I created a duplicate to check CI failures: https://github.com/onnx/optimizer/pull/42

[neginraoof]

@daquexian Also, would you please help us with the CI workflow issue?

[daquexian]

I have merged #42 and released v0.2.6. could you please try again with this latest version?