Feature Request: Optimization for large models

[DA-L3]

Hello,

I have a very large model (>2GB) which I would like to optimize using this library. Unfortunately I cannot shrink my model under 2GB. Therefore while using the optimization, I get the memory exceed error of protobuf onnx.ModelProto exceeds maximum protobuf size of 2GB. Is there a way to use_external_data or some other "tricks" like in the infer_shapes-case mentioned here https://github.com/onnx/onnx/blob/master/docs/PythonAPIOverview.md?

[daquexian]

Great! I'll support it soon

[sumo2017]

I also encounter this problem

[Ponyo1]

me too

[bigbenbird]

me too. Is anybody working on in ?

[JulesBelveze]

+1

[kenny-reyes]

I am trying converting a model whose size in .pkl is 985 MB, after converting with sklearn-onnx (convert was successful)

Connected to pydev debugger (build 213.7172.26)
[convert_sklearn] parse_sklearn_model
[convert_sklearn] convert_topology
[convert_operators] begin
[convert_operators] iteration 1 - n_vars=0 n_ops=3
[call_converter] call converter for 'SklearnScaler'.
[call_converter] call converter for 'SklearnRandomForestClassifier'.
[call_converter] call converter for 'SklearnZipMap'.
[convert_operators] end iter: 1 - n_vars=6
[convert_operators] iteration 2 - n_vars=6 n_ops=3
[convert_operators] end iter: 2 - n_vars=6
[convert_operators] end.
[_update_domain_version] +opset 0: name='', version=9
[_update_domain_version] +opset 1: name='ai.onnx.ml', version=1
[convert_sklearn] end

Process finished with exit code 1

but then trying to save the onnx model with:

onnx.save(onx, "model.onnx", save_as_external_data=True)

and response

Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm 2021.3\plugins\python\helpers\pydev\pydevd.py", line 1483, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2021.3\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Workspace/OnnxTest/pklconversion.py", line 126, in <module>
    onnx.save(onx, "bonus_model.onnx", save_as_external_data=True)
  File "C:\Users\....\AppData\Local\Programs\Python\Python37\lib\site-packages\onnx\__init__.py", line 202, in save_model
    s = _serialize(proto)
  File "C:\Users\....\AppData\Local\Programs\Python\Python37\lib\site-packages\onnx\__init__.py", line 71, in _serialize
    result = proto.SerializeToString()
ValueError: Message onnx.ModelProto exceeds maximum protobuf size of 2GB: 3354372734

I have been finding a fix for this, but I didn't find anything, the question is: Really isn't any solution and you only can't work with onnx with large models?

Thank you very much

[HSQ79815]

Onnx-ptimizer has supported large model(exceed 2G). You can install the latest version from source build.

[michaelroyzen]

@HSQ79815 could you link to a pull request please? I just tried with the latest onnx weekly build and got the same error when trying to export a large model with save_as_external_data=True.

[HSQ79815]

@michaelroyzen you could get more information fromm this pr . In the past, Large model(>2G) is loaded by onnx.load(..., load_external_data=True) and saved by onnx.save(..., save_as_external_data=True), while this functions are implemented byPython. We implement c++ load and save function that support large model.