Packaging C++ functions in MONAI

[masadcv]

There are quite a few C++ functions currently in MONAI that are quite useful in a number of applications. However, all C++ code requires compilation before they can be used (e.g. through JIT compilation or using BUILD_MONAI=1).

In order to use such C++ functions as dependency in other projects, these may need to be packaged as pre-compile binaries as we have already seen a number of issues with dependencies that required compilation (e.g. SImpleCRF dependency issues in MONAI Label see for example: https://github.com/Project-MONAI/MONAILabel/issues/605 and https://github.com/Project-MONAI/MONAILabel/issues/719).

There are of course discussions about moving some useful dependencies for MONAI Label scribbles into MONAI, so they can be maintained properly through community efforts (see for example discussion here: https://github.com/Project-MONAI/MONAILabel/pull/731 and here: https://github.com/Project-MONAI/MONAI/issues/1601)

It may be relevant to address these packaging issue for C++ functionality so that these can be used as dependency in other projects.

Relevant pointers:

• Torch extension builder for CI workflows: https://github.com/charliebudd/torch-extension-builder
• torchmaxflow wrapper for maxflow that is relevant for scribbles functionality in MONAI Label: https://github.com/masadcv/torchmaxflow

cc: @tvercaut @charliebudd

[wyli]

Thanks for bringing this up. I think @charliebudd's solution https://github.com/charliebudd/torch-extension-builder is nice, perhaps we should experiment with it and leverage the project? cc @ericspod @rijobro @Nic-Ma.

But I think in the long term monai's release process couldn't rely on a personal repo with only one maintainer... Is there a more sustainable way of releasing the extension-builder?

(there's a related request https://github.com/Project-MONAI/MONAI/issues/1462)

[tvercaut]

We can certainly transfer the torch-extension-builder repo to sit under the Project-MONAI organisation if there is appetite for MONAI to support it.

[charliebudd]

I'm happy for MONAI to fork the repo. I think we can all agree a good end goal would be to have this supported by PyTorch. Untill that time moving into MONAI would provide better visability. An alternate direction would be to form a new github org ("PyTorch extention community" for example) as this could help to bring in stake holders from other projects. Either way I have added an MIT license to the git repo. I hope to have more time to work on this in the near future.

[wyli]

Sure, we can discuss this at the dev meeting.

To test the feasibility of the precompiling I adapted the tool for monai. The pipeline is working fine now for linux + cuda based environments https://github.com/Project-MONAI/MONAI/actions/runs/2280287297 https://test.pypi.org/project/monai/0.8.1.post0.dev249/#files looks like the end-user's wheel file size will be less than 50Mb and it can support multiple versions of PyTorch/cuda libs.

[charliebudd]

Glad its working 😊

From the looks of it the debug information is being left in. Adding the -g0 flag to the c++ compiler in setup.py will strip this. The wheels for my current project are around 1MB despite including binaries for multiple compilations.

[masadcv]

@charliebudd would it be possible to also include Windows build in this? Or maybe this could be added as a future work?

[charliebudd]

I have spent some time looking into this, Unfortunetly my efforts hit a dead end when I found the github runner cannot run windows docker images. The work around is to install the packages inside the runner for each build. This will be a bit slower than downloading a pre-built docker image but I dont see another way. I'll try to spend some time on this next week.

[masadcv]

Many thanks! Yes, it certainly will be quite useful to have these precompiled for Windows - since most users we have seen have issue compiling source on Windows (see for example: https://github.com/Project-MONAI/MONAILabel/issues/363#issuecomment-1020445110)

[tvercaut]

For the time being, I would suggest to track generic packaging issues with torch-extension-builder in the @charliebudd's repo. E.g. for windows support: https://github.com/charliebudd/torch-extension-builder/issues/13

[wyli]

From the looks of it the debug information is being left in. Adding the -g0 flag to the c++ compiler in setup.py will strip this. The wheels for my current project are around 1MB despite including binaries for multiple compilations.

thanks, this might be an issue for monai, I've enabled cuda10.2/11.3+pytorch1.9/1.10/1.11+multiple archs and the package size becomes 130Mb with the recommended compiler flags. with more than 100Mb, pypi is not supported by default https://test.pypi.org/help/#file-size-limit.

[masadcv]

From the looks of it the debug information is being left in. Adding the -g0 flag to the c++ compiler in setup.py will strip this. The wheels for my current project are around 1MB despite including binaries for multiple compilations.

thanks, this might be an issue for monai, I've enabled cuda10.2/11.3+pytorch1.9/1.10/1.11+multiple archs and the package size becomes 130Mb with the recommended compiler flags. with more than 100Mb, pypi is not supported by default https://test.pypi.org/help/#file-size-limit.

is there any way to reduce this size to get to an acceptable range?

[charliebudd]

From the looks of it the debug information is being left in. Adding the -g0 flag to the c++ compiler in setup.py will strip this. The wheels for my current project are around 1MB despite including binaries for multiple compilations.

thanks, this might be an issue for monai, I've enabled cuda10.2/11.3+pytorch1.9/1.10/1.11+multiple archs and the package size becomes 130Mb with the recommended compiler flags. with more than 100Mb, pypi is not supported by default https://test.pypi.org/help/#file-size-limit.

is there any way to reduce this size to get to an acceptable range?

yes, as I said, the -g0 compiler flag will inform the compiler to strip the debug symbols. This is standard practice for release code and significantly decreases the build size.

[wyli]

I tried flags such as "cxx", ["-g0", "-Wl,-s", "-Os"], the wheel file size is still 126M, mainly within the package there are:

-rw-r--r--@  89M  _C.so.pt110cu102
-rw-r--r--@  78M  _C.so.pt110cu113
-rw-r--r--@  89M  _C.so.pt111cu102
-rw-r--r--@  78M  _C.so.pt111cu113
-rw-r--r--@  89M  _C.so.pt19cu102

[charliebudd]

These are very large file sizes, especially as each of these .so files is a single compilation of the extension. Checking with tool such as readelf/objdump would give clues as to why they are so large. Have we checked no normal compiled size of the MONAI extention?

[vikashg]

maybe out of scope for monai-core? @masadcv Please reopen if needed