HF models integration of MONAI networks

[surajpaib]

Is your feature request related to a problem? Please describe. #7833 Continuing the thread here.

HF has grown largely as a model hosting platform. Offering MONAI networks pretrained weights on HF would be very useful in sharing models with the larger community. I'm envisioning this way to be more a developer friendly way to load models for inference and fine-tuning in their own pipelines - outside the bundle ecosystem.

Describe the solution you'd like #7833 suggests a nice solution, however, further discussion was pending on the implementation specifics.

The Hub suffix for each model was the proposed option by @ericspod. Another way to offer this functionality could be through dynamic class creation using type so the user can just call a wrapper function and add HF capabilities to MONAI models.

Describe alternatives you've considered Creating an independent package with class composition for offering specific models https://github.com/project-lighter/lighter-zoo/blob/master/lighter_zoo/wrappers.py

[ericspod]

We should definitely come back to HF integration. There is also this related tutorial: https://github.com/Project-MONAI/tutorials/pull/1946

[surajpaib]

@ericspod Perfect. This should make the model loading part in the pipeline much more straightforward and offloaded to HF.

Happy to help as needed - this would be of great help to our project!

[ericspod]

Hi @surajpaib we've done some work for HF integration in #8403. I think this and the work on the model zoon to integrate with HF here goes a way towards addressing the requirement here. We've focussed on bundles being on HF, does that help address your requirements here?

[surajpaib]

Thanks for the update @ericspod

As you pointed out, this is focused on bundles on HF.

My use-case is for a MONAI network with pre-trained weights, rather than a bundle, to have the ability to pull from HF. This could be useful for other codebases using MONAI networks to ship weights in a more unified way.

[davanstrien]

Hi, I'm Daniel from Hugging Face. I'm mostly working on data topics but would be very excited about integrating MONAI and the Hub. Particularly, an easy way for people to share a trained model would be very powerful, in my opinion, i.e.,

mondai_model.push_to_hub("username/repo_id")

and

from monai.networks.nets import SwinUNETR

SwinUNETR.from_pre_trained("username/repo_id")

Let us know if there is anything we can do to support this integration from HF's side!

[ericspod]

Hi @davanstrien is this something we can implement now? The issue with #7833 that I'm conscious of now is the need to subclass a lot of networks and so make a lot of minor code changes. I realise this is the pattern established in the HF libraries, but is there an alternative pattern you would recommend?