Define python level abstraction for a Bundle

[Nic-Ma]

Is your feature request related to a problem? Please describe. Currently, we developed many bundles in the model-zoo repo and developed the ConfigParser to load configs. But actually we don't have a clear definition about what a Bundle is in the python level. It would be nice to define the Bundle class to represent a bundle and enable users to follow & define their customized bundle structure, and provide the APIs for other applications to easily load and handle a bundle, like MONAI Label, MONAI FL, etc. Several draft steps to do:

• [x] Develop BundleWorkflow interface and implement config-based subclass, so bundle can be implemented with JSON/YAML config or python code, just need to follow the interface.
• [x] How to expose the CLI entry and args for customized BYOCBundleWorkflow in bundle/scripts/.
• [x] Update FL monai algo to use new workflow.
• [ ] #6720
• [x] Add tutorial to show how to develop a new pure python-based bundle.
• [ ] Develop Bundle interface for the whole bundle, mainly about folder structure, files, download, etc.

[Nic-Ma]

CC @wyli @SachidanandAlle for more discussion.

Thanks.

[vikashg]

Hi @Nic-Ma Thanks for starting this post. As I mentioned that we need to think of usability of bundles from 2 different angles

1. Ease of use from an end-user perspective
2. Ease of building a MONAI Bundle

From an end-user perspective, if the monai model weights should be available from the python script (as they are for pytorch and keras) without explicitly downloading the model. For example

import torchvision
from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
from torchvision.models.detection.mask_rcnn import MaskRCNNPredictor

or for keras

base_model = keras.applications.Xception(
    weights="imagenet",  # Load weights pre-trained on ImageNet.
    input_shape=(150, 150, 3),
    include_top=False,
)  # Do not include the ImageNet classifier at the top.

# Freeze the base_model
base_model.trainable = False

It will be nice to have similar interface for MONAI models. As far as I understand and used, we can create a model architecture in the similar way in MONAI, but they are not populated with model weights. We looked at loading torchvision models and fine-tuning their model weights in this pull request last year with @wyli.

So maybe if we can reach a place where we can load the model weights as follows

from monai.model_zoo import brats_tumor_segmentation_model
model = brats_tumor_segmentation_model.load_weights(dataset="Brats_challenge_2020")

At this point, the user might want to query the model I/O for the purposes of fine-tuning. A user should be able to call something like

print("Model Info")
print(model.info)
print("Model I/O Config")
print(model.config.io)

which should produce an output

Model Info
Model Name: Brats Tumor Segmentation Model
Dataset model is trained on: Brats Challenge 20
Last updated date: 3 July 2021
Author: xxxx xxxx

Model I/O Config
Input Shape: [3, 256, 256, 256]
Intensity Range for input: [0, 1]

Output Labels: [4, 256, 256, 256] 
Produces Binary label for each class.

The user should be able to peel the last layer and attach a different layer for fine-tuning purposes.

[surajpaib]

Hi Team,

I would like to add my 2 cents to the issue based on what @vikashg proposed.

We primarily use MONAI for our config parsing (through ConfigParser), network architecture, and transform requirements, coupled with Pytorch Lightning for training. It's been a great experience working with the MONAI library throughout! Our work focuses on primarily building self-supervised models for medical imaging and we have planned to ship our trained models with MONAI bundle and make it compatible with frameworks like MONAI label so these pre-trained models can be used by the larger community.

However, what we would really like is for users to have simplified access to a model-loading interface. The MONAI bundles on model zoo provide a very nice way to ship models for training and inference but I fear users could be a bit challenged by a new interface. Offering a standard torchvision-like interface, in addition to the existing interface, would be amazing. E.g:

from monai.networks.nets import densenet121
model = densenet121(pretrained="pathology_nuclei_classification") # pretrained / weights 

Where the model now loads pre-trained weights from the pathology_nuclei_classification bundle.

In terms of implementation, I spotted that there is already a load function to get models from bundles. https://github.com/Project-MONAI/MONAI/blob/5feb353030e0bb204e21e1de338cd81b5972bb8a/monai/bundle/scripts.py#L341

Could it be of interest to look at this feature for existing networks in the MONAI library? This way the weights could be loaded for inference (even finetuning, especially in our case as we provide self-supervised models) and users could be pointed toward the full bundle for reproducibility and more advanced use cases. So this would be sort of an easy "entry point" to existing MONAI bundle models.

[surajpaib]

Hi @wyli, do you think something like the above would be of interest?

[wyli]

yes, that's a good point, thanks for the comments @surajpaib, @KumoLiu is designing a new pythonic user experience https://docs.google.com/presentation/d/1z3rVJWX7LMcyTrCH-K9dFehBuEksFj0f8bobmJ-DJPA will let you know when a prototype is ready

[surajpaib]

Thank you very much for sharing this @wyli

This looks very promising @KumoLiu. Let me know if you would like any help with testing

[KumoLiu]

E.g: from monai.networks.nets import densenet121 model = densenet121(pretrained="pathology_nuclei_classification") # pretrained / weights Where the model now loads pre-trained weights from the pathology_nuclei_classification bundle.

Hi @surajpaib, given your comment above, we were able to support getting instantiated nets directly from a bundle in the 1.3 release, which can be used directly in the existing pipeline.

from monai.bundle import load

model = load("pathology_nuclei_classification", bundle_dir="/workspace/", return_state_dict=False)

Feel free to share the feedback.

[wyli]

fyi: feature requests for bundle that are not addressed by monai v1.3:

• [ ] https://github.com/Project-MONAI/tutorials/issues/1453
• [ ] https://github.com/Project-MONAI/tutorials/issues/1454
• [ ] https://github.com/Project-MONAI/MONAI/issues/6724
• [ ] https://github.com/Project-MONAI/MONAILabel/issues/1490
• [ ] https://github.com/Project-MONAI/MONAI/issues/6748
• [ ] https://github.com/Project-MONAI/MONAI/issues/6959
• [ ] https://github.com/Project-MONAI/MONAILabel/issues/1549