[FEA] support of customized components in monai_bundle_inference_operator

[mingxin-zheng]

Is your feature request related to a problem? Please describe.

Currently MONAI bundle allows user to define their components (class) and supports the initiation of the class from the bundle template during inference. We can possibly extend monai_bundle_inference_operator.py to better support these customized components

[MMelQin]

Thanks @mingxin-zheng.

As the MONAI Bundle expands its capability, the App SDK MONAI Bundle (automated) Inference operator will need to follow suite in order to support more use cases (as of now, only pre-processing, post-processing, and inferer attributes, besides metadata.json, are supported).

I see this enhancement of low to medium difficulty, especially after the load and save image logic is separated out from pre and post processing respectively.

[mingxin-zheng]

@MMelQin Thanks for the suggestion and evaluation. I will submit a PR for your review.

By looking at the code, I am confused by the way to get MONAI bundle config and files. MonaiBundleInferenceOperator expects the metadata.json in the extra folder in the zipped bundle package (TorchScript style?) https://github.com/Project-MONAI/monai-deploy-app-sdk/blob/6cdc09d6f1283e8e424bfe9492e5666352ebdd6f/monai/deploy/operators/monai_bundle_inference_operator.py#L94

But the MONAI bundles are in a different format (example).

Is there an example how MONAI deploy can deploy the current MONAI bundle in the model zoo?

[MMelQin]

@mingxin-zheng MONAI Bundle IS a TorchScript with extra config data in JSON.

The example application using MONAI Bundle is the Spleen Seg, and the model is from Model Zoo Spleen Bundle. I have notice the bundle itself has changed over time, but the format remains the same.

[mingxin-zheng]

@MMelQin Thanks for the confirmation. I started with the assumption that bundle isn't always TorchScript. By looking at the storage, it seems most has .ts in the model folder but there are a few that did not follow. For example: https://github.com/Project-MONAI/model-zoo/releases/download/hosting_storage_v1/renalStructures_UNEST_segmentation_v0.2.0.zip

Maybe we make deploy to adapt both (torchscript and non-torchscript)? @MMelQin @Nic-Ma

[MMelQin]

@MMelQin Thanks for the confirmation. I started with the assumption that bundle isn't always TorchScript. By looking at the storage, it seems most has .ts in the model folder but there are a few that did not follow. For example: https://github.com/Project-MONAI/model-zoo/releases/download/hosting_storage_v1/renalStructures_UNEST_segmentation_v0.2.0.zip

Maybe we make deploy to adapt both (torchscript and non-torchscript)? @MMelQin @Nic-Ma

That would be too arbitrary and was the exact reason why we picked the common and standard model archive format.

[mingxin-zheng]

Okay I see. I downloaded all the bundles and check which model actually HAS model.ts file. Here is a snapshot @MMelQin @Nic-Ma

• [x] brats_mri_segmentation_v0.3.0.zip
• [x] endoscopic_inbody_classification_v0.2.0.zip
• [x] endoscopic_tool_segmentation_v0.2.0.zip
• [x] lung_nodule_ct_detection_v0.3.0.zip
• [ ] mednist_gan_v0.4.0.zip
• [x] pancreas_ct_dints_segmentation_v0.3.0.zip
• [x] pathology_tumor_detection_v0.3.0.zip
• [x] prostate_mri_anatomy_v0.3.0.zip
• [ ] renalStructures_UNEST_segmentation_v0.2.0.zip
• [x] spleen_ct_segmentation_v0.3.2.zip
• [x] spleen_deepedit_annotation_v0.3.0.zip
• [ ] swin_unetr_btcv_segmentation_v0.3.0.zip
• [ ] valve_landmarks_v0.4.0.zip
• [x] ventricular_short_axis_3label_v0.3.0.zip
• [ ] wholeBrainSeg_Large_UNEST_segmentation_v0.2.0.zip

I may close this ticket as the issue sounds different that how it was first framed...

[dbericat]

Expand support for more Bundles with additional attributes or components.

Some of them are custom and already supported for Core and Label.

Today we only support pre-transform, inference and post-transform.