monai-deploy-app-sdk icon indicating copy to clipboard operation
monai-deploy-app-sdk copied to clipboard

The App SDK domain specific Image class needs to be improved as it lacks consistency of the image array it serves out

Open MMelQin opened this issue 4 years ago • 0 comments

Is your feature request related to a problem? Please describe. There is a fundamental issue with the Image class as the it lacks precise semantics of the dims of its encapsulated pixel image array, i.e. for a 3D image, which DIM is meant to be the axial/slice direction in DICOM speaking, first DIM to represent the instance, or the last DIM? What happens if some code creates the Image with a pixel array that has the instance as the first DIM.

Anyway, we know the issue with this first version of Image. I had to pay close attention to the what the original image type is, and how and what image array is used when instantiating Image, e.g. DICOM to Volume get a image pixel array with depth dim (slice wise) first, but I had to change that code to swap the depth to last. There is also the swapping of width and height, to as to ensure the shape out of this image.asnumpy() would the same as those from ITK or Nibabel.

This PR may resolve the symptom somewhat, but it is at the mercy of the encapsulated image pixel array. I will ensure an issue is created to address the Image class; I don't think it is usable as intended, as it now simply provides an in-memory storage to the pertinent image array.

Describe the solution you'd like Need to tighten the instantiation of the Image class Provide consistency over access method to get the encapsulated image array as Numpy array, similar to ITK and NiBabel

Describe alternatives you've considered Well, treating the Image simply as an in-mem storage, meaning the creator and consumer of the Image object know exactly the shape of the encapsulated image array. Additional context Built-in operator, PNGConverterOperator has already fall victim of Image, as the operator code assumes the in-mem Image object as the depth as the first dim, however, when used with other operators, image.asnumpy() simply serves out the array that was used to instantiate, and has the depth as the 3rd dim.

MMelQin avatar Oct 09 '21 23:10 MMelQin