Example on Colab not segmenting the tumor

[fepegar]

Hi, and thanks for sharing your work! I have run the notebook you shared on PyTorch Hub and added one cell:

import scipy.ndimage as ndi
import matplotlib.pyplot as plt

pred = output[0, 0].cpu().numpy()
pred_bin = pred > 0.5
pred -= pred.min()
pred /= pred.max()
pred *= 255;
borders = ndi.binary_dilation(pred_bin) ^ pred_bin
input_array = np.array(input_image).copy()
input_array[borders] = input_array.max()

fig, axes = plt.subplots(1, 3, figsize=(12, 8))
axes[0].imshow(input_image)
axes[1].imshow(pred)
axes[2].imshow(input_array)

The segmentation is very inaccurate. Am I doing something wrong?

[mateuszbuda]

Hi @fepegar, thanks for trying the model. The notebook on PyTorch Hub runs inference on one slice. This can be done, but the model was trained on images normalized per MRI volume.

[fepegar]

Thanks. I think that it's detrimental for you and for PyTorch that the provided example is not representative of the original model's performance. Could you update the example with a more appropriate image or with the preprocessing needed for this specific slice? If not? Could you please do that here?

[mateuszbuda]

The problem is that it would require loading the entire 4D MRI volume in the code sample which is run as a test every time there is a PR to pytorch/hub repository. Another option is that I hardcode the mean and standard deviation computed on a volume for the example slice. However, I wanted to show that you have to normalize the input. The fact that you have to normalize the input per volume is described at the beginning of the Example section: https://github.com/pytorch/hub/blob/master/mateuszbuda_brain-segmentation-pytorch_unet.md#example There is also a link to the dataset that contains complete volumes. Anyway, I will compute the mean and std that give expected predictions.

[fepegar]

The problem is that it would require loading the entire 4D MRI volume in the code sample which is run as a test every time there is a PR to pytorch/hub repository.

In my first suggestion (which I just edited, because there was a word missing), I meant that you could replace the current link with one to an already preprocessed slice of that 4D image.

Another option is that I hardcode the mean and standard deviation computed on a volume for the example slice.

This is what I meant by "the preprocessing needed for this specific slice".

The fact that you have to normalize the input per volume is described at the beginning of the Example section: https://github.com/pytorch/hub/blob/master/mateuszbuda_brain-segmentation-pytorch_unet.md#example There is also a link to the dataset that contains complete volumes.

Sure, but I think that it would help you and the users if the example was self-contained, especially if you're explicitly using a sample image for it.

Let me know if I can help.

[fepegar]

For reference, here's a Colab notebook in which I manage to run the segmentation (using TorchIO): https://colab.research.google.com/drive/1XprcAZ59dAYBGFjawgb7h9mx0Y9N6pTj?usp=sharing

I assumed a spacing of 1 x 1 x 9 mm and LPS orientation.

Results look like this:

[fepegar]

By the way, the image I used, assuming 32 bits per voxel, takes 18 MB in memory. The frequency of PRs on pytorch/hub doesn't seem very high, so I think it would be fine to upload the actual 4D image, ideally in a format that supports 4D images such as NIfTI or NRRD. You could use TorchIO to convert the images to any of those formats.

[kruthikakr]

The problem is that it would require loading the entire 4D MRI volume in the code sample which is run as a test every time there is a PR to pytorch/hub repository. Another option is that I hardcode the mean and standard deviation computed on a volume for the example slice. However, I wanted to show that you have to normalize the input. The fact that you have to normalize the input per volume is described at the beginning of the Example section: https://github.com/pytorch/hub/blob/master/mateuszbuda_brain-segmentation-pytorch_unet.md#example There is also a link to the dataset that contains complete volumes. Anyway, I will compute the mean and std that give expected predictions.

If i need to upload a 4D data test image and get the segmentation on that ? How to do it ?

[kruthikakr]

Hi, and thanks for sharing your work! I have run the notebook you shared on PyTorch Hub and added one cell:

import scipy.ndimage as ndi
import matplotlib.pyplot as plt

pred = output[0, 0].cpu().numpy()
pred_bin = pred > 0.5
pred -= pred.min()
pred /= pred.max()
pred *= 255;
borders = ndi.binary_dilation(pred_bin) ^ pred_bin
input_array = np.array(input_image).copy()
input_array[borders] = input_array.max()

fig, axes = plt.subplots(1, 3, figsize=(12, 8))
axes[0].imshow(input_image)
axes[1].imshow(pred)
axes[2].imshow(input_array)

The segmentation is very inaccurate. Am I doing something wrong?

@fepegar I tried this on few samples, it works great . But can the output be displayed as something like this using torchio ? can you please give inputs for this ?

ie overlapping the segmented part with the Brain MRI image

[fepegar]

Yes, you could potentially use TorchIO for that visualization. But it would probably be easier (or more powerful) to use SimpleITK and then matplotlib.

[AK391]

Hi, and thanks for sharing your work! I have run the notebook you shared on PyTorch Hub and added one cell:

import scipy.ndimage as ndi
import matplotlib.pyplot as plt

pred = output[0, 0].cpu().numpy()
pred_bin = pred > 0.5
pred -= pred.min()
pred /= pred.max()
pred *= 255;
borders = ndi.binary_dilation(pred_bin) ^ pred_bin
input_array = np.array(input_image).copy()
input_array[borders] = input_array.max()

fig, axes = plt.subplots(1, 3, figsize=(12, 8))
axes[0].imshow(input_image)
axes[1].imshow(pred)
axes[2].imshow(input_array)

The segmentation is very inaccurate. Am I doing something wrong?

@fepegar I tried this on few samples, it works great . But can the output be displayed as something like this using torchio ? can you please give inputs for this ?

ie overlapping the segmented part with the Brain MRI image

could you please share a colab notebook for this result @kruthikakr @fepegar

[kruthikakr]

There is no co lab notebook done for this . As said i just used the ITK only

[AK391]

@kruthikakr can you share your notebook to get the above result?

[chamecall]

I've made the appropriate normalization. Check the colab.

TCGA_CS_4944_20010208.zip