Use Unet as a net for feeding it to logistichazard

[mahootiha-maryam]

Hi Havard. I have some MRI images from liver. All of them contain tumors. I wanted to predict the survival functions from these images. for images there was one example with mnist dataset. I wanted to use Unet as a net of logistichazard. But first I wanted to use Unet so the model just see the tumor parts for predicting survival functions. I wanted to ask you , is there any way that first I use Unet with my MRI images and my MRI ground truth to extract the features of tumor parts then use the output for another network that contains fully connected layers and finally give them to logistic hazard? My goal is that survival model just look at the tumor parts to understand survival functions. Because when I use simple CNN as an input to logistichazard I do not get good results and C-index. In addition I do not have a lot of dataset. Do you have any suggestion to me and if yes can you explain about the technical parts and coding? For example how can I give two labels to model one the ground truth of images and the other the survival times+events.

[havakv]

First of all, how large is your data set? If it's small, you might be struggling to fit any deeper nets in general, and you might need a net that is pretrain on some similar data.

If I understand you correctly, you want to first fit a classifier (tumor vs not tumor) and the use that net to create survival functions. The simplest way to do that is to first fit the classifier, and then just change the top layer to your desired survival output layer and fit it to the survival target. However, at this point it's very likely that you're overfitting.

This approach is probably simpler that trying to optimize both the classification task and the survival task simultaneously. However, if you want that, you need a net with two output (one for each task), and a loss function that takes both output and a weighted sum of the classification loss and survival loss.

[mahootiha-maryam]

I do not have a large dataset. It contains around 50, 3d images.

Not a classifier. I try to train a segmentation model to get the features of the segmented image from the last layer of the Convolutional neural network. It is like an input to the other network that we can give to cox ph or logistic hazard. I think with this method we give the features to survival model that are related to tumor parts. But I am not sure whether this method works or not.

About the second way, how is it possible to have them simultaneously? Because the output of the segmentation part is the input of the survival part. Can you please explain a bit more?

[havakv]

I'm not very optimistic about fitting a complicated survival model to 50 training points. You really can't expect the survival model to learn much. So maybe this is not the right approach? You'd have a hard time fitting even a simple survival model to so little data. And to train deep neural networks you would need a lot more data than that right? And even if you were able to train it, I'm not sure how you should be able to evaluate your results. If you evaluate on the training set, you can't be sure you haven't just overfitted.

In my mind, fitting a network with the two loss simultaneously is a step in the wrong direction, as that will just complicate things further