score

[iamimage]

I am trying to reproduce the nice result in the paper with this code. What is the best dice score (on validation dataset or testing dataset) achieved by the current code? Is it close to the ones given by the paper?

[IAmSuyogJadhav]

I haven't yet tried the code on the BraTS 2018 dataset, mainly due to computational bottleneck. The author works at Nvidia and had access to a DGX-1. He has described it in the paper. If you happen to have good computational resources, you can give it a try. There are a couple of training specific things he did, like learning rate decay according to the epoch no., You can find them in the paper.

[iamimage]

I haven't yet tried the code on the BraTS 2018 dataset, mainly due to computational bottleneck. The author works at Nvidia and had access to a DGX-1. He has described it in the paper. If you happen to have good computational resources, you can give it a try. There are a couple of training specific things he did, like learning rate decay according to the epoch no., You can find them in the paper.

Thank you SuyogJadhav. I would have a try on our cluster. It could be interesting to see the score. Do you plan to share the data augmentation code as well? That might be also important.

[IAmSuyogJadhav]

I haven't written the data augmentation code yet. The author did standard normalization and nothing much anyways. Citing from the paper directly,

3.7 Data preprocessing and augmentation We normalize all input images to have zero mean and unit std (based on nonzero voxels only). We apply a random (per channel) intensity shift (−0.1..0.1 of image std) and scale (0.9..1.1) on input image channels. We also apply a random axis mirror flip (for all 3 axes) with a probability 0.5.

Should be easy to achieve in Python.

[BigSteve94]

Did someone already tried this code on Brats data and did it work?

[saisubramaniam147]

Did someone already tried this code on Brats data and did it work?

I am trying on BraTS 2017 dataset, but the dice coefficient is very low (0.0035). Anyone else having the same issue?

[saisubramaniam147]

I expanded the slice from 64 to 128 retaining other dimensions as same, the dice score improves.

Intuition is that the tumor is not captured in the early slices and should ideally be large around the middle portion. Comments?

[IAmSuyogJadhav]

You can try adjusting the weights of the loss terms (KL and L2). These seem to be playing pivotal role. Also, you may need to train for around 200-300 epochs for respectable dice loss (I had a discussion with @doc78, who has used the model. He told me this).

[doc78]

I confirm that the model works fine. To get high dice scores you will need to train the model for at least 100-200 epochs, depending on the dataset. I used weights 0.1 for both KL and L2 loss terms.

I trained the model on a dataset of 400 multimodal volumes for training and 85 volumes for testing (Medical Decathlon dataset), using crop size 160x176x112 to fit my GPU, no data augmentation, for 100 epochs (I trained for 300 epochs in this case but get overfitting after 100 epochs) and I got 0.91 dice score. However, I checked better on my Tensorboard logs during different trainings and I see I was able to get 0.7-0.8 dice scoring after just 15-20 epochs. So if after 20 epochs you still have low scores, there should be something wrong in the input or in the normalization of input data.

I also tested it on Brats 2019 Validation dataset and I get 0.89 WT dice score.

[Infinity0106]

Hi, I’m having trouble training and the dice coefficient never goes higher than .03 I’m using BRATS 2018 without any modifications of the data, and also how do you recommend using the training I have a generator and I multiply epochs * numbers of samples for training is that the right way to do it?

I attach the notebook that I changed just for testing and the dice coefficient never went higher than .016 and it was just with 4 subjets jupyter notebook

[ShixianLibai]

Since the image size is 160×192×128, how to predict a new patient data, the data size is 240240155

[bhagi369]

Did someone already tried this code on Brats data and did it work?

I am trying on BraTS 2017 dataset, but the dice coefficient is very low (0.0035). Anyone else having the same issue?

Hi, I am having same problem even more worse.I am getting zero dice score.

[sneh-debug]

Hi, I’m having trouble training and the dice coefficient never goes higher than .03 I’m using BRATS 2018 without any modifications of the data, and also how do you recommend using the training I have a generator and I multiply epochs * numbers of samples for training is that the right way to do it?

I attach the notebook that I changed just for testing and the dice coefficient never went higher than .016 and it was just with 4 subjets jupyter notebook

Try for more number of samples and epoch

[bhagi369]

I confirm that the model works fine. To get high dice scores you will need to train the model for at least 100-200 epochs, depending on the dataset. I used weights 0.1 for both KL and L2 loss terms.

I trained the model on a dataset of 400 multimodal volumes for training and 85 volumes for testing (Medical Decathlon dataset), using crop size 160x176x112 to fit my GPU, no data augmentation, for 100 epochs (I trained for 300 epochs in this case but get overfitting after 100 epochs) and I got 0.91 dice score. However, I checked better on my Tensorboard logs during different trainings and I see I was able to get 0.7-0.8 dice scoring after just 15-20 epochs. So if after 20 epochs you still have low scores, there should be something wrong in the input or in the normalization of input data.

I also tested it on Brats 2019 Validation dataset and I get 0.89 WT dice score.

hi , sneh can help me to increase my dice score and how you got 400 sample for training and 85 for testing sample.I am doing my final year project and I am new to this deep learning.can you please help me?

[bhagi369]

I confirm that the model works fine. To get high dice scores you will need to train the model for at least 100-200 epochs, depending on the dataset. I used weights 0.1 for both KL and L2 loss terms.

I trained the model on a dataset of 400 multimodal volumes for training and 85 volumes for testing (Medical Decathlon dataset), using crop size 160x176x112 to fit my GPU, no data augmentation, for 100 epochs (I trained for 300 epochs in this case but get overfitting after 100 epochs) and I got 0.91 dice score. However, I checked better on my Tensorboard logs during different trainings and I see I was able to get 0.7-0.8 dice scoring after just 15-20 epochs. So if after 20 epochs you still have low scores, there should be something wrong in the input or in the normalization of input data.

I also tested it on Brats 2019 Validation dataset and I get 0.89 WT dice score.

hi can you help me regard this dice score? please.

[srivathsapv]

Adding to @doc78 's point, I think clipping the input off negative values and scaling it between 0 and 1 helped me in solving the 0 dice score problem. However you still have to train it for at least 100 epochs

[mwndrea]

I confirm that the model works fine. To get high dice scores you will need to train the model for at least 100-200 epochs, depending on the dataset. I used weights 0.1 for both KL and L2 loss terms.

I trained the model on a dataset of 400 multimodal volumes for training and 85 volumes for testing (Medical Decathlon dataset), using crop size 160x176x112 to fit my GPU, no data augmentation, for 100 epochs (I trained for 300 epochs in this case but get overfitting after 100 epochs) and I got 0.91 dice score. However, I checked better on my Tensorboard logs during different trainings and I see I was able to get 0.7-0.8 dice scoring after just 15-20 epochs. So if after 20 epochs you still have low scores, there should be something wrong in the input or in the normalization of input data.

I also tested it on Brats 2019 Validation dataset and I get 0.89 WT dice score.

Hi @doc78

Just wondering how big a GPU was needed for the crop size that you used? Or if there are some model parallelisation that you managed to apply to use a relatively large crop size?

Thanks!

[doc78]

This model requires huge GPU power. The author used a V100 equipped with 32 GBytes of Graphic Memory and used a crop size of 160x192x128. In my training I cropped the volumes to the size of 160x176x112 and I was able to fit the model in my NVIDIA P6000 GPU with 24 GBytes of Graphic Memory.

[jayurbain]

doc78 > This model requires huge GPU power. The author used a V100 equipped with 32 GBytes of Graphic Memory and used a crop size of 160x192x128. In my training I cropped the volumes to the size of 160x176x112 and I was able to fit the model in my NVIDIA P6000 GPU with 24 GBytes of Graphic Memory.

I'm struggling getting this model to generate reasonable dice coefficients and would appreciate some feedback. The loss decreases for 15 epochs and just stops improving (scroll to end to see plot after training output) My dataset is BraTS 2018.

If I use a 0.5 threshold on the training output of any of the 3 output segmentation maps, all the values become 1. The output of the VAE is just a light gray blur.

Here are my versions of tf and keras:

tf.__version__ is 1.15.2-dlenv_tfe
tf.keras.__version__ is: 2.2.4-tf

Starting with a size of (4x155x240x240), I crop the images to (4,135, 170, 140). I then resize the cropped images to multiples of 16 that will fit within 16GB T4 nVidia GPU to get (4,112, 144, 112)

# 155x240x240
crop_tuple = ((21, 22), (40, 30), (50,50))
# (135, 170, 140)
resize_shape = (4, 112, 144, 112)
output_channels = 3

After cropping, resizing, and just using standard normalization, here is a sample of my input data (first row) and preprocessed data (second row) and individual segmentations (3rd row) on BraTS 2018.

.

I'm thinking this looks Ok.

This takes me from a size of (155, 240, 240) to (112, 144, 112). I tried @doc78's suggestion of 160x192x128, but I could not fit that in the memory of a 16GB nVidia T4 GPU.

I verified the model is using L2 and KL weights of 0.1 as @doc78 suggested.

Here is my training invocation. I realize, I should be training > 20 epochs, but this is a sanity check for the dice coefficient.

from keras.callbacks import ModelCheckpoint
model_weights_name = "brats2018_vnet_vae_weights_05122020.h5"

checkpoint = [
              ModelCheckpoint(model_weights_name, 
                              verbose=1, 
                              save_best_only=True, 
                              save_weights_only=True)
]

# Train the model, doing validation at the end of each epoch.
epochs = 20
history = model.fit(data, [labels, data], batch_size=1, epochs=epochs, 
                    callbacks=checkpoint, validation_split=0.15)
model.save_weights(model_weights_name)

Here is the training output which I'm troubling to comprehend:

WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

Train on 242 samples, validate on 43 samples
Epoch 1/20
242/242 [==============================] - 930s 4s/step - loss: 0.0446 - Dec_GT_Output_loss: -0.0153 - Dec_VAE_Output_loss: 0.0599 - Dec_GT_Output_dice_coefficient: 0.0153 - Dec_VAE_Output_dice_coefficient: 0.7881 - val_loss: 0.0195 - val_Dec_GT_Output_loss: -0.0080 - val_Dec_VAE_Output_loss: 0.0275 - val_Dec_GT_Output_dice_coefficient: 0.0080 - val_Dec_VAE_Output_dice_coefficient: 0.8797

Epoch 00001: val_loss improved from inf to 0.01947, saving model to brats2018_vnet_vae_weights_05122020.h5
Epoch 2/20
242/242 [==============================] - 888s 4s/step - loss: 0.0270 - Dec_GT_Output_loss: -0.0036 - Dec_VAE_Output_loss: 0.0306 - Dec_GT_Output_dice_coefficient: 0.0036 - Dec_VAE_Output_dice_coefficient: 0.8730 - val_loss: 0.0171 - val_Dec_GT_Output_loss: -0.0080 - val_Dec_VAE_Output_loss: 0.0251 - val_Dec_GT_Output_dice_coefficient: 0.0080 - val_Dec_VAE_Output_dice_coefficient: 0.9062

Epoch 00002: val_loss improved from 0.01947 to 0.01712, saving model to brats2018_vnet_vae_weights_05122020.h5
Epoch 3/20
242/242 [==============================] - 887s 4s/step - loss: 0.0259 - Dec_GT_Output_loss: -0.0036 - Dec_VAE_Output_loss: 0.0295 - Dec_GT_Output_dice_coefficient: 0.0036 - Dec_VAE_Output_dice_coefficient: 0.8790 - val_loss: 0.0173 - val_Dec_GT_Output_loss: -0.0080 - val_Dec_VAE_Output_loss: 0.0253 - val_Dec_GT_Output_dice_coefficient: 0.0080 - val_Dec_VAE_Output_dice_coefficient: 0.8953

Epoch 00003: val_loss did not improve from 0.01712
Epoch 4/20
242/242 [==============================] - 889s 4s/step - loss: 0.0248 - Dec_GT_Output_loss: -0.0036 - Dec_VAE_Output_loss: 0.0284 - Dec_GT_Output_dice_coefficient: 0.0036 - Dec_VAE_Output_dice_coefficient: 0.8838 - val_loss: 0.0152 - val_Dec_GT_Output_loss: -0.0080 - val_Dec_VAE_Output_loss: 0.0232 - val_Dec_GT_Output_dice_coefficient: 0.0080 - val_Dec_VAE_Output_dice_coefficient: 0.9177

Epoch 00004: val_loss improved from 0.01712 to 0.01523, saving model to brats2018_vnet_vae_weights_05122020.h5
Epoch 5/20
242/242 [==============================] - 894s 4s/step - loss: 0.0161 - Dec_GT_Output_loss: -0.0117 - Dec_VAE_Output_loss: 0.0278 - Dec_GT_Output_dice_coefficient: 0.0117 - Dec_VAE_Output_dice_coefficient: 0.8858 - val_loss: 0.0064 - val_Dec_GT_Output_loss: -0.0159 - val_Dec_VAE_Output_loss: 0.0223 - val_Dec_GT_Output_dice_coefficient: 0.0159 - val_Dec_VAE_Output_dice_coefficient: 0.9162

Epoch 00005: val_loss improved from 0.01523 to 0.00638, saving model to brats2018_vnet_vae_weights_05122020.h5
Epoch 6/20
242/242 [==============================] - 895s 4s/step - loss: 0.0100 - Dec_GT_Output_loss: -0.0161 - Dec_VAE_Output_loss: 0.0261 - Dec_GT_Output_dice_coefficient: 0.0161 - Dec_VAE_Output_dice_coefficient: 0.8910 - val_loss: 0.0062 - val_Dec_GT_Output_loss: -0.0158 - val_Dec_VAE_Output_loss: 0.0220 - val_Dec_GT_Output_dice_coefficient: 0.0158 - val_Dec_VAE_Output_dice_coefficient: 0.9123

Epoch 00006: val_loss improved from 0.00638 to 0.00622, saving model to brats2018_vnet_vae_weights_05122020.h5
Epoch 7/20
242/242 [==============================] - 895s 4s/step - loss: -0.0332 - Dec_GT_Output_loss: -0.0586 - Dec_VAE_Output_loss: 0.0254 - Dec_GT_Output_dice_coefficient: 0.0586 - Dec_VAE_Output_dice_coefficient: 0.8936 - val_loss: -0.1996 - val_Dec_GT_Output_loss: -0.2206 - val_Dec_VAE_Output_loss: 0.0210 - val_Dec_GT_Output_dice_coefficient: 0.2206 - val_Dec_VAE_Output_dice_coefficient: 0.9136

Epoch 00007: val_loss improved from 0.00622 to -0.19961, saving model to brats2018_vnet_vae_weights_05122020.h5
Epoch 8/20
242/242 [==============================] - 894s 4s/step - loss: -0.1943 - Dec_GT_Output_loss: -0.2190 - Dec_VAE_Output_loss: 0.0247 - Dec_GT_Output_dice_coefficient: 0.2190 - Dec_VAE_Output_dice_coefficient: 0.8958 - val_loss: -0.2342 - val_Dec_GT_Output_loss: -0.2544 - val_Dec_VAE_Output_loss: 0.0203 - val_Dec_GT_Output_dice_coefficient: 0.2544 - val_Dec_VAE_Output_dice_coefficient: 0.9268

Epoch 00008: val_loss improved from -0.19961 to -0.23417, saving model to brats2018_vnet_vae_weights_05122020.h5
Epoch 9/20
242/242 [==============================] - 894s 4s/step - loss: -0.2116 - Dec_GT_Output_loss: -0.2355 - Dec_VAE_Output_loss: 0.0239 - Dec_GT_Output_dice_coefficient: 0.2355 - Dec_VAE_Output_dice_coefficient: 0.8986 - val_loss: -0.1402 - val_Dec_GT_Output_loss: -0.1606 - val_Dec_VAE_Output_loss: 0.0203 - val_Dec_GT_Output_dice_coefficient: 0.1606 - val_Dec_VAE_Output_dice_coefficient: 0.9192

Epoch 00009: val_loss did not improve from -0.23417
Epoch 10/20
242/242 [==============================] - 894s 4s/step - loss: -0.2350 - Dec_GT_Output_loss: -0.2587 - Dec_VAE_Output_loss: 0.0237 - Dec_GT_Output_dice_coefficient: 0.2587 - Dec_VAE_Output_dice_coefficient: 0.8994 - val_loss: -0.2562 - val_Dec_GT_Output_loss: -0.2758 - val_Dec_VAE_Output_loss: 0.0197 - val_Dec_GT_Output_dice_coefficient: 0.2758 - val_Dec_VAE_Output_dice_coefficient: 0.9270

Epoch 00010: val_loss improved from -0.23417 to -0.25616, saving model to brats2018_vnet_vae_weights_05122020.h5
Epoch 11/20
242/242 [==============================] - 894s 4s/step - loss: -0.2469 - Dec_GT_Output_loss: -0.2701 - Dec_VAE_Output_loss: 0.0232 - Dec_GT_Output_dice_coefficient: 0.2701 - Dec_VAE_Output_dice_coefficient: 0.9012 - val_loss: -0.2454 - val_Dec_GT_Output_loss: -0.2644 - val_Dec_VAE_Output_loss: 0.0190 - val_Dec_GT_Output_dice_coefficient: 0.2644 - val_Dec_VAE_Output_dice_coefficient: 0.9220

Epoch 00011: val_loss did not improve from -0.25616
Epoch 12/20
242/242 [==============================] - 894s 4s/step - loss: -0.2476 - Dec_GT_Output_loss: -0.2702 - Dec_VAE_Output_loss: 0.0225 - Dec_GT_Output_dice_coefficient: 0.2702 - Dec_VAE_Output_dice_coefficient: 0.9035 - val_loss: -0.1973 - val_Dec_GT_Output_loss: -0.2175 - val_Dec_VAE_Output_loss: 0.0202 - val_Dec_GT_Output_dice_coefficient: 0.2175 - val_Dec_VAE_Output_dice_coefficient: 0.9187

Epoch 00014: val_loss did not improve from -0.25616
Epoch 15/20
242/242 [==============================] - 894s 4s/step - loss: -0.2608 - Dec_GT_Output_loss: -0.2831 - Dec_VAE_Output_loss: 0.0223 - Dec_GT_Output_dice_coefficient: 0.2831 - Dec_VAE_Output_dice_coefficient: 0.9046 - val_loss: -0.2375 - val_Dec_GT_Output_loss: -0.2559 - val_Dec_VAE_Output_loss: 0.0184 - val_Dec_GT_Output_dice_coefficient: 0.2559 - val_Dec_VAE_Output_dice_coefficient: 0.9228

Epoch 00015: val_loss did not improve from -0.25616
Epoch 16/20
242/242 [==============================] - 894s 4s/step - loss: -0.2705 - Dec_GT_Output_loss: -0.2929 - Dec_VAE_Output_loss: 0.0224 - Dec_GT_Output_dice_coefficient: 0.2929 - Dec_VAE_Output_dice_coefficient: 0.9043 - val_loss: -0.2460 - val_Dec_GT_Output_loss: -0.2645 - val_Dec_VAE_Output_loss: 0.0186 - val_Dec_GT_Output_dice_coefficient: 0.2645 - val_Dec_VAE_Output_dice_coefficient: 0.9210

Epoch 00016: val_loss did not improve from -0.25616
Epoch 17/20
242/242 [==============================] - 880s 4s/step - loss: nan - Dec_GT_Output_loss: -0.2163 - Dec_VAE_Output_loss: nan - Dec_GT_Output_dice_coefficient: 0.2163 - Dec_VAE_Output_dice_coefficient: 0.7070 - val_loss: nan - val_Dec_GT_Output_loss: -0.0158 - val_Dec_VAE_Output_loss: nan - val_Dec_GT_Output_dice_coefficient: 0.0158 - val_Dec_VAE_Output_dice_coefficient: 0.7577

Epoch 00017: val_loss did not improve from -0.25616
Epoch 18/20
242/242 [==============================] - 851s 4s/step - loss: nan - Dec_GT_Output_loss: -0.0162 - Dec_VAE_Output_loss: nan - Dec_GT_Output_dice_coefficient: 0.0162 - Dec_VAE_Output_dice_coefficient: 0.5848 - val_loss: nan - val_Dec_GT_Output_loss: -0.0158 - val_Dec_VAE_Output_loss: nan - val_Dec_GT_Output_dice_coefficient: 0.0158 - val_Dec_VAE_Output_dice_coefficient: 0.5632

Epoch 00018: val_loss did not improve from -0.25616
Epoch 19/20
242/242 [==============================] - 851s 4s/step - loss: nan - Dec_GT_Output_loss: -0.0162 - Dec_VAE_Output_loss: nan - Dec_GT_Output_dice_coefficient: 0.0162 - Dec_VAE_Output_dice_coefficient: 0.5679 - val_loss: nan - val_Dec_GT_Output_loss: -0.0158 - val_Dec_VAE_Output_loss: nan - val_Dec_GT_Output_dice_coefficient: 0.0158 - val_Dec_VAE_Output_dice_coefficient: 0.5886

Epoch 00019: val_loss did not improve from -0.25616
Epoch 20/20
242/242 [==============================] - 851s 4s/step - loss: nan - Dec_GT_Output_loss: -0.0162 - Dec_VAE_Output_loss: nan - Dec_GT_Output_dice_coefficient: 0.0162 - Dec_VAE_Output_dice_coefficient: 0.5946 - val_loss: nan - val_Dec_GT_Output_loss: -0.0158 - val_Dec_VAE_Output_loss: nan - val_Dec_GT_Output_dice_coefficient: 0.0158 - val_Dec_VAE_Output_dice_coefficient: 0.6099

Epoch 00020: val_loss did not improve from -0.25616

Any and all feedback would be appreciated. Are there other parameterizations or code changes required to make this model work?

Thanks,

Jay

[doc78]

You should not have a "nan" as a loss value. Maybe there is something wrong in your loss function, try double checking it.

[jayurbain]

doc78, thanks for your feedback. I do not understand the logic for seeing "nan" values. That usually happens when your feeding bad values into something like an activation function.

I reduced by input_shape to the values used in the original paper, and I'm starting to see some preliminary results. No "nan' values. At this time, I'm not sure why.

[jayurbain]

Just to follow up. My problem was due to the way I was using the sk-image resize function. Essentially the masks became floating points and the values rounded to zero.

[jayurbain]

I would appreciate some feedback on my training curves. Looks like over-fitting which I thought the VAE would address. L2 and KL are set to 0.1 for the VAE in the loss function. The 'Dec_GT_Output_loss' and 'Dec_GT_Output_dice_coefficient' flat line at zero after ~100 epochs. The VAE portion seems to eventually improve.

After 50 epochs:

Here's an example ground truth:

And the corresponding prediction. Not very good.

After 100 epochs:

After 150 epochs:

Thanks, Jay

[doc78]

In my experiments I trained the model using the loss function as the sum of L=-dice+0.1KL+0.1L2. I wonder if you used only the VAE part of the terms instead of all of them to train the model? Furthermore, you can set the metrics to monitor the Dice, as well as the other terms (do not care about the standard "accuracy" metric used by Keras). Particularly, if you use to save best model weights using checkpoint, you should do so by monitoring the Dice score instead of Accuracy value. While monitoring single terms, after some epochs I see that KL and L2 components plays a minor role in the total loss function and the dice score improves. During my training I found that after just a few epochs the term KL goes many order of magnitude below the Dice and L2 term, so the final loss is dependant mainly by Dice and L2 scores. However, KL term is still useful to keep the VAE model to force latent space to get the gaussian distribution of probabilities, otherwise there is the risk that latent space will not assume that distribution. Hope this helps.

[sneh-debug]

doc78, thanks for your feedback. I do not understand the logic for seeing "nan" values. That usually happens when your feeding bad values into something like an activation function.

I reduced by input_shape to the values used in the original paper, and I'm starting to see some preliminary results. No "nan' values. At this time, I'm not sure why.

i am also faced the same problem. in the model i maintained input shape as used by author whereas in the encoder script i reduced the shape to input_shape=(4,112,112,96) now i am not getting nan but dice coefficient remains zero. m i making some mistake ? although i m not training for both decode and vae as mentioned by @doc78

[doc78]

To help discover the reason of the Nan values, I suggest to check separately the terms of the loss function. You can add more metrics and check which one is having the Nan value during training. It can be due to operations made during the evaluation of KL, or due to wrong labels, or many other reasons that cause the loss function to have wrong values (e.g. division by zero or log of a negative number). Furthermore, keep in mind that loss function should be a differentiable function, otherwise it will have difficulties to converge. This is the reason to use the "soft dice" in the loss of this model instead using the standard dice score formula.

[Godric877]

For anyone still struggling with the dice coefficient going to zero, I found this repository useful : https://github.com/athon2/BraTS2018_NvNet. It does not have the data preprocessing file but you can write one yourself involving normalisation, clipping values between (0,1), resizing etc. It expects the input to the network to be of dimension (4,128,128,128) and trains each of the 3 types of segmentation maps independently, so the output dimension should be (1,128,128,128).

[baaaohaaan]

For anyone still struggling with the dice coefficient going to zero, I found this repository useful : https://github.com/athon2/BraTS2018_NvNet. It does not have the data preprocessing file but you can write one yourself involving normalisation, clipping values between (0,1), resizing etc. It expects the input to the network to be of dimension (4,128,128,128) and trains each of the 3 types of segmentation maps independently, so the output dimension should be (1,128,128,128).

Hi! Did you make preprocessing file run successfully? Could you please share it to me or tell me how to do it? This is the first time I do project like this, so I would be appreciated if you could help!

[Godric877]

I don't know which preprocessing file they have used originally, but I wrote one myself, incorporating it into the dataset class.

I borrowed code from https://github.com/lachinov/brats2019 for this purpose.

` import nibabel as nib import loader_helper

def read_nii(filename): image = nib.load(filename) return np.array(image.get_data())

def read_numpy(filename): return np.load(filename)

def read_nii_header(filename): return nib.load(filename)

def read_multimodal(data_path, series, annotation_path=None, read_annotation=True): suffixes = ['_t1.nii.gz', '_t1ce.nii.gz', '_t2.nii.gz', '_flair.nii.gz']

affine = read_nii_header(os.path.join(data_path, series, series + suffixes[0])).affine
files = [read_nii(os.path.join(data_path, series, series + s)) for s in suffixes]
data = np.stack(files, axis=0).astype(np.float32)
annotation = None
if read_annotation:
    p = os.path.join(data_path, series, series + '_seg.nii.gz')
    if annotation_path is not None and not os.path.isfile(p):
        p = os.path.join(annotation_path, series + '.nii.gz')
    annotation = read_nii(p)

return data, annotation, affine

def bbox3(img):

rows = np.any(img, axis=1)
rows = np.any(rows, axis=1)

cols = np.any(img, axis=0)
cols = np.any(cols, axis=1)

slices = np.any(img, axis=0)
slices = np.any(slices, axis=0)

rows = np.where(rows)
cols = np.where(cols)
slices = np.where(slices)
if (rows[0].shape[0] > 0):
    rmin, rmax = rows[0][[0, -1]]
    cmin, cmax = cols[0][[0, -1]]
    smin, smax = slices[0][[0, -1]]

    return np.array([[rmin, cmin, smin], [rmax, cmax, smax]])
return np.array([[-1,-1,-1],[0,0,0]])

def separate_labels(label_data, config): target_labels = []

for l_idx in range(config["n_labels"]):
    assert config["labels"][l_idx] in [1,2,4],"Wrong label!Expected 1 or 2 or 4, but got {0}".format(config["labels"][l_idx])
    target_labels.append(label_data == config["labels"][l_idx])                
target_labels = np.array(target_labels,dtype=np.float32)
return target_labels

class Brats2019Dataset(Dataset):

def __init__(self,phase,config,
             crop_shape=(128,128,128),
             resize_shape=(128,128,128),
             for_prediction=False):
    self.cache = {}
    self.config = config 
    self.phase = phase
    self.crop_shape = crop_shape
    self.resize_shape = resize_shape
    self.data_paths = []
    self.for_prediction = for_prediction
    self.validation_ids = ['BraTS19_2013_0_1',
                           'BraTS19_2013_12_1',
                           'BraTS19_2013_16_1',
                           'BraTS19_2013_2_1',
                           'BraTS19_2013_23_1',
                           'BraTS19_2013_26_1',
                           'BraTS19_2013_29_1',
                           'BraTS19_CBICA_AAB_1',
                           'BraTS19_CBICA_AAP_1',
                           'BraTS19_CBICA_AMH_1',
                           'BraTS19_CBICA_AQD_1',
                           'BraTS19_CBICA_ATX_1',
                           'BraTS19_CBICA_AZH_1',
                           'BraTS19_CBICA_BHB_1',
                           'BraTS19_TCIA12_101_1',
                           'BraTS19_TCIA01_150_1',
                           'BraTS19_TCIA10_152_1',
                           'BraTS19_TCIA04_192_1',
                           'BraTS19_TCIA08_205_1',
                           'BraTS19_TCIA06_211_1',
                           'BraTS19_TCIA02_222_1',
                           'BraTS19_TCIA12_298_1',
                           'BraTS19_TCIA13_623_1',
                           'BraTS19_CBICA_ANV_1',
                           'BraTS19_CBICA_BBG_1',
                           'BraTS19_TMC_15477_1']
    data_folder = self.config["training_data_folder"]
    self.HGG = os.listdir(data_folder+"/HGG")
    self.LGG = os.listdir(data_folder+"/LGG")
    for folder in self.HGG:
        element = {}
        element["path"] = data_folder + "/HGG/"
        element["grade"] = "HGG"
        element["BraTS19ID"] = folder
        if(self.phase == "train" and folder not in self.validation_ids):
          self.data_paths.append(element)
        elif(self.phase == "valid" and folder in self.validation_ids):
          self.data_paths.append(element)
    for folder in self.LGG:
        element = {}
        element["path"] = data_folder + "/LGG/"
        element["grade"] = "LGG"
        element["BraTS19ID"] = folder
        if(self.phase == "train" and folder not in self.validation_ids):
          self.data_paths.append(element)
        elif(self.phase == "valid" and folder in self.validation_ids):
          self.data_paths.append(element)

def __len__(self):
    return len(self.data_paths)

def __getitem__(self, idx):
    if(idx in self.cache):
        return self.cache[idx]
    sample = {}
    element = self.data_paths[idx]
    if(self.phase == "train" or self.phase == "valid"):
      
      image, label, affine = read_multimodal(element["path"], element["BraTS19ID"], read_annotation=True)
      
      ## Compute bounding box
      bbox = bbox3(label>0)
      borders = np.array(label.shape)
      borders_low = np.array(self.crop_shape) / 2.0 + 1
      borders_high = borders - np.array(self.crop_shape) / 2.0 - 1

      bbox[0] = np.maximum(bbox[0]-50, borders_low)
      bbox[1] = np.minimum(bbox[1]+50, borders_high)
      
      ## Normalising image
      mask = image > 0
      num_voxels = np.sum(mask,axis=(1,2,3))

      mean = np.sum(image / num_voxels[:,None,None,None], axis=(1,2,3))
      mean2 = np.sum(np.square(image) / num_voxels[:,None,None,None], axis=(1,2,3))

      std = np.sqrt(mean2 - mean * mean)

      image = (image - mean.reshape((image.shape[0],1,1,1))) / std.reshape((image.shape[0],1,1,1))
      
      ## Cropping image
      # center = np.random.rand(3)
      center = np.array([0.5,0.5,0.5])

      center = center * (bbox[1] - bbox[0]) + bbox[0]
      left_bottom = center - np.array(self.crop_shape) / 2.0
      left_bottom = left_bottom.astype(np.int32)

      processed_data = image[:,left_bottom[0]:left_bottom[0] + self.crop_shape[0],
                 left_bottom[1]:left_bottom[1] + self.crop_shape[1],
                 left_bottom[2]:left_bottom[2] + self.crop_shape[2]]

      processed_label = label[left_bottom[0]:left_bottom[0] + self.crop_shape[0],
                  left_bottom[1]:left_bottom[1] + self.crop_shape[1],
                  left_bottom[2]:left_bottom[2] + self.crop_shape[2]]
      
      ## Separate the three labels
      processed_labels = separate_labels(processed_label, self.config)
      
      final_data = np.nan_to_num(processed_data)
      if self.config["VAE_enable"]:
        # Concatenate to (7, self.crop_shape) as network output
        final_label = np.nan_to_num(np.concatenate((processed_labels, final_data), axis=0))
      else:
        final_label = np.nan_to_num(processed_labels)
      # final_data = torch.from_numpy(processed_data)
      # final_label = torch.from_numpy(processed_labels)

      if(self.for_prediction):
          sample['bbox']           = [[left_bottom[0],left_bottom[0] + self.crop_shape[0]],
                                      [left_bottom[1],left_bottom[1] + self.crop_shape[1]],
                                      [left_bottom[2],left_bottom[2] + self.crop_shape[2]]]
          sample['affine']         = affine
          sample['final_data']     = final_data
          sample['final_label']    = final_label
          sample['BraTS19ID']      = element["BraTS19ID"]
          sample['original_shape'] = label.shape
          self.cache[idx] = sample
          return sample
      self.cache[idx] = final_data, final_label
      return final_data, final_label

`