Question about the preprecessing

[GWwangshuo]

When I run the 'preprocess.py', I got a new class 7 which is not in 0 - 6. Is this correct or not? Thanks. Moreover, how long and how many GPU you use to train the PSPNet？ Could you please share the trained model for inference? Thanks again.

[hubutui]

The valid label value is 0-6, but there is no image in trainset with label 2. To simplify, you could remap the label or just leave it unchanged. There is no label value 7, you should check your data. Only a TITAN Xp GPU is used for training for 512 epochs. You could finish the training in one day. As for the trained model, we might shared the trained model with a docker image once our paper is accepted.

[GWwangshuo]

The valid label value is 0-6, but there is no image in trainset with label 2. To simplify, you could remap the label or just leave it unchanged. There is no label value 7, you should check your data. Only a TITAN Xp GPU is used for training for 512 epochs. You could finish the training in one day. As for the trained model, we might shared the trained model with a docker image once our paper is accepted.

Thanks for your quick reply. Could you please help me check what happens here when I run the preprocessing.py? Some settings of preprocessing.py when I run it are as follows:

• The --inputdirs is set to ['Maps1_T', 'Maps2_T', 'Maps3_T', 'Maps4_T', 'Maps5_T', 'Maps6_T']
• The --undecidedlabel and --pool-size are set to default.

The problem I encounter is that the output (final annotation) of STAPLE for the image slide007_core056_classimg_nonconvex.png in Maps1_T, Maps2_T, Maps3_T, Maps4_T, Maps5_T contains the pixels with the unexcepted value 7.

I think it may be the problem of sitk.MultiLabelSTAPLE(imgs, 255). The correct result here should only contain the pixel value of 0 to 6. What do you think of this phenomenon?

Furthermore, what's your purpose of using undecidedlabel here and what is the batch_size and the ratio of training images and validation images when you train the PSPNet?

[hubutui]

undecidedlabel is an arg required by sitk.MultiLabelSTAPLE, you could check it here. batch_size=2, 200 images for training, and 44 for validation.

[GWwangshuo]

undecidedlabel is an arg required by sitk.MultiLabelSTAPLE, you could check it here. batch_size=2, 200 images for training, and 44 for validation.

Thanks for your reply. I am still working on the problem of generating the unexcepted pixel value. After reading the SimpleITK document , I found the below hints:

• if more than one label have a maximum probability, then an "undecided" label is assigned to that output pixel.

• By default, the label used for undecided pixels is the maximum label value used in the input images plus one.

Now the problem happens when there exists the situation in which more than one label have a maximum probability. I do the following experiment:

I create two images with pixel value of 0 and 1. By using STAPLE to generate the new annotation, if there is no setting of undecided, the final result should be all 2. But after I set the undecided to 255, the final result is still all 2 which is incorrect.

Is this strange?

[lucky-szj]

Have you run the model successifully?Did you find the paper or report about this?My wechat is szj_lucky0904.

[GWwangshuo]

@lucky-szj , I do not find any papers about this. In this repo, I only encounter problem in the preprecessing stage. I can run the model successfully.

[lucky-szj]

| distributed init (rank 0): 10.212.66.4 Traceback (most recent call last): File "/gs/home/sunzhongjie/Gleason-master/train.py", line 237, in <module> main(args) File "/gs/home/sunzhongjie/Gleason-master/train.py", line 100, in main utils.init_distributed_mode(args) File "/gs/home/sunzhongjie/Gleason-master/utils.py", line 298, in init_distributed_mode torch.distributed.init_process_group(backend=args.dist_backend, init_method=args.dist_url, File "/gs/home/sunzhongjie/.conda/envs/testenv/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 396, in init_process_group rendezvous_iterator = rendezvous(init_method, rank, world_size) File "/gs/home/sunzhongjie/.conda/envs/testenv/lib/python3.8/site-packages/torch/distributed/rendezvous.py", line 73, in rendezvous raise RuntimeError("No rendezvous handler for {}://".format(result.scheme)) RuntimeError: No rendezvous handler for :// Hello,When I run the train.py,I encounter the question,I directly run the train.py firstly,should I run the preprecessing firstly?

[hubutui]

@phuongchi911 I don't understand your question. Did you download the dataset? The masks, which are saved as images, store label value as pixel intensity, usually ranging from 0 to C-1 with C classes. In this dataset, label value 2 is missing, which means that there is no image labeled as Gleason score 2.

[phuongchi911]

@phuongchi911 I don't understand your question. Did you download the dataset? The masks, which are saved as images, store label value as pixel intensity, usually ranging from 0 to C-1 with C classes. In this dataset, label value 2 is missing, which means that there is no image labeled as Gleason score 2.

thanks a lot for the explanation. I think i can understand it now.

[phuongchi911]

@GWwangshuo GWwangshuo may I ask if u have solved the issue with label 7? I also face the same problem for the two cores that resulting mask have 7. Thank you very much

[hubutui]

@phuongchi911 Actually, this is a bug in SimpleITK/ITK. Check this post You may need to wait for a new release to fix it. Or, you could fix it yourself, you could manually update the undecide label value from applying MultiLabelSTAPLE filter with numpy operator.

[phuongchi911]

@phuongchi911 Actually, this is a bug in SimpleITK/ITK. Check this post You may need to wait for a new release to fix it. Or, you could fix it yourself, you could manually update the undecide label value from applying MultiLabelSTAPLE filter with numpy operator.

@hubutui wow thanks so much for the fast response. I will try to reinstall the package to see if it works. By the way, there are two 2 final masks that contains 6 (which are converted to 2 after stapling) while the original masks do not contain 6 at all. Do you think it is normal behaviour? Thanks so much for the insights

[hubutui]

@phuongchi911 I don't think reinstall SimpleITK would fix this issue, since there is no new release since then. Actually, you should check sklearn.preprocessing.LabelEncoder, which encodes target labels with value between 0 and n_classes-1. I would use this method if I had known it.

[phuongchi911]

@phuongchi911 I don't think reinstall SimpleITK would fix this issue, since there is no new release since then. Actually, you should check sklearn.preprocessing.LabelEncoder, which encodes target labels with value between 0 and n_classes-1. I would use this method if I had known it.

Hi fyi u can install it again from the webpage provided by some of the one on ur post. The latest package is 2.0 which produces correct results. Thanks so much for pointing out the post.