dsmil-wsi icon indicating copy to clipboard operation
dsmil-wsi copied to clipboard

about the result are all 0or1

Open hemo0826 opened this issue 2 years ago • 2 comments

Hello, author. When I used your network to train my own WSI, simclr trained 200 epochs. My data is divided into positive and negative, so I set number_ class=1. But in training, the first training result is the best, but after training, the bag_ Predictions are all 0 or 1. Could you give me some advice? thank you

hemo0826 avatar Jun 19 '22 08:06 hemo0826

image image

This is a CSV file extracted by training

hemo0826 avatar Jun 19 '22 08:06 hemo0826

Which script did you use for generating the prediction outputs? It could be that the trained model weights are not loaded correctly.

binli123 avatar Jul 13 '22 02:07 binli123

I incorporated the training/testing into the same pipeline in the latest commit. I also incorporated an orthogonal weights initialization which helps making the training more table. You can set --eval_scheme=5-fold-cv-standalone-test which will perform a train/valid/test like this:

A standalone test set consisting of 20% samples is reserved, remaining 80% of samples are used to construct a 5-fold cross-validation. For each fold, the best model and corresponding threshold are saved. After the 5-fold cross-validation, 5 best models along with the corresponding optimal thresholds are obtained which are used to perform inference on the reserved test set. A final prediction for a test sample is the majority vote of the 5 models. For a binary classification, accuracy and balanced accuracy scores are computed. For a multi-label classification, hamming loss (smaller the better) and subset accuracy are computed.

You can also simply run a 5-fold cv --eval_scheme=5-fold-cv

There were some issues with the testing script when loading pretrained weights (i.e., sometimes the weights are not fully loaded or there are missing weights, setting strict=False can reveal the problems.). The purpose of the testing script is to generate the heatmap, you should now read the performance directly from the training script. I will fix the issues in a couple of days.

binli123 avatar Mar 28 '24 15:03 binli123