Got very different result between process_image.py and chester-xray web with same image.

[Typepluto]

process_image.py result: {'preds': {'Atelectasis': 0.5126234, 'Cardiomegaly': 0.06345778, 'Consolidation': 0.5328131, 'Edema': 0.2958934, 'Effusion': 0.53310466, 'Emphysema': 0.50264, 'Enlarged Cardiomediastinum': 0.5229955, 'Fibrosis': 0.5155035, 'Fracture': 0.51720035, 'Hernia': 0.0015778106, 'Infiltration': 0.5470137, 'Lung Lesion': 0.16751169, 'Lung Opacity': 0.8801794, 'Mass': 0.65471303, 'Nodule': 0.5540755, 'Pleural_Thickening': 0.50598806, 'Pneumonia': 0.033414263, <------------------------------------------------------------------------------ 'Pneumothorax': 0.50849223}}

chester-xray web result: Atelectasis,pred:0.16236889362335205,OP_POINT:0.07422872->normalized:0.5476036444030495 Consolidation,pred:0.14904548227787018,OP_POINT:0.038290843->normalized:0.5575821902452106 Infiltration,pred:0.18629415333271027,OP_POINT:0.09814756->normalized:0.5488697426669435 Pneumothorax,pred:0.03416414186358452,OP_POINT:0.0098118475->normalized:0.5122968015230744 Edema,pred:0.1063307672739029,OP_POINT:0.023601074->normalized:0.5423646990338368 Emphysema,pred:0.01815347746014595,OP_POINT:0.0022490358->normalized:0.507970145973699 Fibrosis,pred:0.02496715821325779,OP_POINT:0.010060724->normalized:0.5075289639347826 Effusion,pred:0.1678045243024826,OP_POINT:0.103246614->normalized:0.5359953535221347 Pneumonia,pred:0.2875974476337433,OP_POINT:0.056810737->normalized:0.8090469355374046 <------------------------ Pleural Thickening,pred:0.039720576256513596,OP_POINT:0.026791653->normalized:0.5066424231236653 Cardiomegaly,pred:0.01207550149410963,OP_POINT:0.050318155->normalized:0.11999149704624136 Nodule,pred:0.13887496292591095,OP_POINT:0.023985857->normalized:0.558856271064256 Mass,pred:0.24335533380508423,OP_POINT:0.01939503->normalized:0.7984534567199927 Hernia,pred:0.00022874104615766555,OP_POINT:0.042889766->normalized:0.0026666156928632527 Lung Lesion,pred:0.020575225353240967,OP_POINT:0.053369623->normalized:0.19276157668605762 Fracture,pred:0.06660972535610199,OP_POINT:0.035975814->normalized:0.5158885595408194 Lung Opacity,pred:0.7050759196281433,OP_POINT:0.20204692->normalized:1 Enlarged Cardiomedia.,pred:0.08681041747331619,OP_POINT:0.05015312->normalized:0.5192964246370511

The result of Pneumonia is very different. Is this a normal behavior?

And what's the difference of web and process_image.py?

[ieee8023]

The main difference is with the preprocessing and resizing because one uses skimage and the other uses built in web browser image resizing. The weights of the models are the same. We did a study in the original chester paper and only found small changes.

To isolate if it is the resizing issue try to scale and crop thr image to 224x224 and then try it wih both methods.

Does this happen for all the images you try? Can you share the image you used?

[Typepluto]

Sorry for the late reply. Does this happen for all the images you try? ->Not as critical as this one. But other images still have some difference.

Can you share the image you used? ->I can send over the image via email, and please provide the email address for further sharing.

[ieee8023]

Did resizing the image fix the issue though? I think this is the main reason for differences.

My email is joseph at josephpcohen.com

[ieee8023]

Ok I figured it out. I forgot I added a scaling only in the web version so it would be more useful in predicting lung opacity and other targets. It scales the upper predictions (0.6 >) which made it easier to interpret the results because physicians didn't see 0.75 as a strong prediction when this is a strong prediction given this model.

https://github.com/mlmed/chester-xray/blob/master/res/js/system.js#L638

The XRV all model has a 1.3 scale: https://github.com/mlmed/chester-xray/blob/master/models/xrv-all-45rot15trans15scale/config.json

[ieee8023]

Let me know if this issue is not resolved. I'll close it for now.