Implement eval_model.py script using a Pytorch ResNet-50 MC Dropout trained model - Diabetic Retinopathy Detection

[amandinesoub]

Hello ! First of all, thank you for having made available all your work.

I am currently trying to implement the eval_model.py script (https://github.com/google/uncertainty-baselines/blob/main/baselines/diabetic_retinopathy_detection/eval_model.py) in order to evaluate my trained model, a ResNet-50 with MC Dropout trained on Kaggle Diabetic Retinopathy Detection dataset. The latter was trained using the following Pytorch training script : https://github.com/google/uncertainty-baselines/blob/main/baselines/diabetic_retinopathy_detection/torch_dropout.py. However, I'm having now trouble implementing the evaluation script as it seems that Pytorch type models are not yet supported, as seen here : https://github.com/google/uncertainty-baselines/blob/main/baselines/diabetic_retinopathy_detection/eval_model.py (lines 230-232)

if use_torch:
    logging.info(f'Loading Torch ResNet-50 {model_type} {ensemble_str}.')
    raise NotImplementedError

Can you confirm me that at this time, a Pytorch ResNet-50 MC Dropout trained model cannot be used in the eval_model.py script ? If not (maybe I missed the existence of such a script), could you tell me where to find such script to perform model evaluation or implement this on my side ?

Thanking you for your help in advance.

Sincerely, Amandine.