Diabetic Retinopathy Model Only Predicts 0

[isaacwasserman]

I tried to reproduce your results, but after loading the pretrained weights, I can only get the model to predict [1,0]. Am I doing this correctly?

import tensorflow as tf
import tensorflow.keras as keras
import numpy as np

model = keras.models.load_model("wb_model.h5")

val_x_file = open("../datasets/DiabeticRetinopathyValData/val_test_x_preprocess.npy", "rb")
val_y_file = open("../datasets/DiabeticRetinopathyValData/val_test_y.npy", "rb")

val_x = np.load(val_x_file)
val_y = np.load(val_y_file)

p = np.random.permutation(len(val_y))
val_x = val_x[p]
val_y = val_y[p]

y_pred = model.predict(val_x)

print("Prediction distribution:", np.histogram(np.argmax(y_pred, axis=1), range=(0,1), bins=2,)[0])
print("Real distribution:", np.histogram(np.argmax(val_y, axis=1), range=(0,1), bins=2,)[0])

Returns:

Prediction distribution: [10644     0]
Real distribution: [8715 1929]

[isaacwasserman]

The CXR model has a similar issue, predicting 7/699 positives. The DERM model does significantly better, predicting 95/135 positives.