ResNeXt-50 accuracy is 0%

[gcunhase]

About

ResNeXt-50 accuracy is 0% instead of 77.36%. Are the pre-trained weights here updated?

Steps to reproduce

1. Install requirements:

pip install tensorflow-gpu==2.8.0
pip install git+https://github.com/qubvel/classification_models.git

2. Load the model as follows:

ResNeXt, preprocess_input = Classifiers.get("resnext50)
model = ResNeXt(
    include_top=True,
    input_shape=(224, 224, 3),
    weights="imagenet"
)

3. Use preprocess_input as the preprocessing function on the ImageNet validation dataset.
4. Compile the model and evaluate:

 model.compile(
    optimizer="sgd",
    loss=tf.keras.losses.SparseCategoricalCrossentropy(),
    metrics=["accuracy"],
)
_, model_accuracy = model.evaluate(val_batches)

Note that there are no issues with the ResNet models when I follow the above steps.

System info

Python 3.8, Ubuntu 18.04

[gcunhase]

Is this repo still active?

[gcunhase]

Anyone still maintaining this repo?

[innat]

Are you training in the image-net dataset? I've just tested with a small-size dataset from tensorflow-dataset and it works fine.

[gcunhase]

Yes, but I’m using ImageNet in tfrecord format. Can you please share the steps tocante it work? How you loaded the data, preprocessing, model instantiation so I can reproduce your steps on my side? Thanks!

[innat]

HERE.

[gcunhase]

@innat Thank you, have you tried evaluating the model directly with ImageNet (include_top=True)? Mine fails in that scenario.

[innat]

I don't have an ImageNet locally to test. But I did a test with include_top=true on the natural image. It did run but the results are kinda non-sense even on the natural images. You can see the above colab file again.

[gcunhase]

@innat Right, this is what I'm experiencing. And then when I iterate over the validation dataset, the accuracy goes to 0%.

[innat]

Oh, I see. Next, the either we can look at the source code to find anomaly or search other possible solution. Can you check other similar architecture on image-net, for example se-resnext.

[gcunhase]

@innat I'm able to get the correct accuracy with seresnext50... Note I'm using pre-processing as follows due to this issue:

tf.keras.applications.imagenet_utils.preprocess_input(image, mode='torch')

I think that the pre-trained weights for ResNext might be incorrect!