Bug: --config.learning.finetune_backbones=True

[HumzaW245]

Hi, When I add --config.learning.finetune_backbones=True I get the following failure. ValueError: Unknown signature default in checkpoints/imagenetr50/ (available signatures: _SignatureMap({'representation': <ConcreteFunction pruned(images) at 0x2B323C2C7BE0>})).

I am trying to just run the finetuning baseline to reproduce some of the results in the paper but cannot get past this error. I was wondering what kind of setup you had to run this experiment since it seems like TF1 Hub does not support trainable=True parameter for some models and it seems like this may be one of them.

I tried pip install --upgrade tf-hub-nightly but still get the error.

Thank you!

[evcu]

Thanks for creating the issue. I need to look at my experiments/notes to remember what I did. I might have converted the checkpoints to tf2 to enable finetuning. I'll check this and get back to you.

[HumzaW245]

For some context which may help identify the issue. When following the steps listed, I ran into some issues related to the tf version and a mismatch with the code's functionality which isn't supported in the tf 2.7 specified in the requirements.txt file.

These are the changes are did to get the general experiments to run (except of course finetuning backbones which is a tf-hub issue that I cannot seem to resolve)

1. Removed reduce_retracing = true from finetune.py (some issue with tf versions)
2. Moved checkpoints folder into first head2toe folder (same directory as where run.sh is located)
3. I then got an error related with something to do with MirroedStrategy -> to fix this, I used import atexit and added atexit.register(self.strategy._extended._collective_ops._pool.close) before with self.strategy.scope(): ... in finetune.py (this above the load_backbones function definition)

After the above steps, I was able to follow the steps you wrote and run some experiments. And now I am having this finetune_backbones=True issue which I'm not sure if is a TF1 vs TF2 hub inconsistency.

Thank you! I'll wait for your response and try some other changes in the meantime.

[evcu]

I totally forgot about this thread. Adding a todo for myself.