Issue about training configuration on Binary Adapter with VTAB-1k Dataset

[leoli646]

Hello, When I tried to replicate your binary_adapter experiment using the VTAB-1k dataset, I was unable to reproduce the results that you reported. I would like to discuss some potential issues with the training configuration that might be causing this discrepancy.

In similar works like VPT and SSF, different hyper-parameters (such as lr_rate, weight-decay, drop-path, etc.) are utilized for various datasets within VTAB-1k. However, the train.sh script in the binary_adapter codebase doesn't seem to account for these variations and applies default hyperparameters universally.

Could you advise on whether I should:

1. Conduct a grid search to find the best hyperparameter set for each dataset?
2. Or, should I use the hyperparameter settings from another public work like SSF, for instance?

Your insights would be greatly appreciated as I continue my experiments.

Looking forward to your reply!