PEViT

PEViT copied to clipboard

Inquiry about KAdaptation Implementation for Different ViTs (e.g., DinoV2)

1

Hello, I am currently working on a project that involves the application of the KAdaptation technique, as detailed in the paper "Parameter-efficient Model Adaptation for Vision Transformers" , to various...

MarioAvolio

Does the repo pick the weights that perform best in val dataset to evaluate in test dataset?

2

Thank you for your solid work. Does the repo implement the function that pick the model weights that perform best in val dataset to evaluate in test dataset? From the...

pierowu

why can't I replicate some results in serveral datasets ？

3

I've tried to replicate the results in LoRA setting by using PEViT/scripts/lora_clip.sh in solid intact. My device is a V100 GPU. The code env is torch 1.7.0 and CUDA 11.0....

pierowu

Question about LoRA alpha

1

Hi, thanks for your great work. I noticed that in your scripts, you hard-coded the lora alpha to be 128 and the rank r to be 4 (therefore leading to...

vishaal27

Questions about KAdaptation implementation

1

Hi, thanks for the great work and releasing the code to reproduce it. I have a few questions regarding the kronecker adaptation forward pass through the adapter modules: (1) The...

vishaal27

How to adjust the hyperparameter r in the Lora modules?

1

Thank you very much for contributing the source code of the paper. When using Lora for fine-tuning, what is the exact value of the hyperparameter r in Lora modules, and...

pzp-dzd

Impressive work. Will you release the weights of the adapted Vision Transformers in Table 3?

XiaohuJoshua

Hint: = is not a valid operator. Did you mean == ?

2

Hi thanks for your work Im trying to start off this project by typing the command "pip install -r requirements.txt" I already installed elevator toolkit from the link you provided,...

jinlovespho