hguan6

Hi Luyu. Can you send a copy of your C-COIL initializer paper to my email: [email protected]? I suppose your coCondenser can also be used to initialize COIL. Did you try...

I think it is better to freeze some parameters to reduce gradient computations. Use something like https://discuss.huggingface.co/t/how-to-freeze-layers-using-trainer/4702/3