Support Conv3d layer

[jsilter]

Feature request

Currently, many adapters support Conv1d and Conv2d layers, but not Conv3d. I would like to be able to use these methods on Conv3d layers.

Motivation

I would like to use PEFT on models which process 3D images (eg video or volumetric images). As it stands, PEFT cannot be applied.

Your contribution

I have an implementation for IA3 which I am happy to contribute. I believe there are 8 total adapter methods which currently support convolutional layers, and each one would need to be modified individually. I can probably find the time for a few other methods as well if there is interest.

[BenjaminBossan]

Adding support for more layers would indeed be a nice addition to PEFT. If you already have an implementation for IA³, please feel free to open a PR. When it comes to other methods, we can add support step by step. Depending on the method, it could be easy or quite hard. The biggest leverage would be if LoRA had support, since that's the most used one and its implementation is quite straightforward.

[jsilter]

Great, I just created a PR at #2082. It includes support for IA^3 and LoRA.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[BenjaminBossan]

Resolved via #2082.