peft
peft copied to clipboard
huggingface
FIX BOFT device error after PR 1742
Fixes failing CI.
PR #1742 introduced the feature that adapters of the same layer can be on different devices. A new method was introduced that is responsible for moving the parameters related to a specific adapter in a consistent way.
In BOFT, however, one parameter was overlooked, boft_P. This parameter is not stored inside a ParameterDict or ModuleDict, hence it was not moved. The reason is (presumably) that this parameter is shared between all BOFT adapters, as it's always identical. However, this clashes with having different adapters on different devices.
To solve this, the parameter is now moved on the fly to the correct device during the forward pass.
Alternatively, we could store this parameter inside a ParameterDict or BufferDict and have one copy per adapter. This would probably be faster at the cost of more memory.
I ran the BOFT tests locally with GPU and they passed, so this should fix the CI.
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
