LORA/MODEL: Use max rank of pattern for `add_weighted_adapter`

[Beinsezii]

Otherwise it's possible that the new merged adapter won't allocate enough space when a patterned rank is larger than config.r.

For example, the cat method uses sum(adapters_ranks) for the empty tensor alloc, so if cat(loras) for a given layer is ever > the sum of config.r the assignment will overflow.

Local test fails but it's all entirely the TestEvaInitialization module which appears completely unrelated.

[bghira]

thanks Ben, i'd requested this pull request be opened since it seems trivial enough but I'm not sure we'll have time to add unit tests here, especially considering none of us are particularly familiar with the unit test ecosystem for PEFT.

[BenjaminBossan]

Thanks for providing more context @bghira. I wrote the code required to test this change:

    @parameterized.expand([IA3Config, LoHaConfig, LoKrConfig, LoraConfig, HRAConfig, BoneConfig])
    def test_add_weighted_adapter_cat_with_rank_pattern(self, config_cls):
        # Fixes a bug described in #2512, which resulted from the rank_pattern not being taken into account
        config0 = LoraConfig(target_modules=["lin0", "lin1"], r=8)
        config1 = LoraConfig(target_modules=["lin0", "lin1"], r=8, rank_pattern={"lin0": 16})
        model = MLP()
        model = get_peft_model(model, config0).to(self.torch_device)
        model.add_adapter("other", config1)
        model.add_weighted_adapter(["default", "other"], weights=[1.0, 1.0], adapter_name="merged", combination_type="cat")

I can best be added in this line. @Beinsezii would you be so kind to add it to your PR?

[Beinsezii]

Verified it fails on main and passes on beinsezii/weighted_adapter_max_rank_pattern

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.