hierarchical adapters

[thistleknot]

⚠️ Please check that this feature request hasn't been suggested before.

• [x] I searched previous Ideas in Discussions didn't find any similar feature requests.
• [x] I searched previous Issues didn't find any similar feature requests.

🔖 Feature description

I have some code that show cases how it's done for lora (my recommendation is with PiSSA)

One for continued pretraining (raw corpus / domain adaptation) which would be frozen when doing completion tuning (qa)

then during inference both are used

✔️ Solution

https://gist.github.com/thistleknot/0d749d423c3d86e33992a711b7c4527e

❓ Alternatives

No response

📝 Additional Context

No response

Acknowledgements

• [x] My issue title is concise, descriptive, and in title casing.
• [x] I have searched the existing issues to make sure this feature has not been requested yet.
• [x] I have provided enough information for the maintainers to understand and evaluate this request.

[NanoCode012]

Your request is to be able to load (and freeze) an adapter while training a separate one right?

I'm open to a PR on this as PEFT does support multi-adapter. Seems like it'll require a new config and to selectively apply the merge_and_unload that we call by default in model loader.

Another thing to check would be any hardcode adapter[0] check (I think there's one in Lora kernels).

[thistleknot]

Here is the pissa version

https://gist.github.com/thistleknot/2aee5dbb84a74564d1b95f48065f777c

Ye id have to digest your codebase

I will consider because I do like your approach to raw v completion

On Tue, Sep 23, 2025, 6:43 AM NanoCode012 @.***> wrote:

NanoCode012 left a comment (axolotl-ai-cloud/axolotl#3168) https://github.com/axolotl-ai-cloud/axolotl/issues/3168#issuecomment-3324082837

Your request is to be able to load (and freeze) an adapter while training a separate one right?

I'm open to a PR on this as PEFT does support multi-adapter. Seems like it'll require a new config and to selectively apply the merge_and_unload that we call by default in model loader.

Another thing to check would be any hardcode adapter[0] check (I think there's one in Lora kernels).

— Reply to this email directly, view it on GitHub https://github.com/axolotl-ai-cloud/axolotl/issues/3168#issuecomment-3324082837, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHKKOTWMH3VCFBWV3NGDZ33UFE7XAVCNFSM6AAAAACG7MVJAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMRUGA4DEOBTG4 . You are receiving this because you authored the thread.Message ID: @.***>

[thistleknot]

But to really be cooking requires the ability to apply weights (i use llm as a judge) to training records (quality scores) for use as grpo but thats above and beyond this process. Scores would require a custom trainer