Anyone tried Dynamic LoRA (DyLoRA) to speedup

[doxav]

I am currently training Alpaca-LoRA as it is, but already wondering if someone tried to speed this up with DyLORA ?

Paper: https://neurips2022-enlsp.github.io/papers/paper_37.pdf Code to replace LoRA by DyLoRA: https://github.com/huawei-noah/KD-NLP/tree/main/DyLoRA

[gururise]

Interested in hearing if anyone had success with this..