Does Keras-NLP support (Q)Dora?

[asmith26]

Hi there,

I recently came across this blog post: https://www.answer.ai/posts/2024-04-26-fsdp-qdora-llama3.html

[QDoRA] is just as memory efficient and scalable as FSDP/QLoRA, and critically is also as accurate for continued pre-training as full weight training.

Does Keras-NLP support/plan to support (Q)Dora?

Many thanks for any help, and this lib!