Hyunwook Yu

@pacman100 please help review.

> Thank you @Splo2t for adding the support to have LoRA layers in Embedding modules 🤗, this is really cool 🔥. Left a few suggestions. > > Are there evaluation...

> Very interesting pointers @Splo2t and @SOCSChamp. And what is the impact on the trainable parameters percentage with and without targeting embedding layer? Excluding the Embedding Layer: trainable params: 8257536...

@pacman100 I've made the suggested changes and rebased on main. And I will post about related performance indicators later.

Hello @pacman100 I have refactored the Lora Embedding layer and fixed some bugs. For everyone, I am sharing the experimental results. I have been working on training a kogpt model...

> Thank you @Splo2t for the detailed experimental results 🔥. A question to clarify, you used this PR for the cases wherein no new tokens were added to the vocab,...

Thank you for your thorough review and valuable feedback @pacman100 !

I'd like to thank you for your input, @flozi00 I'd like to provide some additional information regarding INT8 training. When I mentioned that the issue occurred while resizing the Embedding...

Thank you for your question @Sanster. When training [NSMC](https://github.com/e9t/nsmc) using AutoModelForSequenceClassification, we observed that adjusting the learning rate can lead to a difference of up to 3% in accuracy. Although...