Baoyuan Qi

I have make a pull request to fix this bug.See https://github.com/wurstmeister/storm-docker/pull/18

I came up with the code below. The `id` of `optimizer` changes when call `on_train_epoch_start`. BAD THINGS: Still it needs `lightning` package installed and can only be preformed in a...

I have conducted experiments on llama2-7b using full, lisa_2, lisa_32 methods. From the image above, you can see that the train loss curve decreases and full is the same as...

> Hi @geronimi73 I think you are right. > > I also think the main reason is from the optimizer. Directly using `requires_grad` to dynamically select layers cannot achieve dynamically...

> Hello, Just stumbled into this while trying to figure out how I could change the optimizer / scheduler in the middle of the training. I have managed to do...

I came up with the code below. The `id` of `optimizer` changes when call `on_train_epoch_start`. BAD THINGS: Still it needs `lightning` package installed and can only be preformed in a...