Sebastian Raschka
Sebastian Raschka
Thanks for sharing. This looks nice and elegant. And yes, I think this could indeed be a useful addition. Perhaps under a `--jupyter_env` flag.
Sorry, I have no clue to be honest. It's been years since I used the code and it could just be that the URLs don't work anymore.
Yes, they could be added. We omitted them for simplicity in the CORN paper.
Yes, this looks correct to me. You can also add a default argument so that it performs like before if someone doesn't specify the importance weights: ```python def corn_loss(logits, y_train,...
I greatly simplified this @carmocca . Let me know if that's ok now. It should introduce any changes to the defaults.
> Is it really needed to add an argument for this? Why not just use the initial sanity check as the initial validation loss? I originally had that but others...
Hm, that's a weird one. I am wondering if you are perhaps using an older version of litgpt? Because `evaluate` was just added last week.
Just following up on this, have you been able to run it? Until the PyPI package is updated in the next few days, I recommend installing it from the GitHub...
After our discussion today, I think we should only enable vanilla Galore for now, not worrying about the LoRA support. We can look into LoRA support later if there is...
The combination of LoRA + GaLore doesn't really work yet due precision mismatches when merging the LoRA weighs at the end so it didn't get to the code line that...