Daniel Han

@81549361 Hopefully should be solved! On local PCs, you'll need to update Unsloth via `pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git`. Otherwise Colab / Kaggle no need to reinstall

@muellerzr Thanks again!! Sorry was caught up in stuff - will review today!

@muellerzr Sorry just got to this - I ran T4 and L4 on some examples and nothing seems broken + had a look through the code! In terms of autocast...

Oh yep had a discussion with some researchers about this! Speed wise, because the first and the last get updated, the gradients have to be backpropagated to the start, so...

@risedangel No sorry :( Been stuck on fixing bugs

@ml-maddi Can you screenshot Unsloth's info section

@ml-maddi Oh that's long! It seems like your local installation of CUDA might be broken maybe? Does non Unsloth code paths work as expected, or is this just an Unsloth...

@VatsaDev It should work! We build on top of TRL, so it should work :)

Oh thanks and great work with the notebook!

@VishnuPJ Sadly not - you can do continued pretraining though - https://colab.research.google.com/drive/1ef-tab5bhkvWmBOObepl1WgJvfvSzn5Q?usp=sharing can help. I do not suggest pretraining since you have to spend a lot of compute and money...