Daniel Han

@skmanzg Could you try before running Unsloth: ```python import os os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" os.environ["CUDA_VISIBLE_DEVICES"] = "0" ```

Great!

@regstuff Could you try uninstalling then installing Unsloth ie: ```python pip uninstall unsloth -y pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git ```

Oh interesting on packaging

If this is a major request by the OSS community - I'm more than happy to include some of the changes from Unsloth!

Yes 0% loss in accuracy - we do actual FLOP reductions via our manual autograd engine. I'm actually working with @casper-hansen and some other Axolotl people to put some methods...

:)

@casper-hansen Oh cool - I'll have a look! Ye I'll try to make a PR to axolotl!!

@fakerybakery Sorry not yet - I'll take a look at the PR Casper made, but it might take some time

I don't think that'll work :( We use FA2 and SDPA so the attention output is actually never constructed