derfasthirnlosenick
derfasthirnlosenick
One more benefit this would have: If Modules can be stored in external files, it'd be trivial to edit them using powerful code editors such as VSCode, trace them in...
Hi! Is there an update on this in the meantime? I'm running into exactly the same issue here as well. Given that MetView is OSX and Linux, I can't use...
So similar to #53 ?
My hunch: NF4 requires BitdandBytes, which for now is nVidia specific - whereas FP8 is relatively simple and rubs out of the box. The good news is that Bitsandbytes is...
Yeah I did, a few minutes before the release :D will try the official release when I got the time. Also waiting for the official 6.2 pytorch nightly wheel
NF4 I didn't get running, only fp8 (also w.o. BnB)
Managed to break my venv by trying again :-D got it back working w.o. bitsandbytes, will try again later and report back.
Don't have proper access to my deskto right now so can't test, but in case you're feeling adventurous, there's a ROCm version of xformers as well: https://github.com/ROCm/xformers/ That one apparently...
Ok, can confirm the nf4 working on my 6800xt with the BnB alpha compiled from source myself. Also running Rocm 6.2 with the newest pytorch nightly (hint: only torch, torch...
> Can anyone tell me, which model exactly is meant by the fp8 version? Is there an official version? When I search for it I only find this one https://huggingface.co/Kijai/flux-fp8....