Windows Support

[jim-plus]

Using Python 3.11, pytorch 2.5.1, cuda 12.4 here.

I was able to install a triton wheel of this windows port to ensure compatibility. https://github.com/woct0rdho/triton-windows

I also locally built a complete wheel for flash attention 2.6.3 to support training, not just inference.

The following command gets pretty far:

pip install --upgrade "unsloth[cu124-ampere-torch250] @ git+https://github.com/unslothai/unsloth.git"

The following prerequistes were satisfied:

Requirement already satisfied: bitsandbytes>=0.43.3 in c:\users\jim\appdata\local\programs\python\python311\lib\site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-ampere-torch250]@ git+https://github.com/unslothai/unsloth.git) (0.44.1)
Requirement already satisfied: packaging in c:\users\jim\appdata\local\programs\python\python311\lib\site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-ampere-torch250]@ git+https://github.com/unslothai/unsloth.git) (24.1)
Requirement already satisfied: ninja in c:\users\jim\appdata\local\programs\python\python311\lib\site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-ampere-torch250]@ git+https://github.com/unslothai/unsloth.git) (1.11.1.1)
Requirement already satisfied: flash-attn>=2.6.3 in c:\users\jim\appdata\local\programs\python\python311\lib\site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-ampere-torch250]@ git+https://github.com/unslothai/unsloth.git) (2.6.3)
Requirement already satisfied: torch in c:\users\jim\appdata\local\programs\python\python311\lib\site-packages (from bitsandbytes>=0.43.3->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-ampere-torch250]@ git+https://github.com/unslothai/unsloth.git) (2.5.1+cu124)
Requirement already satisfied: numpy in c:\users\jim\appdata\local\programs\python\python311\lib\site-packages (from bitsandbytes>=0.43.3->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-ampere-torch250]@ git+https://github.com/unslothai/unsloth.git) (1.26.4)
Requirement already satisfied: einops in c:\users\jim\appdata\local\programs\python\python311\lib\site-packages (from flash-attn>=2.6.3->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[cu124-ampere-torch250]@ git+https://github.com/unslothai/unsloth.git) (0.8.0)

Unfortunately, this is the current obstacle:

ERROR: xformers-0.0.28.post2-cp311-cp311-manylinux_2_28_x86_64.whl is not a supported wheel on this platform.