Kohya SS_Flux Not Using GPU

[andy8992]

Package

Kohya_SS_flux

When did the issue occur?

Running the Package

What GPU / hardware type are you using?

3080 ti laptop

What happened?

No matter what I try, Kohya SS flux branch isn't using my GPU, i've tried adding set CUDA_VISIBLE_DEVICES=1 to the bat inside the package folder but nothing.

I have to disable all cache too in order to get the training to start otherwise it hangs on caching latents but never actually does it.

When I bypass that and get right to training I show the first epoch starting but zero activity on my gpu and nothing loaded into vram. Leaving it doesn't change this at all either.

Obviously this could be a kohya issue but things like fluxgym work (kinda lol the training exits before finishing but that's a different issue) and I believe it used kohya as a backend.

So I assume it may have to do with SM's install?

Even if it's not SM related.. do you know how to fix this?

Console output

No response

Version

2.13.1

What Operating System are you using?

Windows

[mohnjiles]

It could be a system RAM issue, I loaded up the sd3-flux.1 branch of kohya_ss and it's currently using 35.5GB of RAM when training. If you have less than that it could be hitting swap which would be very very slow to the point of seeming like it's not doing anything. Even with a 16gb VRAM GPU and 96GB of system RAM, it's taking a few minutes per iteration 😅

Could maybe try one of the fp8 versions of the T5 encoder to reduce the memory load perhaps? Also not sure if kohya supports the NF4 version of Flux, but if it does, that could help too.