Rui Ren

Got the same issue when tried to build `BNB` from source, I followed the instruction from here https://huggingface.co/docs/bitsandbytes/main/en/installation, but ``` CUDA Setup failed despite GPU being available. Please run the...

> > Got the same issue when tried to build `BNB` from source, I followed the instruction from here https://huggingface.co/docs/bitsandbytes/main/en/installation, but > > ``` > > CUDA Setup failed despite...

>Previous PR doesn't take such case related to Squeeze/Unsqueeze into account. Hi @vincent, thank you for your PR. I was planning to add another PR to catch this `squeeze/unsqueeze` scenario....

LGTM, thanks!

BTW, tested this PR with `lora` and `qlora`, no perf degradation regarding throughput and accuracy.

Yes, this implementation is very slow, I have RTX 2060 and cunn, 1 hour can only train 10 episode .

You can use this to update the gpu and Cuda: `pip install tf-nightly-gpu-2.0-preview==2.0.0.dev20191002`

How long it took for 2000 episodes?

> I used a window size of 20, and found that my model converges and profits the most on my 30th episode. All models after 100 seems to diverge and...