Forkoz comments

Results 474 comments of


                                            Forkoz

Not able to load Fastchat 3b model

Will it load if you edit the model config and change "is_encoder_decoder": true, to false?

Not able to load Fastchat 3b model

It's loading as the wrong type. This is a seq2seq.

Numpy 1.22.0 and 1.24.3 Problem with TTS 0.22.0 and Librosa 0.10.0 and 0.10.1

Consider editing the deps.

Numpy 1.22.0 and 1.24.3 Problem with TTS 0.22.0 and Librosa 0.10.0 and 0.10.1

I don't remember having issues with 1.23.5 and new TTS. I run XTTS and numpy keeps getting upgraded. I did upgrade librosa to the latest.

Error when using --load-in-8bit: no kernel image is available for execution on the device

It's out.. you need a new DLL: https://github.com/TimDettmers/bitsandbytes/releases/tag/0.37.0

Training model results in ‘MatmulLtState’ object has no attribute ‘memory_efficient_backward’

Which B*B you got installed.. and which PEFT.. I think the problem isn't in this repo.

RuntimeError: Failed to import transformers.models.llama.modeling_llama

flash attention is broken

RuntimeError: Failed to import transformers.models.llama.modeling_llama

Gotta make sure flash attention and torch version match.

will it work with Nvidia P40 24GB on Linux?

Just tested and perf isn't good. 1.x It/s with no context. Maybe there is a way to just stop doing matmul in FP16? For reference this is how autoGPTQ does...

will it work with Nvidia P40 24GB on Linux?

I've got 2 of them so if you need anything tested I can run it. I wish I was getting 160 t/s, but for some reason I'm not on the...