Marc Sun comments

Results 59 comments of


                                            Marc Sun

Llama3 with LlamaForSequenceClassification - Shape mismatch error

Hi @parasurama, is the issue solved ? LMK if I can chose the issue !

KV cache with CPU offloading

> IMO storing the device makes sense. It's also something that could be interesting for @muellerzr and @SunMarc, an accelerate friendly thing to handle that? 🤗 +1 on @ArthurZucker comments....

How can we resume training from lora model?

Thanks for the reproducer @joris-sense ! I'll try to reproduce and fix the issue !

push_to_hub doesn't push checkpoint folder while training

I'll have a look @amyeroberts !

push_to_hub doesn't push checkpoint folder while training

Hi @pacozaa and @kdcyberdude, could you try to pass `hub_always_push = True `in `TrainingArguments` ? It is strange no checkpoint were uploaded at all. When we are already trying to...

Loading GGUF files support

That's right ! The goal of this feature is to let users load their gguf in transformers so that they can fine-tune them before reconverting them to gguf format !

BitsNBytes 4 bit quantization error message typo and logical errors in error message handling

Thx for the reminder ! I've created the PR !

Saving and loading quantized models doesn't work?

To complete @dacorvo answer, we plan to add support via `save_pretrained` and `from_pretrained` for transformers integration. For now, the only way to save the model is by using quanto.

Safetensor serialization throws "conv_in.weight.qtype is invalid expected torch.Tensor but received string"

I have a [draft PR](https://github.com/huggingface/transformers/pull/30049) which enable serializing quanto model with transformers but we need to modify a bit how transformers saves a model. I will see with a core...