David Xue

icon company Pinterest icon location New York, NY icon location Open source contributor to transformers & GPTQModel || Machine Learning Engineer @ Pinterest. Prev @ HiveAI, Meta & Amazon

Results 31 comments of David Xue

Why LLaMA3-8B after GPTQ test in wikitext2 so bad?

https://github.com/AutoGPTQ/AutoGPTQ/issues/657 this maybe related

Community contribution: enable dynamic resolution input for more vision models.

I can work on [chinese_clip](https://github.com/huggingface/transformers/blob/main/src/transformers/models/chinese_clip/modeling_chinese_clip.py). Will keep the team posted in the next few days. If I get more free time and there are remaining ones by then, happy to...

Community contribution: enable dynamic resolution input for more vision models.

Actually, I can also take [bridgetower](https://github.com/huggingface/transformers/blob/main/src/transformers/models/bridgetower/modeling_bridgetower.py) as well. They will come in as separate PRs though. Shouldn't be more complicated than chinese_clip. So recap: I will work on both bridgetower...

Community contribution: enable dynamic resolution input for more vision models.

@nileshkokane01 @amyeroberts In that case, I will refrain from working on `chinese_clip` and `bridgetower` since both have `# Copied from transformers.models.clip.modeling_clip.CLIPVisionEmbeddings with CLIP` in the comments. I think Kosmos 2...

Community contribution: enable dynamic resolution input for more vision models.

I can take `siglip`. I think some functions are still copied from CLIP but just skimming it, doesn't seem like they will be related to interpolate position encoding code

Community contribution: enable dynamic resolution input for more vision models.

Completed the PR #30719. I actually realized: Should I be referencing this issue directly in my PR? Because if any of our PRs merge then it may end up closing...

[Feature]: GPTQ/AWQ quantization is not fully optimized yet. The speed can be slower than non-quantized models.

Would this have support for 8 bit quants as well or just 4 bit?

[Feature]: GPTQ/AWQ quantization is not fully optimized yet. The speed can be slower than non-quantized models.

Awesome!! Thank you guys for the hard work!

[Feature]: GPTQ/AWQ quantization is not fully optimized yet. The speed can be slower than non-quantized models.

Thank you!!

Llama-3 8B Instruct quantized to 8 Bit spits out gibberish in transformers `model.generate()` but works fine in vLLM?

@Qubitium I tried 4.40.1, it has the same problem. I am also already installing directly from the `transformers` github repo. (4.41.0.dev0) And to add to the above. I get gibberish...