David Xue comments

Results 31 comments of


                                            David Xue

Llama-3 8B Instruct quantized to 8 Bit spits out gibberish in transformers `model.generate()` but works fine in vLLM?

So I feel like may not be related to https://github.com/huggingface/transformers/pull/30380? Since it's already merged? or something else got added that broke things? So far we know transformers 4.38.2, 4.40.1, and...

Llama-3 8B Instruct quantized to 8 Bit spits out gibberish in transformers `model.generate()` but works fine in vLLM?

Yeah so I looked into `nan` logits... the thing is most people are able to get around it by loading in `bfloat16` and that behavior looks normal and correct and...

support gqa

@qwopqwop200 is there help needed on this PR? With the recent Llama 3 release (which all uses gqa), support gqa in GPTQ is becoming more urgent now. I am happy...

Add prompt, response, and messages to Firestore -> Snowflake DAG

I just made a push that fixed the type hinting!

Add prompt, response, and messages to Firestore -> Snowflake DAG

@jlaneve do you want this deployed immediately to backend after merging?

Research:Enhance Testing and Automation for LLM Response Accuracy, accuracy monitoring DAG: automated tests to count docs, test queries, others

Feel free to assign this to me! Happy to take a look

Research:Enhance Testing and Automation for LLM Response Accuracy, accuracy monitoring DAG: automated tests to count docs, test queries, others

### Update on this issue #### Summary Since the inception of RAG-based LLM applications, there's been a constant pursuit for more efficient ways to automate accuracy testing. **Unfortunately, even as...

Research:Enhance Testing and Automation for LLM Response Accuracy, accuracy monitoring DAG: automated tests to count docs, test queries, others

I revised the original issue description and marked the items as completed. I am working on the new issues I have created. At this point I think we can mark...

Uniform kwargs for processors

I can take CLIP and LLaVa

Uniform kwargs for processors

@yonigozlan I am so sorry about the delay. I made some progress locally a while ago, but then completely forgot about this github issue. Will try to get it out...