weijie

Generator Error when evaluating GLUE and superGLUE

8

The error information shows that:  And the corresponding code is:  I guess the error was caused by the parameter , which is a generator, and that was assigned...

Inconsistent evaluation results with Chat Template

4

I evaluated llama3-8b-Instruct using the gsm8k benchmark, and found some interesting phenomenons. 1. Huggingface and vllm has similar results  2. If I use vllm to start an API service,...