Jingwei (Chris) Zuo

icon indicating a website link dr-left.github.io

icon company Tsinghua University icon location Beijing, China icon location Endurance is a much better test of character than acts of heroism

Results 3 issues of Jingwei (Chris) Zuo

Pop up a file everytime I run the s3cmd command line

My environment is Windows10. I installed by pip install s3cmd. Now, everytime I type in "s3cmd xxxxxxxx" command line and hit the enter, it will pop up a file, seemed...

Compatibility with `transformers > 4.36`: error: `AttributeError: 'tuple' object has no attribute 'to_legacy_cache'`

2 comment

This is mentioned in the `transformers` library. [https://github.com/huggingface/transformers/issues/28003](url) & [https://github.com/huggingface/transformers/issues/28045](url) I used `transformers==4.43.3` with `tensor_parallel==2.0.0`, and I loaded the `Llama-3.1-8B-Instruct` model. When I am doing inference, there is an error:...

Flashinfer==0.2.0 precision error when tested on vLLM unit tests

I tested flashiner==0.2.0 with vllm unit tests. When **prefilling**, the precision mismatch seems unacceptable (**~0.04 absolute error**), and all tests on prefilling have failed due to tensor mismatch assertion failed....