kang comments

Repositories
Issues
Comments

Results 4 comments of


                                            kang

Questions about the content of the task and the amount of documentation

Could you give me a rough estimate of when that might be?

Questions about the content of the task and the amount of documentation

Thanks for sharing:)

pip install flash-attn always happens ModuleNotFoundError: No module named 'packaging',but actually i have pip install packaging

I experienced the same issue and it seems to be the issue with the order of installation. In my case, I removed `flash-attn` from requirements.txt and ran `pip install -r...

VRAM Requirements for 72b?

I'm serving with 8 a100(80gb) GPUs, and max_token_len=16384. When I put in 5 images in chat template, the vllm server dies. Sometimes it doesn't die right away, but after a...