islam-nassar comments

Repositories
Issues
Comments

Results 2 comments of


                                            islam-nassar

SIGABRT - Fatal Python error: Aborted when running vllm on llama2-7b with --tensor-parallel-size 2

Getting the same error when running Yarn-llama2-70B-32k on vLLM 0.2.4 Cuda 11.8, V11.8.89 Nvidia Driver 515.105.01 A100 host with 8 GPU cards python 3.9 vLLM 0.2.4

[Bug]: Disk I/O Error when using tools due to shared outlines cache database

hi @AaronFriel , what is the environment variable to set to change this behaviour. I am getting the same error. Edit: found the solution in your original [issue](https://github.com/vllm-project/vllm/issues/4193)