lhs1012

Issue #2244 fixed. In case where a port other than the default port (9000) is used, '-port' option is also added to the command line options.

[Bug]: Converting gguf to state_dict

11

### Your current environment ```text PyTorch version: 2.2.0+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A OS: Ubuntu 22.04.4 LTS (x86_64) GCC...

[Bug]: torch._dynamo.exc.BackendCompilerFailed with command-r-plus

3

### Your current environment aphrodite docker container Setting 1 GPUs: RTX8000 * 2 model: alpindale/c4ai-command-r-plus-GPTQ Quantization: gptq Setting 2 GPUs: A6000 ada * 4 model: CohereForAI/c4ai-command-r-plus Quantization: load-in-smooth ### 🐛...

[Bug]: vllm.engine.async_llm_engine.AsyncEngineDeadError: Background loop has errored already.

31

### Your current environment docker image: vllm/vllm-openai:0.4.2 Model: https://huggingface.co/alpindale/c4ai-command-r-plus-GPTQ GPUs: RTX8000 * 2 ### 🐛 Describe the bug The model works fine until the following error is raised. ------------------------------------------------------- INFO...