Peng Jiang
Peng Jiang
If the error is "input is too large to process", it may be related to this issue https://github.com/gpustack/gpustack/issues/950. It's not a problem on the GPUStack side, but we plan to...
May be related to https://github.com/vllm-project/vllm/issues/28184
Should be considered together with https://github.com/gpustack/gpustack/issues/3293 https://github.com/gpustack/gpustack/issues/3525
Refer to https://github.com/gpustack/gpustack/issues/814
We use Higress as an embedded AI gateway. Besides Higress, there are many other components and features in V2.
Tried with 0.3.9post2, which should be compatible with vllm v0.9.2 with --no-build-isolation according to the compatibility matrix.
It's a Hygon GPU environment, the framework is DTK, which is compatible with ROCm.
You can add customized vllm backend here: The error "The model contains custom code that must be executed to load correctly. If you trust the source, please pass the backend...
1. For Docker deployment, please ensure you mapped the directory to the GPUStack container with the correct path. 2. If there is a standalone server and worker and the file...
You have several options: 1. Provide an NFS share folder and mount it to the same path on all worker nodes or even the server node. Then all the nodes...