Benjamin Bartels

@yzh119 Did you get a chance to have a look? :) Happy to help if you can point me in the right direction

> Hi @bbartels we can't reproduce this issue (even inside the container), seems you don't have write permission to `/usr/local/lib/python3.12/dist-packages/`, may I ask how did you set up the entire...

> @bbartels It would be great if there were a small test case (e.g., a small model to test on a single GPU) to reproduce this. I don’t currently have...

Is there some flag that i can enable to produce some debug/trace logs for you that could help in the meantime?

> I tested this on an 8×H200 node but was not able to reproduce the issue in the `vllm/vllm-openai:nightly-0b25498990f01ea2553c02731d6e2ce2d550156a` container on my node with the command below: > > ```...

@yongwww Managed to replicate it outside of our environment: Dockerfile ``` FROM vllm/vllm-openai:v0.11.2 RUN chmod -R a-w /usr/local/lib/python3.12/dist-packages/flashinfer RUN groupadd -g 1000 vllmuser && \ useradd -m -r -u 1000...

For reference, here is the output of the directories it attempts to create (and fails), when the fs perms are not denied: https://gist.github.com/bbartels/24303a25bb12fd8ff3a630a2ecb2dbde

@yongwww Did you get a chance to check with the setup i had posted? Also, this also seems to require JIT at runtime: https://github.com/vllm-project/vllm/issues/29590 even with cubins preinstalled

@yongwww Thanks for the repeated effort, let me try with modified user permissions to see whether it breaks for me. Though I believe it is primarily due to the fact...

Should deepgemm related JIT binaries appear in `flashinfer-jit-cache`/`flashinfer-cubin` as well?