text-generation-inference

text-generation-inference copied to clipboard

System Info

Any environment, as long as you the TGI container with user != 0.

Information

• [x] Docker
• [ ] The CLI directly

Tasks

• [x] An officially supported command
• [ ] My own modifications

Reproduction

Symptom

I cannot run the current TGI image with user different than root

docker run --platform linux/amd64 -it --user 1000 ghcr.io/huggingface/text-generation-inference:3.1.1

fails immediately with the following error message

unable to refresh ld cache, not a big deal in most cases
/usr/local/bin/text-generation-launcher: error while loading shared libraries: libpython3.11.so.1.0: cannot open shared object file: No such file or directory

Error analysis

• The python library which cannot be found is installed in /root/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/
• The directory /root/ is not readable for non-root users.
• I suspect that the problem has been introduced with #2919 because release v3.0.1 still works.

Expected behavior

I expect to be able to run TGI with non-root users, e.g. on Kubernetes. This was possible previously.