google
Run docker in gVisor with GPU
Description
gVisor allows user to run normal docker workflows inside gVisor sandbox as described at https://gvisor.dev/docs/tutorials/docker-in-gvisor/.
At the same time, users can run AI/ML workflow with docker using GPU (not sandboxed) like https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/sample-workload.html
I would like to gauge interests/demand on running docker workflows using GPU inside gVisor sandbox. Please feel free to let us know what you think about the idea.
Is this feature related to a specific bug?
n/a
Do you have a specific solution in mind?
No response
I think this would be great, but I'm more interested in desktop workloads than server ones.
I was looking into using gVisor to run fully isolated local LLM's for agentic software development. Would be a critical feature for that use case?
@mikeball gVisor already has GPU support: https://gvisor.dev/docs/user_guide/gpu/; so you could spawn gVisor containers which run your LLMs.
I believe this issue is about running the Docker daemon inside a gVisor container and using that inner docker to spawn GPU containers inside of gVisor container (nested GPU access).
@mikeball I can confirm that at least Ollama and vLLM work fine in gVisor with GPU support, without nested Docker. If you find incompatibilities, please report them as separate issues, as this type of workload is intended to be well-supported.
