i still got an error with read-only file system in nvidia docker container

[connor-tan]

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. using docker image: pytorch/pytorch 2.1.0-cuda12.1-cudnn8-runtime
2. execute "docker run -itd --name test --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES="all" pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime"
3. execute docker-entrypoint.sh in container "test" when container start running
4. execute patch-test.sh

Expected behavior patched successfully

Output listings cu->cuMemAlloc(&data, size) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory [AVHWDeviceContext @ 0x55fc0b49f5c0] cu->cuMemAlloc(&data, size) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory [hwupload @ 0x55fc0bd69780] Failed to allocate frame to upload to. [vf#0:0 @ 0x55fc0b49d980] Error while filtering: Cannot allocate memory Failed to inject frame into filter network: Cannot allocate memory Error while filtering: Cannot allocate memory

Environment (please complete the following information):

• OS: [e.g. Ubuntu 22.04]
• GPU model: [e.g. RTX 3080Ti]
• Patch commit used: master branch ,The latest submission
• Nvidia driver version: [e.g. 535.146.02]

Additional context when i execute patch.sh,shows "./patch.sh: line 373: /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.535.146.02: Read-only file system"

[jailuthra]

Can you make the docker filesystem write-able?

[connor-tan]

Can you make the docker filesystem write-able? this directory "/usr/lib/x86_64-linux-gnu/" is write-able

[jailuthra]

You might have to build a custom image based on top of the cuda one you want to use. Please go through the README:

It is possible to use this patch with nvidia-docker containers, even if host machine hasn't patched drivers. See Dockerfile for example.

Essentially all you need to do during build is:

COPY the patch.sh and docker-entrypoint.sh files into your container.

Make sure docker-entrypoint.sh is invoked on container start.

docker-entrypoint.sh script does on-the-fly patching by means of manipulating dynamic linker to workaround read-only mount of Nvidia runtime. Finally it passes original docker command to shell, like if entrypoint was not restricted by ENTRYPOINT directive. So docker run --runtime=nvidia -it mycontainer echo 123 will print 123. Also it can be just invoked from your entrypoint script, if you have any.