docker-env-setup manual not work

[peng-yq]

Additional Information

Providing the following details is crucial to help us assist you effectively. Missing information may delay your support request or result in it being overlooked.

Issue Details

I attempted to install Nydus utilities using Docker by following the instructions in docs/docker-env-setup.md. However, I encountered several issues, likely due to the documentation being outdated. Most errors are related to files not being found.

1. Downloading the Nydus Snapshotter Release Tarball

   # Get the latest version. If this version does not work for you, you can try v0.6.0
   TAG=`curl -s https://api.github.com/repos/containerd/nydus-snapshotter/releases/latest | grep tag_name | cut -f4 -d "\""`
   wget https://github.com/containerd/nydus-snapshotter/releases/download/"$TAG"/nydus-snapshotter-"$TAG"-x86_64.tgz
   tar -xzvf nydus-snapshotter-"$TAG"-x86_64.tgz
   sudo install -D -m 755 nydus-snapshotter/containerd-nydus-grpc /usr/local/bin
   
   wget -O /etc/nydus/nydusd-config.json https://raw.githubusercontent.com/containerd/nydus-snapshotter/"$TAG"/misc/snapshotter/nydusd-config.fusedev.json
   wget -O /etc/nydus/config.toml https://raw.githubusercontent.com/containerd/nydus-snapshotter/"$TAG"/misc/snapshotter/config.toml
   

   • The file name nydus-snapshotter-"$TAG"-x86_64.tgz is incorrect; it should be nydus-snapshotter-"$TAG"-linux-amd64.tar.gz.
   • Extracting the tarball using tar -xvf results in a bin directory, which doesn't align with the subsequent commands and requires adjustments.
   • The commands to download configuration files fail with "/etc/nydus/: No such file or directory". The directory isn't created by running sudo install -D -m 755 nydus-snapshotter/containerd-nydus-grpc /usr/local/bin. It seems the base configuration isn't set up during installation.

2. Fetching the Latest Version of DragonflyOSS Image Service

   # Get the latest version. If this version does not work for you, you can try v2.1.4
   
   TAG=`curl -s https://api.github.com/repos/dragonflyoss/image-service/releases/latest | grep tag_name | cut -f4 -d "\""`
   wget https://github.com/dragonflyoss/image-service/releases/download/"$TAG"/nydus-static-"$TAG"-linux-amd64.tgz
   tar -xzvf nydus-static-"$TAG"-linux-amd64.tgz
   sudo install -D -m 755 nydus-static/* /usr/local/bin
   

   • The command to fetch the latest version tag (TAG=) does not retrieve the tag, preventing subsequent commands from executing.

I have only reached this point in the installation process, so I am unsure if additional issues exist. I recommend the author review the documentation, as it hasn't been updated in a while.

[imeoer]

Sorry for these outdated documents about docker env, if you're interested, could you help submit a PR to improve them, thanks!

For the point 2: the release URL should be changed to https://api.github.com/repos/dragonflyoss/nydus/releases/latest.

[peng-yq]

Sorry for these outdated documents about docker env, if you're interested, could you help submit a PR to improve them, thanks!

For the point 2: the release URL should be changed to https://api.github.com/repos/dragonflyoss/nydus/releases/latest.

Thanks for your reply, I'll work for this issue/bug :)

[peng-yq]

Sorry for these outdated documents about docker env, if you're interested, could you help submit a PR to improve them, thanks!

For the point 2: the release URL should be changed to https://api.github.com/repos/dragonflyoss/nydus/releases/latest.

I thinkI have already resolved the issues prior to step 8, "Run nydus image in docker," and have prepared the environment as instructed. However, when I execute sudo docker run -d --restart=always -p 5000:5000 registry, I encounter the following error:

pyq@ubuntu:~$ sudo docker run -d --restart=always -p 5000:5000 registry
Unable to find image 'registry:latest' locally
docker: Error response from daemon: failed to resolve reference "docker.io/library/registry:latest": failed to do request: Head "https://dockerhub.icu/v2/library/registry/manifests/latest?ns=docker.io": EOF.
See 'docker run --help'.
pyq@ubuntu:~$ sudo docker run -d --restart=always -p 5000:5000 registry
Unable to find image 'registry:latest' locally
latest: Pulling from library/registry
docker: failed to prepare extraction snapshot "extract-714822882-ygRP sha256:d62a02444d39e0408f390b5531241920618333b50d589215edb5e90a4d24f457": Unavailable: connection error: desc = "transport: Error while dialing: dial unix:///run/containerd-nydus/containerd-nydus-grpc.sock: timeout": unavailable.
See 'docker run --help'.
pyq@ubuntu:~$ ls -l /run/containerd-nydus/containerd-nydus-grpc.sock
ls: cannot access '/run/containerd-nydus/containerd-nydus-grpc.sock': No such file or directory
It seems the issue is due to the non-existence of the /run/containerd-nydus/containerd-nydus-grpc.sock file. I'm unsure where the problem lies.

1. Does switching to Nydus affect image pulling? I've tried several image proxies without success, but this might be due to the missing socket file.
2. Additionally, I noticed that after enabling Nydus, the images previously visible with docker images are no longer there. Is this because of the change in the storage system?
3. Lastly, if I want to revert the configuration, can I simply comment out the relevant settings in the Docker and containerd configuration files?

I thinkI have addressed the documentation bug. And which branch I should submit a PR to? Looking forward to your response.

[peng-yq]

Sorry for these outdated documents about docker env, if you're interested, could you help submit a PR to improve them, thanks! For the point 2: the release URL should be changed to https://api.github.com/repos/dragonflyoss/nydus/releases/latest.

I thinkI have already resolved the issues prior to step 8, "Run nydus image in docker," and have prepared the environment as instructed. However, when I execute sudo docker run -d --restart=always -p 5000:5000 registry, I encounter the following error:

pyq@ubuntu:~$ sudo docker run -d --restart=always -p 5000:5000 registry
Unable to find image 'registry:latest' locally
docker: Error response from daemon: failed to resolve reference "docker.io/library/registry:latest": failed to do request: Head "https://dockerhub.icu/v2/library/registry/manifests/latest?ns=docker.io": EOF.
See 'docker run --help'.
pyq@ubuntu:~$ sudo docker run -d --restart=always -p 5000:5000 registry
Unable to find image 'registry:latest' locally
latest: Pulling from library/registry
docker: failed to prepare extraction snapshot "extract-714822882-ygRP sha256:d62a02444d39e0408f390b5531241920618333b50d589215edb5e90a4d24f457": Unavailable: connection error: desc = "transport: Error while dialing: dial unix:///run/containerd-nydus/containerd-nydus-grpc.sock: timeout": unavailable.
See 'docker run --help'.
pyq@ubuntu:~$ ls -l /run/containerd-nydus/containerd-nydus-grpc.sock
ls: cannot access '/run/containerd-nydus/containerd-nydus-grpc.sock': No such file or directory
It seems the issue is due to the non-existence of the /run/containerd-nydus/containerd-nydus-grpc.sock file. I'm unsure where the problem lies.

1. Does switching to Nydus affect image pulling? I've tried several image proxies without success, but this might be due to the missing socket file.
2. Additionally, I noticed that after enabling Nydus, the images previously visible with docker images are no longer there. Is this because of the change in the storage system?
3. Lastly, if I want to revert the configuration, can I simply comment out the relevant settings in the Docker and containerd configuration files?

I thinkI have addressed the documentation bug. And which branch I should submit a PR to? Looking forward to your response.

I just realized that the main branch of the documentation was fixed by you four months ago. The only change of that branch needed might be to replace install with cp and chmod in step three, as install does not support recursive calls and causes errors during execution. And I was referencing the stable2.2 branch, lol.

[imeoer]

@peng-yq Thanks for the feedback.

For 1: Yes. It seems that the snapshotter (listen on /run/containerd-nydus/containerd-nydus-grpc.sock) did not start successfully, could you check the logs of snapshotter? There might be some errors.

For 2: Should be yes, switching snapshotter means change the docker image storage system.

For 3: Should be yes.

[peng-yq]

@peng-yq Thanks for the feedback.

For 1: Yes. It seems that the snapshotter (listen on /run/containerd-nydus/containerd-nydus-grpc.sock) did not start successfully, could you check the logs of snapshotter? There might be some errors.

For 2: Should be yes, switching snapshotter means change the docker image storage system.

For 3: Should be yes.

Thank you for your response, and apologies for the delay due to being busy. It seems the issue is with the snapshotter. I checked the logs, but there doesn't seem to be any useful information.

pyq@ubuntu:~/Downloads/fuse-3.16.2$ sudo systemctl status nydus-snapshotter
× nydus-snapshotter.service - nydus snapshotter
     Loaded: loaded (/etc/systemd/system/nydus-snapshotter.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2024-10-28 10:34:26 CST; 3h 55min ago
   Main PID: 1755 (code=exited, status=1/FAILURE)
        CPU: 45ms

10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 5.
10月 28 10:34:26 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Start request repeated too quickly.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:26 ubuntu systemd[1]: Failed to start nydus snapshotter.

# below is the journal of snapshotter
10月 28 10:34:19 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:20 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:20 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:21 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 1.
10月 28 10:34:21 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:21 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:21 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:21 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:22 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 2.
10月 28 10:34:22 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:22 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:22 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:22 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:23 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 3.
10月 28 10:34:23 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:23 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:23 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:23 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:24 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 4.
10月 28 10:34:24 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:24 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:24 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:24 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 5.
10月 28 10:34:26 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Start request repeated too quickly.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:26 ubuntu systemd[1]: Failed to start nydus snapshotter.

I suspect the issue might be that the FUSE module isn't loaded into the kernel. I've tried installing FUSE via apt and loading it into the kernel according to the manual, but it's not working. Do I need to compile it from source to resolve this or something? Here’s my device information:

pyq@ubuntu:~/Downloads/fuse-3.16.2$ sudo apt-get install fuse
sudo modprobe fuse
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
fuse is already the newest version (2.9.9-5ubuntu3).
The following packages were automatically installed and are no longer required:
  apg ethtool gnome-control-center-faces gnome-online-accounts libcolord-gtk1 libfreerdp-client2-2 libfreerdp-server2-2 libfreerdp2-2 libgnome-bg-4-1 libgsound0 libgssdp-1.2-0 libgupnp-1.2-1 libgupnp-av-1.0-3
  libgupnp-dlna-2.0-4 libmediaart-2.0-0 libmspack0 libntfs-3g89 librygel-core-2.6-2 librygel-db-2.6-2 librygel-renderer-2.6-2 librygel-server-2.6-2 libvncserver1 libwinpr2-2 libxmlsec1 libxmlsec1-openssl
  mobile-broadband-provider-info network-manager-gnome open-vm-tools python3-certifi python3-idna python3-macaroonbakery python3-nacl python3-protobuf python3-pymacaroons python3-requests python3-rfc3339
  python3-tz python3-urllib3 rygel zerofree
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
pyq@ubuntu:~/Downloads/fuse-3.16.2$ lsmod | grep fuse
pyq@ubuntu:~/Downloads/fuse-3.16.2$ uname -a
Linux ubuntu 6.8.0-47-generic #47~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Oct  2 16:16:55 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

[peng-yq]

@peng-yq Thanks for the feedback. For 1: Yes. It seems that the snapshotter (listen on /run/containerd-nydus/containerd-nydus-grpc.sock) did not start successfully, could you check the logs of snapshotter? There might be some errors. For 2: Should be yes, switching snapshotter means change the docker image storage system. For 3: Should be yes.

Thank you for your response, and apologies for the delay due to being busy. It seems the issue is with the snapshotter. I checked the logs, but there doesn't seem to be any useful information.

pyq@ubuntu:~/Downloads/fuse-3.16.2$ sudo systemctl status nydus-snapshotter
× nydus-snapshotter.service - nydus snapshotter
     Loaded: loaded (/etc/systemd/system/nydus-snapshotter.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2024-10-28 10:34:26 CST; 3h 55min ago
   Main PID: 1755 (code=exited, status=1/FAILURE)
        CPU: 45ms

10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 5.
10月 28 10:34:26 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Start request repeated too quickly.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:26 ubuntu systemd[1]: Failed to start nydus snapshotter.

# below is the journal of snapshotter
10月 28 10:34:19 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:20 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:20 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:21 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 1.
10月 28 10:34:21 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:21 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:21 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:21 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:22 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 2.
10月 28 10:34:22 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:22 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:22 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:22 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:23 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 3.
10月 28 10:34:23 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:23 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:23 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:23 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:24 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 4.
10月 28 10:34:24 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:24 ubuntu systemd[1]: Started nydus snapshotter.
10月 28 10:34:24 ubuntu systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
10月 28 10:34:24 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Scheduled restart job, restart counter is at 5.
10月 28 10:34:26 ubuntu systemd[1]: Stopped nydus snapshotter.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Start request repeated too quickly.
10月 28 10:34:26 ubuntu systemd[1]: nydus-snapshotter.service: Failed with result 'exit-code'.
10月 28 10:34:26 ubuntu systemd[1]: Failed to start nydus snapshotter.

I suspect the issue might be that the FUSE module isn't loaded into the kernel. I've tried installing FUSE via apt and loading it into the kernel according to the manual, but it's not working. Do I need to compile it from source to resolve this or something? Here’s my device information:

pyq@ubuntu:~/Downloads/fuse-3.16.2$ sudo apt-get install fuse
sudo modprobe fuse
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
fuse is already the newest version (2.9.9-5ubuntu3).
The following packages were automatically installed and are no longer required:
  apg ethtool gnome-control-center-faces gnome-online-accounts libcolord-gtk1 libfreerdp-client2-2 libfreerdp-server2-2 libfreerdp2-2 libgnome-bg-4-1 libgsound0 libgssdp-1.2-0 libgupnp-1.2-1 libgupnp-av-1.0-3
  libgupnp-dlna-2.0-4 libmediaart-2.0-0 libmspack0 libntfs-3g89 librygel-core-2.6-2 librygel-db-2.6-2 librygel-renderer-2.6-2 librygel-server-2.6-2 libvncserver1 libwinpr2-2 libxmlsec1 libxmlsec1-openssl
  mobile-broadband-provider-info network-manager-gnome open-vm-tools python3-certifi python3-idna python3-macaroonbakery python3-nacl python3-protobuf python3-pymacaroons python3-requests python3-rfc3339
  python3-tz python3-urllib3 rygel zerofree
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
pyq@ubuntu:~/Downloads/fuse-3.16.2$ lsmod | grep fuse
pyq@ubuntu:~/Downloads/fuse-3.16.2$ uname -a
Linux ubuntu 6.8.0-47-generic #47~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Oct  2 16:16:55 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

I have already resolved the issue! It turns out the problem was due to an error in the docker-env-setup documentation from the stable/v2.2 branch, not the FUSE module. It was similar to the issue described in #1481, and the fix was applied in #1586, but only updated in the master branch.

[peng-yq]

@imeoer ,I have already submit a PR to brunch stable v2.2 #1642 .