wsl-distrod
wsl-distrod copied to clipboard
nullpo-head
WSL2 crashes at launch after enabling distrod `[Distrod][ERROR] Failed to spawn the command.`
Here is the error when launching wsl2
Caused by:
No such file or directory (os error 2)
[Distrod][ERROR] Pre_second_fork failed.
Caused by:
0: Failed to enter the init's namespace
1: Failed to open ns/uts
2: ESRCH: No such process
[プロセスはコード 137 (0x00000089) で終了しました]
my distro sort of becomes borked after that.. the only way to get back in was to use wsl --exec to get back in and disable distrod
I am on archlinux and I was using systemd-genie (I uninstalled it to use this) would somebody know how to fix this ?
Thank you very much
Hmm.. It looks systemd crashed for some reason. Possibly due to some Genie's initialization script which wasn't completely removed, but I'm not sure, I'm not so familiar with genie. Or, it could be just a bug of Distrod.
Could you enable the logging of both log_level and kmsg_log_level of Distrod following the doc please? After that, instead of enabling Distrod, run it as a one-shot command by the following commands.
/opt/distrod/bin/distrod-exec /bin/bash bash
It should crash in the same way as before, but with debug logging this time.
Could you share the log of /dev/kmsg then, please? This time I'd like to see the systemd's log as well, so please don't filter the log with Distrod:
sudo cat /dev/kmsg > kmsg_log.txt # this doesn't return. Press ctrl + c after a few seconds
Thank you for the reply ! I have gathered the logs, looks like some mount problem (???) I don't really understand though.. Thank you very much for your help !
[Distrod][DEBUG] starting /init from distrod-exec
[Distrod][DEBUG] WSL envs: "WSLENV" = "WT_SESSION::WT_PROFILE_ID"
[Distrod][DEBUG] WSL envs: "WSL_DISTRO_NAME" = "Archlinux"
[Distrod][DEBUG] WSL envs: "WSL_INTEROP" = "/run/WSL/8_interop"
[Distrod][DEBUG] Container::with_mount source: Some(HostPath("/run/distrod/cmdline")), target: ContainerPath("/proc/cmdline"), fstype: None, flags: MS_BIND, is_file: true
[Distrod][TRACE] mount_distrod_run_files: path: "/opt/distrod/run/systemd"
[Distrod][TRACE] mount_distrod_run_files: path: "/opt/distrod/run/systemd/system"
[Distrod][TRACE] mount_distrod_run_files: path: "/opt/distrod/run/systemd/system/portproxy.service"
[Distrod][DEBUG] Container::with_mount source: Some(HostPath("/opt/distrod/run/systemd/system/portproxy.service")), target: ContainerPath("/run/systemd/system/portproxy.service"), fstype: None, flags: MS_BIND, is_file: true
[Distrod][TRACE] mount_distrod_run_files: path: "/opt/distrod/run/tmpfiles.d"
[Distrod][TRACE] mount_distrod_run_files: path: "/opt/distrod/run/tmpfiles.d/x11.conf"
[Distrod][DEBUG] Container::with_mount source: Some(HostPath("/opt/distrod/run/tmpfiles.d/x11.conf")), target: ContainerPath("/run/tmpfiles.d/x11.conf"), fstype: None, flags: MS_BIND, is_file: true
[Distrod][DEBUG] DistroLauncher::launch
[Distrod][DEBUG] Container::with_mount source: Some(HostPath("/run/distrod/distrod_wsl_env-uid0")), target: ContainerPath("/run/distrod/distrod_wsl_env-uid0"), fstype: None, flags: MS_BIND, is_file: true
[Distrod][DEBUG] Spawning the command or the waiter.
[Distrod][DEBUG] Executing a command in the distro.
[Distrod][TRACE] mounting source: Some(
ContainerPath(
"/run/distrod/cmdline",
),
), mount: ContainerMount { source: Some(HostPath("/run/distrod/cmdline")), target: ContainerPath("/proc/cmdline"), fstype: None, flags: MS_BIND, data: None, is_file: true }
[Distrod][DEBUG] Failed to ignore signal Sys(EINVAL)
[Distrod][TRACE] mounting source: Some(
ContainerPath(
"/opt/distrod/run/systemd/system/portproxy.service",
),
), mount: ContainerMount { source: Some(HostPath("/opt/distrod/run/systemd/system/portproxy.service")), target: ContainerPath("/run/systemd/system/portproxy.service"), fstype: None, flags: MS_BIND, data: None, is_file: true }
[Distrod][DEBUG] Failed to ignore signal Sys(EINVAL)
[Distrod][DEBUG] Distro::exec_command.
[Distrod][DEBUG] Container::exec_command.
[Distrod][TRACE] mounting source: Some(
ContainerPath(
"/opt/distrod/run/tmpfiles.d/x11.conf",
),
), mount: ContainerMount { source: Some(HostPath("/opt/distrod/run/tmpfiles.d/x11.conf")), target: ContainerPath("/run/tmpfiles.d/x11.conf"), fstype: None, flags: MS_BIND, data: None, is_file: true }
[Distrod][TRACE] skipping an identical mount: Some(
ContainerPath(
"/run/distrod/distrod_wsl_env-uid0",
),
), ContainerMount {
source: Some(
HostPath(
"/run/distrod/distrod_wsl_env-uid0",
),
),
target: ContainerPath(
"/run/distrod/distrod_wsl_env-uid0",
),
fstype: None,
flags: MS_BIND,
data: None,
is_file: true,
}
[Distrod][DEBUG] Triple fork done.
[Distrod][ERROR] Pre_second_fork failed.
Caused by:
0: Failed to enter the init's namespace
1: Failed to open ns/mnt
2: ENOENT: No such file or directory
[Distrod][ERROR] Failed to spawn the command.
Caused by:
No such file or directory (os error 2)
[Distrod][DEBUG] The pipe for wait has been closed. Possibly the proxy process has been killed by SIGKILL.
9,333,23059002104,-;Distrod: [DEBUG] starting /init from distrod-exec
9,334,23059002432,-;Distrod: [DEBUG] WSL envs: "WSLENV" = "WT_SESSION::WT_PROFILE_ID"
9,335,23059002443,-;Distrod: [DEBUG] WSL envs: "WSL_DISTRO_NAME" = "Archlinux"
9,336,23059002464,-;Distrod: [DEBUG] WSL envs: "WSL_INTEROP" = "/run/WSL/8_interop"
9,337,23059002752,-;Distrod: [DEBUG] Container::with_mount source: Some(HostPath("/run/distrod/cmdline")), target: ContainerPath("/proc/cmdline"), fstype: None, flags: MS_BIND, is_file: true
9,338,23059002882,-;Distrod: [TRACE] mount_distrod_run_files: path: "/opt/distrod/run/systemd"
9,339,23059002919,-;Distrod: [TRACE] mount_distrod_run_files: path: "/opt/distrod/run/systemd/system"
9,340,23059002951,-;Distrod: [TRACE] mount_distrod_run_files: path: "/opt/distrod/run/systemd/system/portproxy.service"
9,341,23059002969,-;Distrod: [DEBUG] Container::with_mount source: Some(HostPath("/opt/distrod/run/systemd/system/portproxy.service")), target: ContainerPath("/run/systemd/system/portproxy.service"), fstype: None, flags: MS_BIND, is_file: true
9,342,23059003003,-;Distrod: [TRACE] mount_distrod_run_files: path: "/opt/distrod/run/tmpfiles.d"
9,343,23059003031,-;Distrod: [TRACE] mount_distrod_run_files: path: "/opt/distrod/run/tmpfiles.d/x11.conf"
9,344,23059003060,-;Distrod: [DEBUG] Container::with_mount source: Some(HostPath("/opt/distrod/run/tmpfiles.d/x11.conf")), target: ContainerPath("/run/tmpfiles.d/x11.conf"), fstype: None, flags: MS_BIND, is_file: true
9,345,23059003103,-;Distrod: [DEBUG] DistroLauncher::launch
9,346,23059003184,-;Distrod: [DEBUG] Container::with_mount source: Some(HostPath("/run/distrod/distrod_wsl_env-uid0")), target: ContainerPath("/run/distrod/distrod_wsl_env-uid0"), fstype: None, flags: MS_BIND, is_file: true
9,347,23059003775,-;Distrod: [DEBUG] Spawning the command or the waiter.
9,348,23059004154,-;Distrod: [DEBUG] Executing a command in the distro.
9,349,23059004161,-;Distrod: [TRACE] mounting source: Some(\x0a ContainerPath(\x0a "/run/distrod/cmdline",\x0a ),\x0a), mount: ContainerMount { source: Some(HostPath("/run/distrod/cmdline")), target: ContainerPath("/proc/cmdline"), fstype: None, flags: MS_BIND, data: None, is_file: true }
9,350,23059004190,-;Distrod: [DEBUG] Failed to ignore signal Sys(EINVAL)
9,351,23059004224,-;Distrod: [DEBUG] Failed to ignore signal Sys(EINVAL)
9,352,23059004232,-;Distrod: [DEBUG] Distro::exec_command.
9,353,23059004239,-;Distrod: [TRACE] mounting source: Some(\x0a ContainerPath(\x0a "/opt/distrod/run/systemd/system/portproxy.service",\x0a ),\x0a), mount: ContainerMount { source: Some(HostPath("/opt/distrod/run/systemd/system/portproxy.service")), target: ContainerPath("/run/systemd/system/portproxy.service"), fstype: None, flags: MS_BIND, data: None, is_file: true }
9,354,23059004261,-;Distrod: [DEBUG] Container::exec_command.
9,355,23059004278,-;Distrod: [TRACE] mounting source: Some(\x0a ContainerPath(\x0a "/opt/distrod/run/tmpfiles.d/x11.conf",\x0a ),\x0a), mount: ContainerMount { source: Some(HostPath("/opt/distrod/run/tmpfiles.d/x11.conf")), target: ContainerPath("/run/tmpfiles.d/x11.conf"), fstype: None, flags: MS_BIND, data: None, is_file: true }
9,356,23059004320,-;Distrod: [TRACE] skipping an identical mount: Some(\x0a ContainerPath(\x0a "/run/distrod/distrod_wsl_env-uid0",\x0a ),\x0a), ContainerMount {\x0a source: Some(\x0a HostPath(\x0a "/run/distrod/distrod_wsl_env-uid0",\x0a ),\x0a ),\x0a target: ContainerPath(\x0a "/run/distrod/distrod_wsl_env-uid0",\x0a ),\x0a fstype: None,\x0a flags: MS_BIND,\x0a data: None,\x0a is_file: true,\x0a}
9,357,23059004367,-;Distrod: [DEBUG] Triple fork done.
9,358,23059004565,-;Distrod: [ERROR] Pre_second_fork failed.\x0a\x0aCaused by:\x0a 0: Failed to enter the init's namespace\x0a 1: Failed to open ns/mnt\x0a 2: ENOENT: No such file or directory
9,359,23059004610,-;Distrod: [DEBUG] The pipe for wait has been closed. Possibly the proxy process has been killed by SIGKILL.
9,360,23059004613,-;Distrod: [ERROR] Failed to spawn the command.\x0a\x0aCaused by:\x0a No such file or directory (os error 2)
12,361,23118825225,-;WSL2: Performing memory compaction.
Hi, I cannot find anything wrong in your log. I will check if I can reproduce the problem by installing / uninstalling Genie later.
Thank you checking, do you know where it failed so I can try to work around it ?
There are two possibilities as far as I can guess.
- Systemd is crashing immediately after it is launched.
- In this case, I suspect Genie's some systemd generator script is causing crash, because it should assume that Genie is running when it detects systemd running.
- Systemd doesn't crash, but the file
/proc/[pid(e.g: 1)]/ns/mntactually doesn't exist for some reason.- In this case, we must investigate the root cause.
In both cases, there is no workaround. What I can suggest as a workaround is to install a clean new distro by Distrod.
To check whether Systemd crashes or not, could you share the result of ps aux after Distrod crashes, by wsl -e ps aux command? The name of the systemd process is like /sbin/init ....setenv=...
it systemd exists, could you share the output of wsl -e ls /proc/[pid of the systemd you found]/ns as well, please?
I seem to be having the exact same problem running the fedora 35 image from linuxcontainers with distrod.
Here's the output of wsl -e ls /proc/[pid of the systemd you found]/ns:
PS C:\Users\user> wsl -d Distrod -e ps faux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 8944 332 ? Ssl 15:38 0:00 /init
root 7 0.0 0.0 8944 228 tty1 Ss 15:38 0:00 /init
user+ 8 0.0 0.0 233576 1828 tty1 R 15:38 0:00 \_ ps faux
PS C:\Users\user> wsl -d Distrod -e sudo ls /proc/1/ns
ipc mnt net pid user uts
PS C:\Users\user> wsl -d Distrod -e sudo ls /proc/7/ns
ls: cannot access '/proc/7/ns': No such file or directory
There doesn't seem to be any parameters passed to /init though.
@oDn thanks for sharing the outputs! Could you also try wsl -e uname -a, please? I suspect your Linux kernel is too old, or running WSL1, and thus Systemd crashed.
What I can tell from your outputs.
-
/initis not systemd, but WSL's init process, and there's no/sbin/init. That means Systemd crashed. -
You can see> wsl -d Distrod -e sudo ls /proc/1/ns ipc mnt net pid user utspid_for_childrenis missing. This indicates that your WSL kernel is older than 4.12, or your running WSL1.
@nullpo-head, I think you're spot on !
PS C:\Users\user> wsl --list --verbose
NAME STATE VERSION
* Ubuntu Running 2
distrod-fedora Stopped 1
I was sure I'd set WSL's default version to 2 a while ago but that obviously wasn't the case.
PS C:\Users\user> wsl --set-default-version 2
For information on key differences with WSL 2 please visit https://aka.ms/wsl2
The operation completed successfully.
PS C:\Users\user> wsl --unregister distrod-fedora
Unregistering...
Now that I've created a new VM using WSL 2, everything seems to be running fine !
PS C:\Users\user> wsl --list --verbose
NAME STATE VERSION
* Ubuntu Running 2
distrod-fedora Running 2
[user@hostname distrod_wsl_launcher-x86_64]$ systemctl status
● hostname
State: running
Jobs: 0 queued
Failed: 0 units
Since: Mon 2021-11-29 16:18:33 CET; 31s ago
CGroup: /
├─init.scope
│ └─1 /sbin/init systemd.setenv=WSLENV=WT_SESSION::WT_PROFILE_ID systemd.setenv=WSL_INTEROP=/run/WSL/25_interop systemd.setenv=WSL_DISTRO_NAME=distrod-fedora --unit=multi-user.target
└─system.slice
├─systemd-udevd.service
│ └─66 /usr/lib/systemd/systemd-udevd
├─dbus-broker.service
│ ├─78 /usr/bin/dbus-broker-launch --scope system --audit
│ └─91 dbus-broker --log 4 --controller 9 --machine-id f79933900b0e4a7bb4ec85e6727bf35a --max-bytes 536870912 --max-fds 4096 --max-matches 16384 --audit
├─systemd-homed.service
│ └─75 /usr/lib/systemd/systemd-homed
├─systemd-journald.service
│ └─54 /usr/lib/systemd/systemd-journald
└─systemd-logind.service
└─76 /usr/lib/systemd/systemd-logind
Thanks !
By the way, as for the original issue reporter, @eizanprime , it's unlikely the cause is WSL1, since they used genie once, which doesn't run on WSL1. I'm still waiting for their comments.
I am finally back, sorry for being late (was a bit busy these days, I'm actually work in tokyo too)
here is the output of the ps command, the process doesn't look there..
root 1 1.3 0.0 896 528 ? Sl 18:20 0:00 /init
root 16 0.0 0.0 896 84 ? Ss 18:20 0:00 /init
root 17 0.0 0.0 896 84 ? R 18:20 0:00 /init
eizanpr+ 18 0.0 0.0 10292 3396 pts/0 Rs+ 18:20 0:00 ps aux
I would recreate a new distro if I could but my current system is quite customized and I don't really want to restart from scratch (as everything is vmdk in wsl2 I suppose a good old dd on the disk would not work so well)
In attempting to bring up a Fedora 36 or Rawhide container without first upgrading the 35 base from linuxcontainers.org (which BTW puts systemd into a degraded state), I'm seeing the same issue with images from: https://koji.fedoraproject.org/koji/packageinfo?packageID=26387
After extracting layer.tar from the tar.xz and performing the following steps:
> wsl --import rawhide C:\Temp\wslTemp C:\Users\yourusername\Downloads\fedora-rawhide.tar
> wsl -d rawhide
$ dnf update -y && dnf install passwd sudo -y
$ adduser -G wheel yourusername
$ echo -e "[user]\ndefault=yourusername" >> /etc/wsl.conf
$ passwd yourusername
$ exit
> wsl -t rawhide
> wsl -d rawhide
$ curl -L -O "https://raw.githubusercontent.com/nullpo-head/wsl-distrod/main/install.sh"
$ chmod +x install.sh
$ sudo ./install.sh install
I get symlink errors after enabling distrod:
$ sudo /opt/distrod/bin/distrod enable
[Distrod][WARN] Faled to mask systemd-remount-fs.service. Error: Failed to symlink '"/etc/systemd/system/systemd-remount-fs.service"'.
Caused by:
No such file or directory (os error 2)
[Distrod][WARN] Faled to mask systemd-modules-load.service. Error: Failed to symlink '"/etc/systemd/system/systemd-modules-load.service"'.
Caused by:
No such file or directory (os error 2)
[Distrod][WARN] Faled to mask [email protected]. Error: Failed to symlink '"/etc/systemd/system/[email protected]"'.
Caused by:
No such file or directory (os error 2)
[Distrod][WARN] Faled to mask [email protected]. Error: Failed to symlink '"/etc/systemd/system/[email protected]"'.
Caused by:
No such file or directory (os error 2)
[Distrod][WARN] Faled to mask console-getty.service. Error: Failed to symlink '"/etc/systemd/system/console-getty.service"'.
Caused by:
No such file or directory (os error 2)
[Distrod] Distrod has been enabled. Now your shell will start under systemd.
Here are the errors from subsequent launch 1:
> wsl -d rawhide
[Distrod][ERROR] Failed to spawn the command.
Caused by:
No such file or directory (os error 2)
And subsequent launch 2:
> wsl -d rawhide
[Distrod][ERROR] Failed to spawn the command.
Caused by:
No such file or directory (os error 2)
[Distrod][ERROR] Pre_second_fork failed.
Caused by:
0: Failed to enter the init's namespace
1: Failed to open ns/uts
2: ESRCH: No such process
This is from a clean Windows 11 Pro install, so wsl2 is implied and confirmed.
Microsoft Windows [Version 10.0.22000.527]
> wsl -e uname -a
Linux 8600K 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Any progress for this issue? I'm having similar problem. The distro I use is from this https://github.com/machsix/ubuntu-impi. You can download the rootfs here https://github.com/machsix/ubuntu-impi/actions/runs/2157438149
This error just started to appear on my laptop running windows 11. I haven't changed anything: it just doesn't launch anymore.
code 137 (0x00000089)
