qubes-issues icon indicating copy to clipboard operation
qubes-issues copied to clipboard
verified publisher icon QubesOS

Alpine Linux template

Open ayakael opened this issue 4 years ago • 77 comments

description

QubeOS should have an Alpine Linux template. This has been requested in the past, and I've finally decided to tackle it, having packaging experience with this distro. This space is for tracking issues.

current state

  • xorg works
  • memory ballooning works
  • copy / paste works + qvm-copy
  • OpenRC scripts generally working, but things lack finesse
  • tempate builder not implemented yet (now fixed)
  • automatic assignment of ip address broken (now fixed)
  • apk does not work on templates without networking (now fixed)

steps

  • Create qubes-vm packages (repo)
  • Test packages within Alpine HVM with the goal of creating a functional TemplateVM
  • Create builder-alpine (repo)
  • Test within qubes-builder environment with the goal of creating a tar.gz file that matches TemplateVM on test system

hurdles

  • Alpine uses OpenRC as its init system, while afaik every other template uses Systemd. A major part of the work will be updating existing initd scripts, and creating them where they don't exist.
  • Alpine uses musl rather than libc, but this doesn't seem to be a major issue.

ayakael avatar Mar 05 '22 00:03 ayakael

So far, all of the packages have been implemented as an APKBUILD. I'm currently working on initd scripts, with the following, as far as I can tell, mostly working:

  • xendriverdomain
  • qubes-db
  • qubes-meminfo-writer
  • qubes-sysinit
  • qubes-qrexec-agent

My current roadblock is qubes-gui-agent. It is stuck at Waiting on /var/run/xf86-qubes-socket socket. Anyone can give me any pointers on how I should debug this? The initd script is as is:

#!/sbin/openrc-run
  
name=$RC_SVCNAME
cfgfile="/etc/qubes/$RC_SVCNAME.conf"
command="/usr/bin/qubes-gui"
pidfile="/run/qubes/$RC_SVCNAME.pid"
command_background="yes"
error_log=/var/log/qubes/$RC_SVCNAME.log

depend() {
        need qubes-db
}

start_pre() {
	checkpath --directory --owner $command_user:$command_user --mode 0775 \
                /run/qubes /var/log/qubes /var/run/console/user
	# start console-kit-daemon
	/usr/bin/ck-list-sessions > /dev/null 2>&1
	# pretend tha user is at local console
	touch /var/run/console/user
	/bin/sh -c /usr/lib/qubes/qubes-gui-agent-pre.sh
	. /var/run/qubes-service-environment
	command_args="$command_args $GUI_OPTS"
	export DISPLAY=:0
}

ayakael avatar Mar 05 '22 00:03 ayakael

Thank you for working on this. I can't help you with this issue, but maybe this link, how Void Linux done this, can help you. https://github.com/Nexolight/void-tainted-pkgs/blob/qubes/srcpkgs/qubes-gui-agent-linux/files/qubes-gui-agent/run https://github.com/Nexolight/void-tainted-pkgs/tree/qubes/QubesOS

moiselazarus avatar Mar 05 '22 14:03 moiselazarus

Thanks @moiselazarus! I threw an afternoon on it with no success, but that void has suceeded in packaging for Qubes before helps a lot! I havn't implemented the one-shot scripts adequately, so that's where I'm at.

ayakael avatar Mar 08 '22 20:03 ayakael

Back on trying to figure this out. Despite implementing the one-shot scripts, I still am facing the Waiting on /var/run/xf86-qubes-socket socket issue. Anyone could fill me in to exactly what qubes-gui-agent is waiting on, or could give me pointers on where to go to debug this next. Of note, I am running off of the Alpine kernel, as the qubes kernels don't work

ayakael avatar Jun 20 '22 17:06 ayakael

I've made more progress: hvm mode with qubes kernel works using additional kernelopts modules=ext4 rootfstype=ex4 after introducting udev in the initramfs and thus fixing /dev/mapper/dmroot generation. pvh mode is broken, hanging after boot. Still stuck on waiting qubes-socket error ...

ayakael avatar Jun 20 '22 22:06 ayakael

@Nexolight Would you have encountered the issue of Waiting on /var/run/xf86-qubes-socket socket when creating the void template?

ayakael avatar Jun 21 '22 07:06 ayakael

Sounds like either Xorg not starting (or crashing on start?) or not loading qubes driver (generic config file instead of the one from qubes-gui-agent?).

marmarek avatar Jun 21 '22 09:06 marmarek

The /etc/X11/xorg-qubes.conf is the same as my Fedora template, so it can't be the generic config file. Driver are also available at /usr/lib/xorg/modules. At every start, the config file is updated, as well. It just seems like Xorg is never started, as there are no Xorg log (or error) files. In command line qubes-gui-runuser is in sleep (Z) mode.

ayakael avatar Jun 21 '22 19:06 ayakael

Of note, /usr/bin/qubes-gui-runuser /usr/bin/xinit /etc/X11/xinit/xinitrc -- /usr/libexec/Xorg :0 -nolisten tcp vt07 -wr -config /etc/X11/xorg-qubes.conf fails with exit code 255 when executed from shell.

ayakael avatar Jun 22 '22 17:06 ayakael

Good news! Tracked down the issue - it isn't a problem with Xorg, but rather with pam. There needs to be an Alpine version of /etc/pam.d/qubes-gui-agent. I've been trying for the last hour to throw spaguetti at the wall, see if it stick, but alas I do not know anything about pam. I see that they are copies of /etc/pam.d/su, so I went ahead and copied that, but now I get a execve cmd: Permission denied error.

ayakael avatar Jun 28 '22 14:06 ayakael

Doesn't seem to have to do with pam anymore (edit: or maybe it does??)

strace:

execve("/usr/bin/qubes-gui-runuser", ["/usr/bin/qubes-gui-runuser", "user", "/etc/X11", "/bin/sh", "-l", "-c", "exec /usr/bin/xinit /etc/X11/xin"...], 0x7ffdc547b8d0 /* 23 vars */) = 0
arch_prctl(ARCH_SET_FS, 0x77e0c74a5b48) = 0
set_tid_address(0x77e0c74a5f90)         = 2011
brk(NULL)                               = 0x64cf75ec6000
brk(0x64cf75ec8000)                     = 0x64cf75ec8000
mmap(0x64cf75ec6000, 4096, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x64cf75ec6000
open("/etc/ld-musl-x86_64.path", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib/libpam.so.0", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=59288, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\3004\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 65536, PROT_READ, MAP_PRIVATE, 3, 0) = 0x77e0c73ff000
mmap(0x77e0c7402000, 28672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x3000) = 0x77e0c7402000
mmap(0x77e0c7409000, 12288, PROT_READ, MAP_PRIVATE|MAP_FIXED, 3, 0xa000) = 0x77e0c7409000
mmap(0x77e0c740d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0xd000) = 0x77e0c740d000
close(3)                                = 0
mprotect(0x77e0c740d000, 4096, PROT_READ) = 0
mprotect(0x77e0c74a2000, 4096, PROT_READ) = 0
mprotect(0x64cf7503c000, 4096, PROT_READ) = 0
rt_sigprocmask(SIG_UNBLOCK, [RT_1 RT_2], NULL, 8) = 0
rt_sigaction(SIGTERM, {sa_handler=0x64cf7503a506, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x77e0c7455c8a}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGHUP, {sa_handler=0x64cf7503a506, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x77e0c7455c8a}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
open("/etc/passwd", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73fe000
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
read(3, "root:x:0:0:root:/root:/bin/ash\nb"..., 1024) = 1024
read(3, "gin\nsmmsp:x:209:209:smmsp:/var/s"..., 1024) = 579
lseek(3, -325, SEEK_CUR)                = 1278
close(3)                                = 0
munmap(0x77e0c73fe000, 4096)            = 0
stat("/etc/pam.d", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/pam.d/qubes-gui-agent", O_RDONLY|O_LARGEFILE) = 3
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73fe000
read(3, "# basic PAM configuration for Al"..., 1024) = 203
open("/lib/security/pam_rootok.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 4
fcntl(4, F_SETFD, FD_CLOEXEC)           = 0
fstat(4, {st_mode=S_IFREG|0755, st_size=14152, ...}) = 0
read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\20\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 20480, PROT_READ, MAP_PRIVATE, 4, 0) = 0x77e0c73f9000
mmap(0x77e0c73fa000, 4096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 4, 0x1000) = 0x77e0c73fa000
mmap(0x77e0c73fb000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED, 4, 0x2000) = 0x77e0c73fb000
mmap(0x77e0c73fc000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 4, 0x2000) = 0x77e0c73fc000
close(4)                                = 0
mprotect(0x77e0c73fc000, 4096, PROT_READ) = 0
open("/etc/pam.d/base-auth", O_RDONLY|O_LARGEFILE) = 4
read(4, "# basic PAM configuration for Al"..., 1024) = 437
open("/lib/security/pam_env.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 5
fcntl(5, F_SETFD, FD_CLOEXEC)           = 0
fstat(5, {st_mode=S_IFREG|0755, st_size=18248, ...}) = 0
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\21\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 24576, PROT_READ, MAP_PRIVATE, 5, 0) = 0x77e0c73f3000
mmap(0x77e0c73f4000, 8192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 5, 0x1000) = 0x77e0c73f4000
mmap(0x77e0c73f6000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED, 5, 0x3000) = 0x77e0c73f6000
mmap(0x77e0c73f7000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0x3000) = 0x77e0c73f7000
close(5)                                = 0
mprotect(0x77e0c73f7000, 4096, PROT_READ) = 0
open("/lib/security/pam_unix.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 5
fcntl(5, F_SETFD, FD_CLOEXEC)           = 0
fstat(5, {st_mode=S_IFREG|0755, st_size=47000, ...}) = 0
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0%\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 102400, PROT_READ, MAP_PRIVATE, 5, 0) = 0x77e0c73da000
mmap(0x77e0c73dc000, 24576, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 5, 0x2000) = 0x77e0c73dc000
mmap(0x77e0c73e2000, 8192, PROT_READ, MAP_PRIVATE|MAP_FIXED, 5, 0x8000) = 0x77e0c73e2000
mmap(0x77e0c73e5000, 57344, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0xa000) = 0x77e0c73e5000
mmap(0x77e0c73e7000, 49152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x77e0c73e7000
close(5)                                = 0
mprotect(0x77e0c73e5000, 4096, PROT_READ) = 0
open("/lib/security/pam_nologin.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 5
fcntl(5, F_SETFD, FD_CLOEXEC)           = 0
fstat(5, {st_mode=S_IFREG|0755, st_size=14152, ...}) = 0
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\21\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 20480, PROT_READ, MAP_PRIVATE, 5, 0) = 0x77e0c73d5000
mmap(0x77e0c73d6000, 4096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 5, 0x1000) = 0x77e0c73d6000
mmap(0x77e0c73d7000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED, 5, 0x2000) = 0x77e0c73d7000
mmap(0x77e0c73d8000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0x2000) = 0x77e0c73d8000
close(5)                                = 0
mprotect(0x77e0c73d8000, 4096, PROT_READ) = 0
read(4, "", 1024)                       = 0
close(4)                                = 0
open("/etc/pam.d/base-account", O_RDONLY|O_LARGEFILE) = 4
read(4, "# basic PAM configuration for Al"..., 1024) = 69
read(4, "", 1024)                       = 0
close(4)                                = 0
open("/etc/pam.d/base-password", O_RDONLY|O_LARGEFILE) = 4
read(4, "# basic PAM configuration for Al"..., 1024) = 87
read(4, "", 1024)                       = 0
close(4)                                = 0
open("/etc/pam.d/base-session-noninteractive", O_RDONLY|O_LARGEFILE) = 4
read(4, "# basic PAM configuration for Al"..., 1024) = 101
open("/lib/security/pam_limits.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 5
fcntl(5, F_SETFD, FD_CLOEXEC)           = 0
fstat(5, {st_mode=S_IFREG|0755, st_size=22424, ...}) = 0
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\"\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 28672, PROT_READ, MAP_PRIVATE, 5, 0) = 0x77e0c73ce000
mmap(0x77e0c73d0000, 8192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 5, 0x2000) = 0x77e0c73d0000
mmap(0x77e0c73d2000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED, 5, 0x4000) = 0x77e0c73d2000
mmap(0x77e0c73d3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0x4000) = 0x77e0c73d3000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73cd000
mprotect(0x77e0c73d3000, 4096, PROT_READ) = 0
read(4, "", 1024)                       = 0
close(4)                                = 0
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0x77e0c73fe000, 4096)            = 0
open("/etc/pam.d/other", O_RDONLY|O_LARGEFILE) = 3
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73fe000
read(3, "# basic PAM configuration for Al"..., 1024) = 173
open("/etc/pam.d/base-auth", O_RDONLY|O_LARGEFILE) = 4
read(4, "# basic PAM configuration for Al"..., 1024) = 437
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73cc000
read(4, "", 1024)                       = 0
close(4)                                = 0
open("/etc/pam.d/base-account", O_RDONLY|O_LARGEFILE) = 4
read(4, "# basic PAM configuration for Al"..., 1024) = 69
read(4, "", 1024)                       = 0
close(4)                                = 0
open("/etc/pam.d/base-password", O_RDONLY|O_LARGEFILE) = 4
read(4, "# basic PAM configuration for Al"..., 1024) = 87
read(4, "", 1024)                       = 0
close(4)                                = 0
open("/etc/pam.d/base-session-noninteractive", O_RDONLY|O_LARGEFILE) = 4
read(4, "# basic PAM configuration for Al"..., 1024) = 101
read(4, "", 1024)                       = 0
close(4)                                = 0
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0x77e0c73fe000, 4096)            = 0
ioctl(0, TIOCGWINSZ, {ws_row=25, ws_col=80, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(0, TIOCGWINSZ, {ws_row=25, ws_col=80, ws_xpixel=0, ws_ypixel=0}) = 0
readlink("/proc/self/fd/0", "/dev/tty1", 32) = 9
stat("/dev/tty1", {st_mode=S_IFCHR|0600, st_rdev=makedev(0x4, 0x1), ...}) = 0
fstat(0, {st_mode=S_IFCHR|0600, st_rdev=makedev(0x4, 0x1), ...}) = 0
clock_gettime(CLOCK_REALTIME, {tv_sec=1656427127, tv_nsec=903365906}) = 0
getuid()                                = 0
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 3
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73fe000
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 24) = -1 ENOENT (No such file or directory)
close(3)                                = 0
munmap(0x77e0c73fe000, 4096)            = 0
open("/etc/group", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73fe000
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
read(3, "root:x:0:root\nbin:x:1:root,bin,d"..., 1024) = 919
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0x77e0c73fe000, 4096)            = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[], NULL, 8) = 0
setgroups(1, [1000])                    = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73fe000
open("/etc/passwd", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
read(3, "root:x:0:0:root:/root:/bin/ash\nb"..., 1024) = 1024
read(3, "gin\nsmmsp:x:209:209:smmsp:/var/s"..., 1024) = 579
lseek(3, -325, SEEK_CUR)                = 1278
close(3)                                = 0
prlimit64(0, RLIMIT_CPU, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_FSIZE, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_DATA, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_CORE, NULL, {rlim_cur=0, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_RSS, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_NPROC, NULL, {rlim_cur=3625, rlim_max=3625}) = 0
prlimit64(0, RLIMIT_NOFILE, NULL, {rlim_cur=1024, rlim_max=4*1024}) = 0
prlimit64(0, RLIMIT_MEMLOCK, NULL, {rlim_cur=8192*1024, rlim_max=8192*1024}) = 0
prlimit64(0, RLIMIT_AS, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_LOCKS, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_SIGPENDING, NULL, {rlim_cur=3625, rlim_max=3625}) = 0
prlimit64(0, RLIMIT_MSGQUEUE, NULL, {rlim_cur=800*1024, rlim_max=800*1024}) = 0
prlimit64(0, RLIMIT_NICE, NULL, {rlim_cur=0, rlim_max=0}) = 0
prlimit64(0, RLIMIT_RTPRIO, NULL, {rlim_cur=0, rlim_max=0}) = 0
prlimit64(0, RLIMIT_RTTIME, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
getpriority(PRIO_PROCESS, 0)            = 20
open("/etc/security/limits.conf", O_RDONLY|O_LARGEFILE) = 3
read(3, "# /etc/security/limits.conf\n#\n#T"..., 1024) = 1024
read(3, " the soft limits\n#        - \"har"..., 1024) = 1024
read(3, "ity\n#\n#<domain>      <type>  <it"..., 1024) = 378
read(3, "", 1024)                       = 0
close(3)                                = 0
open("/etc/security/limits.d/", O_RDONLY|O_LARGEFILE|O_CLOEXEC|O_DIRECTORY) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77e0c73ca000
getdents64(3, 0x77e0c73ca0b8 /* 3 entries */, 2048) = 88
getdents64(3, 0x77e0c73ca0b8 /* 0 entries */, 2048) = 0
close(3)                                = 0
munmap(0x77e0c73ca000, 8192)            = 0
open("/etc/security/limits.d/90-qubes-gui.conf", O_RDONLY|O_LARGEFILE) = 3
read(3, "# Qubes GUI agent needs to mlock"..., 1024) = 158
read(3, "", 1024)                       = 0
close(3)                                = 0
prlimit64(0, RLIMIT_MEMLOCK, {rlim_cur=131072*1024, rlim_max=131072*1024}, NULL) = 0
setpriority(PRIO_PROCESS, 0, 0)         = 0
getuid()                                = 0
open("/etc/login.defs", O_RDONLY|O_LARGEFILE) = 3
read(3, "USERGROUPS_ENAB yes\n", 1024)  = 20
read(3, "", 1024)                       = 0
close(3)                                = 0
open("/etc/passwd", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
read(3, "root:x:0:0:root:/root:/bin/ash\nb"..., 1024) = 1024
read(3, "gin\nsmmsp:x:209:209:smmsp:/var/s"..., 1024) = 579
lseek(3, -325, SEEK_CUR)                = 1278
close(3)                                = 0
getuid()                                = 0
socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 3
connect(3, {sa_family=AF_UNIX, sun_path="/dev/log"}, 12) = 0
clock_gettime(CLOCK_REALTIME, {tv_sec=1656427127, tv_nsec=906160763}) = 0
sendto(3, "<86>Jun 28 14:38:47 : pam_unix(q"..., 107, 0, NULL, 0) = 107
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[], ~[KILL STOP RTMIN RT_1 RT_2], 8) = 0
fork()                                  = 2012
rt_sigprocmask(SIG_SETMASK, ~[KILL STOP RTMIN RT_1 RT_2], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
wait4(2012, [{WIFEXITED(s) && WEXITSTATUS(s) == 127}], 0, NULL) = 2012
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2012, si_uid=1000, si_status=127, si_utime=0, si_stime=0} ---
getuid()                                = 0
open("/etc/login.defs", O_RDONLY|O_LARGEFILE) = 4
read(4, "USERGROUPS_ENAB yes\n", 1024)  = 20
read(4, "", 1024)                       = 0
close(4)                                = 0
clock_gettime(CLOCK_REALTIME, {tv_sec=1656427127, tv_nsec=909603199}) = 0
sendto(3, "<86>Jun 28 14:38:47 : pam_unix(q"..., 86, 0, NULL, 0) = 86
munmap(0x77e0c73fe000, 4096)            = 0
munmap(0x77e0c73cc000, 4096)            = 0
exit_group(127)                         = ?
+++ exited with 127 +++

ayakael avatar Jun 28 '22 14:06 ayakael

Can you push the openrc files to your repo. I would like to test, maybe some hints or steps to build the template.

moiselazarus avatar Jun 30 '22 11:06 moiselazarus

@moiselazarus Apologies for the delay, I was without my laptop for a little while. You will find the repo updated with latest openrc files. Following this guide to setting up an Alpine HVM proved useful: https://github.com/Nexolight/void-tainted-pkgs/tree/qubes/QubesOS. A few notes:

  • kernelopts should be set via qvm-prefs -s $vmname "kernelopts modules=ext4 rootfstype=ext4"
  • There should be no need for linux-qubes, as qvm-vm-core now sets CONFIG_TRANSPARENT_HUGEPAGE_MADVISE as Y at startup
  • qubes-mkinitfs is needed to allow inclusion of udev in initramfs. Without this, /dev/mapper/dm_mapper is never created in standalone VMs.
  • Make sure created partition is GPT with following partition table: xvda1 200M EFI System xvda2 2048K BIOS boot partition xvda3 (whatever) Root filesystem (note the name for xvda3 as "Root filesystem")
  • To create getty@hvc0 (xen tty), add to /etc/inittab: hvc0::respawn:/sbin/getty -L hvc0 115200 vt220
  • make sure to rc-update add udev + udev-trigger
  • make sure to add "eudev" to /etc/mkinitfs/mkinitfs.conf features line and regenerate initramfs with mkinitfs

By memory, that should get your VM to a similar state as mine. Don't desitate in case of questions

ayakael avatar Jul 06 '22 17:07 ayakael

Current state of qubes-gui-runuser issue:

  • fixed error connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 24) = -1 ENOENT (No such file or directory) by installing musl-nscd, executing echo 'hosts: files dns' > /etc/nsswitch.conf, and executing nscd
  • fixed error open("/etc/ld-musl-x86_64.path", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory) by executing echo '/lib:/usr/local/lib:/usr/lib' > /etc/ld-musl-x86_64.path

That the issue still persists suggests that these error were non-fatal.

@marmarek Having created qubes-gui-runuser binary, would you have any pointers as to the source of the wait4(1994, execve cmd: Permission denied error in the strace?

ayakael avatar Jul 11 '22 21:07 ayakael

Debugged the issue: the qubes-gui-runuser issue had to do with two things: one was an error of mine in misunderstanding the command syntax, and the other was that qubes-run-xorg was trying to start xorg with ash (via /bin/sh link) rather than with bash. Of course, it'd be too easy for it to just work, as Xorg runs now but then crashes shortly after. See log below:

[    19.332] 
X.Org X Server 1.21.1.3
X Protocol Version 11, Revision 0
[    19.332] Current Operating System: Linux alpine 5.16.18-2.fc32.qubes.x86_64 #1 SMP PREEMPT Fri Apr 1 22:28:01 CEST 2022 x86_64
[    19.332] Kernel command line: root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH rd.plymouth.enable=0 plymouth.enable=0 xen_scrub_pages=0 modules=ext4 rootfstype=ext4
[    19.332]  
[    19.332] Current version of pixman: 0.40.0
[    19.332] 	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
[    19.332] Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[    19.332] (==) Log file: "/home/user/.local/share/xorg/Xorg.0.log", Time: Thu Jul 14 16:15:09 2022
[    19.332] (++) Using config file: "/etc/X11/xorg-qubes.conf"
[    19.332] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[    19.332] (==) ServerLayout "Default Layout"
[    19.332] (**) |-->Screen "Screen0" (0)
[    19.332] (**) |   |-->Monitor "Monitor0"
[    19.332] (**) |   |-->Device "Videocard0"
[    19.332] (**) |-->Input Device "qubesdev"
[    19.332] (==) Automatically adding devices
[    19.332] (==) Automatically enabling devices
[    19.332] (==) Automatically adding GPU devices
[    19.332] (==) Automatically binding GPU devices
[    19.332] (==) Max clients allowed: 256, resource mask: 0x1fffff
[    19.333] (WW) The directory "/usr/share/fonts/TTF" does not exist.
[    19.333] 	Entry deleted from font path.
[    19.333] (==) FontPath set to:
	/usr/share/fonts/misc,
	/usr/share/fonts/100dpi:unscaled,
	/usr/share/fonts/75dpi:unscaled,
	/usr/share/fonts/Type1
[    19.333] (==) ModulePath set to "/usr/lib/xorg/modules"
[    19.333] (II) The server relies on udev to provide the list of input devices.
	If no devices become available, reconfigure udev or disable AutoAddDevices.
[    19.333] (II) Module ABI versions:
[    19.333] 	X.Org ANSI C Emulation: 0.4
[    19.333] 	X.Org Video Driver: 25.2
[    19.333] 	X.Org XInput driver : 24.4
[    19.333] 	X.Org Server Extension : 10.0
[    19.334] (--) PCI:*(0@0:4:0) 1234:1111:1af4:1100 rev 2, Mem @ 0xf1000000/16777216, 0xf2012000/4096, BIOS @ 0x????????/131072
[    19.334] (WW) Open ACPI failed (/var/run/acpid.socket) (No such file or directory)
[    19.334] (II) "glx" will be loaded by default.
[    19.334] (II) LoadModule: "fb"
[    19.334] (II) Module "fb" already built-in
[    19.334] (II) LoadModule: "glx"
[    19.335] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[    19.345] (II) Module glx: vendor="X.Org Foundation"
[    19.345] 	compiled for 1.21.1.3, module version = 1.0.0
[    19.345] 	ABI class: X.Org Server Extension, version 10.0
[    19.345] (II) LoadModule: "dummyqbs"
[    19.345] (II) Loading /usr/lib/xorg/modules/drivers/dummyqbs_drv.so
[    19.345] (II) Module dummyqbs: vendor="X.Org Foundation"
[    19.345] 	compiled for 1.21.1.2, module version = 0.3.6
[    19.345] 	Module class: X.Org Video Driver
[    19.345] 	ABI class: X.Org Video Driver, version 25.2
[    19.345] (II) LoadModule: "qubes"
[    19.345] (II) Loading /usr/lib/xorg/modules/drivers/qubes_drv.so
[    19.345] (II) Module qubes: vendor="X.Org Foundation"
[    19.345] 	compiled for 1.21.1.2, module version = 0.0.1
[    19.345] 	Module class: X.Org XInput Driver
[    19.345] 	ABI class: X.Org XInput driver, version 24.4
[    19.345] (II) DUMMYQBS: Driver for Dummy chipsets: dummy
[    19.345] (WW) Falling back to old probe method for dummyqbs
[    19.345] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
[    19.345] (II) DUMMYQBS(0): Chipset is a DUMMY
[    19.346] (**) DUMMYQBS(0): Depth 24, (--) framebuffer bpp 32
[    19.346] (==) DUMMYQBS(0): RGB weight 888
[    19.346] (==) DUMMYQBS(0): Default visual is TrueColor
[    19.346] (==) DUMMYQBS(0): Using gamma correction (1.0, 1.0, 1.0)
[    19.346] (DB) xf86MergeOutputClassOptions unsupported bus type 0
[    19.346] (**) DUMMYQBS(0): Option "GUIDomID" "0"
[    19.346] (**) DUMMYQBS(0): VideoRAM: 28000 kByte
[    19.346] (--) DUMMYQBS(0): Max Clock: 300000 kHz
[    19.346] (II) DUMMYQBS(0): Monitor0: Using hsync range of 49.00-50.00 kHz
[    19.346] (II) DUMMYQBS(0): Monitor0: Using vrefresh range of 46.00-47.00 Hz
[    19.346] (II) DUMMYQBS(0): Clock range:  11.00 to 300.00 MHz
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x350" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "320x175" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x400" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "320x200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "720x400" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "360x200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x480" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "320x240" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x480" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "320x240" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x480" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "320x240" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x480" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "320x240" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "400x300" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "400x300" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "400x300" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "400x300" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "400x300" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x768i" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "512x384i" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "512x384" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "512x384" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "512x384" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "512x384" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1152x864" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "576x432" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x960" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x480" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x960" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x480" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x1024" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x512" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x1024" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x512" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x1024" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x512" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x1200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x1200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x1200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x1200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x1200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1792x1344" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "896x672" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1792x1344" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "896x672" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1856x1392" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "928x696" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1856x1392" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "928x696" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1440" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x720" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1440" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x720" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "832x624" (vrefresh out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "416x312" (vrefresh out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1400x1050" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "700x525" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1400x1050" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "700x525" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1440" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x720" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2048x1536" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2048x1536" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2048x1536" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x360" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "320x180" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x360" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "320x180" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "720x405" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "360x202" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "720x405" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "360x202" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "864x486" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "432x243" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "864x486" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "432x243" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x540" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "480x270" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x540" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "480x270" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x576" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "512x288" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x576" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "512x288" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x720" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x360" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x720" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x360" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1368x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "684x384" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1368x768" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "684x384" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x900" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x450" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x900" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "800x450" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1080" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x540" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1080" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x540" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2048x1152" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x576" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2048x1152" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1024x576" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2560x1440" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x720" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2560x1440" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x720" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2880x1620" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1440x810" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2880x1620" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1440x810" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "3200x1800" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x900" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "3200x1800" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1600x900" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "3840x2160" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1080" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "3840x2160" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1080" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "4096x2304" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2048x1152" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "4096x2304" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2048x1152" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "5120x2880" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2560x1440" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "5120x2880" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2560x1440" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "7680x4320" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "3840x2160" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "7680x4320" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "3840x2160" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "15360x8640" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "7680x4320" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "15360x8640" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "7680x4320" (insufficient memory for mode)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x800" (vrefresh out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x400" (vrefresh out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x800" (vrefresh out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "640x400" (vrefresh out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1400x900" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "700x450" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1400x900" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "700x450" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1680x1050" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "840x525" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1680x1050" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "840x525" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1920x1200" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "960x600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2560x1600" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x800" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Not using default mode "2560x1600" (bad mode clock/interlace/doublescan)
[    19.346] (II) DUMMYQBS(0): Not using default mode "1280x800" (hsync out of range)
[    19.346] (II) DUMMYQBS(0): Virtual size is 1920x1080 (pitch 1920)
[    19.346] (**) DUMMYQBS(0): *Mode "QB1920x1080": 96.0 MHz, 49.9 kHz, 46.1 Hz
[    19.346] (II) DUMMYQBS(0): Modeline "QB1920x1080"x46.1   96.00  1920 1921 1922 1923  1080 1081 1082 1083 (49.9 kHz z)
[    19.346] (==) DUMMYQBS(0): DPI set to (96, 96)
[    19.346] (II) Loading sub module "fb"
[    19.346] (II) LoadModule: "fb"
[    19.346] (II) Module "fb" already built-in
[    19.346] (II) Loading sub module "ramdac"
[    19.346] (II) LoadModule: "ramdac"
[    19.346] (II) Module "ramdac" already built-in
[    19.346] (EE) DUMMYQBS(0): Failed to open xengntshr: Permission denied!
[    19.346] (EE) 
Fatal server error:
[    19.346] (EE) AddScreen/ScreenInit failed for driver 0
[    19.346] (EE) 
[    19.346] (EE) 
Please consult the The X.Org Foundation support 
	 at http://wiki.x.org
 for help. 
[    19.346] (EE) Please also check the log file at "/home/user/.local/share/xorg/Xorg.0.log" for additional information.
[    19.347] (EE) 
[    19.347] (EE) Server terminated with error (1). Closing log file.

Failed to open xengntshr: Permission denied! relates to the following bit of code in DUMMYQBS:

    dPtr->xgs = xengntshr_open(NULL, 0);
    if (dPtr->xgs == NULL) {
        xf86DrvMsg(pScrn->scrnIndex, X_ERROR, "Failed to open xengntshr: %s!\n", strerror(errno));
        return FALSE;
    }

Unfortunately, my debugging hits a new wall...

ayakael avatar Jul 14 '22 14:07 ayakael

Failed to open xengntshr: Permission denied!

Missing udev rules from https://github.com/QubesOS/qubes-linux-utils/blob/master/udev/udev-qubes-misc.rules ? Or maybe user isn't a member of qubes group?

marmarek avatar Jul 14 '22 14:07 marmarek

  • fixed error connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 24) = -1 ENOENT (No such file or directory) by installing musl-nscd, executing echo 'hosts: files dns' > /etc/nsswitch.conf, and executing nscd

  • fixed error open("/etc/ld-musl-x86_64.path", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory) by executing echo '/lib:/usr/local/lib:/usr/lib' > /etc/ld-musl-x86_64.path

I don't think any of those require "fixing" - both are optional features and everything should work just fine without them (and indeed the process continues to run past those attempts).

marmarek avatar Jul 14 '22 14:07 marmarek

Failed to open xengntshr: Permission denied!

Missing udev rules from https://github.com/QubesOS/qubes-linux-utils/blob/master/udev/udev-qubes-misc.rules ? Or maybe user isn't a member of qubes group?

Adding user to qubes group did the trick, Xorg now works, although qvm-run alpine urxvt does not work. I'm able to execute sudo -u user urxvt from within the VM via xl console alpine, which opens it fortunately.

Another issue is that while qvm-copy to alpine works, it sends file to /root/QubesIncoming.

  • fixed error connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 24) = -1 ENOENT (No such file or directory) by installing musl-nscd, executing echo 'hosts: files dns' > /etc/nsswitch.conf, and executing nscd
  • fixed error open("/etc/ld-musl-x86_64.path", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory) by executing echo '/lib:/usr/local/lib:/usr/lib' > /etc/ld-musl-x86_64.path

I don't think any of those require "fixing" - both are optional features and everything should work just fine without them (and indeed the process continues to run past those attempts).

Indeed, this was desperation. I was barking up the wrong tree.

ayakael avatar Jul 14 '22 14:07 ayakael

qubes-session process should run as user, and should have that user environment (including HOME=/home/user).

qvm-run alpine urxvt does not work

You can add -p option to get the command output, hopefully that will help. If it just hangs, it probably waits for qubes.WaitForSession service - that would be related to the qubes-session process above, and especially qrexec-fork-server started from there (it should also run as user).

marmarek avatar Jul 14 '22 15:07 marmarek

qubes-session process should run as user, and should have that user environment (including HOME=/home/user).

qvm-run alpine urxvt does not work

You can add -p option to get the command output, hopefully that will help. If it just hangs, it probably waits for qubes.WaitForSession service - that would be related to the qubes-session process above, and especially qrexec-fork-server started from there (it should also run as user).

initrd starts qubes-session under user, but it stalls. htop shows xfsettingsd --replace subprocess under qubes-session. XFCE4 is installed. Log:

(gsd-xsettings:2617): Gdk-CRITICAL **: 17:31:26.664: gdk_atom_intern: assertion 'atom_name != NULL' failed

(gsd-xsettings:2617): Gdk-CRITICAL **: 17:31:26.664: gdk_atom_intern: assertion 'atom_name != NULL' failed

(gsd-xsettings:2617): xsettings-plugin-WARNING **: 17:31:26.672: You can only run one xsettings manager at a time; exiting
Failed to start: Could not initialize xsettings manager.
2022-07-14 17:31:26,952 icon-sender: running: ['qrexec-client-vm', 'dom0', 'qubes.WindowIconUpdater']

(evolution-alarm-notify:2568): Gdk-CRITICAL **: 17:31:27.142: gdk_atom_intern: assertion 'atom_name != NULL' failed

(evolution-alarm-notify:2568): Gdk-CRITICAL **: 17:31:27.142: gdk_atom_intern: assertion 'atom_name != NULL' failed

(evolution-alarm-notify:2568): Gdk-WARNING **: 17:31:28.268: ../gdk/x11/gdkproperty-x11.c:224 invalid X atom: 1701998435

(evolution-alarm-notify:2568): Gdk-WARNING **: 17:31:28.268: ../gdk/x11/gdkproperty-x11.c:224 invalid X atom: 1852403568

(xfsettingsd:2526): Gdk-WARNING **: 17:31:28.268: ../gdk/x11/gdkproperty-x11.c:224 invalid X atom: 1701998435

(xfsettingsd:2526): Gdk-WARNING **: 17:31:28.268: ../gdk/x11/gdkproperty-x11.c:224 invalid X atom: 1852403568
xfsettingsd: No window manager registered on screen 0.

(xfsettingsd:2526): xfsettingsd-WARNING **: 17:31:31.629: Failed to get the _NET_NUMBER_OF_DESKTOPS property.

ayakael avatar Jul 14 '22 15:07 ayakael

starts qubes-session under user, but it stalls

It should remain running, with various children processes (mostly dependent on /etc/xdg/autostart content). The question is whether qrexec-fork-server is started as one of them.

marmarek avatar Jul 14 '22 15:07 marmarek

starts qubes-session under user, but it stalls

It should remain running, with various children processes (mostly dependent on /etc/xdg/autostart content). The question is whether qrexec-fork-server is started as one of them.

Fixed it, it was a permission issue in /var/run/qubes (group owner root, not qubes) making the start of qrexec-fork-server fail

ayakael avatar Jul 14 '22 18:07 ayakael

Memory ballooning doesn't work, any pointers that I could follow?

Note that no initcpio hooks have been adapted to Alpine. I suspect it may have something to do with that. Also note, AppVM does announce succesful start, so connection to qmemman should be restarted.

ayakael avatar Jul 16 '22 10:07 ayakael

Memory ballooning doesn't work, any pointers that I could follow?

That's about qubes-meminfo-writer service.

marmarek avatar Jul 16 '22 12:07 marmarek

Memory ballooning doesn't work, any pointers that I could follow?

That's about qubes-meminfo-writer service.

Indeed, /sbin/meminfo-writer is started on boot, but it doesn't do anything when memory is maxed out via tail /dev/zero, A workaround I've found is executing kill -SIGUSR1 $(cat /run/qubes/qubes-meminfo-writer.pid) as a start_post() command in the OpenRC init script, but that immediately allocates the maximum amount of memory, defeating the point of memory ballooning

ayakael avatar Jul 17 '22 19:07 ayakael

After converting to template, qvm-run appvm-alpine-315 urxvt doesn't work in AppVM. qrexec-fork-server gives following error:

2022-07-17 22:33:04.813 qrexec-fork-server[12043]: qrexec-agent-data.c:244:handle_new_process_common: executed: QUBESRPC qubes.VMShell dom0 (pid 12045)
2022-07-17 22:33:04.816 qrexec-fork-server[12043]: qrexec-agent-data.c:272:handle_new_process_common: pid 12045 exited with 2

dom0 error after qvm-run:

Running 'urxvt' on appvm-alpine-315
appvm-alpine-315: command failed with code 2

I can execute urxvt without issue via sudo -u user urxvt through xl console appvm-alpine-315.

ayakael avatar Jul 17 '22 20:07 ayakael

Fixed the qvm-run issue - still related to Alpine's sh pointing to Busybox. For previous version of this issue, I simply changed the shebang. For now I'm seeing if Alpine minds if sh points to bash. A better fix for qubes is to patch i nthe APKBUILD every shebang so that it points to /bin/bash rather than /bin/sh

ayakael avatar Jul 18 '22 13:07 ayakael

A workaround I've found is executing kill -SIGUSR1 $(cat /run/qubes/qubes-meminfo-writer.pid) as a start_post()

qrexec agent should do that on the first call (first qvm-run). Have you changed the pid file path? I see meminfo-writer.pid there, not qubes-meminfo-writer.pid.

but that immediately allocates the maximum amount of memory, defeating the point of memory ballooning

That's absolutely expected and desirable, if you have enough memory on the host. qmemman distributes as much memory as possible to running VMs (up to their limits).

marmarek avatar Jul 18 '22 13:07 marmarek

A workaround I've found is executing kill -SIGUSR1 $(cat /run/qubes/qubes-meminfo-writer.pid) as a start_post()

qrexec agent should do that on the first call (first qvm-run). Have you changed the pid file path? I see meminfo-writer.pid there, not qubes-meminfo-writer.pid.

but that immediately allocates the maximum amount of memory, defeating the point of memory ballooning

That's absolutely expected and desirable, if you have enough memory on the host. qmemman distributes as much memory as possible to running VMs (up to their limits).

Yep, moving the pid file pack to the expected location worked. Thank you!

ayakael avatar Jul 18 '22 14:07 ayakael

Quick update on this project: I've been running Alpine Linux as my Qubes OS template for a few weeks now, and so far it's been quite a success! I've (moslty) reached parity with my personal Fedora-based template, so I'm quite happy with this development. I have felt like Qubes using the template is snappier, but that might be placebo. The next big hurdle is implementing the qubes-builder hooks for building the template RPM. That's likely going to be slow-going as I work on refining my implementation of the qubes packages. As the 20/80 rule goes, it takes 20% of the time to get to 80%, 80% of the time to finish the last 20%. 80% of the work is done.

ayakael avatar Aug 17 '22 15:08 ayakael