traceback with enabled http2
Since supposedly the curl bug that was causing HTTP2 framing errors is now resolved and Fedora Infra has re-enabled HTTP2 I decided to compile a rawhide ostree version with HTTP2 enabled. I did that and then installed it on a test rawhide vagrant box. Initial testing looked good but when I tried to do a rebase I got a core dump:
[root@vanilla-rawhide-atomic ~]# rpm-ostree rebase fedora-atomic:fedora/28/x86_64/atomic-host
Receiving objects: 56% (5848/10368) 2.5 MB/s 178.9 MB
error: Bus owner changed, aborting.
Versions of software below. ostree-libs-2018.5-2.fc29.x86_64 was a scratch build with this code change
[root@vanilla-rawhide-atomic ~]# rpm -q ostree-libs rpm-ostree libcurl
ostree-libs-2018.5-2.fc29.x86_64
rpm-ostree-2018.5-1.fc29.x86_64
libcurl-7.60.0-1.fc29.x86_64
[root@vanilla-rawhide-atomic ~]# rpm-ostree status
State: idle; auto updates disabled
Deployments:
● ostree://fedora-atomic:fedora/rawhide/x86_64/atomic-host
Version: Rawhide.20180615.n.0 (2018-06-15 09:39:15)
BaseCommit: 707bd0e1213f8e713c676a6018e0d7308b217ec757367faaab115a15e02126da
ReplacedBasePackages: ostree-libs ostree ostree-grub2 2018.5-1.fc29 -> 2018.5-2.fc29
ostree://fedora-atomic:fedora/rawhide/x86_64/atomic-host
Version: Rawhide.20180615.n.0 (2018-06-15 09:39:15)
Commit: 707bd0e1213f8e713c676a6018e0d7308b217ec757367faaab115a15e02126da
traceback from the journal is below:
Jun 15 21:17:17 vanilla-rawhide-atomic rpm-ostree[1397]: Initiated txn Rebase for client :1.16 (uid 0): /org/projectatomic/rpmostree1/fedora_atomic
Jun 15 21:17:39 vanilla-rawhide-atomic kernel: SELinux: Context system_u:object_r:container_runtime_exec_t:s0 is not valid (left unmapped).
Jun 15 21:18:03 vanilla-rawhide-atomic kernel: SELinux: Context system_u:object_r:container_unit_file_t:s0 is not valid (left unmapped).
Jun 15 21:18:31 vanilla-rawhide-atomic kernel: show_signal_msg: 53 callbacks suppressed
Jun 15 21:18:31 vanilla-rawhide-atomic kernel: pool[1549]: segfault at 228 ip 00007f5715195ef7 sp 00007f57073dfe50 error 4 in libcurl.so.4.5.0[7f5715143000+85000]
Jun 15 21:18:31 vanilla-rawhide-atomic kernel: Code: 48 39 c2 75 10 48 8b 87 c8 51 00 00 48 39 87 e0 0a 00 00 74 26 48 8d 6c 24 0c 48 89 ee e8 31 ff ff ff 31 f6 48 89 e9 48 89 df <41> 8b 94 24 28 02 00 00 e8 1c 9e fb ff 85 c0 75 08 48 89 df e8 e0
Jun 15 21:18:31 vanilla-rawhide-atomic audit[1397]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:install_t:s0 pid=1397 comm="pool" exe="/usr/bin/rpm-ostree" sig=11 res=1
Jun 15 21:18:31 vanilla-rawhide-atomic systemd[1]: Created slice system-systemd\x2dcoredump.slice.
Jun 15 21:18:31 vanilla-rawhide-atomic systemd[1]: Started Process Core Dump (PID 1612/UID 0).
Jun 15 21:18:31 vanilla-rawhide-atomic audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@0-1612-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jun 15 21:18:32 vanilla-rawhide-atomic systemd[1]: rpm-ostreed.service: Main process exited, code=killed, status=11/SEGV
Jun 15 21:18:32 vanilla-rawhide-atomic systemd[1]: rpm-ostreed.service: Failed with result 'signal'.
Jun 15 21:18:32 vanilla-rawhide-atomic audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=rpm-ostreed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Jun 15 21:18:33 vanilla-rawhide-atomic systemd-coredump[1613]: Process 1397 (rpm-ostree) of user 0 dumped core.
Stack trace of thread 1549:
#0 0x00007f5715195ef7 n/a (libcurl.so.4)
#1 0x00007f571519671a n/a (libcurl.so.4)
#2 0x00007f5715197927 n/a (libcurl.so.4)
#3 0x00007f5715163539 n/a (libcurl.so.4)
#4 0x00007f57151635b8 n/a (libcurl.so.4)
#5 0x00007f571519494a n/a (libcurl.so.4)
#6 0x00007f5715168852 n/a (libcurl.so.4)
#7 0x00007f571517ad23 n/a (libcurl.so.4)
#8 0x00007f571517bea9 n/a (libcurl.so.4)
#9 0x00007f571517c0c9 curl_multi_socket_action (libcurl.so.4)
#10 0x00007f5718c7f138 timer_cb (libostree-1.so.1)
#11 0x00007f5717272371 g_timeout_dispatch (libglib-2.0.so.0)
#12 0x00007f57172718ad g_main_context_dispatch (libglib-2.0.so.0)
#13 0x00007f5717271c78 g_main_context_iterate.isra.21 (libglib-2.0.so.0)
#14 0x00007f5717271d10 g_main_context_iteration (libglib-2.0.so.0)
#15 0x00007f5718c4faee ostree_repo_pull_with_options (libostree-1.so.1)
#16 0x0000559e34aa2284 n/a (rpm-ostree)
#17 0x0000559e34a9cd39 n/a (rpm-ostree)
#18 0x0000559e34a909fd n/a (rpm-ostree)
#19 0x00007f571781f937 g_task_thread_pool_thread (libgio-2.0.so.0)
#20 0x00007f571729a933 g_thread_pool_thread_proxy (libglib-2.0.so.0)
#21 0x00007f5717299f2a g_thread_proxy (libglib-2.0.so.0)
#22 0x00007f5716bf0644 start_thread (libpthread.so.0)
#23 0x00007f5716923f8f __clone (libc.so.6)
Stack trace of thread 1399:
#0 0x00007f57169194c9 __poll (libc.so.6)
#1 0x00007f5717271be6 g_main_context_iterate.isra.21 (libglib-2.0.so.0)
#2 0x00007f5717271fa2 g_main_loop_run (libglib-2.0.so.0)
#3 0x00007f571786276a gdbus_shared_thread_func (libgio-2.0.so.0)
#4 0x00007f5717299f2a g_thread_proxy (libglib-2.0.so.0)
#5 0x00007f5716bf0644 start_thread (libpthread.so.0)
#6 0x00007f5716923f8f __clone (libc.so.6)
Stack trace of thread 1398:
#0 0x00007f57169194c9 __poll (libc.so.6)
#1 0x00007f5717271be6 g_main_context_iterate.isra.21 (libglib-2.0.so.0)
#2 0x00007f5717271d10 g_main_context_iteration (libglib-2.0.so.0)
#3 0x00007f5717271d61 glib_worker_main (libglib-2.0.so.0)
#4 0x00007f5717299f2a g_thread_proxy (libglib-2.0.so.0)
#5 0x00007f5716bf0644 start_thread (libpthread.so.0)
#6 0x00007f5716923f8f __clone (libc.so.6)
Stack trace of thread 1603:
#0 0x00007f571691e999 syscall (libc.so.6)
#1 0x00007f57172b86ee g_cond_wait_until (libglib-2.0.so.0)
#2 0x00007f57172440f1 g_async_queue_pop_intern_unlocked (libglib-2.0.so.0)
#3 0x00007f571729aa72 g_thread_pool_thread_proxy (libglib-2.0.so.0)
#4 0x00007f5717299f2a g_thread_proxy (libglib-2.0.so.0)
#5 0x00007f5716bf0644 start_thread (libpthread.so.0)
#6 0x00007f5716923f8f __clone (libc.so.6)
Stack trace of thread 1397:
#0 0x00007f57169194c9 __poll (libc.so.6)
#1 0x00007f5717271be6 g_main_context_iterate.isra.21 (libglib-2.0.so.0)
#2 0x00007f5717271d10 g_main_context_iteration (libglib-2.0.so.0)
#3 0x0000559e34a89b8c n/a (rpm-ostree)
#4 0x0000559e34a4ee3e n/a (rpm-ostree)
#5 0x0000559e34a3acf6 n/a (rpm-ostree)
#6 0x00007f571684d1bb __libc_start_main (libc.so.6)
#7 0x0000559e34a3b00a n/a (rpm-ostree)
Stack trace of thread 1601:
#0 0x00007f571691e999 syscall (libc.so.6)
#1 0x00007f57172b86ee g_cond_wait_until (libglib-2.0.so.0)
#2 0x00007f57172440f1 g_async_queue_pop_intern_unlocked (libglib-2.0.so.0)
#3 0x00007f571729aa72 g_thread_pool_thread_proxy (libglib-2.0.so.0)
#4 0x00007f5717299f2a g_thread_proxy (libglib-2.0.so.0)
#5 0x00007f5716bf0644 start_thread (libpthread.so.0)
#6 0x00007f5716923f8f __clone (libc.so.6)
Stack trace of thread 1602:
#0 0x00007f571691e999 syscall (libc.so.6)
#1 0x00007f57172b86ee g_cond_wait_until (libglib-2.0.so.0)
#2 0x00007f57172440f1 g_async_queue_pop_intern_unlocked (libglib-2.0.so.0)
#3 0x00007f571729aa72 g_thread_pool_thread_proxy (libglib-2.0.so.0)
#4 0x00007f5717299f2a g_thread_proxy (libglib-2.0.so.0)
#5 0x00007f5716bf0644 start_thread (libpthread.so.0)
#6 0x00007f5716923f8f __clone (libc.so.6)
Default assumption should be it's a bug in ostree probably. Our CI doesn't really cover HTTP2 unfortunately.
I'd like to fix this but it's going to be back burner for a bit.
@dustymabe does this still repro?