update to mirage 4.2.0 & mirage-xen 8.0.0

[palainp]

~~With these changes it compiles with mirage 4.0 and runs on qubes 4.1.~~ EDIT: This PR is now a general update to mirage 4.2+mirage-xen 8.0.0 and update of qubes-builder + docker scripts.

[xaki23]

as part of the whole mess around (the lower parts of) https://github.com/QubesOS/qubes-issues/issues/6162 and https://github.com/Solo5/solo5/pull/516 i can confirm this branch works at least as much as any other, if not better.

tl;dr: LGTM, please merge

[palainp]

Thanks @hannesm for the mirage update. I manually pinned mirage.4.2 until the merge in opam-repository and updated the memory_pressure.ml + config.ml for using mirage-xen.8.0.0. It seems to works correctly with the new quick_stat estimation. For @xaki23 if you want to try before opam-repository is updated, you may:

opam update -yu && \
opam pin functoria.4.2.0 git+https://github.com/mirage/mirage.git#main -y && \
opam pin functoria-runtime.4.2.0 git+https://github.com/mirage/mirage.git#main -y && \
opam pin mirage-runtime.4.2.0 git+https://github.com/mirage/mirage.git#main -y && \
opam pin mirage.4.2.0 git+https://github.com/mirage/mirage.git#main -y

before compiling with the last commit. EDIT: the various pins are no longer needed since the new mirage version is actually out, opam update -yu should be enough.

[palainp]

As @Szewcson I have a correct running here with:

[user@dom0 Desktop]$ sha256sum /var/lib/qubes/vm-kernels/mirage-test/vmlinuz 
588e921b9d78a99f6f49d468a7b68284c50dabeba95698648ea52e99b381723b  /var/lib/qubes/vm-kernels/mirage-test/vmlinuz

[2022-08-12 08:20:21] Solo5: Xen console: port 0x2, ring @0x00000000FEFFF000
[2022-08-12 08:20:21]             |      ___|
[2022-08-12 08:20:21]   __|  _ \  |  _ \ __ \
[2022-08-12 08:20:21] \__ \ (   | | (   |  ) |
[2022-08-12 08:20:21] ____/\___/ _|\___/____/
[2022-08-12 08:20:21] Solo5: Bindings version v0.7.3
[2022-08-12 08:20:21] Solo5: Memory map: 64 MB addressable:
[2022-08-12 08:20:21] Solo5:   reserved @ (0x0 - 0xfffff)
[2022-08-12 08:20:21] Solo5:       text @ (0x100000 - 0x31bfff)
[2022-08-12 08:20:21] Solo5:     rodata @ (0x31c000 - 0x386fff)
[2022-08-12 08:20:21] Solo5:       data @ (0x387000 - 0x544fff)
[2022-08-12 08:20:21] Solo5:       heap >= 0x545000 < stack < 0x4000000
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.rexec] waiting for client...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.gui] waiting for client...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.db] connecting to server...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.db] connected
...

@tommytran732 can you copy/paste your vm configuration and the last part of the qubes log to help understanding the crash please?

[TommyTran732]

As @Szewcson I have a correct running here with:

[user@dom0 Desktop]$ sha256sum /var/lib/qubes/vm-kernels/mirage-test/vmlinuz 
588e921b9d78a99f6f49d468a7b68284c50dabeba95698648ea52e99b381723b  /var/lib/qubes/vm-kernels/mirage-test/vmlinuz

[2022-08-12 08:20:21] Solo5: Xen console: port 0x2, ring @0x00000000FEFFF000
[2022-08-12 08:20:21]             |      ___|
[2022-08-12 08:20:21]   __|  _ \  |  _ \ __ \
[2022-08-12 08:20:21] \__ \ (   | | (   |  ) |
[2022-08-12 08:20:21] ____/\___/ _|\___/____/
[2022-08-12 08:20:21] Solo5: Bindings version v0.7.3
[2022-08-12 08:20:21] Solo5: Memory map: 64 MB addressable:
[2022-08-12 08:20:21] Solo5:   reserved @ (0x0 - 0xfffff)
[2022-08-12 08:20:21] Solo5:       text @ (0x100000 - 0x31bfff)
[2022-08-12 08:20:21] Solo5:     rodata @ (0x31c000 - 0x386fff)
[2022-08-12 08:20:21] Solo5:       data @ (0x387000 - 0x544fff)
[2022-08-12 08:20:21] Solo5:       heap >= 0x545000 < stack < 0x4000000
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.rexec] waiting for client...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.gui] waiting for client...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.db] connecting to server...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.db] connected
...

@tommytran732 can you copy/paste your vm configuration and the last part of the qubes log to help understanding the crash please?

Sorry for the late reply, where do you get those logs?

I am using the same VM configuration in the README. It works when I build Mirage 3.10, but when I replace the vmlinuz with the 4.2 version it just crashes shortly after it launches.

[palainp]

The logs come from tail -f /var/log/xen/console/guest-mirage-test.log. With a correct running with mirage 3.10 (post solo5+pvh) I don't see what can be your issue now, so the logs will certainly be useful :)

[TommyTran732]

I figured out what was wrong. I forgot to set qvm-features mirage-firewall no-default-kernelopts 1 :D For some reason it will happily run with the 3.10 kernel but it crash with 4.2. Anyhow, this was user error on my part. The PR is good to go I think :P

[TommyTran732]

@palainp Mirage 4.2.1 just came out a few hours ago. Is there any chance you can update this PR? (Would be a great opportunity to bump the ocaml docker image version too) :D

[palainp]

@tommytran732 @Szewcson: The build should be fine with mirage 4.2.1 now. I ran into troubles for the compilation, maybe you will have to correctly clean the repository. The shasum has not been changed because the mirage update only affects the build system.

[hannesm]

Thanks for all your hard work! :)

[palainp]

Thanks @hannesm! I hope it can also be built fine with orb at robbur. The next step will be to work on #115 :)

[hannesm]

Yes, see https://builds.robur.coop/job/qubes-firewall/build/4ead4483-4236-4b87-9498-b86ed931df8b -- indeed I just sent you a mail how to approach #115