Why isn't the Mirage Qubes Firewall available as a Qubes Template in the upstream repos

[ContemplatingSuicide]

For example it would be cool to just have to do

[user@dom0 ~]$ sudo qubes-dom0-update --enablerepo=qubes-templates-community qubes-template-mirage-firewall

to get it installed and ready (would also be much better than manual updating).

[one7two99]

On Mon, 20 Jul 2020 at 10:34, ContemplatingSuicide < [email protected]> wrote:

For example it would be cool to just have to do

[user@dom0 ~]$ sudo qubes-dom0-update --enablerepo=qubes-templates-community qubes-template-mirage-firewall

to get it installed and ready (would also be much better than manual updating).

I agree would be great if we can just download it from the / a repository.

799

[mlhanot]

Wouldn't the announcement of the Qubes OS contributed package repository facilitate this issue?

https://www.qubes-os.org/news/2020/10/05/qubes-os-contributed-packages/

[hannesm]

Thanks for the link @mlhanot. This looks like a good path forward.

[xaki23]

there is still a benefit to deploying as a template: you can install templates without risking a dom0 compromise.

so my mission there is still to have a mirage-firewall template that can be installed without touching dom0 at all. (which can still use the contrib-pkg repo as distribution mechanism for the package)

[quantumpacket]

Is there any update on getting an official template for this? I'm hesitant to use this if there is no easy update mechanism and I have to download files manually to dom0, which is risky.

Also, how can we trust that the code or releases have not been tampered with using the current install/build instructions? At least with having a template in https://github.com/QubesOS/qubes-template-configs/tree/master/R4.1/templates-community or https://github.com/QubesOS-contrib it will have to use a GPG key to ensure integrity of the source repository, tagged releases, or any binaries.

Users could potentially be installing a backdoored firewall in QubesOS if nothing is being signed :man_shrugging:. The only signatures I see are using GitHub's key, but signing using their key is not trustworthy for obvious reasons.

[hannesm]

That is indeed a very good question, how to move forward. I personally don't use QubesOS, but I'm eager that installing and using MirageOS Qubes Firewall is as easy as it can get. For what it is worth, we have trustworthy reproducible binary builds of the firewall at https://builds.robur.coop/job/qubes-firewall/ that always use the main branch of this repository and the current latest releases of all other OCaml packages. Each build includes several files: build-environment (the environment variables that were set), system-packages (the system packages installed via apt (or any other system package manager)), and opam-switch (the set of OCaml packages). The latter includes URLs and hashes of all tarballs (and git commit IDs for e.g. this repository).

Now, how could that be uploaded to the qubes-contrib repository? Any takers? On each successful build which binary changed we could sign that binary and upload it somewhere? /cc @marmarek

[hannesm]

Have a look at https://builds.robur.coop/job/qubes-firewall/build/09b47dd7-b6c9-404c-a534-e2d44d733d5c/ for an example which build artifacts are available.

[reynir]

It may be worth looking at qubes-builder-mirage. I don't understand very well how qubes-builder works, and I don't have a qubes setup available at this moment :/

[hannesm]

Hmm, I'm wondering (maybe @marmarek @palainp @xaki23 have comments):

• qubes-builder-mirage uses pvgrub, is this correct with recent (2020-10) MirageOS Xen development (PVH)
• we're happy to publish reproducible binaries, what would the path be? is there some sort of repository we can provide (similar to apt.robur.coop), so that a qubes user only needs to add a repo and receive updates whenever this updates? what should be in that repository? some sort of rpm? anyone with a shell script / Makefile how to generate that from a .xen unikernel? :)
• the infrastructure we have in place is builds.robur.coop which captures all hashes of tarballs, git commits, system packages, and is reproducible (using orb and builder from the github.com/roburio organization)

[palainp]

I think that grub2 boot with pvh will be correct, I tried some times ago with success on qubes 4.0 https://github.com/QubesOS/qubes-issues/issues/6162#issuecomment-938678401

I made some changes into qubes-mirage-firewall to have the qubes-buidler script work with qubes-mirage-firewall, ocaml 4.14 and mirage 4: the changes are in https://github.com/mirage/qubes-mirage-firewall/pull/140 (I updated the PR for mirage 4 for simplicity, and since the qubes-builder compilation is related I hope this is ok :) ).

I also made a minor update into the script in qubes-builder: https://github.com/palainp/qubes-builder/commit/8eebb00ffe0b126841c104f1d73f2777bf66fa06 but there still something not working, I ended with a rpm file but the installation in dom0 does not go well (I still have to dig this, I have a qadminexec error in the qvm-template-postprocess script, I now have a /var/lib/qubes/vm-template/tmp-mirage-firewall folder with data but no template seen in qubes).

[palainp]

Update: The failure in the template installation may be fixed directly into qubes at some point (https://github.com/QubesOS/qubes-issues/issues/7426). I now have a reliable template qubes-build+installation but I fail to boot the kernel (although it booted fine less than 6 months ago :/).

What puzzles me is that the code to handle multiboot seems to be correctly compiled into the unikernel, and I'm sure that the kernel is correctly found by the grub script (If I rename it in the grub.cfg file in the disk image it clearly fails to boot for a "kernel not found" reason). @marmarek I found into https://github.com/Solo5/solo5/pull/482 that the multiboot header have to be into the first 8k of the unikernel which seems to be the case (first bytes at 4k):

$ hexdump /mnt/boot/kernel | grep "b002 1bad " # grep for MULTIBOOT_HEADER_MAGIC
0001000 b002 1bad 0003 0001 4ffb e451 0000 0010

Do you have any clues on how I can investigate and fix this regression?

---- "proofs" for multiboot code inside the unikernel ----

$ objdump -D /mnt/boot/kernel |head
/mnt/boot/kernel:     file format elf64-x86-64
Disassembly of section .text:
0000000000100000 <_stext>:
  100000:	02 b0 ad 1b 03 00    	add    0x31bad(%rax),%dh
  100006:	01 00                	add    %eax,(%rax)
  100008:	fb                   	sti    

is coherent with the MULTIBOOT_HEADER_MAGIC (value 0x1BADB002) and MYMULTIBOOT_FLAGS and we can found something that seems to be the inlined version of parse_multiboot at the begining of platform_init (from solo5):

...
00000000001054c0 <platform_init>:
...
  1054de:       48 89 44 24 28          mov    %rax,0x28(%rsp)
  1054e3:       8b 07                   mov    (%rdi),%eax
  1054e5:       3d 78 c5 6e 33          cmp    $0x336ec578,%eax        ; <- this is XEN_HVM_START_MAGIC_VALUE
  1054ea:       0f 85 2f 02 00 00       jne    10571f <platform_init+0x25f>       ; <- if not equal goto parse_multiboot 
  1054f0:       41 b8 ff 5f 58 00       mov    $0x585fff,%r8d
...
  10571f:       a8 04                   test   $0x4,%al              ; <- MULTIBOOT_INFO_CMDLINE is 4 and parse_multiboot starts with a comparison to that
  105721:       75 68                   jne    10578b <platform_init+0x2cb>
...

vs

...
void platform_init(const void *arg)
{
    bool is_direct_pvh_boot;

    if (*((uint32_t*)arg) == XEN_HVM_START_MAGIC_VALUE)
        is_direct_pvh_boot = true;
    else
        is_direct_pvh_boot = false;

    /*
     * If booted via multiboot, parse multiboot structure before setting up
     * page tables as its location may not be mapped later.
     */
    if (!is_direct_pvh_boot)
        parse_multiboot(arg);
...

[Szewcson]

I saw that the patch for that size of the rootfs problem got his way to stable repo of qubes. Will it work now? I realy hope for that mirage firewall will be avaliable as qubes community template soon :)

[palainp]

Yes the patch to fix the "small rootfs template" works correctly and the installation is now perfect :) Unfortunately the issue occurs when I try to boot the unikernel (with grub mode). I'll keep investigating this point.

[resulin]

@hannesm I'm pretty sure you'll get answers for those questions if you ask in the qubes-devel mailing list https://www.mail-archive.com/[email protected]/

[Szewcson]

So is there any blocker now for add it to qubes-community-templates after merging latest PR?

[hannesm]

@Szewcson no, and there's a qubes-devel thread https://groups.google.com/g/qubes-devel/c/ZcR01kc3dz4 that proposes to add qubes-mirage-firewall :)