VPN-6301: Flatpak builds on taskcluster

[oskirby]

Description

It might be possible to get flatpaks building on Taskcluster by making use of a generic-worker instead of going through Docker. Let's explore this idea.

Reference

JIRA issue: VPN-6301

Checklist

• [ ] My code follows the style guidelines for this project
• [ ] I have not added any packages that contain high risk or unknown licenses (GPL, LGPL, MPL, etc. consult with DevOps if in question)
• [ ] I have performed a self review of my own code
• [ ] I have commented my code PARTICULARLY in hard to understand areas
• [ ] I have added thorough tests where needed

[oskirby]

Neat! It works! It's not exactly the friendliest with regards to downloading dependencies in the build tasks, but it does produce builds.

[ahal]

This is very cool!

What are the expectations for this build, is it just to have it available or do we eventually want this to be released? There are two potential issues here if we eventually want to ship these builds:

1. Using --privileged. I think this might actually be ok because AIUI podman doesn't run as root and (unlike docker-worker), generic-worker creates a new task user for each task. So we get task isolation from the system and don't need to rely on containers. That said, I think this will require some more thought and possibly an RRA with SecOps.

2. Chain of Trust. The fact this runs custom commands to load and launch the image means the image is not verifiable by Chain of Trust. Which in turn means that any downstream scriptworker tasks are going to fail (e.g a theoretical pushflatpakscript task). If we decide that --privileged is ok, then it should be possible to convert this to a D2G task on a pool with the allowPrivileged config enabled. Unfortunately Ben recently found an issue with D2G, so currently D2G tasks are also not CoT verifiable. We'll be pushing for a fix there however. Alternatively we can decide that CoT doesn't need to verify images, but this would be a step backwards security wise.

Neither problem is insurmountable. Just beware that you're at the forefront of release tasks using generic-worker on Linux here :p

[oskirby]

My personal goal for this task is to provide build artifacts that:

1. We can direct QA at for testing purposes, either as a part of release and regression testing or just for general debug.
2. To provide an automated build pipeline that will alert us to breakage.
3. And some day to run a functional test suite in taskcluster (the big and impossible dream) that consumes these artifacts.

The chain-of-trust isn't strictly necessary for these goals as a true flatpak release wouldn't go through this pipeline and we can just push the sources to flathub to do the real building (IIRC Mozilla already has a helper to do this), but I feel uncomfortable doing that without some other CI pipeline to test that the sources are really buildable, which is where this work comes in.

And I guess the other goal is really just to test out what we can do in a generic linux worker - though a big part of this feels like I am re-inventing the wheel and just creating a new docker-worker but with more permissions.

[oskirby]

FWIW, another approach to take here would be to have the flatpak and flatpak-builder packages installed in the base operating system, in which case this task wouldn't need the step of creating a podman worker as we can just rely on the flatpak isolation framework but that wouldn't have been as fun :)

[ahal]

FWIW, another approach to take here would be to have the flatpak and flatpak-builder packages installed in the base operating system, in which case this task wouldn't need the step of creating a podman worker as we can just rely on the flatpak isolation framework but that wouldn't have been as fun :)

This might be a good approach to move to, though we probably don't want to have a special VM image just for this.. We often use toolchain tasks for this kind of thing. Using toolchains to install flatpak and flatpak-builder at task runtime is probably the cleanest way to do it.

But for now this approach seems fine.