qubes-issues

qubes-issues copied to clipboard

QubesOS is the most secure operating system available, by far. However, it unfortunately only runs on the x86 instruction set, which runs on unauditable and insecure firmware. The Power Architecture is a much more secure ISA. Products like the Talos II (edit: and now much more affordable Blackbird) with the Power9 CPU are fully open, with auditable schematics, firmware, and software - and being able to run QubesOS on such devices would be a huge win for the infosec community.

There are various ways to achieve this compatibility, so I thought that this issue could be a way to track them/discuss.

1 - Xen could have a ppc64 port (Raptor Computing Systems has offered free hardware to incentivize) 2 - Using the seL4 microkernel (https://github.com/QubesOS/qubes-issues/issues/3894), which is already looking into supporting the Power Architecture 3 - Qubes' Hypervisor Abstraction Layer (HAL), which utlizes libvirt to support multiple hypervisors, yet currently only supports Xen, could be expanded to support KVM, to run on ppc64.

March 26, 2022: We are now all in agreement for Xen+Power (option 1).

Funds available as of May 7th, 2022: I (Robert Spigler) have 0.35 bitcoin & Blackbird Bundle @leo-lb has pledged 0.8 btc (need to confirm) Total 1.15 btc

@madscientist159 Has offered to do the Xen port for 2 btc (just Xen port; no Qubes integration yet)

Power Foundation has made a statement of support (https://twitter.com/OpenPOWERorg/status/1504112361975730186?s=20), but this needs to be clarified.

We will be moving from Github -> Gitlab for development. (https://gitlab.com/groups/xen-project/-/epics/6)

We have made a Mailing List and Matrix Room: [email protected]; https://lists.riseup.net/www/info/qubes_port https://matrix.to/#/#qubes-port:matrix.org

We have now adopted this milestone approach for this Port: (done here)

1. Phase 1: 0.65BTC. Build tooling, minimal boot to serial console of a Xen kernel on a single core (no SMP, missing drivers, core locked at 100% power).

2. (Proposed) Phase 1.5: 0.65BTC (Pricing subject to change due to economic fluctuations): SMP, some driver integration (possible power state management?) required to get a usable system in preparation for Phase 2

I (Robert) donated 0.65 bitcoin out of my remaining 1 bitcoin bounty to fulfill the Phase 1 requirement. See here

@Rudd-O donated the entirety of his bounty (0.5 bitcoin) towards Phase 1.5. He no longer has any remaining pledge, and Phase 1.5 has 0.15 btc left to fulfill. See here

We are still waiting for @leo-lb to re-confirm his pledge.

Last updated May 7th, 2022

---

Details/History of Funding Below:

Please see the below chronological updates to funding:

• 1
• 2
• 3

In summary, we have a 3 bitcoin bounty, ~~and an additional 0.5 bitcoin remaining for matching funds~~ (deadline passed with 0.5 matching funds filled out of 1 bitcoin matching funds offered - see here). The match offer expired on July 28th 2021.

Details of the bounty are below:

@leo-lb paid @shawnanastasio 0.2 btc out of his 1 bitcoin bounty here: https://github.com/QubesOS/qubes-issues/issues/4318#issuecomment-630972681

@Rspigler (me) paid Shawn 0.5 bitcoin out of his 1.5 bitcoin bounty here. I have also offered hardware (Blackbird mainboard and one 4 core Power9 CPU) for a developer who will use it towards this project. See post here.

@Rudd-O pledged 0.5 bitcoin here (has paid 0).

~~I (Robert) have a remaining 0.5 matching bitcoin offer that expires on July 28th 2021.~~

Last updated: July 31st, 2022

My vote would be seL4. Not only are they responsive to the threat posed by closed, highly privileged firmware, but the kernel is already being used in very high security installations.

I would truly love seL4 as well and hope that is the end goal; however, I believe that that course of action would take the longest of all 3. I would argue that porting Qubes for ppc64 (by way of 1 or 3) is of higher priority than working on substituting the seL4 microkernel for Xen, since that offers the quickest route to Qubes running a truly open platform. The security benefit of seL4 could then come after.

I could be wrong though.

Of the two, I suspect 3 is the fastest and probably a decent stopgap solution. 1 would be duplicating effort that could go into the seL4 port IMO.

If needed, I can provide a significant quantity of build and test infrastructure for this effort.

There are various ways to achieve this compatibility, so I thought that this issue could be a way to track them/discuss.

The qubes-devel mailing list is the appropriate place for this sort of discussion. By contrast, qubes-issues is reserved for actionable issues as described in our issue reporting guidelines. If this issue is to remain open, it needs to be action-oriented rather than just a place for tracking and discussion. I'll change the title to reflect this.

I humbly propose this thread: https://groups.google.com/d/msg/qubes-devel/IFLyyCUbLmQ/_HtgOJK4AQAJ as a good place to continue the discussion!

I have had some difficulty getting to Google Groups, but I'd like to put my vote in as well for a port to the Talos II, whichever the professionals decide it will take the form of. I'm not well-versed enough in hypervisor design to be able to make any meaningful comment on which one would be best, but on the PPC64el, Debian runs KVM quite well with Libvirt for running other Debian and CentOS guests and is happily usable as a desktop with a minimum of work. I'd be very happy to beta-test a port to the PPC64, if you guys need testers.

Not only that but the Talos Lite is now here, which is about the price of high-end business grade or one of those high-end name-branded gaming laptops, and offers far better upgradeability for the future than a gaming laptop even when we don't look at the owner control/openness aspect of it. For many people interested in and serious about security, the price equivalent of a good quality laptop isn't a high one to pay, especially if the user isn't a firmware hacker that has the skills and know how to create their own "trusted stick" like what Joanna Rutkowska was suggesting for a stateless, verified boot x86 laptop.

The OTF funding form is here.

Time to tag developers, fund grant filling people (@mfc!) that might help so the discussion can follow on google groups and the information for the grant gets filled for the next OTF round!

hi all, i don't have any capacity to help out right now, but realistically i don't think OTF would fund such effort.

their target group is non-tech-savvy users in at-risk environments. these users have the current computers they own, they don't have the money/luxury of buying new hardware. Qubes' existing hardware requirements are already a very high barrier for these users and their target audience.

so if anything they would be more interested in funding Qubes development onto phones than Qubes on ppc64.

not stopping you all from submitting a proposal to OTF of course. maybe this is more something for a crowdfunding approach, for example.

Understandable. Going ahead with both might be possible. If a crowdfunding approach were to be done, what would be the estimated target goals for the KVM approach, and the seL4 approach? (seL4 could possibly be set as a stretch goal).

As for KVM:

Well, the first non-trivial task would be research what really needs to be done. There are two main parts here:

1. add KVM support in general
2. solve ppc64 specific problems (including verification of devices isolation etc)

The first one is much more predictable, I think it basically is:

• implementing vchan using KVM-specific mechanism
• implement "driver domains" (running a disk/net/usb backend in separate VM, instead of the host) - this is major feature missing in KVM compared to Xen; there are possible workarounds sacrificing some security
• add support for KVM to GUI agent/daemon - one hypervisor specific part there is accessing actual window composition buffers, using shared memory mechanism provided by particular hypervisor
• adjust startup scripts, libvirt configuration etc
• test everything and fix bugs

I think we should assume 50-100kEUR goal for this part alone. Note it may be also beneficial for other architectures, not only ppc64.

The second part may take several months of work, but also may turn out to be trivial. Require someone knowing ppc64 architecture (which I don't know).

For seL4, I don't even try to provide any estimate, without doing some research on the current state of it and what features crucial for Qubes are missing. See FAQ for some hints.

Thank you for your detailed response and all your work on Qubes.

What would your estimate be for approach #1? (ppc64 port of Xen?)

What would your estimate be for approach #1? (ppc64 port of Xen?)

Take a look at this xen-devel thread

GSoC proposal, anyone?

In theory deadline for setting up GSoC idea page was Feb 6, but in practice period for students to choose projects haven't started yet. Feel free to open PR against gsoc.md. It's ok to have a proposal with very basic description, although this ticket already contain content that could be re-used.

Added here.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

I am placing 1 bitcoin as a bounty for completing this task in porting Qubes.

As multiple developers will most likely be involved, I will divide the bounty between based on work done as I see fit. -----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwsYOJ56G8Q1Wl3glNc4P5sIUGCMFAlxjj+AACgkQNc4P5sIU GCNDWw//YeGQtVH2xc6+qNRezvcBG3M4Ql0DaPFndSPqQ3tBaHsaBoAH7BT6o3Ih EU/pTCBWDAItC13yot4FejYp2eLRY4VtHPd3wHB+e5WVCZdKbW7Z1F2SWsPJutDo NSKC+Rb8CEZO0sxSe9CYP7plc6a6GK+WF+ZjgiSN0ISeBf/VMr1V7aXJgZ+cIjSp eaJCzQ0XSah0oO2brJXUTvPOHUKsFHv3vJLcZag+N5T3HZ5P6Zt6jYorCGRtRBzK aWVaqGb8MVR9CQUOcLJ8c0ubyRHwwpTWMgr0QQ0QUzwRTBKMyt8jAfPVshI1l/T5 xzr067/sX1cUIWXGp5kpvK32aX9Gb+Sh7e+BqYour4hGiSpq2Vzz2UAakLutM2s4 hkFfnZA1HiZsqS0bxR9/iNi1+dvEBc2U4KUd6ou++p4osve+PpHqWXJGd5Acewgf fcdhNDPSSif5ON56gZRzvEMhtbE4MXfIdEI8NQGihZfdyO+R+muKxDRDPq9HFK3s hH1RcAYx6jC1T7E4UENZMzw2n9xKIcBnbkURmx6byzRhZACcX/I6YMciZks5Xxdb yIItqpTqjW7IdabmDxEYUlG7g+9G8kzZMNFNiqbbxmvsd83O4OUuPOcYj9XAIRtw w5QmHMVKbPMoKPN9s15i3KXGTnpIQ80E0nGWai8NykgzzzMOH3c= =Tehn -----END PGP SIGNATURE-----

As for KVM:

Well, the first non-trivial task would be research what really needs to be done. There are two main parts here:

1. add KVM support in general

2. solve ppc64 specific problems (including verification of devices isolation etc)

The first one is much more predictable, I think it basically is:

* implementing vchan using KVM-specific mechanism

I have been working on a reimplementation of vchan using KVM with the ivshmem shared memory backend, available here (comments welcome!).

I originally started it to assist in porting individual Qubes projects (mainly qubes-gui-daemon), but it would be great if it was useful for a port of Qubes as a whole.

I have been working on a reimplementation of vchan using KVM with the ivshmem shared memory backend, available here (comments welcome!).

Here you are: https://github.com/shawnanastasio/libkvmchan/issues/1

In the Qubes Architecture spec (Version 0.3), in the rational for choosing Xen over KVM, the authors mention how KVM relies on the Linux kernel to provide isolation. They also argue that Xen is much easier to audit due to a smaller code base, and that the I/O emulator, networking code, and drivers can be moved out of Dom0, further minimizing TCB.

This document was written 2010. These concerns still present?

Mostly yes. In the meantime, some extra precautions were developed to isolate device emulator (I think the most relevant is seccomp filtering), so it is slightly better now. But the arguments of a) Linux vs Xen code size, and b) backend drivers on the host system still stands. The "b" point is especially bad, as it makes it hard to have host system isolated from the outside world (networking, usb etc).

On the other hand, a lot of development have happened on KVM since 2010, some of it very relevant for Qubes. For example much better support for GPU passthrough. And (the main point of this discussion) support for other architectures.

Very interesting development: KVM support for Xen guests, including PV drivers. Given that Xen PV drivers supports backends in another guest, there is a chance this mechanism will support that too. Especially those patches emulates grant tables and event channels, which are designed with non-host backend in mind.

I'll add up another Bitcoin to the bounty, split across people same as above..

At current market rate, the bounty is worth more than $20,000 - I think that's good money.

A quick update. My vchan implementation for KVM is nearing completion. Once it's in a usable state, I plan on testing it with qrexec and other miscellaneous qubes utilities. If all goes well I'll look into getting qubes-gui working which will require a mechanism to map guest pages from the host which I'm still not sure how to (safely) do on KVM.

it's y'all's money of course, but as someone who doesn't see ppc ever becoming mainstream, while the project is rather under-staffed (considering the number of open issues on the tracker doubled since late 2016), I do think it's a strange priority to not want to give this to the Qubes project in general, but only to someone willing & able to port Qubes to a yuppie platform.

it's y'all's money of course, but as someone who doesn't see ppc ever becoming mainstream, while the project is rather under-staffed (considering the number of open issues on the tracker doubled since late 2016), I do think it's a strange priority to not want to give this to the Qubes project in general, but only to someone willing & able to port Qubes to a yuppie platform.

I am not interested in furthering Qubes OS support on x86. The architecture is doomed in regards to providing freedom to it's users. OpenPOWER is not a "yuppie" platform, I use it as my main workstation for day to day programming and administrative work. It would not benefit me at all to donate to the Qubes OS project if it's only goal is supporting x86 because I don't use x86 hardware anymore. So when Qubes OS is ported to ppc64[le] then I can explore donating for the rest of the project, otherwise, it's of no use to me..

A quick update. My vchan implementation for KVM is nearing completion. Once it's in a usable state, I plan on testing it with qrexec and other miscellaneous qubes utilities. If all goes well I'll look into getting qubes-gui working which will require a mechanism to map guest pages from the host which I'm still not sure how to (safely) do on KVM.

Awesome!

to clarify: by 'yuppie' I mean 'a platform for the quite well-off (who tell themselves Must Have Total Platform Security Or I Might As Well Move In With The NSA)', as 1300$ (ex VAT, shipping) for a 4c CPU in 2019 just means you have money to burn. Not that you aren't allowed to tell yourself these things, of course, but the thing you're afraid of is the govt and intelligence complex (which is of course interwoven with the Tech companies), not the arch and its shortcomings. Anyway, I've said my piece, so I'll leave you to it.

to clarify: by 'yuppie' I mean 'a platform for the quite well-off (who tell themselves Must Have Total Platform Security Or I Might As Well Move In With The NSA)', as 1300$ (ex VAT, shipping) for a 4c CPU in 2019 just means you have money to burn. Not that you aren't allowed to tell yourself these things, of course, but the thing you're afraid of is the govt and intelligence complex (which is of course interwoven with the Tech companies), not the arch and its shortcomings. Anyway, I've said my piece, so I'll leave you to it.

The CPU is cheaper than Intel in that regard, what costs money is the motherboard, and if I don't order RaptorCS offerings it will last even longer at that price, so that's meant to evolve of course. Personally I'm not afraid of surveillance or anything like that, I just think that by principle a machine you own means a machine you can hack. Not the case with Intel, AMD etc. You can't hack it.

I don't agree with the simplistic "thing you're afraid of". Governments are claiming justification for their expanded reach partly on the great success criminal syndicates have had in expanding theirs. And the former will (these days) typically pay ransoms for expediency over ethics, thus feeding a vicious circle.

Price points are an issue, but there are examples about how open source products can gain traction in the marketplace and eventually reduce costs via economies of scale. What we should avoid is becoming obsessed with attaining price-parity vs WinTel (the platform that offers us cheap-and-nasty, or cheap-and-nasty-on-steroids).