qubes-issues
qubes-issues copied to clipboard
QubesOS
Change the OS used in dom0
We have discussed this numerous times but don't have an issue to track these discussions. It would be worth understanding what would be needed to change dom0 from Fedora to Debian (say Debian 8). Benefits include:
- increased hardware compatibility
- incorporate serious work taken towards reproducible builds
- better firstboot installer
- better (slower) release cycle than Fedora with longer-term support
- other things?
This ticket does not encompass modifying the desktop environment.
the problem is , why anyone would take fedora which is very known for its issues (from about 2000) and their enterprise lockin on the code (redhat sucking stuff...) ?
is there anyone can give me good reasons why taking fedora at the first place ? why not debian ? why not even gentoo (which is regarding of code easiness/freedomcy better than fedora)?
im afraid the one who took the first (lets call it "bad") decision is still continuing to take these decisions which will make qubes a disaster even if u shifted to debian or X distro.
so i think consultations on these decisions is very important. and fedora to be the core of qubes or any other linux distro is a mistaken step.
Fedora is currently hosting the graphical environment and, like it or not, it has the best compatibility with graphic cards. Debian generally has old X drivers and older kernel. This is especialy problematic with Qubes, because the hardware selection is already constrained by virtualisation extension.
We will surely consider switching the distro when we implement GUI domain. It is on out roadmap and depends on GPU passthrough, which does not currently work, at least not without patching Xen and libvirt. We this it will be hard enough, and the effort should be directed there.
When switching, it will be neither Fedora nor Debian or any general purpose distro, but something very minimal, like Alpine.
Take a look at this thread - it's a good summary of GPU passthrough and PCI passthrough to HVM at all: https://groups.google.com/d/topic/qubes-devel/mvZBIUuyjv0/discussion
It would be really nice to have a choice here. Like @woju I think it would be best to have a minimal distribution. However it might be nice to try something more declarative like NixOS.
I guess you wouldn't want to maintain different distributions, at least not for the official distribution? Do you have an idea how many Fedora specific configuations are in the qubes builder and the applications?
I guess you wouldn't want to maintain different distributions, at least not for the official distribution?
Exactly.
Do you have an idea how many Fedora specific configuations are in the qubes builder and the applications?
During the current Fedora 20 -> 23 upgrade I'm trying to collect that list. Actually two lists:
- customization of GUI domain (which currently is the same as dom0) - this list is almost completed based on #1784
- things customized in dom0 in general
I think the only really Fedora specific thing is update mechanism, other parts are just some system configuration which may need some adjustment when porting to other distribution (like config file locations etc).
Yes, we're considering NixOS here, but no decision has been made yet.
I think the only really Fedora specific thing is update mechanism, other parts are just some system configuration which may need some adjustment when porting to other distribution (like config file locations etc).
I've got some experience here as I've been running Debian in dom0 for a while. It's an unholy mix of packages, alien and hand crafted stuff,(hand crafted like the Borja Jesus), and not full Qubes. But it works well enough for my slight requirements. I'm hoping that the upgrade to 3.1 will be somewhat closer to full Qubes. So far I would say there is almost nothing Fedora specific that comes outwith the normal repackaging process.
It's interesting that @mfc suggests that Debian provides increased hardware compatibility, while @woju says Fedora has best compatibility with graphic cards. Is there any evidence either way?
The Borja jesus :joy: That still sounds somehow encouraging! What are your reasons for prefering Debian on the dom0? I just started using qubes and I had much more experience with Debian, Arch, ... but basically none with Fedora. It took a short while to get accustomed - but in the end it feels all the same with some minor package manager differences. Since qubes takes a more radical approach in general, I think it would also make sense to consider a change to something conceptually "better" or more advanced like nix.
unman, could you please share more details about your setup, maybe even commits? ;-) Are you running this on a jessie based dom0 or stretch? How did you build+install the qubes specific stuff?
I'm still pondering doing the same, and looking at the cubes git repos I've identified these as being needed for dom0:
- qubes-core-admin
- qubes-core-admin-linux
- qubes-core-agent-linux
- qubes-core-libvirt
- qubes-core-vchan-xen
Is that correct? Which are missing? Cause once dom0 is working one should be able to deploy VMs from a backup… ;-)
On Wed, May 18, 2016 at 09:03:43AM -0700, Holger Levsen wrote:
I'm still pondering doing the same, and looking at the cubes git repos I've identified these as being needed for dom0:
- qubes-core-admin
- qubes-core-admin-linux
- qubes-core-agent-linux
- qubes-core-libvirt
- qubes-core-vchan-xen
Is that correct? Which are missing?
In addition to above:
- qubes-vmm-xen (yes, you need this, not native Debian packages because of Qubes-specific patches, and parts in Debian packages - libxenvchan, stubdomains)
- qubes-gui-common (required to build gui-daemon)
- qubes-gui-daemon
- qubes-core-qubesdb
- qubes-manager
And mgmt-salt stuff (everything with "base" and "dom0").
Some of them already have debian packaging.
Packaging Xen for dom0 would be tricky because of stubdomains. It consists of some external libraries/tools built as part of Xen build and statically linked. Something that AFAIR is against Debian policy (for example there are multiple source tarballs, from different projects).
And template packages needs to be converted to deb. Or not packaged at all and installed some other way - which I think would be even better option (installing templates as packages have a lot of side effects...).
Cause once dom0 is working one should be able to deploy VMs from a backup… ;-)
Yes, it should be possible.
Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
Marek, thanks for the hints!
For a start I don't mind violating Debian policy if needed. That said, while statically linking and embedded code copies are discouraged in general, there are also exceptions to these rules.
But really, at first I just want to be able to use a Debian based system as dom0.
@h01ger Worked through the build logs from fedora and the spec files. I followed the same build order and worked through each package in turn. I started with a base jessie install, and built up from there. I tried cherry picking Xen packages from sid and applying the Qubes patches, and that worked reasonably but I found difficult to incorporate stubdomains, and got pretty lost. In the end I basically built everything from source. It was relatively easy to identify Debian equivalents for the fedora requirements, and referencing the native Debian packages also helped, particularly for Xen. I had some success using alien for simpler stuff, although some paths and scripts needed to be bodged by hand. I didn't build manager or salt stack, but I cant imagine they would be difficult. Some stuff just doesn't work at all - can't boot from iso in HVM, for example. And nothing is really packaged so it's probably fragile on updates.
I'm currently working on 3.1, and trying to do the right thing, packaging properly and (sometimes) using qubes-builder framework, but I have limited time before my next trip. When I have anything sensible to share I will certainly commit, but I'm a way off that at present.
When I started there was a clear thought that Qubes would be moving to Debian, and I wanted to see what would be involved. Now, of course, that's no longer the case, so I suspect time is better spent working on Debian issues on the vm side.
I thought one of the main reasons to switch from Fedora to Debian are reproducible builds. Debian made and make a lot of progress in this area.
The other question which was discussed before is moving from KDE to GTK (Gnome) for the desktop.
@woju proposed above a switch to Alpine Linux. I understand that Alpine Linux has nice security features, but it seems to be a radical switch. A switch to Alpine would be less radical if the management GUI and dom0 would be separated. In the case the separation between GUI and dom0 would be possible and implemented, dom0 would be Alpine, but what about the GUI domain?
Why not going step by step. First moving from Fedora to Debian, because of the ability to create reproducible builds. Then moving from KDE to GTK (Gnome). The final step would be to separate Dom0 into separate domains. The management domain would be Alpine whereas the graphical engagement domain would be still Debian.
Another good point of @woju is the problem with drivers, older kernel and drivers in Debian. This is true for Debian stable, but Debian has a testing, unstable and experimental repository which contain more up to date packages.
Debian and Linux in general has many distribution derivatives. Why not using one of the derivatives which fulfills the requirements:
- based on Debian
- static release model
- up-to-date driver
Tanglu is a distribution which fulfills all the requirements. Tanglu is based on Debian, but uses software packages from testing, unstable and sometimes experimental to provide the newest drivers and other software packages, but is still based on Debian stable. Tanglu has three main editions KDE, GNOME and core. In addition Tanglu uses Calamares the independent system installer.
For reference:
- https://wiki.debian.org/Qubes
- https://wiki.debian.org/Qubes/Devel
https://wiki.debian.org/Qubes/Devel is the relevant URL for this issue.
https://wiki.debian.org/Qubes just describe how to use Qubes from the point of view of someone familar with Debian - and that wiki page is also a bit outdated.
@unman in https://github.com/QubesOS/qubes-issues/issues/1781#issuecomment-262856034
I don't think that #1919 is still a target.
How did you come to that conclusion?
In any case it seems to me that most users should be kept well away from dom0 and so what is running there is irrelevant. If anyone has the nous to tinker in dom0 then they should be able to handle the differences.
I don't think Fedora can be as transparent as that. Once one wants to do certain things, one example is #1375 and there are others, one is back to learning about Fedora. Same when issues happen or when debug info is requested. And that probably is not going to go away. Also higher effort for developers to know two systems. For example makes getting grsecurity in Qubes harder since an rpm package / repository is required.
Once one wants to do certain things, one example is #1375
No, it isn't - if you have enabled USB VM. Generally we want to keep user away from manipulating dom0 as much as possible. The extreme case (on the roadmap!) is GUI domain, where user have no way for direct interaction with dom0. Anyway, we're not going to switch dom0 to any other distribution until implementation of GUI domain.
No, it isn't - if you have enabled USB VM.
Which does not work for me due to effectively only 1 usb controller and all usb devices including internal keyboard connected to that, no ps2 either.
Other issues #2387 / #2360 also require quite some Fedora knowledge to sort out.
Anyway, we're not going to switch dom0 to any other distribution until implementation of GUI domain.
That sounds great and also sounds like there is still hope for dom0 Debian.
That sounds great and also sounds like there is still hope for dom0 Debian.
Yes, that's right. But because of complexity of GUI domain, it will not be soon.
@adrelanos re 1 USB controller: have you tried https://github.com/QubesOS/qubes-app-linux-input-proxy?
It provides questionable security benefit, but (of particular relevance here) does further the goal of users not needing to do much in dom0.
What about Ubuntu LTS? It provides 5Y support with releases every 2Y. So, we could stick with one version for 3Y-5Y.
It seems that there is some support of new hardware: https://wiki.ubuntu.com/Kernel/LTSEnablementStack
Ubuntu has the wrong mindset privacy wise: https://www.eff.org/deeplinks/2012/10/privacy-ubuntu-1210-amazon-ads-and-data-leaks
Ubuntu is problematic license wise:
- https://www.whonix.org/wiki/Dev/Operating_System#Switch_from_Ubuntu_to_Debian
- http://www.networkworld.com/article/2226367/opensource-subnet/making-sense-of-the-ubuntu-licensing-fiasco.html
That's also the reason why there is no Ubuntu template for Qubes.
https://www.qubes-os.org/doc/templates/ubuntu/
@v6ak Ubuntu LTS or Debian "Stable" does not matter. Ubuntu is based on Debian "Stable". Additionally, both offer support for 3/5 years. However, as @adrelanos pointed out there are license and privacy issues regarding Ubuntu.
Fedora is a nice choice, but the 9 months release cycle is a problem for dom0 in Qubes OS. The release cycle of Fedora is too short as a result most Qubes OS versions ship with an soon to be outdated or outdated version of Fedora as Dom0. Debian "Stable" and Ubuntu LTS on the other hand do not always support new hardware and kernel/packages are even on release date kind of out-of-date. I would prefer if Qubes OS would use a Debian "Testing" based OS for Dom0 with a semi-rolling or rolling release model. SparkleLinux (https://sparkylinux.org) could be an option.
@v6ak https://github.com/v6ak Ubuntu LTS or Debian "Stable" does not matter. Ubuntu is based on Debian "Stable". Additionally, both offer support for 3/5 years.
AFAIK Debian does not provide 5Y support for new hardware.
However, as @adrelanos https://github.com/adrelanos pointed out there are license and privacy issues regarding Ubuntu.
Yes, this is a good point.
as a result most Qubes OS versions ship with an soon to be outdated or outdated version of Fedora as Dom0.
If Fedora has a better support for new hardware, it is effectively nullified by having outdated version, itn't is?
I would prefer if Qubes OS would use a Debian "Testing" based OS for Dom0 with a semi-rolling or rolling release model. SparkleLinux (https://sparkylinux.org) could be an option.
Well, uh rolling release in dom0? I want my dom0 to be working and stable. I know this might be a little bit contradictory (support for new hardware requires changes), but still. Introducing a new drivers to dom0 is probably rather wanted, major upgrades of some packages (especially Xfwm/Kwin) are however likely to be breaking in context of Qubes.
Sorry, for using the term outdated. Unsupported is the right term. Fedoras release model is as following: "Release X is supported until one month after the release of Release X+2." [1] as a result Fedora 24 will reach it's end of life (EOL) approx. one month after the Fedora 26 release. The current supported versions are Fedora 24 and 25. Therefor Fedora 23 which is currently used for Dom0 is unsupported by the Fedora team. Unsupported does not mean Fedora is outdated. Fedora comes always a lot of new software in every release. The hardware support is still superior compared to other distribution.
Kalilinux is doing pretty well since they based their OS on the Debian "Testing" branch. Other distributions are successful doing the same. The Debian "Testing" branch is far more conservative compared to Arch Linux. Arch Linux has new software shortly after it is available. With other words Arch Linux follows a pretty aggressive rolling release model. Debian "Testing" on the other hand accepts packages only if they have been proven to work in the Debian "Unstable" branch [2]. As a result, I would consider Debian "Testing" as having conservative rolling release model. In addition, I think it fits into Qubes OS release model. Qubes OS receives minor upgrades and features even during the lifetime of a major release. For example, Qubes VM Manager updates or kernel updates during the lifetime of Qubes OS 3.2.
[1] https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle [2] https://wiki.debian.org/DebianReleases
Therefor Fedora 23 which is currently used for Dom0 is unsupported by the Fedora team. Unsupported does not mean Fedora is outdated.
It depends. With my new laptop, Fedora 23 is surely outdated and a newer kernel was needed.
Debian "Testing" on the other hand accepts packages only if they have been proven to work in the Debian "Unstable" branch [2].
This doesn't make it suitable for Qubes, see my note on Xfwm/Kwin. Maybe this could be solved by adding Xfwm/Kwin with high version number, but I am not sure about all the consequences.
Sent from my fruity BlackBerry pocket computer powered by Android with K-9 Mail. Please excuse my brevity.
@v6ak I do not get your point of Xfwm/Kwin. Please elaborate on that a little bit more.
Whonix was based on Debian testing. (now on stable) The short summary "testing was absolutely horrible". Long summary:
https://www.whonix.org/wiki/Dev/Operating_System#Why_is_Whonix_based_on_Debian_Stable.2C_not_Debian_Testing.3F
@adrelanos that make sense. Thanks. I thought Debian "testing" is more reliable.