qubes-issues
qubes-issues copied to clipboard
QubesOS
Design and create salt formulas distribution method
Since we've completed VM configuration stack, we need some method for distributing salt formulas. Currently the base formulas (default VMs, Whonix etc) are installed as rpm packages. But this doesn't scale well, especially for community-developed formulas (because rpm package can do a lot more than putting a simple text file in /srv/salt, so one should install packages in dom0 only from ultimately trusted places).
Standard approach for Saltstack is using git or Salt Package Manager. We may use something based on either of them, but need to add a "qrexec proxy", which will verify signatures (similar to qubes-dom0-update) (it isn't clear to me whether SPM support signatures at all...). Or we may design/use something different.
Then, we'd need some UI to import/select trusted keys. The whole point here is to support 3rd-party formulas, not only those provided by ITL. While Salt formula can do whatever it like to the system (same as rpm package), it is much easier to audit, because it is just a text file, usually very short.
A single formula may consists of multiple files. Example formula for installing Martus, based on https://github.com/QubesOS/qubes-issues/issues/1836#issuecomment-213575740:
https://gist.github.com/marmarek/29f9a4a1f3a7a457cf2b449ab0b0e2f4
(place those files in /srv/salt/martus, then execute qubesctl top.enable martus && qubesctl --all state.highstate - only in Qubes 3.2)
cc @mfc @bnvk @andrewdavidwong
Would it be possible to adapt Split-GPG for signature verification?
I don't think it would be good idea - in the end we need dom0 to verify the signature, otherwise some VM may in practice gain control over dom0 (if you compromise formulas verification path, you can configure dom0 to do anything). Also Splig-GPG is mostly useful for using privatekeys, not just verification.
So, basically, it's the same problem as the one faced in designing the backup-restore model, right? We have some input which has to be verified in dom0, but we don't want anything in dom0 to parse potentially malicious input before it gets verified. Would it make sense to approach this in the same way?
In case of backup it is additionally encrypted. Here we want only signature (not simple HMAC).
Then, we'd need some UI to import/select trusted keys. The whole point here is to support 3rd-party formulas, not only those provided by ITL. While Salt formula can do whatever it like to the system (same as rpm package), it is much easier to audit, because it is just a text file, usually very short.
I'd been assuming we'd a UI for the browsing / installing / removing of salt formulas from a users machine, thus this would involve more than just the key management, and by keys I assume PGP keys correct @marmarek ? In both cases, I was thinking making this engaging and welcoming with icons and names.
Since the .sls files appear to just be YML format, are they flexible enough to just add values to the existing fields like:
martus:
recipe:
- title: Martus
- description: Martus is a free, open source, secure information collection and management tool that empowers rights activists to be stronger in their fight against injustice and abuse.
- icon: path/to/logo.png
qvm.vm:
- present:
- template: debian-8-martus
- label: blue
And then to improve the usability of the key handling could we also store that data in same .sls file or some other file with the package?
maintainer:
- name: Emma G.
- email: [email protected]
- website: https://anarchy.org
- keyid: 0x3B0F8CCA
- fingerprint: 1B5BAEEA01DBE9EC8420E5B668B2C4A23B0F8CCA
- avatar: path/to/avatar.jpg
My reasoning is that users tend to understand the concept of interacting with people, companies, or projects a bit better than importing a cryptographic key and of course we could still show the full fingerprint under a hidden menu (for developers we've established trust with) or shown for untrusted sources
As per my work on the New "Qubes Manager" in #1870 I envisioned calling these Recipes and clicking on the button would launch a separate window that. Now that I've got the hang of Gtk + python a bit I could pretty easily make a little interface showing these concepts with test data
In my view, users do not understand well to decide to trust various individuals or companies. They want to get task x, y, z done. For the goal towards mass adaption of Qubes, rather than inventing a key and trust scheme, I would suggest to have some well tested, well supported official recipes for common cases we want to support.
In my view, users do not understand well to decide to trust various individuals or companies. They want to get task x, y, z done. For the goal towards mass adaption of Qubes, rather than inventing a key and trust scheme, I would suggest to have some well tested, well supported official recipes for common cases we want to support.
Yes, on the second thought it seems to be a better option.
As for enhancing formulas with additional metadata - it isn't that simple, because salt would try to parse it and obviously will fail. Anyway we definitely need some GUI with name, description, icon of a formula to install. That metadata can probably go as a separate file.
@adrelanos @marmarek what i'm envisioning would look and feel like a normal "app store" experience, e.g. Apple App Store, Google Play, etc... for the trusted recipes. I think it would be nice to down the line offer installing recipes that are not official, but are still secure :)
As for enhancing formulas with additional metadata - it isn't that simple, because salt would try to parse it and obviously will fail.
I'll just make a small example with the metadata and how I see it being implemented toward the above goal.
are we actually targeting R4.1 for this? I've moved to R4.2.
once there is an agreed-upon data structure, @ninavizz it would be great to get your input on what that could look like. also i'm not clear exactly where it should live -- yes referenced from the Qubes Manager but should there also be a dedicated widget or it being linked from some other existing widget? just as all other functionality is decomposed elsewhere.
and once there is basic agreement, i would be happy to work with community to draft some example recipes.
Where is the value in a GUI for Salt recipes? I honestly don't even really know what Salt is (I have a vague understanding that it has to do with package management and config scripts), and suspect that anyone who would, would be comfortable managing that in the CLI?
TL;DR, there are A LOT of things that Qubes needs GUI'd and made more usable. I'd like to get a centralized pool of such things, so that we can maybe better prioritize?
@andrewdavidwong could this get a UX tag? :D
On Wed, Mar 31, 2021 at 05:20:27PM -0700, Nina Eleanor Alter wrote:
Where is the value in a GUI for Salt recipes? I honestly don't even really know what Salt is, and suspect that anyone who would, would be comfortable managing that in the CLI?
TL;DR, there are A LOT of things that Qubes needs GUI'd and made more usable. I'd like to get a centralized pool of such things, so that we can maybe better prioritize?
@andrewdavidwong could this get a UX tag? :D
Salt is a configuration tool - you use it when you update your system, and you used it (behind the scenes) when you first set up your Qubes system.
You can use Salt to easily setup qubes, or to install particular capabilities in to templates or qubes. You can install salt states using a standard fedora package - whether you would want this to automatically implement the state is worth discussing.There's been discussion in the Forum - I prefer to allow users to review the state before running it, but then I mostly want users to understand what they are doing ,( and learn more), rather than just click and install. This doesn't make me a wonk, security or otherwise.
@unman I won't tell anyone you're a wonk, promise! ;) The SecureDrop team (whom I also work with) use Salt to configure our bespoke Qubes Workstations, so it's a term I'm familiar with... I just somewhat suck at comprehending things I can't visualize. But it's worth it to understand those things to me, if it can help me give things a visual framework for others to better understand them with.
@mfc Honestly, the best GUI would come from the designer working with a handful of formulas the community agrees are good/solid to use as a basis for building a GUI from. Data structuring, how exceptions/rules are established, etc. Do you press "Go" on them, or what actions get taken? Stuff like that.
Also, something like this I envisioned going into the proposed #6483, likely under the "Advanced" pulldown (or in place of it).
ok makes sense, i will try to coordinate the creation of some example salt recipes. would someone at FPF be interested in contributing some more general journalist example recipes? any interest from @unman @tlaurion @tasket ?
FYI this is an old mock-up but i think starting from scratch more systematically as you describe is the way to go.
I don't think any of FPF's journalist users have ever actually used Salt? Most are non-technical Mac users. Less curious/excited about technology that doesn't come with an 800 number to call when help is needed, and simply wanting stuff "to work" so they can focus on their investigative projects. Which I say with no doubts of Salt as a great opportunity, but more in acknowledgement of it as a current strange-unknown to our core users (and, heck, to me!).
Libre software's appeal is in its trustworthiness wrt security. The decentralized community-driven part of it of course "why" that is, yet it also imposes new behavioral and expectations shifts for end users that can feel abrupt and burdensome.
I really love the idea of creating an app-store like experience (wrt ease/access) for browsing, consuming, and sharing recipes! It feels like a terrific vehicle to present the libre ecosystem's roots in community support as an appealing thing, vs a draining thing in its otherness.
Back to Salt, tho— @conorsch is a huge Salt fan, and I mentioned this ticket to him earlier today. I definitely think he'd be a great contributor to this thread!
This may be a related topic: https://groups.google.com/g/qubes-devel/c/VQOKxsMG0-Q/m/mZ2KIpJUCQAJ
I really love the idea of creating an app-store like experience (wrt ease/access) for browsing, consuming, and sharing recipes! It feels like a terrific vehicle to present the libre ecosystem's roots in community support as an appealing thing, vs a draining thing in its otherness.
I absolutely agree, but with one caveat: we need to ensure that the recipes shared are legitimate, not malicious. That means that recipes must be cryptographically signed by their creators, and that there must be a mechanism for the reporting and removal of malicious ones. The Update Framework is ideal for the first part.
I absolutely agree, but with one caveat: we need to ensure that the recipes shared are legitimate, not malicious. That means that recipes must be cryptographically signed by their creators, and that there must be a mechanism for the reporting and removal of malicious ones.
This solves the issue only partially. We'd need also a process for approving new creators or maybe even validating each recipe individually. Some stores (for example Firefox addons) have explicit markings which packages were validated and are more trustworthy - we could do a similar thing. But there is also a middle ground - some recipes may not not be strictly malicious, but may increase attack surface as a side effect (I believe this applies to most network services for example - like a recipe to setup web development environment intentionally exposed to your LAN). We could have some rating system based on what features given recipe use (idea similar to https://developers.home-assistant.io/docs/add-ons/presentation#security, although specific criteria will surely be different). Anyway, I don't think the latter should be a blocker. Rather, something necessary for this to scale.
All of this is important feedback to consider for both the UI itself, but also for the service design aspect of such a store. Off the top of my head, some kind of a verification system and probably a separate classification system and approved auditors/verifiers, will need to exist. That said, this does not sound like a small undertaking, and I question its priority over other more urgent/basic usability concerns.
On Thu, Apr 01, 2021 at 05:28:18AM -0700, Michael Carbone wrote:
ok makes sense, i will try to coordinate the creation of some example salt recipes. would someone at FPF be interested in contributing some more general journalist example recipes? any interest from @unman @tlaurion @tasket ?
FYI this is an old mock-up but i think starting from scratch more systematically as you describe is the way to go.
No problem in generating recipes - we already have them for printer, caching proxy, multimedia player, qubes-builder, split-ssh etc etc. Would be more useful to start with what outcomes are wanted, and then contributors would know where to put their efforts. I know people like a nice GUI but why couldn't this just be assimilated in to the general (and familiar) package install process? It's easy to build packages that drop salt files, or even install the states.
Would be more useful to start with what outcomes are wanted, and then contributors would know where to put their efforts.
The above is so incredibly important for all of Qubes, before undertaking any specific effort(s). It's step 0 in most product design efforts in both the digital and analog worlds.
I've also been thinking... the only folks likely to think to themselves "I should run a Salt script!" are going to be native Linux users, IT folks, or developers. Folks also familiar with CLI interfaces—so, why should a GUI for this, be a priority now? It's just not a natural mental model for most users to want to run a script to configure things; you just go into each app's preferences to set things up, install PPD files, etc. After running "an installer" aka a package.
Much as I love the idea of the Salt store, I also question investing time on a project like this, when other areas of Qubes that new-to-Linux and new-to-VM users are more likely to encounter frustration with, have yet to receive usability love. Which isn't to suggest punting on this idea, entirely; only to not prioritize it ahead of other things, like a redesign of Qubes Manager, a redesign of the Devices Widget, major improvements to Clipboard and Split-GPG, and possibly a global File Manager or qubes-specific Task Bar.
In a couple weeks we'll begin interviewing users, as our first significant user research step following the deployment of the initial survey. From that, we'll learn a lot more about what users need/want from Qubes. I'd suggest re-considering UX priorities for the project, after findings are assembled, for +4.2 and beyond. Oh ya... and next-up on my own plate after finishing the app menu, is the Policies GUI. :)
Much as I love the idea of the Salt store, I also question investing time on a project like this, when other areas of Qubes that new-to-Linux and new-to-VM users are more likely to encounter frustration with, have yet to receive usability love. Which isn't to suggest punting on this idea, entirely; only to not prioritize it ahead of other things, like a redesign of Qubes Manager, a redesign of the Devices Widget, major improvements to Clipboard and Split-GPG, and possibly a global File Manager or qubes-specific Task Bar.
I wholeheartedly agree with this. From what I can tell, QubesOS’s biggest limitations are (in no particular order):
- Limited hardware support.
- No hardware graphics acceleration.
- UI/UX.
The real question isn’t “Is QubesOS easy for us to use?”. It is, “Is QubesOS nearly as easy to use as Windows or macOS?”. If I cannot recommend QubesOS to my non-technical friends, that is a problem. That includes those who have a limited budget and are heavy gamers. They need an OS that can run AAA games on reasonably priced hardware, and does not require them to use a command line.
On Mon, Apr 05, 2021 at 01:33:15AM -0700, Demi Marie Obenour wrote:
Much as I love the idea of the Salt store, I also question investing time on a project like this, when other areas of Qubes that new-to-Linux and new-to-VM users are more likely to encounter frustration with, have yet to receive usability love. Which isn't to suggest punting on this idea, entirely; only to not prioritize it ahead of other things, like a redesign of Qubes Manager, a redesign of the Devices Widget, major improvements to Clipboard and Split-GPG, and possibly a global File Manager or qubes-specific Task Bar.
I wholeheartedly agree with this. From what I can tell, QubesOS???s biggest limitations are (in no particular order):
- Limited hardware support.
- No hardware graphics acceleration.
- UI/UX.
The real question isn???t ???Is QubesOS easy for us to use????. It is, ???Is QubesOS nearly as easy to use as Windows or macOS????. If I cannot recommend QubesOS to my non-technical friends, that is a problem. That includes those who have a limited budget and are heavy gamers. They need an OS that can run AAA games on reasonably priced hardware, and does not require them to use a command line.
I think that the "heavy gamers" requirement is relatively unimportant in the great scheme of things. (Note that "relatively").
As to the salt store, it's a neat way of providing outcomes without using the command line except that for some outcomes configuration is required - e.g gpg, vpn. We could prompt during installation of course, assuming that users understood what information was required.
There is a strong tendency amongst Qubes users (and some devs) to think it is difficult to use. This has been played out in the mailing list for years. And there is an assumption that Windows and Macs are easy to use - they aren't, but familiarity has bred contempt. With good support, Qubes can be, and is, used successfully by non-technical users. If you are recommending Qubes and not providing the right level of support then you are doing your friends and Qubes a disservice.
They need an OS that can run AAA games on reasonably priced hardware I would consider any energy spend in this direction at this point in the development a total waste.
Agree with hardware support and UI/UX points and expect a 'salt store' if done right to be a major improvement to the UX and certainly 1000x more important than enabling gaming.
They need an OS that can run AAA games on reasonably priced hardware I would consider any energy spend in this direction at this point in the development a total waste. Agree with hardware support and UI/UX points and expect a 'salt store' if done right to be a major improvement to the UX and certainly 1000x more important than enabling gaming.
“Enabling gaming” in this context refers to allowing qubes to use hardware-accelerated graphics, which is becoming more and more important with time. It’s really hard to drive a 4K monitor at any decent rate with software alone.
allowing qubes to use hardware-accelerated graphics
I'd use hardware acceleration if we had it, of course. But it's hardly a priority given target audience and current state.
It’s_really_ hard to drive a 4K monitor at any decent rate with software alone.
Not sure what you mean.
I have a 4+ year old ThinkPad with integrated Intel HD 630 graphics running Qubes OS R4.0 with a 4K monitor at 60 Hz. There was nothing to it other than plugging it into the correct port (for some reason 3840x2160 only works on the display port but not the HDMI port).
VLC crashes when I tell it to go full screen, but other than that I have no issues with it. I just played the first 4K trailer YouTube showed me in fullscreen, no issues (maybe a very slight stutter in action scenes).
With good support, Qubes can be, and is, used successfully by non-technical users. If you are recommending Qubes and not providing the right level of support then you are doing your friends and Qubes a disservice.
So: The above, by itself, is a significant caveat that I feel needs to be strongly considered in multiple contexts.
Not only does Qubes require "support," but it requires "community support;" which is completely new to folks also new to the libre ecosystem, and can be a major time commitment—while also not providing immediate/emergency options. It feels like a caveat that too many high-risk users will not be able to make work for them, for various reasons (from general patience, to risks that impose upon job needs, etc). Which I don't say to be a negative-nancy or to be difficult. It just feels like a simple reality that is not only a "Qubes problem," but more broadly also a linux and a libre problem.
Qubes' community support, really does seem to stand-out as the best of the best. Which I've heard in multiple places, and think is wonderful... and I realize has also been no small feat for Michael or Andrew to achieve. So, please also know, I have a very difficult time offering the above, as I don't want to diminish from the wonderfulness of what so many volunteers and paid contributors have busted their humps to make possible.
Do we "want" to impose that mandate of support-required onto all new and future users of Qubes?
Do we "want" to impose that mandate onto all new and future users of Qubes?
Do you think there is a way to design the UI/UX so users will just "get" how to use Qubes OS correctly?
Obviously there is work to be done to make it easier to interact with Qubes OS and also to make it prettier. There are also the obvious issues to just get started (hardware support, installation, certified/pre-installed laptops etc).
After all the is done, we are still pretty much at 0 I think.
In my imagination a new user needs something like a 5 minute video that explains why one would want to use Qubes, how it is structured, and how some basic workflow works (very high-level, just the basics) and how to find more help.
Then it would be very nice to have "wizards" (I know you will shriek but I don't know a better word) that kind of guide the user through setting up her first compartments, installs the software she needs and makes sure her theme choices (or just one standard theme) is applied in dom0 and all her qubes.
All of that stuff would obviously use salt in the background. It's also a huge task. And the more choice you want to offer the more work it will become.
So back to the original thread: having a way to review and distribute salt formulas would kind of crowd-source a lot of that work to the community to create formula for all kinds of applications and scenario upon which the above vision could be built.
Any way you slice it, if the goal is to empower new users to get going without a human guide or strong community support ... salt is the "magic" behind the scenes that will make that happen.
On Tue, Apr 06, 2021 at 02:57:50PM -0700, Nina Eleanor Alter wrote:
With good support, Qubes can be, and is, used successfully by non-technical users. If you are recommending Qubes and not providing the right level of support then you are doing your friends and Qubes a disservice.
So: The above, by itself, is a significant caveat that I feel needs to be strongly considered in multiple contexts.
Not only does Qubes require "support," but it requires "community support;" which is completely new to folks also new to the libre ecosystem, and can be a major time commitment. It feels like a caveat that too many high-risk users simply do not have either the time nor the patience to wrangle. Which I don't say to be a negative-nancy or difficult. It just feels like a simple reality that is not only a "Qubes problem," but more broadly also a linux and a libre problem.
Do we "want" to impose that mandate onto all new and future users of Qubes?
You need only scan the myriad mailing lists, fora, and magazines, aimed at Windows and Mac users to see that these things are not new to people new to libre, unless you mean to distinguish the generally welcoming and helpful tone of Qubes community support ( my contributions excepted).
I've worked in a variety of support positions, and my view of Qubes is based on providing support to many new users. In my view, (and I've said this many times), the focus on individual qubes, in Qube manager and the default menu, is counterproductive for many new users. Providing a customised menu which is focussed on activities, setting up specific configurations (e.g. links to network drives, vpns), these are the things which I count as support.
You need only scan the myriad mailing lists, fora, and magazines, aimed at Windows and Mac users to see that these things are not new to people new to libre, unless you mean to distinguish the generally welcoming and helpful tone of Qubes community support ( my contributions excepted).
I have no idea what "things" you're referencing. Could you elaborate?
I mean, I'm a native Mac user, and could go on for hours about how about Qubes is alienating and strange, or simply "difficult;" when, it just doesn't have to be. Same, with most "Security" software. Which I don't say to be a know-it-all. I appreciate how hard it is to keep the Qubes project going, and how little funding there is to spread-across requisite resources. Yet friction = abandonment. It's' why people revert to using insecure products.
People love Macs, because they "simply work." Why not prioritize making that a goal for Qubes? Agency and empowerment to get done what each person uniquely need to get done, is what users desire; not the ability to do "anything and everything," nor a dependency upon others.
I appreciate that forums and mailing list are ongoing communications with users, but without actually studying how people use a tool and asking them questions throughout that process, it's not possible to understand why a thing is "hard" or how to shape a tool to better serve a user. With the Salt Recipes store, my gut response is "Why not just make settings and preferences easier for folks to manage?" Or, make it clearer that you install stuff on a template that is then 'activated' at the per-qube level.
Come to think—async comms sucks, and I'd really love to just do a call sometime. @unman you have my contact info, I think. If you'd be comfortable, I have an account to use an encrypted meeting tool called whereby, and I'd be happy to make time to chat at any time you could. Same to you @SvenSemmler (even tho I don't think you have my contact info). I need to re-key the research at qubes-os dot com email, otherwise I'd say just ping me there (unless you're ok pinging me there w/o GPG).
With the Salt Recipes store, my gut response is "Why not just make settings and preferences easier for folks to manage?" Or, make it clearer that you install stuff on a template that is then 'activated' at the per-qube level.
We share the same goal.
I am reacting to "Where is the value in a GUI for Salt recipes? I honestly don't even really know what Salt is..."
It's NOT technically correct, but think of it as a scripting language. You can write scripts (formula) that automate installs and setting of preferences to be reused by others so they don't have to learn the details. These scripts need to be reviewed and then published in a trusted way to prevent abuse.
The difference between a script and a formula is that instead of a step-by-step description of what needs to be done, the formula describes the end result and let's Salt figure out how to do it. But for the purpose of this discussion that's irrelevant.
When you are done designing the settings dialog, there needs to be code that does something. When that something is about settings and preferences ... it will very likely be a salt formula, because that's the mechanism already built into Qubes OS for exactly that purpose.
Whenever you want to install something that's a bit more involved like a VPN tunnel, the ability to tether your phone to sys-net ... you would best do that using a salt formula that was reviewed by people you trust. Specifically so you don't have to research and find out yourself.
I see "design and create salt formulas distribution method" as the first step in a journey that will result in:
- having a better idea which features and functions are in high demand
- the best way to do it
- not insignificant amount of verification of that method
- the ability to pick and choose "the best" and make them built-in configuration options in Qubes... which brings us to "Why not just make settings and preferences easier for folks to manage?"