[RFE] CRC libvirt related resources should be in the user namespace, not root

[cjeanner]

General information

• OS: Linux
• Hypervisor: KVM
• Did you run crc setup before starting it (Yes/No)? Yes
• Running CRC on: Laptop

CRC version

CRC version: 2.11.0+823e40d

CRC status

DEBU CRC version: 2.11.0+823e40d                  
DEBU OpenShift version: 4.11.13                   
DEBU Podman version: 4.2.0                        
DEBU Running 'crc status'                         
CRC VM:          Running
OpenShift:       Running (v4.11.13)
RAM Usage:       7.157GB of 9.399GB
Disk Usage:      16.12GB of 32.74GB (Inside the CRC VM)
Cache Usage:     15.99GB
Cache Directory: /home/cjeanner/.crc/cache

CRC config

- consent-telemetry                     : no
- kubeadmin-password                    : 12345678
- pull-secret-file                      : /home/cjeanner/work/github/install_yamls/devsetup/pull-secret.txt
- skip-check-daemon-systemd-sockets     : true
- skip-check-daemon-systemd-unit        : true

Host Operating System

NAME="Fedora Linux"
VERSION="36 (Thirty Six)"
ID=fedora
VERSION_ID=36
VERSION_CODENAME=""
PLATFORM_ID="platform:f36"
PRETTY_NAME="Fedora Linux 36 (Thirty Six)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:36"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f36/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=36
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=36
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
SUPPORT_END=2023-05-16

Steps to reproduce

1. as a user, deploy CRC following the documentation
2. once CRC setup is over, as a user, run virsh list --all as well as virsh net-list
3. as a user, run sudo virsh list --all and sudo virsh net-list
4. Compare both outputs

Expected

Virtual machine managed in libvirt should be visible from the user without need for sudo.

Actual

The VM (and related resources such as network) are running in root namespace. This requires privileges escalation while they aren't actually needed. In order to allow a user to manage VM in libvirt, crc may have to push a polkit rule as follow:

 # cat /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules
polkit.addRule(function(action, subject) {
        if (action.id == "org.libvirt.unix.manage" &&
            subject.user == ">>> current_user <<<") {
                return polkit.Result.YES;
                polkit.log("action=" + action);
                polkit.log("subject=" + subject);
        }
});

Though it's not mandatory depending on the current_user rights (part of libvirt groups, for instance). A way to check the right access level is to run, for instance, virsh net-list - it shouldn't return any error if we have sufficient accesses. It may happen the user will need to create the "default" network if it's used within CRC. That network has an OS provided xml located, at least on Fedora, here: /usr/share/libvirt/networks/default.xml

Context

To give some more context: we're needing to deploy crc in an automated way (using ansible). Due to the way crc is working, we can't really do it, because of the call for sudo during the process.

We've detected some others, such as:

• create NetworkManager dispatcher (but afaik we can avoid it if we pre-create that dispatcher)
• inject a line in the /etc/hosts (but we may be able to avoid it as well)
• create the crc network and related VM with libvirt

At this point, most of the "config" steps can be reached with preparation steps we can handle, using the "become" parameter in ansible, but for the libvirt related content, it will be a bit hard...

Thank you for your attention :).

Cheers,

C.

[cjeanner]

Note: after some more digging, it seems virsh is called using --connect=qemu:///system - this can be also called from the shell directly.

Maybe mentioning it to the user, and exposing they may export the following in their env would be sufficient to solve that one: export VIRSH_DEFAULT_CONNECT_URI=qemu:///system (Note that LIBVIRT_DEFAULT_URI may also be something to consider)

[praveenkumar]

@cjeanner Looking at the below context to the issue, I think it is not only the libvirt calls which require the sudo permission but other steps also.

To give some more context: we're needing to deploy crc in an automated way (using ansible). Due to the way crc is working, we can't really do it, because of the call for sudo during the process.

Following preflight checks need sudo permission for the first time which run as part of crc setup.

• Putting the dnsmasq config to /etc/NetworkManager/dnsmasq.d/crc.confg
• Add a dispatcher to /etc/NetworkManager/dispatcher.d/99-crc.sh
• Loading the kvm module if it is not loaded by default
• Adding user to libvirt group if it not part of.
• Adding setsuid for admin helper binary

So I am hoping when you say At this point, most of the "config" steps can be reached with preparation steps we can handle, using the "become" parameter in ansible, but for the libvirt related content, it will be a bit hard... . It means only the libvirt connection calls is creating issue.

As part of https://github.com/crc-org/crc/issues/3541#issuecomment-1466302900 does using export VIRSH_DEFAULT_CONNECT_URI=qemu:///system solve the libvirt issue for ansible also now?

[cjeanner]

Hello @praveenkumar ! Thanks for your answer. So, it's a bit more complicated actually: I've checked some of the sudo calls, and the actual result. I'm not 100% sure we can prepare everything using ansible before we actually run crc setup or start.

Now, I'm wondering: Could it be possible to run crc setup with sudo, and passing a username? That would then make the command look like this:

$ sudo crc setup --user fooBar

The result would be:

• files are created in fooBar home directory, like it's done now
• tasks needing privileges will be launched as root
• this would allow to run a single ansible task with become: true
• this would make the sudoers really easier to maintain, with only one entry (if we want): fooBar ALL: NOPASSWD /path/to/crc setup --user fooBar

Would that approach be doable? My golang is rusted (no pun intended), but I may have a look (if I have time, that is... -.-)

As for the virsh content, it's all good, we can export the env var and see/manage the VM/network, so that's solved for us :)

Cheers,

C.

[cfergeau]

Ideally you'd be able to do the tasks listed by Praveen directly with ansible, and then use the skip-* config options to tell crc setup this is already done. If some are not possible to do with ansible, it will be easier to know which ones are problematic so that we can have a more specific discussion regarding how to solve them. I don't know much about ansible, but you say:

• this would allow to run a single ansible task with become: true
• this would make the sudoers really easier to maintain, with only one entry (if we want): fooBar ALL: NOPASSWD /path/to/crc setup --user fooBar

As an alternative, could you add fooBar ALL: NOPASSWD /path/to/crc setup to sudoers, and then use become_user: fooBar to run crc setup? (disclaimer, I don't' know much about ansible, I've searched from 'become' since you mentioned it and read https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_privilege_escalation.html )

[cjeanner]

I'll give it a try. Though we can't run sudo crc since it checks for the UID, and fails if it's running as root. But we get our way in, I think. We may probably close that issue, though a proper doc as to what is done (with commands and use) are running with privileges - that would help :).

Cheers,

C.

[praveenkumar]

@cjeanner can you put the details how you get your way and if you have that ansible role can you share it, in case someone also want to look into this?

[cjeanner]

@praveenkumar all will be in a new project - https://github.com/openstack-k8s-operators/ci-framework. For now, the "way to go" is to inject a temporary file in /etc/sudoers.d/ in order to allow just a subset of precise commands to be launched without password:

• https://github.com/openstack-k8s-operators/ci-framework/blob/main/ci_framework/roles/rhol_crc/tasks/sudoers_grant.yml
• https://github.com/openstack-k8s-operators/ci-framework/blob/main/ci_framework/roles/rhol_crc/vars/main.yml#L38-L46

Those authorizations should cover the usecase.

[praveenkumar]

@cjeanner Thanks, and how are you handing the libvirt sudo permissions for it? Using https://github.com/crc-org/crc/issues/3541#issuecomment-1466302900 ?

[cjeanner]

@praveenkumar yep - so using the system connection works perfectly - that's providing the same context access. Now, it MAY be possible the user running things has to be part of a group - we may need to investigate this, but so far, nothing has been needed. But my own env is a bit custom - will need to deploy a fresh system and double-check. If needed, we may need to inject a polkit config, but it's mostly "harmless" compared to calling sudo at this point :).

[gbraad]

@cfergeau @praveenkumar this is an issue which Zeeshan worked on. We still have some work that he did and some of the open questions.