"crc setup" then "crc start" fail when SELinux user is confined

[rmetrich]

General information

A Red Hat customer having his user confined to staff_u:staff_r:staff_t is trying to use crc setup command.
Confining users is MANDATORY when having STIG (DoD customers, etc.), it's hence important to fix this.

The tarball was downloaded from developers.redhat.com.

It appears not much works, for multiple reasons (non-exhaustive list, I dropped the investigation due to having too many issues in the end).

Operating System

Linux

Hypervisor

KVM

Did you run crc setup before crc start?

yes

Running on

VM

Steps to reproduce

Please see below the procedure I followed after downloading the tarball.

I tried installing as my user staff, mapped to staff_u SELinux user, to suit STIG requirements:

# useradd -Z staff_u staff
# echo redhat | passwd --stdin staff
# ssh staff@localhost
...
$ id -Z
staff_u:staff_r:staff_t:s0-s0:c0.c1023

---

crc setup

1. Initially crc setup fails because it tries to sudo to change an ownership

$ ./crc setup
[...]
ERRO Unable to set ownership of /home/staff/.crc/bin/crc-admin-helper-linux-amd64 to root:  exit status 1: chown: cannot access '/home/staff/.crc/bin/crc-admin-helper-linux-amd64': Permission denied 

The reason for failure is ownership cannot be changed because sudo executes in the context of the user, staff_t, but ownership can only be changed by sysadm_t.
A workaround is to have sudo rule below to switch the user, but in practice users won't be able to do this: (/etc/sudoers)

%wheel  ALL=(ALL)       TYPE=sysadm_t ROLE=sysadm_r ALL

2. Then crc setup fails when trying to start the systemd --user vsock socket

INFO Setting up crc daemon systemd socket units
ERRO Executing systemctl action failed:  exit status 1: Job failed. See "journalctl -xe" for details.

The reason for failure is missing rules in the SELinux policy:

# cat local_crc.te 
policy_module(local_crc, 1.0)

gen_require(`
    type staff_t;
')

allow staff_t self:vsock_socket create_stream_socket_perms;

# make -f /usr/share/selinux/devel/Makefile local_crc.pp
# semodule -i local_crc.pp

3. Later crc setup fails because it tries to execute udevadm

INFO Using root access: Reloading udev rules database 
ERRO udevadm executable not found

Here, the issue is due to SELinux: the program cannot execute udevadm because the latter is labeled with udev_exec_t but staff_t (the context of crc setup) cannot execute such context:

# ls -Z /usr/bin/udevadm 
system_u:object_r:udev_exec_t:s0 /usr/bin/udevadm

# sesearch -A -s staff_t -t udev_exec_t -c file -p execute
--> nothing

To bypass the error, I went with creating a wrapper to execute under sudo (so that it executes with sysadm_r role, see workaround for step 1):

$ mkdir ~/bin
$ cat > ~/bin/udevadm << EOF
#!/bin/sh
exec sudo /usr/bin/udevadm "\${@:-}"
EOF
$ chmox +x ~/bin/udevadm

With this, crc setup completed with success.

crc start

1. crc start failed immediately

WARN Preflight checks failed during `crc start`, please try to run `crc setup` first in case you haven't done so yet 
ERRO vsock udev rule does not exist

With strace, I could see the "Preflight" warning was due to yet another missing rule:

36832 16:09:48.169951 newfstatat(AT_FDCWD</home/staff/crc-linux-2.55.1-amd64>, "/etc/udev/rules.d/99-crc-vsock.rules", 0xc0008e7ca8, AT_SYMLINK_NOFOLLOW) = -1 EACCES (Permission denied) <0.000151>

# ls -lZ /etc/udev/rules.d/99-crc-vsock.rules
-rw-r--r--. 1 root root staff_u:object_r:udev_rules_t:s0 59 Nov 15 16:10 /etc/udev/rules.d/99-crc-vsock.rules

# sesearch -A -s staff_t -t udev_rules_t -c file -p open
--> nothing

But it appears more stuff is required, stracing shows a lot of EACCES.

2. In the end, in Permissive + dontaudit disabled, I got this below, but probably some are not needed or maybe some are still missing (since I didn't do anything after crc started):

#============= staff_t ==============
allow staff_t http_port_t:tcp_socket name_bind;

#!!!! This avc can be allowed using the boolean 'staff_use_svirt'
allow staff_t kvm_device_t:chr_file getattr;
allow staff_t udev_rules_t:file getattr;

#!!!! This avc can be allowed using one of the these booleans:
#     nis_enabled, selinuxuser_tcp_server
allow staff_t unreserved_port_t:tcp_socket name_bind;
allow staff_t vsock_device_t:chr_file { read write };

#============= svirt_t ==============
allow svirt_t virtd_t:dir search;
allow svirt_t virtd_t:file { getattr open read };

CRC version

https://developers.redhat.com/content-gateway/rest/mirror/pub/openshift-v4/clients/crc/latest/crc-linux-amd64.tar.xz

CRC status

CRC config

Host Operating System

RHEL9 + SELinux + Confined users

Expected behavior

Users mapped to SELinux staff_u users can start crc

Actual behavior

"EACCES 13 Permission denied" in many places

CRC Logs

Additional context

This is a copy of RHEL JIRA CRC-1375

[praveenkumar]

@rmetrich Thank you for the issue and steps to workaround it do the user have sudo access on that even it is confined? I am thinking if this is something we should document it or automate to make it part of crc binary. If sudo access is not available then it might be hard to do changes on socket side or even udev side.

[rmetrich]

Yes he has sudo access, but actually I'm wondering why there is need for sudo and such, if, in the end, all this "crc" is running as the user (I could see some part is running as a systemd user instance service).