atomic-system-containers
atomic-system-containers copied to clipboard
projectatomic
WIP: firewalld system containers
The original thought came from https://bugzilla.redhat.com/show_bug.cgi?id=1403331
The container is still work in progress. The docker file can be compiled successfully. When running the command, the service will not error out. However, the command will time out eventually (not executing properly). I have a few thoughts for debugging, will take a while to figure it out.
Have it here first for some early feedback :). (will modify the commit message once lifting the WIP )
Addressed most of the comments above. Now the service seem to run with active running on Atomic Host shown below: ( will add README.md sooner and likely tmr )
[root@localhost firewalld]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2017-12-04 04:30:33 UTC; 9min ago
Process: 3852 ExecStop=/bin/runc --systemd-cgroup kill firewalld (code=exited, status=0/SUCCESS)
Main PID: 3865 (runc)
Tasks: 7 (limit: 4915)
Memory: 5.1M
CPU: 13ms
CGroup: /system.slice/firewalld.service
└─3865 /bin/runc --systemd-cgroup run firewalld
Dec 04 04:30:33 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 04 04:30:33 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
[root@localhost firewalld]# rpm-ostree status
State: idle
Deployments:
fedora-atomic:fedora/26/x86_64/atomic-host
Version: 26.150 (2017-10-14 23:19:12)
BaseCommit: d518b37c348eb814093249f035ae852e7723840521b4bcb4a271a80b5988c44a
Commit: 14af07323295072070ab4964bd5df50ea1745a068b0ed80c3c52326b215d9296
GPGSignature: Valid signature by E641850B77DF435378D1D7E2812A6B4B64DAB85D
LayeredPackages: git vim
● fedora-atomic:fedora/26/x86_64/atomic-host
Version: 26.150 (2017-10-14 23:19:12)
BootedBaseCommit: d518b37c348eb814093249f035ae852e7723840521b4bcb4a271a80b5988c44a
Commit: 0511d91a45a2733667caae26c4263183d52a30e828eb4c18a648c12c3dda7be7
LiveCommit: 14af07323295072070ab4964bd5df50ea1745a068b0ed80c3c52326b215d9296
GPGSignature: Valid signature by E641850B77DF435378D1D7E2812A6B4B64DAB85D
LayeredPackages: git
fedora-atomic:fedora/26/x86_64/atomic-host
Version: 26.150 (2017-10-14 23:19:12)
BaseCommit: d518b37c348eb814093249f035ae852e7723840521b4bcb4a271a80b5988c44a
Commit: 0511d91a45a2733667caae26c4263183d52a30e828eb4c18a648c12c3dda7be7
GPGSignature: Valid signature by E641850B77DF435378D1D7E2812A6B4B64DAB85D
LayeredPackages: git
However, one problem remain is that, when I run systemctl stop firewalld, it shows me a failed status:
[root@localhost firewalld]# systemctl stop firewalld
[root@localhost firewalld]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2017-12-04 04:42:07 UTC; 3s ago
Process: 4119 ExecStop=/bin/runc --systemd-cgroup kill firewalld (code=exited, status=0/SUCCESS)
Process: 3865 ExecStart=/bin/runc --systemd-cgroup run firewalld (code=exited, status=143)
Main PID: 3865 (code=exited, status=143)
CPU: 30ms
Dec 04 04:30:33 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 04 04:30:33 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
Dec 04 04:42:07 localhost.localdomain systemd[1]: Stopping firewalld - dynamic firewall daemon...
Dec 04 04:42:07 localhost.localdomain systemd[1]: firewalld.service: Main process exited, code=exited, status=143/n/a
Dec 04 04:42:07 localhost.localdomain systemd[1]: Stopped firewalld - dynamic firewall daemon.
Dec 04 04:42:07 localhost.localdomain systemd[1]: firewalld.service: Unit entered failed state.
Dec 04 04:42:07 localhost.localdomain systemd[1]: firewalld.service: Failed with result 'exit-code'.
I run a check to look for journal look, there is one suscipious line:
<warn> [1512362701.1380] firewall: [0x55f9b3941f00,change:"eth0"]: complete: request failed (An SELinux policy prevents this sender from sending this message to this recipient, 0 matched rules; type="method_call", sender=":1.2" (uid=0 pid=709 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") interface="org.fedoraproject.FirewallD1.zone" member="changeZone" error name="(unset)" requested_reply="0" destination=":1.82" (uid=0 pid=4220 comm="/usr/bin/python3 -Es /usr/sbin/firewalld --nofork " label="system_u:system_r:container_runtime_t:s0"))
and this is the link for entire log, any ideas why is that happening ? https://paste.fedoraproject.org/paste/49rl5Pgd4bxCV~y5n9bIjw.
I am personally not familiar with firewalld. So, also wonders, other than the stopping failure, is the current behavior for firewalld behaving correctly?
Thanks for your time =).
could you try with KillMode=none in the systemd unit file? Does that solve the issue you are seeing?
Another thing I was wondering about, does firewalld load kernel modules? If yes, I think we should bind mount /usr/lib/modules from the host, to avoid mismatches between the host and the modules used by the container.
Thanks for the review/commenting =).
could you try with KillMode=none in the systemd unit file?
~~hmm.. I tried the container again on my f27 host, now it seems not recognizing /usr/local/share~~ ~~'s actions...... any resolutions to that? Unfortunately, on my f26 host ( without denied error)~~ ~~, killmode = none seems also not solving the problem of failing from systemctl stop firewalld,~~ ~~I still suspect that it might be due to the selinux problem reported earlier.~~
~~Below is the info from my f27 host. Thoughts? ( only add one kill mode = none , and nothing else)~~
[root@localhost firewalld.0]# runc --systemd-cgroup run firewalld
2017-12-04 16:29:30 ERROR: Exception DBusException: org.freedesktop.DBus.Error.AccessDenied: Connection ":1.98" is not allowed to own the service "org.fedoraproject.FirewallD1" due to security policies in the configuration file
[root@localhost firewalld.0]# ll -a /usr/local/share/polkit-1/actions/
total 12
drwxr-xr-x. 2 root root 164 Dec 4 16:28 .
drwxr-xr-x. 3 root root 21 Dec 4 15:58 ..
-rw-r--r--. 1 root root 3916 Jan 1 1970 org.fedoraproject.FirewallD1.desktop.policy.choice
-rw-r--r--. 1 root root 4024 Jan 1 1970 org.fedoraproject.FirewallD1.policy
-rw-r--r--. 1 root root 4024 Jan 1 1970 org.fedoraproject.FirewallD1.server.policy.choice
[root@localhost firewalld.0]# rpm-ostree status
State: idle
Deployments:
* fedora-atomic:fedora/27/x86_64/atomic-host
Version: 27.16 (2017-11-28 23:08:35)
BaseCommit: 86727cdbc928b7f7dd0e32f62d3b973a8395d61e0ff751cfea7cc0bc5222142f
GPGSignature: Valid signature by 860E19B0AFA800A1751881A6F55E7430F5282EE4
LayeredPackages: git
fedora-atomic:fedora/27/x86_64/atomic-host
Version: 27.16 (2017-11-28 23:08:35)
Commit: 86727cdbc928b7f7dd0e32f62d3b973a8395d61e0ff751cfea7cc0bc5222142f
GPGSignature: Valid signature by 860E19B0AFA800A1751881A6F55E7430F5282EE4
Another thing I was wondering about, does firewalld load kernel modules? If yes, I think we should bind mount /usr/lib/modules from the host,
ah, good point, I think we do need it =), https://github.com/firewalld/firewalld/blob/master/src/firewall/functions.py#L336. we just bind as ro mount right? ( and rprivate in the options), because we do not want to modify it?
EDIT The failure for dbus access error seems to be a flake... please disregard it for now =)
a read-only rslave bind mount is fine, in case it is ever overriden on the host, we will get the changes.
I've not seen the SELinux failure and the KillMode= helped me to solve the failure on exit.
Could you try to temporarily disable selinux and see if it works? We can address the selinux issue later, for example you could try to set the SELinux label in the config.json.template file to something like "system_u:system_r:firewalld_t:s0"
a read-only rslave bind mount is fine, in case it is ever overriden on the host, we will get the changes
sure, I will try to find usages of rslave, and apply it to the config files =D.
I've not seen the SELinux failure and the KillMode= helped me to solve the failure on exit
... Now the KillMode=none solves the problem for me. i guess it was probably due to the internal errors I had some where earlier, sorry for my misunderstanding =(. But now the container seems to work =D, will try and test it a bit more, and going to complete the README.md.
Thanks for the info and your time =D.
@peterbaouoft let me know when you are ready for others to test. I'll be happy to give it a go!
@ashcrow, sure, I still have some doubts currently about some parts of the system containers, may want to confirm with gisueppe about those. After those doubts are cleared, I think this system container should be ready for test =).
@peterbaouoft sounds good. And to reiterate, good work so far! Hope you're learning quite a bit 😁
Pushed a fixup ⬆️, and also a documentation for installation instructions. Now the system container seems no longer have the failed state when running systemctl stop firewalld
There are also a few doubts that I came across while making this container. Hopefully those won't bother you too much =).
Q1: It might be a doubt that is related to previously asked questions, but I still think I am a bit unsure about the concept.. The doubt is that what might be the purpose that we copy /etc/firewalld into the host? I noticed during the execution of firewalld, files in /usr/lib/firewalld are used within the container.
I wonder if /etc/firewalld can be used by the firewalld binary in the container. (without copying it to the host).
My Guess: we might want user to configure the /etc/firewalld themselves, that might be one of the reasons why we copy it to the host
Q2: For my other testing machines that are not atomic host, some of them seems not including policy kit by default, leading to some "service not found" warnings during execution. If that is the case, do we want to worry about the case where polkit is not installed?
Q3: Minor: I noticed that there are some empty folders left over when running atomic containers delete firewalld. I see we have https://github.com/projectatomic/atomic/pull/1131 that address this problem, so do we still have empty folders left over when the patch is applied? ( sorry, can't run a easy check, it seemed hard to build and apply upstream atomic into Atomic Host).
The output is as following: as you can see /etc/firewalld and its empty subdirectories are not deleted
[root@localhost ~]# atomic containers list -a
[root@localhost ~]# cd /etc/firewalld
bash: cd: /etc/firewalld: No such file or directory
[root@localhost ~]# atomic install --system --system-package=no --name=firewalld firewalld
Extracting to /var/lib/containers/atomic/firewalld.0
File /./etc already exists.
File /./usr already exists.
File /./var already exists.
File /etc/dbus-1 already exists.
File /etc/sysconfig already exists.
File /etc/dbus-1/system.d already exists.
File /etc/sysconfig/firewalld already exists.
File /usr/local already exists.
File /usr/local/share already exists.
File /usr/local/share/polkit-1 already exists.
File /usr/local/share/polkit-1/actions already exists.
File /var/log already exists.
Created file /etc/dbus-1/system.d/FirewallD.conf
Created file /etc/firewalld
Created file /etc/firewalld/firewalld-server.conf
Created file /etc/firewalld/firewalld-standard.conf
Created file /etc/firewalld/firewalld-workstation.conf
Created file /etc/firewalld/firewalld.conf
Created file /etc/firewalld/helpers
Created file /etc/firewalld/icmptypes
Created file /etc/firewalld/ipsets
Created file /etc/firewalld/lockdown-whitelist.xml
Created file /etc/firewalld/services
Created file /etc/firewalld/zones
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.desktop.policy.choice
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.policy
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.server.policy.choice
Created file /var/log/firewalld
systemctl daemon-reload
systemctl enable firewalld
[root@localhost ~]# atomic containers delete firewalld
Do you wish to delete the following images?
ID NAME IMAGE_NAME STORAGE
firewalld firewalld firewalld ostree
Confirm (y/N) y
systemctl disable --now firewalld
systemctl daemon-reload
[root@localhost ~]# ll /etc/firewalld/
total 0
drwxr-xr-x. 2 root root 6 Dec 4 21:01 helpers
drwxr-xr-x. 2 root root 6 Dec 4 21:01 icmptypes
drwxr-xr-x. 2 root root 6 Dec 4 21:01 ipsets
drwxr-xr-x. 2 root root 6 Dec 4 21:01 services
drwxr-xr-x. 2 root root 6 Dec 4 21:01 zones
-
yes, the reason why we copy the files to
/etcand bind mount them into the container is that in this way users can configure it in the usual way. Otherwise it would not be possible to modify/etcin the container. -
good point. Should we make the dependency on policykit optional?
-
Patch #1131 should fix that. If you want to try it, you can run atomic directly from the local directory. Clone the repository and use
./atomicinstead of/usr/bin/atomic.
Patch #1131 should fix that. If you want to try it, you can run atomic directly from the local directory. Clone the repository and use ./atomic instead of /usr/bin/atomic.
$ sudo python ./atomic
good point. Should we make the dependency on policykit optional?
I am not sure how hard it would be to do optional dependency checking =), but willing to do so if needed. But currently this container is targeting for Atomic Host right? If so, we don't necessarily need to install the polkit as we already have polkit as base package in Atomic Host. ( we can just assume polkit is there :p)
So my opinion is, we don't need to have this optional dependency( we can assume polkit is there on host) unless people outside atomic host wants it? WDYT @giuseppe
Patch #1131 should fix that. If you want to try it, you can run atomic directly from the local directory. Clone the repository and use ./atomic instead of /usr/bin/atomic.
cool, lemme try that :p
$ sudo python atomic
thx, @ashcrow :-)
I was suggesting about having the systemd dependency as optional, so that it is used only when polkit is available (i.e. Wants= in the systemd service file)
I was suggesting about having the systemd dependency as optional, so that it is used only when polkit is available (i.e. Wants= in the systemd service file)
ah oops, I misunderstood earlier. Sorry for my poor english. In this case, do we want Require= or Wants=, I think both should be ok since Wants is A weaker version of Requires=
so it will be Wants=polkit.service or Requires=polkit.service right? I would try with Wants=polkit.service first since that is what you suggested :p.
It is odd..... now when I install the container to a new (freshly) installed Atomic Host, it can't seem to find /usr/local/share's policy file via dbus any more =(. ( DBusAcessError for org.fedoraproject.FirewallD1)
But then, if ... I uninstall the container, and reinstall back, it runs without a problem..(#facepalm). Then every reinstall after that seems ok.... Interesting... Need investigating/testing more to see if the behavior can always be reproduced. (Sorry for the delay so far...)
...After some painful debugging..., I have found two ways currently that a system container for firewalld could fail to start. But I am unaware if those causes will be the same. Both errors appear to happen consistently ( at least had three tries, hopefully it is not just only me =( ).
For case 1, the reproducing step is too long, so did not have a log of it...
1: Fresh install firewalld on a freshly provisioned Atomic Host (i.e rpm-ostree install git, then follow instructions for installing it). The first systemctl start firewalld will fail. Then by atomic uninstall firewalld and atomic install --system --system-package=no --name=firewalld firewalld, the service will likely have an active status. ( shown below for an example)
2: This one is reproducible without having a freshly installed Atomic Host. It only requires you having a "working" firewalld system container.
The steps to reproduce are as follows: on a f27 atomic host, steps to do after seeing a failed firewalld service
[root@localhost ~]# atomic uninstall firewalld
systemctl disable --now firewalld
systemctl daemon-reload
[root@localhost ~]# atomic install --system --system-package=no --name=firewalld firewalld
Extracting to /var/lib/containers/atomic/firewalld.0
Created file /etc/dbus-1/system.d/FirewallD.conf
Created file /etc/firewalld
Created file /etc/firewalld/firewalld-server.conf
Created file /etc/firewalld/firewalld-standard.conf
Created file /etc/firewalld/firewalld-workstation.conf
Created file /etc/firewalld/firewalld.conf
Created file /etc/firewalld/helpers
Created file /etc/firewalld/icmptypes
Created file /etc/firewalld/ipsets
Created file /etc/firewalld/lockdown-whitelist.xml
Created file /etc/firewalld/services
Created file /etc/firewalld/zones
Created file /etc/sysconfig/firewalld
Created file /usr/local/share/polkit-1
Created file /usr/local/share/polkit-1/actions
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.desktop.policy.choice
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.policy
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.server.policy.choice
Created file /var/log/firewalld
systemctl daemon-reload
systemctl enable firewalld
[root@localhost ~]# systemctl start firewalld
[root@localhost ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2017-12-05 21:37:56 UTC; 3s ago
Main PID: 1536 (runc)
Tasks: 8 (limit: 4915)
Memory: 468.0K
CPU: 2ms
CGroup: /system.slice/firewalld.service
└─1536 /bin/runc --systemd-cgroup run firewalld
Dec 05 21:37:55 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 05 21:37:56 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
After that, the failure can be reproduced through the following commands:
[root@localhost ~]# systemctl stop dbus
Warning: Stopping dbus.service, but it can still be activated by:
dbus.socket
[root@localhost ~]# systemctl stop polkit
[root@localhost ~]# systemctl start dbus
[root@localhost ~]# systemctl start polkit
[root@localhost ~]# systemctl start firewalld
Job for firewalld.service failed because the control process exited with error code.
See "systemctl status firewalld.service" and "journalctl -xe" for details.
[root@localhost ~]# date
Tue Dec 5 21:40:41 UTC 2017
=====================Output in /var/log/firewalld which tracks error messages============
2017-12-05 21:40:29 ERROR: Exception DBusException: org.freedesktop.DBus.Error.AccessDenied: Connection ":1.10" is not allowed to own the service "org.fedoraproject.FirewallD1" due to security policies in the configuration file
2017-12-05 21:40:30 ERROR: Exception DBusException: org.freedesktop.DBus.Error.AccessDenied: Connection ":1.15" is not allowed to own the service "org.fedoraproject.FirewallD1" due to security policies in the configuration file
2017-12-05 21:40:31 ERROR: Exception DBusException: org.freedesktop.DBus.Error.AccessDenied: Connection ":1.20" is not allowed to own the service "org.fedoraproject.FirewallD1" due to security policies in the configuration file
2017-12-05 21:40:32 ERROR: Exception DBusException: org.freedesktop.DBus.Error.AccessDenied: Connection ":1.25" is not allowed to own the service "org.fedoraproject.FirewallD1" due to security policies in the configuration file
Any clues what could be the causes for the problem? I am not sure at all how to solve this atm.. Thanks in advance =). @giuseppe can you take a look when you have time? :-). P.S: I can provide slightly more details if needed, do not want to make the content look too long here
EDIT: the dockerfile I tested is mostly similar to the current one, except that the base image is From fedora (I could not download the image from registry.fedoraproject.org today)
the selinux issue we were seeing is fixed by:
https://github.com/projectatomic/atomic/pull/1143
Could you please verify if it works for you?
Also, dbus must be reloaded before it considers the new configuration. I am fine if you just document this step. After the atomic install, the user needs to run "killall -SIGHUP dbus-daemon".
Doesn't the same issue happen with the firewalld rpm? I don't see any special handling in the post scripts for it.
Could you please verify if it works for you?
looks like it is working \o/, now the second case(dbus) no longer fails out after the patch is applied. (On my testing machine)
[root@localhost ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2017-12-06 20:46:46 UTC; 25s ago
Main PID: 2234 (runc)
Tasks: 7 (limit: 4915)
Memory: 1.3M
CPU: 7ms
CGroup: /system.slice/firewalld.service
└─2234 /bin/runc --systemd-cgroup run firewalld
Dec 06 20:46:46 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 06 20:46:46 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
[root@localhost ~]# systemctl stop dbus
Warning: Stopping dbus.service, but it can still be activated by:
dbus.socket
[root@localhost ~]# systemctl start dbus
[root@localhost ~]# systemctl start polkit
[root@localhost ~]# systemctl start firewalld
[root@localhost ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2017-12-06 20:47:34 UTC; 3s ago
Process: 2309 ExecStop=/bin/runc --systemd-cgroup kill firewalld (code=exited, status=0/SUCCESS)
Main PID: 2331 (runc)
Tasks: 7 (limit: 4915)
Memory: 1.3M
CPU: 7ms
CGroup: /system.slice/firewalld.service
└─2331 /bin/runc --systemd-cgroup run firewalld
Dec 06 20:47:33 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 06 20:47:34 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
Also, dbus must be reloaded before it considers the new configuration. I am fine if you just document this step. After the atomic install, the user needs to run "killall -SIGHUP dbus-daemon".
ah a bit weird, the above worked without me trying the killall command, I will also document it later just in case =).
Doesn't the same issue happen with the firewalld rpm? I don't see any special handling in the post scripts for it.
for rpm, I am not seeing this behavior. But since the patch kinda solves the problem for me, let's not worry about it then?
EDIT: now it appears the fresh install problem also got fixed :D. here is the log if you are interested. Note: it is really long as I need to apply upstream changes with the selinux fix: https://paste.fedoraproject.org/paste/X2m3Meu2AI-DJhHB9JwF-A
@giuseppe , Seems like with the new patch applied, both failing cases before all passes as well =)
[root@localhost atomic]# rpm-ostree status
State: idle
Deployments:
* fedora-atomic:fedora/27/x86_64/atomic-host
Version: 27.16 (2017-11-28 23:08:35)
BaseCommit: 86727cdbc928b7f7dd0e32f62d3b973a8395d61e0ff751cfea7cc0bc5222142f
GPGSignature: Valid signature by 860E19B0AFA800A1751881A6F55E7430F5282EE4
LayeredPackages: PyYAML gcc gcc-go git golang-github-cpuguy83-go-md2man libffi-devel ostree-devel pylint python-dbus python-devel python-gobject-base python-slip-dbus python2-coverage python2-dateutil python2-pylint python3-pylint redhat-rpm-config rpm-python
Unlocked: development
fedora-atomic:fedora/27/x86_64/atomic-host
Version: 27.16 (2017-11-28 23:08:35)
Commit: 86727cdbc928b7f7dd0e32f62d3b973a8395d61e0ff751cfea7cc0bc5222142f
GPGSignature: Valid signature by 860E19B0AFA800A1751881A6F55E7430F5282EE4
[root@localhost atomic]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2017-12-08 18:54:39 UTC; 2min 44s ago
Process: 1951 ExecStop=/bin/runc --systemd-cgroup kill firewalld (code=exited, status=0/SUCCESS)
Main PID: 1970 (runc)
Tasks: 8 (limit: 4915)
Memory: 5.4M
CPU: 19ms
CGroup: /system.slice/firewalld.service
└─1970 /bin/runc --systemd-cgroup run firewalld
Dec 08 18:54:38 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 08 18:54:39 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
[root@localhost atomic]# systemctl stop dbus
Warning: Stopping dbus.service, but it can still be activated by:
dbus.socket
[root@localhost atomic]# systemctl start firewalld
[root@localhost atomic]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2017-12-08 18:57:32 UTC; 4s ago
Process: 2153 ExecStop=/bin/runc --systemd-cgroup kill firewalld (code=exited, status=0/SUCCESS)
Main PID: 2172 (runc)
Tasks: 7 (limit: 4915)
Memory: 5.3M
CPU: 19ms
CGroup: /system.slice/firewalld.service
└─2172 /bin/runc --systemd-cgroup run firewalld
Dec 08 18:57:31 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 08 18:57:32 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
This is the host applying atomic upstream changes, the firewalld container might be ready for other folks to test =). The service is passing. Tho, tbh, I am not too certain what will the expected firewalld functionality be, so if there is any functionality issues, please let me know :p.
Also Note: currently only firewalld binary is shipped, (i.e no firewall-cmd or other related commands is there with it yet).
Very close. I ended up hitting a missing directory and an SELinux issue. Here is how I found them ...
Install
[root@fedora-27-ah-beta ~]# systemctl status firewalld
Unit firewalld.service could not be found.
[root@fedora-27-ah-beta ~]# atomic install --system --system-package=no -n firewalld --storage ostree docker:firewalld:latest
Extracting to /var/lib/containers/atomic/firewalld.0
Created file /etc/dbus-1/system.d/FirewallD.conf
Created file /etc/firewalld/firewalld-server.conf
Created file /etc/firewalld/firewalld-standard.conf
Created file /etc/firewalld/firewalld-workstation.conf
Created file /etc/firewalld/firewalld.conf
Created file /etc/firewalld/lockdown-whitelist.xml
Created file /etc/sysconfig/firewalld
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.desktop.policy.choice
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.policy
Created file /usr/local/share/polkit-1/actions/org.fedoraproject.FirewallD1.server.policy.choice
Created file /var/log/firewalld
systemctl daemon-reload
systemctl enable firewalld
[root@fedora-27-ah-beta ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: inactive (dead)
[root@fedora-27-ah-beta ~]# systemctl start firewalld
Job for firewalld.service failed because the control process exited with error code.
See "systemctl status firewalld.service" and "journalctl -xe" for details.
Look at issue
Investigate
[root@fedora-27-ah-beta ~]# cd /var/lib/containers/atomic/firewalld.0/
[root@fedora-27-ah-beta firewalld.0]# /bin/runc --systemd-cgroup run 'firewalld'
container_linux.go:265: starting container process caused "process_linux.go:348: container init caused \"rootfs_linux.go:57: mounting \\\"/run/firewalld\\\" to rootfs \\\"/var/lib/containers/atomic/firewalld.0/rootfs\\\" at \\\"/run/firewalld\\\" caused \\\"stat /run/firewalld: no such file or directory\\\"\""
Create dir and try again
[root@fedora-27-ah-beta firewalld.0]# sudo mkdir /run/firewalld
[root@fedora-27-ah-beta firewalld.0]# /bin/runc --systemd-cgroup run 'firewalld'
2017-12-08 19:36:47 ERROR: Exception DBusException: org.freedesktop.DBus.Error.AccessDenied: Connection ":1.207" is not allowed to own the service "org.fedoraproject.FirewallD1" due to security policies in the configuration file
Run ausearch -l
type=AVC msg=audit(1512761962.841:424): avc: denied { read } for pid=1 comm="systemd" name="firewalld" dev="dm-0" ino=28120247 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:container_share_t:s0 tclass=file permissive=0
Later I remembered this comment about using a specific version of atomic. I used it but got the same error.
Very close. I ended up hitting a missing directory and an SELinux issue
Ouch, and thanks for the feedback =), the missing directory is expected as that should be autogenerated by systemd with
RuntimeDirectory=${NAME} line in service.template file.
Tho, for selinux is a bit weird... it seems working on my side T_T. I might need more info to debug here, hopefully you won't mind =). Did you try killall -SIGHUP dbus-daemon after applying the patch ? Also, do note the patch does not fix the issue in place, you need to reinstall the container because selinux label applied to files seems not get reset with the patch.
The patch for restoring selinux label should be effective at installation time(copying files) to the host.
So the steps what I usually do on a fresh installed Atomic Host for firewalld container is:
1: Layer all the packages needed for atomic upstream so I can do make install
2: git clone atomic and git clone atomic-system-containers
3: checkout specific version of atomic, and firewalld
4: do an docker build, atomic pull, atomic install path
5: reload dbus configuration
6: Start the service
Wondering do you also follow a similar path? @ashcrow. P.S: The set up is a bit messy here, sorry for the confusion.
Following exactly the steps in the paste (installing packages, unlocking, overwriting atomic rather than running from source, killing dbus) did end up working. I'm not sure if it's due to an update in packages or if the source runs didn't work as expected. 👍
This probably is blocked on https://github.com/projectatomic/atomic/pull/1143 at this point.
Update: Now another problem arises for the container ( so many cases to consider about #facepalm):
The log is as follows: ( this is after applying the selinux fix patch)
[root@localhost firewalld]# rpm-ostree status
State: idle
Deployments:
* fedora-atomic:fedora/27/x86_64/atomic-host
Version: 27.16 (2017-11-28 23:08:35)
BaseCommit: 86727cdbc928b7f7dd0e32f62d3b973a8395d61e0ff751cfea7cc0bc5222142f
GPGSignature: Valid signature by 860E19B0AFA800A1751881A6F55E7430F5282EE4
LayeredPackages: PyYAML gcc gcc-go git golang-github-cpuguy83-go-md2man libffi-devel pylint python-dbus python-devel python-gobject-base python-slip-dbus python2-coverage python2-dateutil python2-pylint python3-pylint redhat-rpm-config rpm-python
Unlocked: development
fedora-atomic:fedora/27/x86_64/atomic-host
Version: 27.16 (2017-11-28 23:08:35)
Commit: 86727cdbc928b7f7dd0e32f62d3b973a8395d61e0ff751cfea7cc0bc5222142f
GPGSignature: Valid signature by 860E19B0AFA800A1751881A6F55E7430F5282EE4
[root@localhost firewalld]# killall -SIGHUP dbus-daemon
[root@localhost firewalld]# systemctl start firewalld
[root@localhost firewalld]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/etc/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2017-12-15 15:31:11 UTC; 5s ago
Main PID: 2254 (runc)
Tasks: 8 (limit: 4915)
Memory: 144.0K
CPU: 1ms
CGroup: /system.slice/firewalld.service
└─2254 /bin/runc --systemd-cgroup run firewalld
Dec 15 15:31:09 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 15 15:31:11 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
[root@localhost firewalld]# atomic run firewalld firewall-cmd --state
ERROR:dbus.proxies:Introspect error on :1.41:/org/fedoraproject/FirewallD1: dbus.exceptions.DBusException: org.freedesktop.PolicyKit1.Error.Failed: Action org.fedoraproject.FirewallD1.info is not registered
Error: Action org.fedoraproject.FirewallD1.config is not registered
=====================================================================
A suspicious warning in journal log is as follows:
localhost.localdomain NetworkManager[716]: <warn> [1513351872.5476] firewall: [0x55fdb04f3180,change:"eth0"]: complete: request failed (Action org.fedoraproject.FirewallD1.config is not registered
I suspect one of the causes may be that applications do not tend to recognize the policy files copied /usr/local/share. The default location is /usr/share, but we able unable to copy files into /usr on atomic host. Currently investigating if a fix can exist for this problem
if you don't find an answer, could you try asking on the polkit-devel mailing list?
For a better understanding, I've tried to make the changes required into polkit. I've tried locally with the version of polkit from my branch and it solves the problem you are seeing. I changed the system container to copy the files under /etc/polkit-1/actions:
https://github.com/giuseppe/polkit/tree/polkit-alternative-actions-directory
Let's see where the discussion goes, and I could cleanup the patch if needed. I am not sure it is an acceptable version yet since it changes the API.
@peterbaouoft could you file a bugzilla for polkit to allow an alternative directory for reading these files?
@peterbaouoft could you please followup with the polkit people so to enable this use case for system containers on Atomic?