Firewalld not working in latest releases

[nkinchev]

Upgraded from 36420 to latest (36510) and Firewalld stopped working. Replicated the issue on second server.

Unfortunately there is not too much useful information in the logs:

systemctl status firewalld
× firewalld.service - firewalld - dynamic firewall daemon
     Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Sun 2022-06-19 08:41:35 PDT; 4min 57s ago
       Docs: man:firewalld(1)
    Process: 7578 ExecStart=/usr/bin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=1/FAILURE)
   Main PID: 7578 (code=exited, status=1/FAILURE)

Jun 19 08:41:35 ****** systemd[1]: Starting firewalld - dynamic firewall daemon...
Jun 19 08:41:35 ****** systemd[1]: firewalld.service: Main process exited, code=exited, status=1/FAILURE
Jun 19 08:41:35 ****** systemd[1]: firewalld.service: Failed with result 'exit-code'.
Jun 19 08:41:35 ****** systemd[1]: Failed to start firewalld - dynamic firewall daemon.

journalctl -xeu firewalld.service

Jun 19 08:41:35 ******* systemd[1]: Starting firewalld - dynamic firewall daemon...
░░ Subject: A start job for unit firewalld.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit firewalld.service has begun execution.
░░ 
░░ The job identifier is 684.
Jun 19 08:41:35 ****** systemd[1]: firewalld.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStart= process belonging to unit firewalld.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Jun 19 08:41:35 ******* systemd[1]: firewalld.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit firewalld.service has entered the 'failed' state with result 'exit-code'.
Jun 19 08:41:35 ******* systemd[1]: Failed to start firewalld - dynamic firewall daemon.
░░ Subject: A start job for unit firewalld.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit firewalld.service has finished with a failure.
░░ 
░░ The job identifier is 684 and the job result is failed.

I’m guessing could be related to the python3 version. Do not think Firewalld was updated between latest versions some dependency has changed.

[bryteise]

Looks like firewalld changed the default backend to nftables. I've added in the dependency to the package but you can go back to using iptables in the config I think and it might be okay.

[nkinchev]

Thank you for the reply. I'm not using iptables only nftables. I could reproduce with freshly installed firewalld with no/default rules (could be still loading some default nftables rules). I rolled back to 36420. Let me know what version to test.

[nkinchev]

Not sure is default backend issue as default to nftables was done long time ago and clr firewalld code was not changed since Jan 22. Could be python version issue, it was updated beginning of the month.

[bryteise]

Hrm well I don't see the package ever having nftables as a dependency. So I'd guess a change in firewalld somewhere causes the import of the nftables module that wasn't occurring before.

[hksdpc255]

Latest version update (36560) doesn't solve the problem.

[hksdpc255]

# firewalld --nofork --nopid
Traceback (most recent call last):
  File "/usr/bin/firewalld", line 215, in <module>
    main()
  File "/usr/bin/firewalld", line 210, in main
    startup(args)
  File "/usr/bin/firewalld", line 163, in startup
    from firewall.server import server
  File "/usr/lib/python3.10/site-packages/firewall/server/server.py", line 40, in <module>
    from firewall.server.firewalld import FirewallD
  File "/usr/lib/python3.10/site-packages/firewall/server/firewalld.py", line 30, in <module>
    from firewall.core.fw import Firewall
  File "/usr/lib/python3.10/site-packages/firewall/core/fw.py", line 34, in <module>
    from firewall.core import nftables
  File "/usr/lib/python3.10/site-packages/firewall/core/nftables.py", line 35, in <module>
    from nftables.nftables import Nftables
ModuleNotFoundError: No module named 'nftables'

$ firewall-offline-cmd --version
Traceback (most recent call last):
  File "/usr/bin/firewall-offline-cmd", line 37, in <module>
    from firewall.core.fw import Firewall
  File "/usr/lib/python3.10/site-packages/firewall/core/fw.py", line 34, in <module>
    from firewall.core import nftables
  File "/usr/lib/python3.10/site-packages/firewall/core/nftables.py", line 35, in <module>
    from nftables.nftables import Nftables
ModuleNotFoundError: No module named 'nftables'

[hksdpc255]

@bryteise Is there anything temporary workaround? It's a very serious security regression for my server.

[bryteise]

Ah looks like nftables is a bit broken. You can try and do a mv /usr/lib/python3.10/site-packages/nftables-*.egg/nftables /usr/lib/python3.10/site-packages/ and see if that gets you in better shape.

[hksdpc255]

I tried this and firewalld successfully started

ln -sr /usr/lib/python3.10/site-packages/nftables-*.egg/nftables /usr/lib/python3.10/site-packages/nftables

But firewalld companies about this

ERROR: Failed to load '/etc/firewalld/firewalld.conf': [Errno 2] No such file or directory: '/etc/firewalld/firewalld.conf'
WARNING: [Errno 2] No such file or directory: '/etc/firewalld/firewalld.conf'
WARNING: Using fallback firewalld configuration settings.
WARNING: INVALID_IPV: 'ipv4' is not a valid backend or is unavailable
WARNING: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
WARNING: COMMAND_FAILED: INVALID_IPV: 'ipv4' is not a valid backend or is unavailable
WARNING: COMMAND_FAILED: INVALID_IPV: 'ipv4' is not a valid backend or is unavailable

[bryteise]

We don't (and never have as far as I can tell) ship a default config for firewalld. The others don't appear to be fatal (though nftables is the only available backend it looks like).

[bryteise]

That said on 36570 I see the following

root@clr-f458ef7a08224da5801a9a22188816f5 ~ # firewalld --nofork
2022-06-28 20:46:17 ipset not usable, disabling ipset usage in firewall. Other set backends (nftables) remain usable.
2022-06-28 20:46:17 iptables is not usable.
2022-06-28 20:46:17 ip6tables is not usable.
2022-06-28 20:46:17 ebtables is not usable.

[hksdpc255]

Do I need to touch /etc/firewalld/firewalld.conf? Or just ignore this error?

[bryteise]

Looking at the journal error when I start it with systemctl it seems it uses a fallback config. I'd imagine you'd want to actually configure it though for it to be useful.

[nkinchev]

What are there plans to fix the iptables , ip6tables, ipset .. ? Rules can be mixed if need. As example Docker uses iptables only , does not work with nftables .

[bryteise]

I can add those, should be enabled as part of the next release.

[nkinchev]

Thanks

[Pamalosebi]

Hmm... This is an older thread, but it seems to still be open, and it seems to be going in the right direction as well.

firewalld doesn't work on CL anymore. I have several systems where firewalld was running and none of them are running anymore.

When trying to start firewalld manually:

sudo /usr/bin/firewalld --nofork --nopid $FIREWALLD_ARGS
Password: 
Traceback (most recent call last):
  File "/usr/bin/firewalld", line 293, in <module>
    main()
  File "/usr/bin/firewalld", line 287, in main
    startup(args)
  File "/usr/bin/firewalld", line 237, in startup
    from firewall.server import server
  File "/usr/lib/python3.12/site-packages/firewall/server/server.py", line 25, in <module>
    from firewall.server.firewalld import FirewallD
  File "/usr/lib/python3.12/site-packages/firewall/server/firewalld.py", line 15, in <module>
    from firewall.core.fw import Firewall
  File "/usr/lib/python3.12/site-packages/firewall/core/fw.py", line 18, in <module>
    from firewall.core import nftables
  File "/usr/lib/python3.12/site-packages/firewall/core/nftables.py", line 41, in <module>
    from nftables.nftables import Nftables
ModuleNotFoundError: No module named 'nftables'

The only message in the log is:

Nov 04 17:12:01 cl_sys_02 (irewalld)[3564]: firewalld.service: Referenced but unset environment variable evaluates to an empty string: FIREWALLD_ARGS
Nov 04 17:12:02 cl_sys_02 systemd[1]: firewalld.service: Main process exited, code=exited, status=1/FAILURE
Nov 04 17:12:02 cl_sys_02 systemd[1]: firewalld.service: Failed with result 'exit-code'.
Nov 04 17:12:02 cl_sys_02 systemd[1]: Failed to start firewalld.service.

When searching for "firewalld.service: Referenced but unset environment variable evaluates to an empty string: FIREWALLD_ARGS" I get a result from NixOS, where the problem also occurred. They seem to have fixed it with this patch: https://github.com/NixOS/nixpkgs/issues/293778#issuecomment-2186709371 and this links to --> https://github.com/NixOS/nixpkgs/pull/205380

[hksdpc255]

I can confirm this. But this time I cannot find the module named nftables.

This workaround fix the issue:

sudo mkdir -p "$(cat /usr/lib/python3*/site-packages/usrlocal.pth | head -n 1)/nftables"
curl -L 'https://git.netfilter.org/nftables/plain/py/src/__init__.py?h=v1.1.1' | sudo tee "$(cat /usr/lib/python3*/site-packages/usrlocal.pth | head -n 1)/nftables/__init__.py"
curl -L 'https://git.netfilter.org/nftables/plain/py/src/nftables.py?h=v1.1.1' | sudo tee "$(cat /usr/lib/python3*/site-packages/usrlocal.pth | head -n 1)/nftables/nftables.py"
curl -L 'https://git.netfilter.org/nftables/plain/py/src/schema.json?h=v1.1.1' | sudo tee "$(cat /usr/lib/python3*/site-packages/usrlocal.pth | head -n 1)/nftables/schema.json"

Remember to run sudo rm -r /usr/local/lib/python3.*/site-packages/nftables && sudo rmdir -p --ignore-fail-on-non-empty /usr/local/lib/python3.*/site-packages/ after official fix proposed.

[Pamalosebi]

@hksdpc255 Thank you very much! That worked. 👍🏻

I was stuck when https://pypi.org/project/ansibleguy-nftables/ didn't work...

[bryteise]

Should be fixed in the next release.