[BR]: fail2ban relies on deprecated, soon-to-be-removed setuptools install functionality

[thesamesam]

Environment:

• Fail2Ban version : 1.1.0, master
• OS, including release name/version : Gentoo Linux
• [X] Fail2Ban installed via OS/distribution mechanisms
• [X] You have not applied any additional foreign patches to the codebase

The issue:

fail2ban currently relies on setuptools setup.py install, but this has been marked deprecated for quite some time. In setuptools-80.1.0, they've finally set a removal date for 2025-10-31.

Steps to reproduce

/tmp/fail2ban $ python setup.py install --prefix="/usr" --root="/tmp/foo"
/usr/lib/python3.13/site-packages/setuptools/_distutils/dist.py:289: UserWarning: Unknown distribution option: 'test_suite'
  warnings.warn(msg)
running install
/tmp/fail2ban/setup.py:111: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  install.initialize_options(self)
[...]

After 2025-10-31, that deprecation warning will become fatal, and setup.py install won't work at all.

In summary, the Python ecosystem decided that the setup.py install way of doing things was too haphazard. There are two models now:

1. Installing software via PEP517, which has restrictions on the installation of data files and other auxiliary files, and is intended for software that is meant to be imported as a library, or a Python tool with little-to-no configuration or extra files;
2. Installing software via a real build system like Meson, that happens to just be installing software written in Python.

A lot of software written in Python was relying on setup.py install because it seemed like the right thing to do, but there's no reason that software just written in Python that isn't intended to be imported or must live in site-packages has to be installed by the Python packaging setup (especially if it's not intended to be installed via pip or on pypi at all).

I personally recommend Meson for this case.

Expected behavior

No deprecation warnings.

Observed behavior

Deprecation warnings indicating imminent breakage.

Any additional information

See also https://github.com/fail2ban/fail2ban/issues/3361, https://github.com/pypa/setuptools/issues/2088#issuecomment-2845099972, and https://github.com/pypa/packaging-problems/issues/576#issuecomment-1058464665

[sebres]

You seem to misunderstand the deprecation: it is not the setuptools what is deprecated, but setup.py install method. See Python Packaging :: Should setup.py be deleted?. With other words, one have to use python -m pip install . instead of python setup.py install.

Particularly if I try this on my box with newest setuptools, I don't see any warning and it simulates the installation without any issue:

sudo python3 -m pip install --dry-run --break-system-packages --report /tmp/inst-report.txt .

Also note that the packaging is rather a matter of distribution maintainers and their common mechanisms or tools used by distribution. For instance debian maintainers (and we when building our releases) use pybuild/dh to build fail2ban. Other distros may use other stuff to build it.

We have surely to adjust the README and the wiki for "new" syntax of manual install, but the assumption "fail2ban relies on deprecated, soon-to-be-removed setuptools install functionality" is not correct.

[eli-schwartz]

Hello.

Please show me the contents of your systemd service file that is installed by pip install.

[thesamesam]

I haven't confused them. Please see the links I gave and my description above. The wheel format can't actually handle auxiliary data files like configuration in /etc correctly at all. Using pip install will use setuptools-via-PEP517.

Also note that the packaging is rather a matter of distribution maintainers and their common mechanisms or tools used by distribution.

I am such a distribution maintainer, and I'm reaching out to explain the problem. Right now, fail2ban's build system requires manual hacks (which every distribution will have to do) to handle the files fail2ban needs outside of site-packages.

[sebres]

Please show me the contents of your systemd service file that is installed by pip install.

We don't install it (and never did it) by manual install - as the README and wiki say, the build folder would contain fail2ban.service which could be then copied (if needed) to /etc/systemd/system/. Or do you mean the new build facilities would not generate fail2ban.service from fail2ban.service.in? Although the task is and was always the task of maintained packages (not of common manual install).

Right now, fail2ban's build system requires manual hacks (which every distribution will have to do) to handle the files fail2ban needs outside of site-packages

Doesn't gentoo have build tools like pybuild/dh? Or is that mentioned Meson such a tool?

And are this "manual hacks" not just few rules for packaging tool to package paths like this?

./config ==> /etc/fail2ban
./build/scripts ==> /usr/bin/

Anyway, why don't you provide your build scripts (Meson or whatever), so other maintainers would not require manual hacks? Welcome with a PR...

[eli-schwartz]

As was mentioned by the setuptools maintainer on the linked ticket:

I'd really like to know why a drop-in replacement for setup.py install using PEP 517 isn't viable today.

The issue is miscellaneous packages that were relying on setup.py install kind of "just because this thing is written in Python" and install auxiliary data files, as mentioned in #2088 (comment). fail2ban is an example, cloud-init is another.

I ran build -w on fail2ban and it built a wheel.

Is that wheel not viable for installation? I can see it has usr/share/doc and etc/fail2ban, which would require special handling by a distributor's installer, but it sure seems close to being drop-in replacement.

My reply there was very specific about the issues which exist in practice, for fail2ban itself:

fail2ban creates build/fail2ban.service, which already needs to be manually installed as it isn't included in a wheel nor even copied over as a data_files. Really it should have, historically, been in data_files so that it would be installed to /usr/lib/systemd/system/. But if it did, that wouldn't help install from a wheel, because... here are the contents of the file when using python -m build:

[Unit]
Description=Fail2Ban Service
Documentation=man:fail2ban(1)
After=network.target iptables.service firewalld.service ip6tables.service ipset.service nftables.service
PartOf=iptables.service firewalld.service ip6tables.service ipset.service nftables.service

[Service]
Type=simple
Environment="PYTHONNOUSERSITE=1"
ExecStartPre=/bin/mkdir -p /run/fail2ban
ExecStart=build/bdist.linux-x86_64/wheel/fail2ban-1.1.1.dev1.data/scripts/fail2ban-server -xf start
# if should be logged in systemd journal, use following line or set logtarget to sysout in fail2ban.local
# ExecStart=build/bdist.linux-x86_64/wheel/fail2ban-1.1.1.dev1.data/scripts/fail2ban-server -xf --logtarget=sysout start
ExecStop=build/bdist.linux-x86_64/wheel/fail2ban-1.1.1.dev1.data/scripts/fail2ban-client stop
ExecReload=build/bdist.linux-x86_64/wheel/fail2ban-1.1.1.dev1.data/scripts/fail2ban-client reload
PIDFile=/run/fail2ban/fail2ban.pid
Restart=on-failure
RestartPreventExitStatus=0 255

[Install]
WantedBy=multi-user.target

Those paths are totally wrong, and are normally deduced by python setup.py install --prefix=/usr. That needs replacing here.

You can see this buried in the logs that you pasted:

running install_scripts
creating build/bdist.linux-aarch64/wheel/fail2ban-1.1.1.dev1.data/scripts
copying build/scripts-3.13/fail2ban-server -> build/bdist.linux-aarch64/wheel/fail2ban-1.1.1.dev1.data/scripts
copying build/scripts-3.13/fail2ban-regex -> build/bdist.linux-aarch64/wheel/fail2ban-1.1.1.dev1.data/scripts
copying build/scripts-3.13/fail2ban-client -> build/bdist.linux-aarch64/wheel/fail2ban-1.1.1.dev1.data/scripts
copying build/scripts-3.13/fail2ban-testcases -> build/bdist.linux-aarch64/wheel/fail2ban-1.1.1.dev1.data/scripts
WARNING: Cannot find root-base option, check the bin-path to fail2ban-scripts in "fail2ban.service" and "fail2ban-openrc.init".
Creating build/fail2ban.service (from fail2ban.service.in): @BINDIR@ -> build/bdist.linux-aarch64/wheel/fail2ban-1.1.1.dev1.data/scripts
Creating build/fail2ban-openrc.init (from fail2ban-openrc.init.in): @BINDIR@ -> build/bdist.linux-aarch64/wheel/fail2ban-1.1.1.dev1.data/scripts
creating fail2ban-python binding -> build/bdist.linux-aarch64/wheel/fail2ban-1.1.1.dev1.data/scripts

If I run in legacy mode and pretend it's a Makefile:

$ python setup.py install --prefix=/usr --root=$PWD/pkg

running install_scripts
creating /tmp/fail2ban/pkg/usr/bin
copying build/scripts-3.12/fail2ban-testcases -> /tmp/fail2ban/pkg/usr/bin
copying build/scripts-3.12/fail2ban-regex -> /tmp/fail2ban/pkg/usr/bin
copying build/scripts-3.12/fail2ban-server -> /tmp/fail2ban/pkg/usr/bin
copying build/scripts-3.12/fail2ban-client -> /tmp/fail2ban/pkg/usr/bin
Creating build/fail2ban.service (from fail2ban.service.in): @BINDIR@ -> /usr/bin
Creating build/fail2ban-openrc.init (from fail2ban-openrc.init.in): @BINDIR@ -> /usr/bin
creating fail2ban-python binding -> /tmp/fail2ban/pkg/usr/bin

Note the paths are now /usr/bin

[eli-schwartz]

So, using pip install produces wheels, which tells fail2ban to produce the wrong file contents for the systemd service and openrc init file.

They can't be used. Gentoo manually copies them over, but we also use setup.py install to make sure the file contents are correct.

[eli-schwartz]

Note this problem cannot be solved by pip because pip now requires that you do NOT install software directly, but first build a generic "wheel" package with unfixed relative paths.

As discussed at https://github.com/pypa/packaging-problems/issues/576#issuecomment-1058464665 -- the python packaging community has officially deprecated and removed support for installing system integration. If your software doesn't install into a python virtualenv as standalone, it isn't a good fit for pip. If you have /etc files, you aren't supported. If you need to install GUI /usr/share/applications/*.desktop, you're not supported. If you need to install systemd services, those won't work in ~/.venvs/app_foo/lib/systemd/system/ so you are, once again, not supported.

Anyway, why don't you provide your build scripts (Meson or whatever), so other maintainers would not require manual hacks? Welcome with a PR...

I may look into this soon, if nobody else beats me to it. :)

[sebres]

How about this one?.. In branch fix-pythonic-build-install I extended setup.py with 2 new commands - build-ex and install-ex (not yet ready, only builds at the moment), that internally uses setuptools (build and egg_info) to build the library and hereafter create the fail2ban.service and fail2ban-openrc.init files, and copy the auxiliary data (/etc, /var, etc) to --build-base=... folder (default build in root of fail2ban).

For instance, to make build folder (very similar to deprecated install):

python3 setup.py build-ex --build-base=build --without-tests

(it is also covered in GHA now - see actions/runs/14814502138 for the output)

Use build-ex --help for more detailed usage with all options:

$ python3 setup.py build-ex --help
usage: setup.py comand [options]
Commands:
  build-ex        build the package underneath ./build (or --build-base)
  install-ex      will install the package
Options:
  --quiet (-q)    run quietly (turns verbosity off)
  --dry-run (-n)  don't actually do anything
  --help (-h)     show detailed help message
Options of 'install-ex' and 'build-ex' commands:
  --prefix=            installation prefix
  --build-base= (-b)   base directory for build (default ./build)
  --root=              install everything relative to this
                       alternate root directory
  --lib=               directory for library
  --bin=               build directory for binary
  --without-tests      don't enclose fail2ban test-suite

(--prefix and --root are not implemented at the moment, but new options --lib and --bin which act relative --build-base)

This doesn't rely on deprecated mechanisms anymore.

If it is OK, I'd fulfill the install-ex method for manual install and release it.

[sebres]

small amend: commands f2b-build and f2b-install renamed in build-ex and install-ex (comment above adjusted)

[sebres]

Option --prefix and command install-ex (with --root) are also implemented now:

python3 setup.py install-ex --root=./pkg --build-base=./build --prefix=/usr --without-tests

This would firstly build to folder "./build" (using build-ex command) and then install everything from ./build to given --root ("./pkg"). The files fail2ban.service and fail2ban-openrc.init remain as by original install only in folder "./build", but in opposite to deprecated install, they would be generated now on the phase of build-ex... And because it also handles --prefix now, one could also use build-ex instead of install-ex now to prepare the package.

Anyway it looks completed to me now, reviews and tests are welcome.

[sebres]

Did someone test it? Some comments?

[sebres]

@thesamesam, @eli-schwartz ping... How it looks for you? @fail2ban/maintainers Any opinions or objections?

I'd like to make a next release and it'd be nice if this can be clarified.

[thesamesam]

Let me check it out. Thanks.

[chla28]

I need to build a RPM, not clear if build-ex/install-ex is able to do this...

[sebres]

I need to build a RPM, not clear if build-ex/install-ex is able to do this...

Indirectly yes, if you'd supply your spec file and change there:

- python3 setup.py install --single-version-externally-managed -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES
+ python3 setup.py install-ex --single-version-externally-managed -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES

Alternatively we could provide some env-var, like F2B_ALT_INSTALL=1, so if set to 1 it'd use install-ex instead of install (and build-ex instead of build).

However this all is PoC at the moment. As well as I don't know how it'd help you with your issue #4030 right now (your issue is on stage of RPM build after install)... And as I already wrote in https://github.com/fail2ban/fail2ban/issues/4030#issuecomment-3035812226 - no idea whether bdist_rpm would rely on install in the future once it gets removed (install is deprecated in setuptools, but related to this bdist_rpm shall still remain. Although who knows, because in the same article they recommend to replace bdist_wheel.

[hobbes1069]

Late to this conversation but Fedora is now actively moving away from the python setup.py install method. I have an interesting result in that nothing is being installed using the wheel method:

+ /usr/bin/python3 -m pip install --root /builddir/build/BUILD/fail2ban-1.1.0-build/BUILDROOT --prefix /usr --no-deps --disable-pip-version-check --progress-bar off --verbose --ignore-installed --no-warn-script-location --no-index --no-cache-dir --find-links /builddir/build/BUILD/fail2ban-1.1.0-build/fail2ban-1.1.0/pyproject-wheeldir fail2ban==1.1.0
Using pip 25.1.1 from /usr/lib/python3.14/site-packages/pip (python 3.14)
Looking in links: /builddir/build/BUILD/fail2ban-1.1.0-build/fail2ban-1.1.0/pyproject-wheeldir
Processing ./pyproject-wheeldir/fail2ban-1.1.0-py3-none-any.whl
Installing collected packages: fail2ban
Successfully installed fail2ban-1.1.0

This was a minimal attempt in converting using a Fedora contributor pyprojectize migration tool.

[sebres]

that nothing is being installed using the wheel method

I have no idea what exactly that pip-install call or its wheel pip module would use internally (probably setup.py bdist_wheel and co), but I thought only install and sdist were deprecated, and other "build" facilities shall now "communicate" using correct setuptools commands internally (that could miss "fail2ban.service", but anyway create something in root). But really wondering why it said "successful" if nothing at all is done.

Cannot it somehow make this using prebuilt (with new commands build-ex or install-ex) version?.. I mean could the build or install subcommand configured somewhere in options of wheel config or spec file?.. (I never used them, so not familiar, because built deb-package with pybuild/dh) If yes could you try it with branch fix-pythonic-build-install. If no then we'd try with my idea with env variable, like F2B_ALT_INSTALL=1 (or replace deprecated install with new facilities of install-ex)... So you could patch setup.py (of that branch) with something like - sed -i -e 's/install-ex/install/g' setup.py and then try your pip-install again. I'm curious about the result.

[hobbes1069]

It's worth noting that most of this is handled using build macros within the RPM build process so I have limited ways of influencing the build options. In case I haven't shared this before, this is what I'm using to try to "port" the fail2ban build over to the wheel method: https://fedoraproject.org/wiki/Changes/DeprecateSetuppyMacros

This is much deeper in the weeds that I'm accustomed to for Python projects so I'm learning as I go. It looks like the build process is working (find %build in the log) but the install is not (%install).

https://kojipkgs.fedoraproject.org//work/tasks/1520/135811520/build.log

[merwok]

I have no idea what exactly that pip-install call or its wheel pip module would use internally (probably setup.py bdist_wheel and co),

No – there is an interface defined so that pip (or any build frontend: tox, hatch, ill-named build tool…) can call any build backend (setuptools, flit-core, hatchling… and many more that don’t use setup.py): https://peps.python.org/pep-0517/

[sebres]

No – there is an interface defined so that pip (or any build frontend: tox, hatch, ill-named build tool…) can call any build backend (setuptools, flit-core, hatchling… and many more that don’t use setup.py):

And this interface to setuptools obviously uses setup.py (or did it previously)... Otherwise how they'd build project specific stuff, declared in setup.py using setuptools API?

If it shall mean that legacy setup.py (in code declaration) is not supported anymore or also deprecated, I can understand that, but I'm not maintainer (RPM, wheels, or whatever packaging system), so I may be dependent on foreign assistance (and, sorry, but with the best will it is not a link to long-read article like pep-0517, even if I had the time to read it and references).

In any case it is strange to claim it is successful, where it did basically nothing.

[merwok]

It does not use setup.py directly: it calls functions following the interface in setuptools, which itself does what it wants using its run_setup function, reading data from setup.py and/or setup.cfg and pyproject.toml. So yes a project using setuptools can still define a setup.py with custom commands, but pip does not know about setup.py anymore.

But because the goal of this interface is to build wheels and sdists, customizations to the install commands in setup.py will have no effect in a pep 517 mode. These are the features that need to be replaced or removed.

[sebres]

customizations to the install commands in setup.py will have no effect in a pep 517 mode.

That was clear from the beginning. The questions are:

• either how exactly one'd do that it would have the effect, e. g. to force it to do the customization;
• or whether it is possible to do the customization before (e. g. build it before) and then create the wheels/sdists from that build.

I guess the latter could be easier, but, again, not familiar with that and no time to investigate deeper into this maintainer world.

These are the features that need to be replaced or removed.

This is really a brilliant insight about how to circumvent with deprecated stuff :)

[hobbes1069]

No good "fix" seems to be available.

https://bugzilla.redhat.com/show_bug.cgi?id=2377254#c9

Basically, install everything in the prefix first and then move the files that need to be outside of the prefix with a post install script...

[sebres]

Basically, install everything in the prefix first and then move the files that need to be outside of the prefix with a post install script...

Well, the new install-ex facility of setup.py in mentioned branch doing exactly that, doesn't it?

[hobbes1069]

Without understanding the inner workings, I guess it's not clear how creating a new command works around wheel's inability to install files outside of the prefix.

[sebres]

I only spoke about "install everything in the prefix first". Since old install is deprecated and will be removed, I created new install-ex what doing that. And as already said, I have no idea how pip or setuptools would construct everything around wheels. I'm not a maintainer and if needed packaging (for instance to release fail2bans deb-package here) I used pybuild/dh, that seems not have any issue till now.

Can you please provide pyproject-wheeldir and/or ./pyproject-wheeldir/fail2ban-1.1.0-py3-none-any.whl file you used for build and the result (ls -la) over the directory built by /usr/bin/python3 -m pip install ... previously? I mean as it was still OK.

[hobbes1069]

Here's the wheel file: https://hobbes1069.fedorapeople.org/fail2ban-1.1.0-py3-none-any.whl

$ ll -a /var/lib/mock/fedora-rawhide-x86_64/root/builddir/build/BUILD/fail2ban-1.1.0-build/BUILDROOT/
total 0
drwxr-xr-x. 1 build mock  24 Aug 18 07:08 .
drwxr-xr-x. 1 build mock 364 Aug 18 07:08 ..
drwxr-xr-x. 1 build mock  22 Aug 18 07:08 etc
drwxr-xr-x. 1 build mock  16 Aug 18 07:08 run
drwxr-xr-x. 1 build mock  22 Aug 18 07:08 usr
drwxr-xr-x. 1 build mock   6 Aug 18 07:08 var

Also just in case it's helpful, here's the buildroot: https://hobbes1069.fedorapeople.org/f2b-buildroot.txt

[hobbes1069]

Well, this is becoming a little clearer... Some things that are supposed to be installed in %prefix is actually being installed under %python3_sitelib...

[hobbes1069]

Confirmed, moving the etc and usr/share parts out of %python3_sitelib manually and a few other tweaks fixed the build. Now a lot of comparison and testing to make sure I found everything.

[sebres]

Some things that are supposed to be installed in %prefix is actually being installed under %python3_sitelib...

Can not some parameters/options of bdist_wheel be used with python -m build (I guess pip would use it internally), e. g. like -C KEY[=VALUE]) to supply another path and to move them back to %prefix?