Migrate flake8 config to pyproject.toml

[johnthagen]

If/when the flake8 developers decide to support pyproject.toml, should move flake8 configuration to that to consolidate configurations.

• https://github.com/PyCQA/flake8/issues/234 (locked)

Current Status

This comment lists blockers the Flake8 developers have listed:

• https://github.com/PyCQA/flake8/issues/234#issuecomment-812800722

• [ ] "pip to change its behaviour so mere presence of the file does not change functionality"

  • https://github.com/pypa/pip/issues/9175
  • https://github.com/pypa/pip/issues/8102

• [x] "neither toml nor the python implementations are even 1.0"

  • The TOML language is 1.0

• [x] "python to consider adding a standard library toml implementation"

  • tomllib has been added to Python 3.11

This comment lists a non-technical reason a Flake8 developer has against TOML (against the creator of TOML himself):

• https://github.com/PyCQA/flake8/issues/1467#issuecomment-1044442694

[drhagen]

Issue for flake8 pyproject.toml support was migrated and locked.

[Halkcyon]

@drhagen reading through that with the broken links is quite a mess. Has any progress been made in the year+ since that issue was closed on supporting pyproject.toml like a lot of other popular packages have?

[agronholm]

As a workaround, you can use pyproject-flake8.

[johnthagen]

bpo-40059 is tracking the discussion about adding TOML parsing to the stdlib, which has historically been one of the blockers that the flake8 maintainer has listed as a requirement for supporting pyproject.toml.

Also see:

• https://github.com/hukkin/tomli/issues/141
• https://github.com/python/steering-council/issues/92
• https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/88
• https://github.com/uiri/toml/issues/361#issuecomment-999076587

[johnthagen]

tomli has been accepted into the stdlib for 3.11+ as tomllib

• https://github.com/hukkin/tomli/issues/141#issuecomment-1047797677

[johnthagen]

pyproject-flake8 does not seem very well maintained (last commit 1 year ago).

[agronholm]

It's a crutch until flake8 itself gets TOML support. Now that TOML support is in stdlib, there shouldn't be any excuse left for them not to add pyproject.toml support.

[johnthagen]

It appears that the recent acceptance of tomllib into the std library has not been enough to motivate flake8 devs to begin adding support at this time:

• https://github.com/PyCQA/flake8/issues/1570
• https://github.com/PyCQA/flake8/issues/1571
• https://github.com/PyCQA/flake8/issues/1467#issuecomment-1044442694

[agronholm]

This looks like a people problem, rather than a technical problem.

[johnthagen]

#91 uses pyproject-flake8 to move flake8 configuration into pyproject.toml. I am going to leave this issue open until flake8 adds native support for pyproject.toml, as there are certain edge cases that (e.g. editors shelling out to flake8) this does not fully solve.

[flying-sheep]

The problems with pyproject-flake8 are that it’s not upstream, and therefore needs more configuration than flake8, and is less integrated than it. E.g.:

• Its versions don’t reflect flake8 versions, so pre-commit autoupdate doesn’t bump its version, which means the cache will continue to hold the flake8 version that was downloaded when the newest pyproject-flake8 was released.
• IDE/Editor plugins like VS Code’s need to be configured to use pflake8 instead of flake8.

I’m very much a fan of having all development configuration in pyproject.toml and wonder how we can get every project to support it.

[agronholm]

Which is why it would be nice to get that support in upstream flake8 now that all of the blockers should be gone.

[johnthagen]

Summary of recent closed requests:

• https://github.com/PyCQA/flake8/issues/1598
• https://github.com/PyCQA/flake8/issues/1632
• https://github.com/PyCQA/flake8/issues/1651
• https://github.com/PyCQA/flake8/issues/1655

[agronholm]

It is clear now that this is not about technical issues, but Anthony's personal problems.

[flying-sheep]

I think there’s two facets to this:

1. The opaque approach the PyCQA people are taking here with closing & locking mostly* without even hinting at the reason. They don’t treat people with respect, they don’t acknowledge any of the arguments, they just cut off communication as if everyone was a bad faith actor or obviously wrong. I think that’s inexcusable. People write these issues because they want to make things better, not to spite Anthony or something.

   *The only hint I saw was in @NeilGirdhar’s issue https://github.com/PyCQA/flake8/issues/1632#issuecomment-1200525722 where Anthony says “neither of the pip issues you linked are related”.

2. One of the things Anthony considers a hard blocker still remains: Pip is still branching behavior based on the presence of pyproject.toml, specifically build isolation.

   @NeilGirdhar’s issue above does indeed link unrelated pip issues, as can be glanced from https://github.com/pypa/pip/issues/8437#issuecomment-805321274: Pip uses build isolation independently from PEP 517: --no-use-pep517 doesn’t turn it off.

To fix the bad blood, the PyCQA people need to start respecting people filing issues and explicitly link the pip issues that point out that their blocker is still not resolved.

To fix the underlying technical issue, pip would need to switch build isolation on or off based on a criterion other than Path('pyproject.toml').is_file(). Unconditionally on would work for the flake8 maintainers (as stated in an email), alternatively I assume switching based on the [build-system] table would also work.

[agronholm]

Hm, I was under the impression that nowadays pip only does an isolated build if the build-system section is found in pyproject.toml. Am I mistaken?

[flying-sheep]

You are! I checked the current main branch code:

1. The build_isolation config value is True by default and means “does the user allow pip to use build isolation?” (can be turned off via --no-build-isolation)
2. A requirement’s use_pep517 defaults to has_pyproject or not importlib.util.find_spec("setuptools") (can be specified via --use-pep517/--no-use-pep517)
3. pip will use build isolation if self.req.use_pep517 and build_isolation

Therefore without specifying any of the mentioned CLI options and with setuptools installed, build isolation depends on the existence of pyproject.toml, not its contents.

[agronholm]

Then it was at least discussed somewhere. Would seem like a good change to make in pip, yes?

[flying-sheep]

Seems like that’ll happen:

Once we kill setup.py install and setup.py develop, we should be good to flip the switch on build isolation and pyproject.toml-based builds as well.

See here: https://github.com/pypa/pip/issues/9175#issuecomment-1212489222

[NeilGirdhar]

[off topic]

[johnthagen]

Hey all, happy to use this as a forum to track status and discuss technical productive ways to improve the situation, but please refrain from personal attacks of any kind.

Thank you for your understanding.

[johnthagen]

I am migrating to this Flake8 pyproject.toml tool instead:

• https://github.com/john-hen/Flake8-pyproject
• https://github.com/john-hen/Flake8-pyproject/issues/2

The main advantages being:

• It still allows the user to run flake8 rather than pflake8
  • This removes one more thing to teach about the differences of how to run flake8 with pyproject.toml
  • This helps with some editor integration
• It doesn't monkey patch, so it might be a more stable way to get this functionality than pyproject-flake8

Disadvantages:

• It is more complex than pyproject-flake8

Edit, done in:

• #140

[flying-sheep]

I just came across https://github.com/charliermarsh/ruff, which seems like a great and fast alternative to flake8, isort, …

[johnthagen]

I just came across https://github.com/charliermarsh/ruff,

Yes, I've been keeping my eye on Ruff as well. It looks very promising.

[johnthagen]

I have decided to migrate from Flake8 to Ruff for a variety of reasons. Ruff natively supports pyproject.toml so this particular papercut is solved with it.

To look at how I choose to do the migration, see

• #164