Use 3.11 for Github actions

[Pierre-Sassoulas]

Type of Changes

	Type
✓	:sparkles: New feature
✓	:scroll: Docs

Description

Closes #5920

[DanielNoord]

This is still broken. dill isn't 3.11 compatible yet. You can test this locally, but for some reason in Github Actions we don't fail but instead hang.

[DanielNoord]

dill has released a new version but we still need an astroid update for these tests to pass.

[Pierre-Sassoulas]

dill is still broken it seems.

[DanielNoord]

They are releasing at the end of June.

[hugovk]

py311 also needs adding to tox.ini

[Pierre-Sassoulas]

Milestone for the required release of dill : https://github.com/uqfoundation/dill/milestone/17

[Pierre-Sassoulas]

Re-trying as dill 0.3.5.1 was released and it look like the milestone we had to follow is not the one I thougth.

[Pierre-Sassoulas]

We do need the next version of dill.

[DanielNoord]

We also need astroid 2.12 😄

[AdamWill]

There's an astroid 2.12.1 now, which passes all its own tests on Python 3.11. Current git HEAD dill is good with 3.11, but a release hasn't been tagged yet.

[AdamWill]

With a recent snapshot of dill, and astroid 2.12.1, I'm down to 43 failures in the tests when building pylint 2.14.4:

https://kojipkgs.fedoraproject.org/work/tasks/1561/89381561/build.log

edit: the ones related to deprecated modules/methods are likely downstream-specific (we exclude similar tests from 3.7 and 3.8 in our pytest command), others are more likely to also affect upstream.

[DanielNoord]

We are working on getting support for astroid 2.12.x in pylint, progress of which you can track in #7153.

dill seems to have missed its release date on the 27th on June. I thought about sending them a reminder, but I think they know we're waiting on their release so I don't want to bug them too much.

[AdamWill]

OK, so with new astroid and dill and #7153 , here's some really-python3.11 issues:

• telnetlib is deprecated in 3.11, that breaks access_attr_before_def_false_positive.py
• asyncio.coroutine was removed in 3.11, that breaks iterable_context_py3.py
• I think due to changes in Enum upstream, we're now getting an unexpected super-init-not-called message for enum_self_defined_member_5138

[AdamWill]

Hmm, this is a fun one. the pylint output for the syntax_error.py test has changed, which breaks both that test and import_error (which imports it). It changed from just "invalid syntax" to "expected '('". The new error seems if anything better? But not sure why it changed, or how to allow the output to differ across Python versions (if that's desired).

[DanielNoord]

Hmm, this is a fun one. the pylint output for the syntax_error.py test has changed, which breaks both that test and import_error (which imports it). It changed from just "invalid syntax" to "expected '('". The new error seems if anything better? But not sure why it changed, or how to allow the output to differ across Python versions (if that's desired).

This is an annoying issue which I encountered before. We should probably merge https://github.com/PyCQA/pylint/pull/7097 first which deals with a similar issue and then any PR that moves syntax-error functional tests to unittests is much appreciated!

[AdamWill]

Thanks. I'm trying to work on fixes for the things I'm sure are python 3.11 (and not just general running-the-tests-downstream stuff, or already-fixed-since-2.14.4 stuff), but I don't know the codebase very well so I may be doing it wrong. :P I'll send a PR when I'm done.

[DanielNoord]

Thanks. I'm trying to work on fixes for the things I'm sure are python 3.11 (and not just general running-the-tests-downstream stuff, or already-fixed-since-2.14.4 stuff), but I don't know the codebase very well so I may be doing it wrong. :P I'll send a PR when I'm done.

Much appreciated! If you need any direction feel free to ping me, that can also be on your local fork. It's often much easier with respect to time and effort for me (or other maintainers) to comment on a WIP PR than create one ourselves.

[AdamWill]

Thanks! The fork is https://github.com/AdamWill/pylint/commits/py311 , with my attempts at fixes for the telnetlib and asyncio.coroutine issues so far. I'll send a PR when I'm done looking through the other few failures that are left.

edit: oh, and yeah, the syntaxerror change is indeed inherited from Python itself:

[adamw@xps13k pylint (py311)]$ python3 tests/functional/s/syntax/syntax_error.py 
  File "/home/adamw/local/pylint/tests/functional/s/syntax/syntax_error.py", line 1
    def toto # [syntax-error]
             ^^^^^^^^^^^^^^^^
SyntaxError: expected '('
[adamw@xps13k pylint (py311)]$ python3.10 tests/functional/s/syntax/syntax_error.py 
  File "/home/adamw/local/pylint/tests/functional/s/syntax/syntax_error.py", line 1
    def toto # [syntax-error]
             ^^^^^^^^^^^^^^^^
SyntaxError: invalid syntax

so I guess I'll look at your idea of moving it and see if I can do that.

[AdamWill]

@DanielNoord could we say that test_stdin_syntaxerror in tests/test_self.py already covers the same ground as the syntaxerror functional test?

edit: hmm, but that doesn't account for import_error's use of it. mmf.

[DanielNoord]

@DanielNoord could we say that test_stdin_syntaxerror in tests/test_self.py already covers the same ground as the syntaxerror functional test?

edit: hmm, but that doesn't account for import_error's use of it. mmf.

I think there are some more syntax-error tests currently. For example there is one for tokenize and one for imports (you can find them by grepping for syntax-error). If they don't break let's leave them, otherwise those should probably also be ported to unittests.

[AdamWill]

With the changes I've submitted so far, plus a downstream patch for those two specific cases where the syntax error text changed, the test suite fully passes. So I don't think any of the others are a problem.

I think the intent of syntax-error is just to test that passing through a very basic SyntaxError from the interpreter works, so we could potentially change it to any simple syntax error whose text is the same on every tested version of Python, but it seems hard to find one :| I tried for like half an hour to find a variation on the theme ("an obviously erroneous function definition") which has the same text representation in Python 3.6 and 3.11, and struck out.

The unknown_encoding_jython one is Jython-only, and the tokenize_error one is meant to test a case where Python itself is OK but tokenize throws the exception. So I don't think we can quite sub in one of those two and throw out syntax-error, sadly.

Oooh! But. Ironically - I think the one that's used in the unit test is actually good! The unit test uses a file that's just this:

for

i.e. for and a newline. That gives SyntaxError: invalid syntax on both Python 3.6 and 3.11 for me (the widest delta I can handily test). Maybe we could change the syntax-error functional test to use that.

[DanielNoord]

With the changes I've submitted so far, plus a downstream patch for those two specific cases where the syntax error text changed, the test suite fully passes. So I don't think any of the others are a problem.

I think the intent of syntax-error is just to test that passing through a very basic SyntaxError from the interpreter works, so we could potentially change it to any simple syntax error whose text is the same on every tested version of Python, but it seems hard to find one :| I tried for like half an hour to find a variation on the theme ("an obviously erroneous function definition") which has the same text representation in Python 3.6 and 3.11, and struck out.

The unknown_encoding_jython one is Jython-only, and the tokenize_error one is meant to test a case where Python itself is OK but tokenize throws the exception. So I don't think we can quite sub in one of those two and throw out syntax-error, sadly.

Oooh! But. Ironically - I think the one that's used in the unit test is actually good! The unit test uses a file that's just this:

for

i.e. for and a newline. That gives SyntaxError: invalid syntax on both Python 3.6 and 3.11 for me (the widest delta I can handily test). Maybe we could change the syntax-error functional test to use that.

Sorry I should have been clearer. I meant using the same code in the unittests but using an assert with if sys.version > ["3", "10"]: ... else: ... The unittests allow testing the different results on different interpreter versions a little easier.

[AdamWill]

My problem with moving it to the unittests is then how do we keep the bit of import-error that tries to import syntax-error.py and expects it to throw specific errors? That seems a bit harder to move to the unit tests.

[DanielNoord]

https://github.com/PyCQA/pylint/blob/eb57dedb622909ca2ccf00516b595df2ae85b75e/tests/test_self.py#L260

This is a test that uses a data file and then tests for emitted messages. I think we should be able to use something like that?

[AdamWill]

Was the Enum.__init__ thing resolved somehow? My PR wasn't merged and the upstream comment hasn't had a reply yet..

[DanielNoord]

I think the recent change of not raising super-init-not-called on "abstract" classes fixes this. We consider methods that only have pass as being abstract, so that should have fixed it.

Sorry I should have tagged you.

Edit: See #7227. It hasn't actually been merged but it should fix the issues!

[cclauss]

Given that Python 3.11 release candidate 2 is still experimental, would it be possible on Python 3.11 only in the GitHub Actions to pip install https://github.com/uqfoundation/dill.git@master just to make sure our tests will pass when dill makes its next release?

[Pierre-Sassoulas]

pip install https://github.com/uqfoundation/dill.git@master

Yeah, you're right, that would make a lot of sense. We could not release a 3.11 compatible version but at least we'll detect the issue with 3.11 before the new release of dill !

[Pierre-Sassoulas]

Well sadly it's not doable: https://github.com/pypa/pip/issues/5566#issuecomment-402417467

[cclauss]

That issue is from four years ago.

Currently .github/workflow/tests.yaml at line 57 says:

      - name: Run pytest
        run: |
          . venv/bin/activate
          pytest --durations=10 --benchmark-disable --cov --cov-report= tests/

Just before those lines, add:

      - if: "contains(matrix.python-version, '-dev')"
        run: |
          . venv/bin/activate
          pip install https://github.com/uqfoundation/dill.git@master