Add official support for Python 3.12

[astrojuanlu]

Close #3287.

Description

Development notes

First attempt, I haven't ran the test suite locally yet. But pip install -e .[test] worked first time on Apple Silicon.

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

• [x] Read the contributing guidelines
• [x] Signed off each commit with a Developer Certificate of Origin (DCO)
• [x] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• [x] Updated the documentation to reflect the code changes
• [x] Added a description of this change in the RELEASE.md file
• [x] Added tests to cover my changes
• [ ] Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

[astrojuanlu]

Tests failing on 3.11 instead 🤔

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/multiprocessing/queues.py", line 244, in _feed
    obj = _ForkingPickler.dumps(obj)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'PluginManager.add_hookcall_monitoring.<locals>.traced_hookexec'

rings a bell but I don't have time to look at this right now, will do so later.

[merelcht]

Tests failing on 3.11 instead 🤔

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/multiprocessing/queues.py", line 244, in _feed
    obj = _ForkingPickler.dumps(obj)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'PluginManager.add_hookcall_monitoring.<locals>.traced_hookexec'

rings a bell but I don't have time to look at this right now, will do so later.

This is the flaky test that occasionally fails. I think @lrcouto was seeing this as well on her PR for adding the _EPHEMERAL attribute.

[astrojuanlu]

E               AttributeError: 'called_once_with' is not a valid assertion. Use a spec for the mock if 'called_once_with' is meant to be an attribute.. Did you mean: 'assert_called_once_with'?

Interesting 😄

[astrojuanlu]

One test failure left, only happening on Python 3.12:

    @use_config_dir
    def test_load_core_config_get_syntax(self, tmp_path):
        """Make sure core config can be fetched with .get()"""
        conf = OmegaConfigLoader(
            str(tmp_path), base_env=_BASE_ENV, default_run_env=_DEFAULT_RUN_ENV
        )
        params = conf.get("parameters")
        catalog = conf.get("catalog")
    
>       assert params["param1"] == 1
E       TypeError: 'NoneType' object is not subscriptable

[lrcouto]

Tests failing on 3.11 instead 🤔

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/multiprocessing/queues.py", line 244, in _feed
    obj = _ForkingPickler.dumps(obj)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'PluginManager.add_hookcall_monitoring.<locals>.traced_hookexec'

rings a bell but I don't have time to look at this right now, will do so later.

This is the flaky test that occasionally fails. I think @lrcouto was seeing this as well on her PR for adding the _EPHEMERAL attribute.

This test is different than the one I've seen. There was one that was caused by a requirement version, which Nok fixed, and another that happened in the tests for the starters. This one is new to me.

[ankatiyar]

One test failure left, only happening on Python 3.12:

    @use_config_dir
    def test_load_core_config_get_syntax(self, tmp_path):
        """Make sure core config can be fetched with .get()"""
        conf = OmegaConfigLoader(
            str(tmp_path), base_env=_BASE_ENV, default_run_env=_DEFAULT_RUN_ENV
        )
        params = conf.get("parameters")
        catalog = conf.get("catalog")
    
>       assert params["param1"] == 1
E       TypeError: 'NoneType' object is not subscriptable

Did a little digging - https://github.com/python/cpython/issues/105524

[astrojuanlu]

Looks like this will require a bit of work. I'm not pursuing this PR for the time being.

[merelcht]

Looks like this will require a bit of work. I'm not pursuing this PR for the time being.

@astrojuanlu Maybe you can add some context on the challenges you're seeing in https://github.com/kedro-org/kedro/issues/3287 and then we can properly plan it for a sprint?

[astrojuanlu]

Good point, done https://github.com/kedro-org/kedro/issues/3287#issuecomment-1956164676