Does dask-kubernetes compatible with newer version of k8rs?

[dbalabka]

Describe the issue:

Using "NodePort" leads to the following error:

File [.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:82](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:82), in <listcomp>(.0)
     [79](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:79) def _get_port(service, port_name, is_node_port=False):
     [80](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:80)     """NodePort is a special case when we have to use node_port instead of node"""
     [81](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:81)     [port] = [
---> [82](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:82)         port.port if not is_node_port else port.node_port
     [83](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:83)         for port in service.spec.ports
     [84](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:84)         if port.name == service.metadata.name or port.name == port_name
     [85](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:85)     ]
     [86](.venv/lib/python3.10/site-packages/dask_kubernetes/operator/networking.py:86)     return port

File [.venv/lib/python3.10/site-packages/box/box.py:647](.venv/lib/python3.10/site-packages/box/box.py:647), in box.box.Box.__getattr__()

BoxKeyError: "'Box' object has no attribute 'node_port'"

The Box class was introduced in some newer versions of k8rs and has different attribute notation "nodePort"

Minimal Complete Verifiable Example:

TBD

Anything else we need to know?:

Environment:

• Dask version: 2023.11.0
• Python version: 3.10
• Operating System: Linux
• Install method (conda, pip, source): poetry

[dbalabka]

While I believe that #849 is a proper fix, the open constraints like kr8s>=0.9.0 should be avoided because they might introduce problems with library compatibility: https://github.com/dask/dask-kubernetes/blob/main/requirements.txt#L8 Probably, the correct constraint should be kr8s>=0.9.0,<0.13 (depending on supported version)

[jacobtomlinson]

Thanks @dbalabka. I've switched back to a hard pin on the kr8s dependency for now, but I hope to remove it again as things continue to become more stable in the future.

Adding upper bounds to dependencies is a highly debated subject. In Dask we typically do not do this.

[dbalabka]

@jacobtomlinson, it seems another place should be fixed with the same problem. Please see #858

[jacobtomlinson]

Updated kr8s to the latest version in #880