Nok Lam Chan

Realistically does everything stay the same except the session creation step? I am fine with adding support, the problem tho is that we don't have a databrick environments that we...

We can keep the discussion here, I can transfer the issue to the correct repository.

Basically I don't have problem with this, because it won't affect existing user but add new support to DatabricksConnect. I am slightly uncomfortable with the way to check the version....

Maybe somewhere within the spark/ modules, since datasets can be installed independently.

@KrzysztofDoboszInpost it's not, our CI is running on pyspark 3.4 SPARK = "pyspark>=2.2,

Pre-requisite of * #2048

kedro-org/kedro#2417 Related

@astrojuanlu How would we validate it as a whole? In this case, if `tensorflow` never release a Python3.11 version do we just don't release newer versions or do we bump...

@astrojuanlu 1. We have `SnowparkTableDataSet` which support Python 3.8 only. (thus build docs only works on Python 3.8, if one day A dataset works on Python 3.8 only and B...

Is this a valid way to do interpolation? It works in config but the example here is pure python code, could you use f-string instead?