Casey Artner

Error on mv using DirFileSystem with fs=SMBFileSystem

4

I am attempting to use the `DirFileSystem` to wrap `SMBFileSystem` so I can operate in my SMB folder as though it's my base directory. I can use most methods I...

Config Value spark.sql.mapKeyDedupPolicy not Supported by Databricks SQL Warehouse

1

Getting the error `spark.sql.mapKeyDedupPolicy` is not supported by Databricks SQL Warehouses when using ibis pyspark with a Databricks SQL Warehouse Cluster. See: https://community.databricks.com/t5/data-engineering/spark-settings-in-sql-warehouse/td-p/7959 Set in do_connnect: https://github.com/ibis-project/ibis/blame/e425ad57899f8ebbea29b57bb53cedb40ebd7193/ibis/backends/pyspark/__init__.py#L180 ```python self._session.conf.set("spark.sql.mapKeyDedupPolicy", "LAST_WIN")...