ucx icon indicating copy to clipboard operation
ucx copied to clipboard

[EPIC] Lint problems related to previous Databricks Runtime versions

Open nfx opened this issue 1 year ago • 0 comments

There are breaking changes sometimes. Lint for breaking changes and deprecations:

  • [ ] https://spark.apache.org/docs/latest/sql-migration-guide.html#upgrading-from-spark-sql-34-to-35
  • [ ] https://spark.apache.org/docs/latest/ss-migration-guide.html
  • [ ] https://spark.apache.org/docs/latest/ml-migration-guide.html
  • [ ] https://spark.apache.org/docs/latest/api/python/migration_guide/pyspark_upgrade.html
  • [ ] https://spark.apache.org/docs/latest/api/python/migration_guide/koalas_to_pyspark.html
  • [ ] https://docs.databricks.com/en/release-notes/runtime/14.3lts.html#spark-catalog-apis-now-fully-supported-in-shared-access-mode
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/14.0.html#default-current-working-directory-has-changed
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/14.0.html#introducing-spark-connect-in-shared-cluster-architecture
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/13.1.html#remove-start-version
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/13.1.html#parquet-failonunknownfields-no-longer-silently-drop-data-on-type-mismatch
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/13.0.html#pip-does-not-automatically-restart-python
  • [ ] https://docs.databricks.com/en/release-notes/runtime/13.3lts.html#block-schema-overwrite-when-using-dynamic-partition-overwrites
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/13.2.html#unity-catalog-support-for-python-and-pandas-user-defined-functions-udfs
  • [ ] https://docs.databricks.com/en/release-notes/runtime/12.2lts.html#behavior-changes-with-the-new-lateral-column-alias-feature
  • [ ] https://docs.databricks.com/en/release-notes/runtime/12.2lts.html#implicit-lateral-column-aliasing-support
  • [ ] https://docs.databricks.com/en/release-notes/runtime/11.3lts.html#structured-streaming-trigger-once-is-deprecated
  • [ ] https://docs.databricks.com/en/release-notes/runtime/11.3lts.html#applyinpandaswithstate-throws-an-error-if-the-query-has-a-shuffle-after-the-operator
  • [ ] https://docs.databricks.com/en/release-notes/runtime/11.3lts.html#hadoop-cache-for-s3a-is-now-disabled
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/11.0.html#https-scheme-now-enforced-when-abfs-client-uses-sas-token
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/11.0.html#log4j-is-upgraded-from-log4j-1-to-log4j-2
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/11.0.html#the-table-property-external-is-now-reserved
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/11.0.html#null-values-in-csv-files-are-now-written-as-unquoted-empty-strings-by-default
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/11.0.html#string-format-in-format_string-and-printf-no-longer-allows-0
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/11.2.html#change-to-alter-table-permissions-in-table-access-controls-table-acls
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/11.0.html#dropping-a-missing-delta-table-constraint-now-raises-an-error
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/10.0.html#breaking-changes-for-all-python-users
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/10.0.html#breaking-changes-for-spark-sql-users-who-enable-ansi-mode
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/10.0.html#breaking-changes-for-all-spark-sql-users
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/10.5.html#deprecation-of-koalas
  • [ ] https://docs.databricks.com/en/archive/runtime-release-notes/9.0.html#bug-fixes
  • [x] https://docs.databricks.com/en/archive/runtime-release-notes/8.0.html#delta-is-now-the-default-format-when-a-format-is-not-specified

nfx avatar Mar 26 '24 15:03 nfx