[Feature Request]: Add support for Delta Lake

[mattcasters]

What would you like to happen?

Using the new standalone driver of Databricks, it should be possible to support both reading and writing data to Delta Lake.

https://docs.delta.io/latest/delta-standalone.html

We can add Databricks server metadata to allow us to automatically configure the Hadoop Configuration objects with respect to connectivity. This might also be helpful for issue #2639 (DBFS).

The trick will be to re-use code and functionality of our Parquet Writer transform to avoid code duplication.

Issue Priority

Priority: 3

Issue Component

Component: Transforms