spark-daria

spark-daria copied to clipboard

Should work like `printAthenaCreateTable`. Should be something like `printCreateTable`. We can add an option like `catalog_type: "hms" | "unity" | "iceberg"` to support various catalog CREATE TABLE syntaxes.

MrPowers

Sbt version is too old to work with specially from updated IDE's

alfonsorr

Fix #172

zeotuan

assertNotNull spark sql functions

1

Making AssertNotNull Expression available as spark sql functions. This this particular helpful when spark is unable to know that a column not nullable when working with Schema on write format...

zeotuan

Description: Currently, updating the nullability metadata of a DataFrame schema in Spark is most commonly done via: ```scala spark.createDataFrame(df.rdd, newSchema) ``` This approach is inefficient and low-level, especially when working...

zeotuan

zeotuan