hudi
hudi copied to clipboard
[SUPPORT] does hudi have plans to support default values
Our dataset requires that no field is allowed to have null values, but hudi currently does not support default values, resulting in new fields having null values when querying historical data. The only way currently is to update historical data.
Environment Description
-
Hudi version : 0.12.1
-
Spark version : 3.1.3
-
Hive version : 3.1.0
-
Hadoop version : 3.1.1
-
Storage (HDFS/S3/GCS..) : HDFS
-
Running on Docker? (yes/no) : no
@JoshuaZhuCN Do you have any idea/design how can that be supported or have any thoughts in your mind? Adding @nsivabalan @yihua @xushiyan @danny0405 @codope
In SQL, a tricky solution is when pasing the SQL, if a column defines a default value, the parser would translate and wrapper it as a function with a null fallback as the default.