hudi
hudi copied to clipboard
[SUPPORT]--source-ordering-field should be deprecated to honor precombine field
I am using Hudi 0.15, I find that ORDERING_FIELD has been deprecated with PRECOMBINE_FIELD_NAME,
/** @deprecated Use {@link HoodieWriteConfig#PRECOMBINE_FIELD_NAME} and its methods instead */
@Deprecated
public static final ConfigProperty<String> ORDERING_FIELD = ConfigProperty
.key(PAYLOAD_ORDERING_FIELD_PROP_KEY)
.defaultValue("ts")
.markAdvanced()
.withDocumentation("Table column/field name to order records that have the same key, before "
+ "merging and writing to storage.");
but HoodieStreamer still use --source-ordering-field as parameter, I think HoodieStreamer should also deprecate --source-ordering-field and introduce a new parameter to honor precombine field to make concept consistent
@Parameter(names = {"--source-ordering-field"}, description = "Field within source record to decide how"
+ " to break ties between records with same key in input data. Default: 'ts' holding unix timestamp of record")
public String sourceOrderingField = "ts";
Agree, maybe you can fire a fix for it, we can firstly deprecate it.
@bithw1 Do you want to contribute on this?
IIRC source ordering field config in hudi streamer is being used to set precombine field, so this is really just a naming mismatch/confusion (?)