materialize
materialize copied to clipboard
sql-parser: move minor `CREATE SOURCE` options to `WITH` block
More context in Slack, but the main idea is to move options like REMOTE
and TIMELINE
to the new WITH (OPT1 VAL1 OPT2 VAL2)
block that supports order-independent settings, similar to CREATE CONNECTION
already works.
#14082 is done, pulling REMOTE
to live with SIZE
in the new WITH
block.
I'm trying to figure out which other options we'd want to roll in here, if any. timeline
lives in the legacy WITH
block; is the idea that we'd roll everything that lives in legacy_with_options
over to the new format and style? Only hoist timeline
over? Some other thing?
cc @benesch as the original idea haver.
We should cleanly separate the options! Some of the legacy with options (like start_offset
) ought to move to a Kafka-specific option block. Only the options that are unrelated to the connector, format, or envelope should move to the new top-level WITH
block. I think that's just REMOTE
, SIZE
, and TIMELINE
, but not sure. TIMELINE
is also not something we want to expose to users quite yet, I don't think, so no particular urgency to move it.
REMOTE
, SIZE
, and TIMELINE
have been moved, and we've decided against moving epoch_ms_timeline
... so it looks like we're done here!
There are a few other odds and ends that are moving to source- or envelope-specific option blocks; see https://github.com/MaterializeInc/materialize/issues/14341 for some related work.