[FLINK-22315][table] Support add/modify column/constraint/watermark for ALTER TABLE statement

[LadyForest]

What is the purpose of the change

This pull request aims to support the impl of

ALTER TABLE table_name  ADD | MODIFY { <schema_component> | (<schema_component> [, ...]) }

More specifically, this work is a subtask of FLINK-21634 and is based on FLINK-26813.

Brief change log

• Changes made on module flink-sql-parser

  • Remove the TODO left in parserImpl and convert alter table add constraint to the SqlAlterTableSchema representation, instead of SqlAlterTableAddConstraint
  • Fix typo in SqlAlterTableModify
  • Let SqlAlterTableSchema also implement ExtendedSqlNode to perform a type spec rewrite during the validation. This aligns with the behavior of SqlCreateTable, which changes column nullability when the column is a primary key.
  • Introduce a util class SqlConstraintValidator to reuse code.

• Changes made on module flink-table-api-java

  • Let AlterTableSchemaOperation#asSummaryString use catalogTable#getUnresolvedSchema. This is because getSchema is deprecated, and the default method is to return null, which causes NPE for subclasses that do not override this method.

• Changes made on module flink-table-common

  • Make public constructors of UnresolvedPhysicalColumn, UnresolvedComputedColumn and UnresolvedMetadataColumn, to build a list of unresolved columns in arbitrary order.

• Changes made on module flink-table-planner

  • Add AlterTableSchemaUtil to perform the conversion. This util method performs a basic check on adds/modifies duplicate primary keys/adds an existed column/modify nonexistent column/refers to a nonexistent column etc.
  • Convert SqlAlterTableSchema to AlterTableSchemaOperation

As discussed on the Jira page, the planner does not validate the incompatible type changes such like modify the column type from STRING to BIGINT etc.

Verifying this change

This change added tests and can be verified as follows:

• Adapt FlinkSqlParserImplTest, add one test to cover nested type, and add one case to verify add/modify duplicate pk will throw the expected exception.
• Add tests in SqlToOperationConverterTest to verify all the behaviors are expected.
• Add tests in TableEnvironmentTest to verify the expected altered table schema.
• Add tests in table.q to verify SqlCLI works as expected.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): yes, change the constructor's access modifier
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduces a new feature? yes
• If yes, how is the feature documented? FLINK-22320 will document this new feature

[flinkbot]

CI report:

• 1c78796663642cd0479bb37532e9418b6575fa35 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[LadyForest]

@flinkbot run azure

[LadyForest]

Hi @lsyldliu, do you have time to take a look?

[lsyldliu]

Hi @lsyldliu, do you have time to take a look?

Thanks for your contribution, I will review it as soon as possible.

[LadyForest]

@flinkbot run azure

[LadyForest]

@flinkbot run azure

[LadyForest]

Hi @lsyldliu, could you spare some time to take a look?

[LadyForest]

@flinkbot run azure

[LadyForest]

@flinkbot run azure

[LadyForest]

@flinkbot run azure

[LadyForest]

Update 9545b73 to 03a65c64

[LadyForest]

Update fb4e41c to 6470f1cc

[LadyForest]

Hi @fsk119, I've addressed the comments. Would you have time to revisit it? By the way, ALTER TABLE MODIFY will be tracked by FLINK-22316