[FLINK-37913][table] Add built-in OBJECT_OF function
What is the purpose of the change
This pull request implements the OBJECT_OF function as specified in FLIP-520 to enable creation of structured types in SQL and Table API. The function allows users to construct structured objects from key-value pairs without requiring UDFs, making structured type creation SQL-serializable and improving the overall usability of structured types in Flink.
Brief change log
- Added
OBJECT_OFfunction definition toBuiltInFunctionDefinitions - Implemented
ObjectOfInputTypeStrategyfor input validation (odd argument count, string keys, unique field names) - Implemented
ObjectOfTypeStrategyfor output type inference (creates structured types) - Added
ObjectOfFunctionruntime implementation that createsRowDatafrom key-value pairs - Enhanced
Expressions.javawithobjectOf()methods for Table API support - Added Python API support with
object_of()function inexpressions.py - Updated SQL functions documentation with
OBJECT_OFfunction details
Verifying this change
This change added tests and can be verified as follows:
- Added unit tests for
ObjectOfInputTypeStrategyto validate argument count, type validation, and field name uniqueness - Added integration tests in
StructuredFunctionsITCaseto test end-to-end functionality - Verified SQL usage:
SELECT OBJECT_OF('com.example.User', 'name', 'Bob', 'age', 42) - Verified Table API usage:
objectOf(User.class, "name", "Bob", "age", 42)
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
@Public(Evolving): yes (additions toExpressions.javaandBuiltInFunctionDefinitions) - The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs / JavaDocs (updated SQL functions documentation, comprehensive JavaDocs for all new classes and methods, Python API documentation)
CI report:
- a49b6ada6d0a5009347d2a8290c66a8d7078def9 Azure: SUCCESS
Bot commands
The @flinkbot bot supports the following commands:@flinkbot run azurere-run the last Azure build
I think there's a checkstyle violation making the CI fail. Make sure to run ./mvnw checkstyle:check locally
@twalthr There is also another case that I have prepared, but so far, it has been stashed on my branch. In the FLIP-520, you mentioned under the table API, this signature:
Expressions.objectOf(DataType, Object... kv);
In SQL, this would be equivalent to:
SELECT OBJECT_OF(OBJECT_OF(...)));
Is this a case we should also support in SQL? Does this mean that we can construct structure types from structured types?