starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Feature] Support Paimon view

Open zyl891229 opened this issue 10 months ago • 8 comments

Why I'm doing:

Paimon has supported view to Hive Catalog https://github.com/apache/incubator-paimon/pull/4340

What I'm doing:

Implement Paimon view in Starrocks, includes the following:

  1. Support create/replace/drop Paimon view;
  2. Support show/desc Paimon view;
  3. Support read Paimon view;

Optimized the return value to make the appearance of the table building information clearer show create table adds catalog and database to the return values of ext table and ext view

Fixes #56080

What type of PR is this:

  • [ ] BugFix
  • [x] Feature
  • [ ] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [x] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [x] 3.4
    • [x] 3.3
    • [ ] 3.2
    • [ ] 3.1
    • [ ] 3.0

[!NOTE] Add Paimon view support (create/list/drop/show/read) and include catalog.db in external table/view DDL; introduce Paimon type converters and HMS getTables implementations.

  • Paimon Views:
    • Add TableType.PAIMON_VIEW, PaimonView (extends ConnectorView).
    • PaimonMetadata: list views, createView, dropTable handling views, getView retrieval.
    • Update auth (Authorizer) and MV analyzer to recognize PAIMON_VIEW.
  • Type Conversion:
    • ColumnTypeConverter: add fromPaimonSchemas, toPaimonRowType, toPaimonDataType for bi-directional Paimon<->StarRocks type mapping.
  • DDL Formatting:
    • AstToStringBuilder: SHOW CREATE for external tables/views now prints catalog.db.table.
    • ConnectorView: expose getCatalogName/getCatalogDBName for view DDL.
  • Hive Metastore Client:
    • Implement getTables(...) variants and integrate MetaStoreFilterHook.
  • Tests:
    • Add/extend Paimon metadata and type converter tests (incl. views).
    • Update Hive/Iceberg golden results to include catalog and db.
    • Enhance SQL test harness to persist and reuse UUID variables in expectations.

Written by Cursor Bugbot for commit 99625a1a23f9073c1b89b306d13e42c85b638845. This will update automatically on new commits. Configure here.

zyl891229 avatar Feb 19 '25 11:02 zyl891229

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

sonarqubecloud[bot] avatar Feb 28 '25 08:02 sonarqubecloud[bot]

Is this PR still being maintained? We also need this feature recently. If no one maintains it, we will develop it based on this. @zyl891229 @stephen-shelby @mxdzs0612

zenoyang avatar Apr 21 '25 12:04 zenoyang

Is this PR still being maintained? We also need this feature recently. If no one maintains it, we will develop it based on this. @zyl891229 @stephen-shelby @mxdzs0612

Thanks, we will continue to review and merge this PR

miomiocat avatar Apr 22 '25 03:04 miomiocat

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

sonarqubecloud[bot] avatar May 06 '25 03:05 sonarqubecloud[bot]

[FE Incremental Coverage Report]

:x: fail : 66 / 93 (70.97%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 0 11 00.00% [355, 357, 358, 359, 360, 361, 362, 902, 908, 914, 927]
:large_blue_circle: com/starrocks/sql/analyzer/AstToStringBuilder.java 1 3 33.33% [406, 407]
:large_blue_circle: com/starrocks/catalog/Table.java 2 3 66.67% [134]
:large_blue_circle: com/starrocks/connector/paimon/PaimonMetadata.java 23 31 74.19% [97, 183, 184, 185, 197, 198, 199, 200]
:large_blue_circle: com/starrocks/connector/ColumnTypeConverter.java 40 45 88.89% [608, 614, 649, 653, 669]

github-actions[bot] avatar Dec 03 '25 09:12 github-actions[bot]

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 03 '25 09:12 github-actions[bot]

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 03 '25 09:12 github-actions[bot]

@cursor review

alvin-celerdata avatar Dec 03 '25 17:12 alvin-celerdata