starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Feature] Implement variant type casting functions

Open xxubai opened this issue 2 weeks ago • 8 comments

Why I'm doing:

What I'm doing:

  1. Support casting variant type to primitive types, decimal, array, struct, map and JSON.
  2. Introduce some query functions: get_variant_string, get_variant_bool, get_variant_int, get_variant_double etc.

Will add some documents in the following PRs

What type of PR is this:

  • [ ] BugFix
  • [x] Feature
  • [ ] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [x] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [ ] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [ ] I have checked the version labels which the pr will be auto-backported to the target branch
    • [ ] 4.0
    • [ ] 3.5
    • [ ] 3.4
    • [ ] 3.3

[!NOTE] Adds comprehensive VARIANT casting (to numeric, boolean, string, decimal, JSON, ARRAY/MAP/STRUCT) and new query helpers get_variant_{bool,int,double,string}.

  • Casting/Type System:

    • Implement casting from VARIANT to primitives (BOOLEAN/INT/FLOAT/DOUBLE/STRING), decimals (DECIMAL32/64/128), JSON, and complex types (ARRAY/MAP/STRUCT).
    • Add CastVariantToArray, CastVariantToMap, CastVariantToStruct expressions and integrate into VectorizedCastExprFactory.
    • Extend decimal cast pipeline to handle VARIANT -> DECIMAL*.
    • Update string/JSON cast paths to accept VARIANT and disable JIT for VARIANT casts.
    • FE: adjust TypeManager and PrimitiveType implicit/compatibility rules for VARIANT (including complex types) and JSON interactions.
  • Variant utilities:

    • Introduce variant_converter for casting Variant to target types; expose cast_variant_value_to helpers.
    • Enhance VariantPathParser::seek validation and Parquet VariantMetadata::get_index binary search logic.
  • Functions/APIs:

    • Add builtins: get_variant_bool, get_variant_int, get_variant_double, get_variant_string; wire prepare/close and codegen entries.
    • Extend VariantFunctions to support typed queries via shared path parsing and casting.
  • Build/Plumbing:

    • Register new sources in CMake; update function registry (gensrc/script/functions.py).

Written by Cursor Bugbot for commit 888c1afb383441823e9ff10776483674a4c7e088. This will update automatically on new commits. Configure here.

xxubai avatar Dec 10 '25 04:12 xxubai

🧪 CI Insights

Here's what we observed from your CI run for 888c1afb.

🟢 All jobs passed!

But CI Insights is watching 👀

mergify[bot] avatar Dec 10 '25 04:12 mergify[bot]

@cursor review

alvin-celerdata avatar Dec 10 '25 06:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 10 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 11 '25 15:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 15 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 16 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 17 '25 05:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 17 '25 18:12 alvin-celerdata

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Dec 18 '25 07:12 CLAassistant

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Dec 18 '25 10:12 github-actions[bot]

[FE Incremental Coverage Report]

:white_check_mark: pass : 14 / 14 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/sql/common/TypeManager.java 12 12 100.00% []
:large_blue_circle: com/starrocks/connector/iceberg/IcebergApiConverter.java 2 2 100.00% []

github-actions[bot] avatar Dec 18 '25 10:12 github-actions[bot]

[BE Incremental Coverage Report]

:x: fail : 366 / 500 (73.20%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: be/src/exprs/cast_expr.h 19 38 50.00% [217, 218, 219, 220, 222, 223, 227, 239, 245, 267, 277, 278, 279, 280, 281, 282, 285, 286, 290]
:large_blue_circle: be/src/exprs/decimal_cast_expr.h 63 102 61.76% [464, 465, 466, 469, 477, 478, 479, 480, 481, 483, 491, 492, 493, 494, 495, 497, 498, 499, 500, 501, 502, 504, 505, 506, 507, 508, 509, 511, 518, 521, 523, 525, 529, 530, 531, 570, 576, 579, 580]
:large_blue_circle: be/src/exprs/cast_expr.cpp 135 193 69.95% [195, 261, 262, 267, 268, 275, 276, 277, 559, 648, 672, 696, 1159, 1160, 1224, 1408, 1776, 1777, 1778, 1779, 1817, 1818, 1819, 1820, 1821, 1822, 1825, 1826, 1827, 1828, 1829, 1830, 1837, 1838, 1839, 1858, 1859, 1860, 1871, 1872, 1873, 1874, 1889, 1892, 2001, 2005, 2006, 2007, 2008, 2011, 2012, 2013, 2014, 2018, 2019, 2020, 2021, 2022]
:large_blue_circle: be/src/exprs/cast_expr_map.cpp 54 64 84.38% [101, 102, 123, 124, 147, 148, 149, 150, 151, 152]
:large_blue_circle: be/src/exprs/cast_expr_array.cpp 44 48 91.67% [318, 340, 341, 376]
:large_blue_circle: be/src/exprs/cast_expr_struct.cpp 51 55 92.73% [136, 172, 193, 205]

github-actions[bot] avatar Dec 18 '25 11:12 github-actions[bot]