starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Enhancement] Support Named Parameters in Scala Functions

Open EdwardArchive opened this issue 3 weeks ago • 11 comments

Issue: #66227
Related PR: #66206
Status: Developing on SSRF (Named Parameter done)
Authors: @EdwardArchive, with review from @alvin-celerdata

Why I'm doing:

Named Arguments Support for Scalar Functions

Summary

This PR implements Named Arguments support for scalar functions in StarRocks, using http_request() as the first function to leverage this feature.

Features

  • Named Arguments syntax: function(param => value, ...)
  • Default values: Optional parameters can be omitted
  • Positional calls: Traditional positional syntax still works
  • User-friendly error messages: Clear hints for common mistakes

Usage Examples

-- Named Arguments (any order, omit optional params)
SELECT http_request(url => 'https://api.example.com');
SELECT http_request(url => 'https://api.example.com', method => 'POST', body => '{}');
SELECT http_request(method => 'POST', url => 'https://api.example.com');  -- order doesn't matter

-- Positional Arguments (still supported) SELECT http_request('https://api.example.com');

<clipboard-copy aria-label="Copy" class="ClipboardButton btn js-clipboard-copy m-2 p-0" data-copy-feedback="Copied!" data-tooltip-direction="w" value="-- Named Arguments (any order, omit optional params) SELECT http_request(url => 'https://api.example.com'); SELECT http_request(url => 'https://api.example.com', method => 'POST', body => '{}'); SELECT http_request(method => 'POST', url => 'https://api.example.com'); -- order doesn't matter

-- Positional Arguments (still supported) SELECT http_request('https://api.example.com');" tabindex="0" role="button" style="box-sizing: border-box; position: relative; display: inline-block; padding: 0px !important; font-size: 14px; font-weight: 500; line-height: 20px; white-space: nowrap; vertical-align: middle; cursor: pointer; user-select: none; border: 1px solid rgb(209, 217, 224); border-radius: 6px; appearance: none; color: rgb(37, 41, 46); background-color: rgb(246, 248, 250); box-shadow: none; transition: color 80ms cubic-bezier(0.33, 1, 0.68, 1), background-color, box-shadow, border-color; margin: 8px !important;"><svg aria-hidden="true" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-copy js-clipboard-copy-icon m-2">

Error Messages

Scenario Error Message
Missing required param http_request() required parameter 'url' is missing
Unknown param (with hint) http_request() unknown parameter 'URL'. Did you mean 'url'?
Duplicate param http_request() duplicate parameter 'url'
NULL for required param http_request() required parameter 'url' cannot be NULL
No arguments http_request() requires at least 1 argument(s). Missing required parameter(s): 'url'

Architecture Flow

SQL: http_request(url => '...', method => 'POST')
│
▼
┌─────────────────────────────────────┐
│ 1. Parser (StarRocks.g4)            │
│    Parse param =&gt; value syntax    │
└─────────────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 2. AstBuilder.java                  │
│    Create NamedArgument AST nodes   │
└─────────────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 3. FunctionParams.java              │
│    Separate exprs[] + exprsNames[]  │
└─────────────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 4. ExpressionAnalyzer.java          │
│    Branch: Named vs Positional      │
└─────────────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 5. FunctionAnalyzer.java            │
│    Validate & lookup function       │
└─────────────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 6. FunctionParams.java              │
│    Reorder args & append defaults   │
└─────────────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 7. Backend (C++)                    │
│    Receive all 8 columns            │
└─────────────────────────────────────┘
<svg aria-hidden="true" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-copy js-clipboard-copy-icon m-2">

Backward Compatibility

  • No impact on existing functions: Only functions with named_args defined use this path
  • Positional calls still work: http_request('url') works as expected
  • Varargs functions excluded: Functions like concat() are not affected

Constraints

  • No mixed mode: All arguments must be either named or positional (not both)
  • Case-sensitive: Parameter names are case-sensitive (url  URL)
  • Varargs not supported: Functions with variable arguments cannot use Named Arguments

[!NOTE] Adds named-argument syntax and end-to-end handling for scalar functions, including parsing, validation, reordering, default-value filling, friendlier errors, codegen metadata, and tests.

  • SQL Parsing / AST:
    • Support named-argument syntax in StarRocks.g4 with argumentList/namedArgumentList and namedArgument.
    • AstBuilder adds helpers to read mixed arg lists and rewrites function-call handling to work with named args across many functions.
  • Analyzer:
    • ExpressionAnalyzer: detects named args, looks up function by names, validates structure, reorders params, appends defaults, re-analyzes children, validates NULL constraints; adds positional fallback with defaults and friendlier errors.
    • FunctionAnalyzer: new APIs for named args (getAnalyzedFunctionForNamedArgs, validateNamedArguments*, reorderNamedArgAndAppendDefaults, appendDefaultsForPositionalArgs, getNamedArgStr, friendly error helpers); relaxes arg-count check to enable named-arg fallback.
  • Function Params:
    • FunctionParams exposes setExprsNames and preserves extracted names from NamedArgument.
  • Codegen:
    • gen_functions.py emits FE metadata for named args, including setArgNames and default Expr values into generated VectorizedBuiltinFunctions.
  • Tests:
    • Add NamedArgumentValidatorTest and FunctionParamsTest covering validation, reordering, defaults, NULL checks, and mixing detection.

Written by Cursor Bugbot for commit 5d6232f2e8761b26037f3c41ea7fd4d06b6506cc. This will update automatically on new commits. Configure here.

This could lead to runtime errors or unexpected behavior.

What I'm doing:

This PR adds comprehensive validation for named arguments in scalar functions:

  1. Validates that all required parameters (without defaults) are provided
  2. Checks for duplicate parameter names
  3. Validates that required parameters cannot be NULL
  4. Provides user-friendly error messages with suggestions for typos
  5. Moves validation logic from FunctionParams to FunctionAnalyzer to resolve module dependency issues

What type of PR is this:

  • [ ] BugFix
  • [ ] Feature
  • [x] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [x] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [x] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [ ] I have checked the version labels which the pr will be auto-backported to the target branch
    • [x] 4.0
    • [ ] 3.5
    • [ ] 3.4
    • [ ] 3.3

EdwardArchive avatar Dec 04 '25 19:12 EdwardArchive

🧪 CI Insights

Here's what we observed from your CI run for e8ead0e6.

🟢 All jobs passed!

But CI Insights is watching 👀

mergify[bot] avatar Dec 04 '25 19:12 mergify[bot]

@alvin-celerdata Hi, here’s a separate PR for the Named Parameters in Scala functions. Please take a look when you have some time.

EdwardArchive avatar Dec 04 '25 20:12 EdwardArchive

@cursor review

alvin-celerdata avatar Dec 04 '25 21:12 alvin-celerdata

@EdwardArchive, one thing: could you add some SQL tests to verify that your changes work?

alvin-celerdata avatar Dec 04 '25 21:12 alvin-celerdata

@EdwardArchive, one thing: could you add some SQL tests to verify that your changes work? Hi @alvin-celerdata, do you have any ideas for creating test SQL on this PR? This PR doesn't have any Scala function changes, so should the test SQL be for running non-named parameter functions?

EdwardArchive avatar Dec 05 '25 04:12 EdwardArchive

@EdwardArchive, one thing: could you add some SQL tests to verify that your changes work? Hi @alvin-celerdata, do you have any ideas for creating test SQL on this PR? This PR doesn't have any Scala function changes, so should the test SQL be for running non-named parameter functions?

After suggesting, I found out that there are no functions that have named argument functions, right now. It makes it difficult to add SQL test, but I believe UT should be added to cover the changes.

alvin-celerdata avatar Dec 05 '25 17:12 alvin-celerdata

@EdwardArchive, one thing: could you add some SQL tests to verify that your changes work? Hi @alvin-celerdata, do you have any ideas for creating test SQL on this PR? This PR doesn't have any Scala function changes, so should the test SQL be for running non-named parameter functions?

After suggesting, I found out that there are no functions that have named argument functions, right now. It makes it difficult to add SQL test, but I believe UT should be added to cover the changes.

Hi I just added.

EdwardArchive avatar Dec 06 '25 15:12 EdwardArchive

@cursor review

alvin-celerdata avatar Dec 06 '25 16:12 alvin-celerdata

@alvin-celerdata Hi, is there any comment on my code? thanks

Done!

EdwardArchive avatar Dec 07 '25 13:12 EdwardArchive

@cursor review

alvin-celerdata avatar Dec 09 '25 17:12 alvin-celerdata

@cursor review

alvin-celerdata avatar Dec 10 '25 17:12 alvin-celerdata

@alvin-celerdata Hi, this PR(https://github.com/StarRocks/starrocks/pull/66206) depends on the Named Arguments feature I'm working on in another branch. I'd like to set that branch as the base for this PR (stacked PRs), but I can only select branches from the upstream repo, not from my fork. Is there a recommended way to handle dependent PRs here?

EdwardArchive avatar Dec 12 '25 22:12 EdwardArchive

@cursor review

alvin-celerdata avatar Dec 14 '25 00:12 alvin-celerdata