[Feature] Stream load support JSON to STRUCT/MAP/ARRAY
Why I'm doing:
Now, the SR does't support loading JSON to complex type column.
What I'm doing:
This PR adds the support of JSON format loading to STRUCT/MAP/ARRAY type column.
Here are some examples.
create table tbl_map (key1 int, key2 map<string,string>);
curl --location-trusted -u root: 'http://127.0.0.1:18040/api/db0/tbl_map/_stream_load'
\-X PUT \-H 'format: json' -H "columns:key1,key2"
-d '{"key1":1,"key2":{"key3":"aaa"}}'
create tablet tbl_struct (key1 int, key2 struct<key3, string>);
curl --location-trusted -u root: 'http://127.0.0.1:18040/api/db0/tbl_struct/_stream_load'
\-X PUT \-H 'format: json' -H "columns:key1,key2"
-d '{"key1":1,"key2":{"key3":"aaa"}}'
create tablet tbl_array (key1 int, key2 array<string>);
curl --location-trusted -u root: 'http://127.0.0.1:18040/api/db0/tbl_array/_stream_load'
\-X PUT \-H 'format: json' -H "columns:key1,key2"
-d '{"key1":1,"key2":["aaa","bbb"]}'
Fixes https://github.com/StarRocks/starrocks/issues/43101
What type of PR is this:
- [ ] BugFix
- [x] Feature
- [ ] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [x] Yes, this PR will result in a change in behavior.
- [ ] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [x] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [x] I have added test cases for my bug fix or my new feature
- [x] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
- [ ] 3.3
- [ ] 3.2
- [ ] 3.1
- [ ] 3.0
- [ ] 2.5
@mergifyio rebase
rebase
✅ Branch has been successfully rebased
@mergifyio rebase
rebase
✅ Branch has been successfully rebased
[FE Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
[BE Incremental Coverage Report]
:white_check_mark: pass : 118 / 134 (88.06%)
file detail
| path | covered_line | new_line | coverage | not_covered_line_detail | |
|---|---|---|---|---|---|
| :large_blue_circle: | be/src/formats/json/nullable_column.cpp | 38 | 46 | 82.61% | [213, 214, 234, 235, 249, 250, 267, 268] |
| :large_blue_circle: | be/src/formats/json/struct_column.cpp | 16 | 19 | 84.21% | [29, 30, 31] |
| :large_blue_circle: | be/src/formats/json/map_column.cpp | 25 | 28 | 89.29% | [30, 31, 32] |
| :large_blue_circle: | be/src/exec/json_scanner.cpp | 39 | 41 | 95.12% | [279, 550] |
https://github.com/Mergifyio backport branch-3.3
backport branch-3.3
✅ Backports have been created
- #46448 [Feature] Stream load support JSON to STRUCT/MAP/ARRAY (backport #45406) has been created for branch
branch-3.3
https://github.com/Mergifyio backport branch-3.2
backport branch-3.2
✅ Backports have been created
- #47233 [Feature] Stream load support JSON to STRUCT/MAP/ARRAY (backport #45406) has been created for branch
branch-3.2