doris icon indicating copy to clipboard operation
doris copied to clipboard

[Enhancement](load) add hidden_columns in stream load param

Open yixiutt opened this issue 3 years ago • 0 comments

Stream load will ignore invisible columns if no http header columns specified, but in some case user cannot get all columns if columns changed frequently。 Add a hidden_columns header to support hidden columns import。User can set hidden_columns such as DORIS_DELETE_SIGN and add this column in stream load data so we can delete this line. For example: curl -u root -v --location-trusted -H "hidden_columns: DORIS_DELETE_SIGN" -H "format: json" -H "strip_outer_array: true" -H "jsonpaths: ["$.id", "$.name","$.DORIS_DELETE_SIGN"]" -T 1.json http://{beip}:{be_port}/api/test/test1/_stream_load

Proposed changes

Issue Number: close #xxx

Problem summary

Describe your changes.

Checklist(Required)

  1. Does it affect the original behavior:
    • [ ] Yes
    • [ ] No
    • [ ] I don't know
  2. Has unit tests been added:
    • [ ] Yes
    • [ ] No
    • [ ] No Need
  3. Has document been added or modified:
    • [ ] Yes
    • [ ] No
    • [ ] No Need
  4. Does it need to update dependencies:
    • [ ] Yes
    • [ ] No
  5. Are there any changes that cannot be rolled back:
    • [ ] Yes (If Yes, please explain WHY)
    • [ ] No

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

yixiutt avatar Aug 09 '22 09:08 yixiutt