doris icon indicating copy to clipboard operation
doris copied to clipboard

[fix](ParquetReader) definition level of repeated parent is wrong

Open AshinGau opened this issue 1 year ago • 9 comments

Proposed changes

Fix three bugs:

  1. repeated_parent_def_level should be the definition of its repeated parent.
  2. Failed to parse schema like decimal(p, s)
  3. Fill wrong offsets for array type

Checklist(Required)

  • [ ] Does it affect the original behavior
  • [ ] Has unit tests been added
  • [ ] Has document been added or modified
  • [ ] Does it need to update dependencies
  • [ ] Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

AshinGau avatar Mar 02 '23 08:03 AshinGau

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Mar 02 '23 08:03 github-actions[bot]

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Mar 02 '23 10:03 github-actions[bot]

run buildall

AshinGau avatar Mar 02 '23 10:03 AshinGau

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Mar 02 '23 10:03 github-actions[bot]

TeamCity pipeline, clickbench performance test result: the sum of best hot time: 33.76 seconds stream load tsv: 469 seconds loaded 74807831229 Bytes, about 152 MB/s stream load json: 41 seconds loaded 2358488459 Bytes, about 54 MB/s stream load orc: 76 seconds loaded 1101869774 Bytes, about 13 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230306005759_clickbench_pr_108901.html

hello-stephen avatar Mar 02 '23 11:03 hello-stephen

run buildall

AshinGau avatar Mar 03 '23 01:03 AshinGau

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Mar 03 '23 01:03 github-actions[bot]

run buildall

AshinGau avatar Mar 03 '23 11:03 AshinGau

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Mar 03 '23 11:03 github-actions[bot]

run buildall

AshinGau avatar Mar 06 '23 00:03 AshinGau

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Mar 06 '23 00:03 github-actions[bot]

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar Mar 06 '23 00:03 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar Mar 06 '23 00:03 github-actions[bot]