doris icon indicating copy to clipboard operation
doris copied to clipboard

[fix](parquet)Fix the be core issue when reading parquet unsigned types. (#39926)

Open hubgeter opened this issue 1 year ago • 3 comments

bp #39926

Proposed changes

Since Doris does not have an unsigned type, we convert parquet uint32 type to doris bigint (int64) type. When reading the parquet file, the byte size stored in parquet and the byte size of the data type mapped by doris are inconsistent, resulting in be core. Fix: When reading, we read according to the byte size stored in parquet, and then convert it to the data type mapped by doris.

Mapping relationship description: parquet -> doris
UInt8 -> Int16 UInt16 -> Int32 UInt32 -> Int64 UInt64 -> Int128.

Proposed changes

Issue Number: close #xxx

hubgeter avatar Aug 29 '24 10:08 hubgeter

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar Aug 29 '24 10:08 doris-robot

run buildall

hubgeter avatar Aug 29 '24 10:08 hubgeter

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Aug 29 '24 10:08 github-actions[bot]