arrow icon indicating copy to clipboard operation
arrow copied to clipboard

ARROW-17995: [C++] Fix json decimals not being rescaled based on the explicit schema

Open stiga-huang opened this issue 3 years ago • 2 comments

arrow::json::DecimalConverter::Convert() currently read the decimal values using the parsed precision and scale. This produces wrong results if the parsed scale doesn't match the output scale (specified by explicit schema).

More details on how to reproduce the issue are in the JIRA description. This patch fixes json::DecimalConverter::Convert() to rescale the values based on the output scale. Unit tests are added as well.

stiga-huang avatar Oct 12 '22 00:10 stiga-huang

https://issues.apache.org/jira/browse/ARROW-17995

github-actions[bot] avatar Oct 12 '22 00:10 github-actions[bot]

:warning: Ticket has not been started in JIRA, please click 'Start Progress'.

github-actions[bot] avatar Oct 12 '22 00:10 github-actions[bot]

Thanks for your review! @pitrou

stiga-huang avatar Oct 15 '22 23:10 stiga-huang

Benchmark runs are scheduled for baseline = aeba61663fdd82719e6cc0945aba216958ad6970 and contender = 289e0c92d7a14c279b15a22fca5ce4839b404116. 289e0c92d7a14c279b15a22fca5ce4839b404116 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. Conbench compare runs links: [Finished :arrow_down:0.0% :arrow_up:0.0%] ec2-t3-xlarge-us-east-2 [Failed :arrow_down:0.0% :arrow_up:0.56%] test-mac-arm [Finished :arrow_down:0.54% :arrow_up:0.0%] ursa-i9-9960x [Finished :arrow_down:0.75% :arrow_up:0.07%] ursa-thinkcentre-m75q Buildkite builds: [Finished] 289e0c92 ec2-t3-xlarge-us-east-2 [Failed] 289e0c92 test-mac-arm [Finished] 289e0c92 ursa-i9-9960x [Finished] 289e0c92 ursa-thinkcentre-m75q [Finished] aeba6166 ec2-t3-xlarge-us-east-2 [Failed] aeba6166 test-mac-arm [Finished] aeba6166 ursa-i9-9960x [Finished] aeba6166 ursa-thinkcentre-m75q Supported benchmarks: ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True test-mac-arm: Supported benchmark langs: C++, Python, R ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

ursabot avatar Oct 17 '22 17:10 ursabot

['Python', 'R'] benchmarks have high level of regressions. test-mac-arm ursa-i9-9960x

ursabot avatar Oct 17 '22 17:10 ursabot