[VL] Results are mismatch with vanilla Spark when get_json_object({"dScore":0.0215434648799772}, "$.dScore")
Backend
VL (Velox)
Bug description
The following SQL might lead to wrong results, but it's not yet certain if there are other factors involved. I will try to reproduce later.
select get_json_object(factor_context, '$.factor_trust_model_score') as model_score
FROM mart_finrisk.dwd_risk_antifraud_approve_scene_log_inc_d;
Spark version
None
Spark configurations
No response
System information
No response
Relevant logs
|gluten_model_score | vanilla_model_score
+--------------------+-----------------------
| 0.0215435 | 0.0215434648799772
| 0.0128806 | 0.012880573063864434
| 0.0114058 | 0.011405845787963208
| 0.00863517 | 0.008635174896913173
The following unit test case can reproduce the issue. I'm sure that got wrong value while parsing double in function SIMDGetJsonObjectFunction::extractStringResult().
More info about UT can refer to this commit https://github.com/kecookier/velox/commit/f87115d074772f3c00d326cd607742125b101beb#diff-fdbb1f97f88c92ea26b933c047486a61efcf71cc9cd1995f77d09fd3c7578d7aR39
EXPECT_EQ(
"0.0215434648799772",
getJsonObject(R"({"dScore":0.0215434648799772})", "$.dScore"));
---------------------------------------------------------------
[zk] dv:0.0215435 numberResult:0.0215435
Expected equality of these values:
"0.0215434648799772"
Which is: 0x53d043d
getJsonObject(R"({"dScore":0.0215434648799772})", "$.dScore")
Which is: ("0.0215435")
Hi @PHILO-HE , reviewing the commit history, I believe you have more expertise in this section. Would you be willing to assist in resolving this issue, please?
Thanks for reporting this issue! I will take a look.
Hi @kecookier, it looks the below small patch can fix this issue. Please help verify it. Thanks!
https://github.com/PHILO-HE/velox/commit/a1e9be059ceb03ca4e1c68145e516a9bb07ed956
Hi @kecookier, it looks the below small patch can fix this issue. Please help verify it. Thanks!
@PHILO-HE Thanks for your help, I'll try it out later.
Hi @PHILO-HE , I have tested it, and that patch can fix the bug.
Hi @kecookier, it looks the below small patch can fix this issue. Please help verify it. Thanks! PHILO-HE/velox@a1e9be0
@PHILO-HE Thanks for your help, I'll try it out later.