hive icon indicating copy to clipboard operation
hive copied to clipboard
verified publisher icon apache

HIVE-27370: support 4 bytes characters

Open ryukobayashi opened this issue 2 years ago • 9 comments

What changes were proposed in this pull request?

If a SUBSTR UDF has a 4-byte characters in its parameter, the behavior is different between vectorized and non-vectorized. The vectorized version handles 4-byte characters properly, but the non-vectorized version does not, so similar logic is needed. And these fixes use vectorized logic: https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringSubstrColStartLen.java#L89-L130 https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringSubstrColStart.java#L78-L109

Why are the changes needed?

Vectorized and non-vectorized have different results.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Added pattern tests to itest for these to work correctly.

ryukobayashi avatar Jan 12 '24 06:01 ryukobayashi

Quality Gate Passed Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

2 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

sonarqubecloud[bot] avatar Jan 12 '24 08:01 sonarqubecloud[bot]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.

github-actions[bot] avatar Mar 13 '24 00:03 github-actions[bot]

@ryukobayashi, could you please rebase

deniskuzZ avatar Apr 09 '24 11:04 deniskuzZ

@deniskuzZ I updated it to the latest.

ryukobayashi avatar Apr 10 '24 08:04 ryukobayashi

Quality Gate Passed Quality Gate passed

Issues
10 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

sonarqubecloud[bot] avatar Apr 11 '24 09:04 sonarqubecloud[bot]

Hi @deniskuzZ I would if you could check about this when you have time.

ryukobayashi avatar Apr 26 '24 07:04 ryukobayashi

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.

github-actions[bot] avatar Jun 26 '24 00:06 github-actions[bot]

sorry, I'll try to review ASAP, but i have many other PRs on me pending review @SourabhBadhya, could you please check this

deniskuzZ avatar Jun 27 '24 13:06 deniskuzZ