HIVE-28118: Hive Insert Into S3 with Viewfs overload scheme fails wit…

[eshwitha]

…h MoveTask Error

What changes were proposed in this pull request?

During hive staging, isPathEncrypted of the source tmp path returns false, check for isPathEncrypted of destination tmp path leads to resolvePath which throws a FileNotFoundException leading to MoveTask error. We add a check for file existence before call to isPathEncrypted.

Why are the changes needed?

When ViewFs Overload scheme is enabled and HDFS encryption is enabled (hadoop.security.key.provider.path is not null), hive insert into fails in below cases without the fix:

1. Encryption zone is not enabled for hive warehouse directory.
2. Insert Into is done for the partition present in S3 (mount link created using Viewfs overload).

Does this PR introduce any user-facing change?

Is the change a dependency upgrade?

No

How was this patch tested?

Tests to verify the behaviour are added.

[eshwitha]

@ayushtkn Can you please review the changes? Thanks.

[eshwitha]

The test -> TestEncryptedHDFSCliDriver passes in my local branch. Command used: mvn test -Dtest=TestEncryptedHDFSCliDriver -Dqfile=encryption_load_data_to_encrypted_tables.q -pl itests/qtest -Pitests

[sonarqubecloud[bot]]

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[sonarqubecloud[bot]]

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.

[eshwitha]

Request to kindly review @ayushtkn @deniskuzZ

[sonarqubecloud[bot]]

Quality Gate passed

Issues
8 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[deniskuzZ]

@eshwitha, new tests fail with NullPointer https://ci.hive.apache.org/job/hive-precommit/job/PR-5137/10/

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.